protocol

package
v0.27.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 22, 2023 License: Apache-2.0 Imports: 13 Imported by: 9

README

This is a Golang implementation of the Choria protocol. It does not implement any networking or transport, just the protocol parts.

This is in use by the go-choria project that builds a new mcollectivelike server, broker and eventually clients.

Protocol Design

The messages in a Choria network are made of up many layers, for example should we wish to send hello world to a remote host we have to do quite a bit of work to get there:

( transport like NATS
  ( transport packet that travels over the transport
    ( security plugin that securely wraps the payload in SSL signatures etc
        ( choria message that has details about sender, filters, etc
            ( message body like `hello world` )
        )
    )
  )
)

So similar to how lets say REST lives in HTTP lives in TCP etc which allows the actual data to travel anything from high speed fibre to pigeons the Choria protocol allows similar flexibility in choosing a transport.

The middle most message is where the MCollective RPC system lives and like with HTTP, FTP etc multiple protocols can cohabit this network.

In Choria with NATS the above looks like this:

( NATS with TLS
    ( JSON encoded choria:transport:1
        ( JSON encoded choria:secure:request:1 or choria:secure:reply:1
            ( JSON encoded choria:request:1 or choria:reply:1
                ( payload for/from a given agent)
            )
        )
    )
)

The strings like choria:request:1 means it's a V1 protocol choria:request message and maps to a constant like protocol.RequestV1.

The protocol also supports Federation which further complicates matters as in federated networks there are additional wrapping of packets going on - in practice it's just data copied into the above structure rather than more wrapping.

JSON Schemas for the whole version 1 protocol can be found in the repo, these schemas are used to validate every step of the way.

Examples

Create a request and package it for transport:

// a request to the agent test_agent sent from a machine called my.host.name and a user
// identifying itself as having a certificate rip.mcollective.  The message may live for
// 120 seconds and is targetted at a sub collective de_collective
request, _ := v1.NewRequest("test_agent", "my.host.name", "choria=rip.mcollective", 120, "unique_req_id", "de_collective")

// the payload the request will copy for us
request.SetMessage("hello world")

// at this point you can claim to be anyone with any cert, no validation is done yet,
// this follows at the security layer, this will assert that the cert you give does actually
// match rip.mcollective in name and it will sign the message and fingerprint it, this will
// fail if you are unable to present a certificate that match what you claimed above
srequest, _ := v1.NewSecureRequest(request, "path/to/pubcert.pem", "path/to/privatecert.pem")

// now this message is validated that you have a matching cert and it cannot be tampered with
// by anyone, we can turn it into a transport
trequest, _ := v1.NewTransportMessage("rip.mcollective")
trequest.SetRequestData(srequest)

// finally we can get the JSON data to send over the wire using whatever means we like
j, _ := trequest.JSON()

Now the JSON above gets sent to a node using any means you like, Choria uses NATS. Decoding this is done as follows:

// read the choria config
cfg, _ := choria.NewConfig(choria.UserConfig())

// j here is what was received over the wire, we now have our Choria transport
trequest, _ := v1.NewTransportFromJSON(j)

// we parse the transport as a request which gives us a secure request - and validates the sender is
// signed by our CA etc, validates it matches the allowed cert regexes and determines if its a super
// user request or not
srequest, _ := v1.NewSecureRequestFromTransport(trequest, "/path/to/ca.pem", "/path/to/ssl_cache", cfg.Choria.CertnameWhitelist, cfg.Choria.PrivilegedUsers, false)

// we now get a request and inside it is the payload
request, _ := v1.NewRequestFromSecureRequest(srequest)

// prints "hellow world"
fmt.Println(request.Message())

This is to be honest a bit verbose, the Choria framework in go-choria has a bunch of helpers to make this much easier for you and also support things like detecting protocol versions and doing the right thing for you so in reality you'd probably use it via those, but it's totally usable without as you see.

The same basic process is followed for replies, you just need to keep unwrapping the onion :)

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrSchemaUnknown indicates the schema could not be found
	ErrSchemaUnknown = errors.New("unknown schema")
	// ErrSchemaValidationFailed indicates that the validator failed to perform validation, perhaps due to invalid schema
	ErrSchemaValidationFailed = errors.New("validation failed")
)
View Source 
var ClientStrictValidation = false

ClientStrictValidation gives hints to the protocol implementations that a client does not wish to be fully validated, this is because validation can often be very slow so clients can elect to disable that.

It's not mandatory for a specific version of implementation of the protocol to do anything with this, so it's merely a hint

View Source 
var Secure = "true"

Secure controls the signing and validations of certificates in the protocol

Functions

func CopyFederationData 

func CopyFederationData(from Federable, to Federable)

CopyFederationData copies the Federation related data from one message to another

func IsRegistrationAgent added in v0.25.0 

func IsRegistrationAgent(agent string) bool

IsRegistrationAgent determines if agent is the registration target agent

func IsRemoteSignerAgent added in v0.24.0 

func IsRemoteSignerAgent(agent string) bool

IsRemoteSignerAgent determines if agent is the standard remote signer

func IsSecure 

func IsSecure() bool

IsSecure determines if this build will validate senders at protocol level

func SchemaBytes added in v0.26.2 

func SchemaBytes(protocol ProtocolVersion) ([]byte, error)

SchemaBytes returns the JSON schema matching a specific protocol definition like `ReplyV1`

func SchemaValidate added in v0.26.2 

func SchemaValidate(protocol ProtocolVersion, data []byte) (valid bool, errors []string, err error)

SchemaValidate validates data against the JSON schema for protocol

Types

type FactFilter 

type FactFilter struct {
	Fact     string `json:"fact"`
	Operator string `json:"operator"`
	Value    string `json:"value"`
}

FactFilter is how a fact match is represented to the Filter

type Federable 

type Federable interface {
	SetFederationRequestID(id string)
	SetFederationReplyTo(reply string)
	SetFederationTargets(targets []string)
	SetUnfederated()

	FederationRequestID() (string, bool)
	FederationReplyTo() (string, bool)
	FederationTargets() ([]string, bool)

	RecordNetworkHop(in string, processor string, out string)
	NetworkHops() [][3]string

	IsFederated() bool
}

Federable is any kind of message that can carry federation headers

type Filter 

type Filter struct {
	Fact     []FactFilter          `json:"fact"`
	Class    []string              `json:"cf_class"`
	Agent    []string              `json:"agent"`
	Identity []string              `json:"identity"`
	Compound [][]map[string]string `json:"compound"`
	// contains filtered or unexported fields
}

Filter is a Choria filter

func NewFilter 

func NewFilter() *Filter

NewFilter creates a new empty filter

func (*Filter) AddAgentFilter 

func (f *Filter) AddAgentFilter(agent string)

AddAgentFilter appends a filter to the agent filters

func (*Filter) AddClassFilter 

func (f *Filter) AddClassFilter(class string)

AddClassFilter appends a filter to the class filters

func (*Filter) AddCompoundFilter 

func (f *Filter) AddCompoundFilter(query string) error

AddCompoundFilter appends a filter to the compound filters, the filter should be an expr string representing a valid choria filter

func (*Filter) AddFactFilter 

func (f *Filter) AddFactFilter(fact string, operator string, value string) (err error)

AddFactFilter appends a filter to the fact filters

func (*Filter) AddIdentityFilter 

func (f *Filter) AddIdentityFilter(id string)

AddIdentityFilter appends a filter to the identity filters

func (*Filter) AgentFilters 

func (f *Filter) AgentFilters() []string

AgentFilters retrieve the list of agent filters

func (*Filter) ClassFilters 

func (f *Filter) ClassFilters() []string

ClassFilters retrieve the list of class filters

func (*Filter) CompoundFilters 

func (f *Filter) CompoundFilters() [][]map[string]string

CompoundFilters retrieve the list of compound filters

func (*Filter) Empty 

func (f *Filter) Empty() bool

Empty determines if a filter is empty - that is all its contained filter arrays are empty

func (*Filter) FactFilters 

func (f *Filter) FactFilters() [][3]string

FactFilters retrieve the list of fact filters

func (*Filter) IdentityFilters 

func (f *Filter) IdentityFilters() []string

IdentityFilters retrieve the list of identity filters

func (*Filter) MatchAgents 

func (f *Filter) MatchAgents(knownAgents []string) bool

MatchAgents determines if the filter would match a list of agents

func (*Filter) MatchClasses 

func (f *Filter) MatchClasses(knownClasses []string, _ Logger) bool

MatchClasses determines if the filter would match against the list of classes

func (*Filter) MatchClassesFile 

func (f *Filter) MatchClassesFile(file string, log Logger) bool

MatchClassesFile determines if the filter would match a list of classes

func (*Filter) MatchCompound added in v0.19.0 

func (f *Filter) MatchCompound(facts json.RawMessage, knownClasses []string, knownAgents []string, fm ddl.FuncMap, log Logger) bool

MatchCompound determines if the filter would match against classes, facts and agents using an expr expression

func (*Filter) MatchCompoundFiles added in v0.20.0 

func (f *Filter) MatchCompoundFiles(factsFile string, classesFile string, knownAgents []string, log Logger) bool

MatchCompoundFiles determines if the filter would match against classes, facts and agents using an expr expression

func (*Filter) MatchFacts 

func (f *Filter) MatchFacts(factsj json.RawMessage, log Logger) bool

MatchFacts determines if the filter would match a given set of facts found in given JSON data

func (*Filter) MatchFactsFile 

func (f *Filter) MatchFactsFile(file string, log Logger) bool

MatchFactsFile determines if the filter would match a given set of facts found in a file

func (*Filter) MatchIdentity 

func (f *Filter) MatchIdentity(ident string) bool

MatchIdentity determines if the filter would match a given identity

func (*Filter) MatchServerRequest added in v0.20.0 

func (f *Filter) MatchServerRequest(request Request, si ServerInfoSource, log Logger) bool

type Logger 

type Logger interface {
	Warnf(format string, args ...any)
	Debugf(format string, args ...any)
	Errorf(format string, args ...any)
}

Logger provides logging facilities

type ProtocolVersion added in v0.26.2 

type ProtocolVersion string

ProtocolVersion defines known protocol versions 

const (
	RequestV1          ProtocolVersion = "choria:request:1"
	ReplyV1            ProtocolVersion = "choria:reply:1"
	SecureRequestV1    ProtocolVersion = "choria:secure:request:1"
	SecureReplyV1      ProtocolVersion = "choria:secure:reply:1"
	TransportV1        ProtocolVersion = "choria:transport:1"
	RequestV2          ProtocolVersion = "io.choria.protocol.v2.request"
	ReplyV2            ProtocolVersion = "io.choria.protocol.v2.reply"
	SecureRequestV2    ProtocolVersion = "io.choria.protocol.v2.secure_request"
	SecureReplyV2      ProtocolVersion = "io.choria.protocol.v2.secure_reply"
	TransportV2        ProtocolVersion = "io.choria.protocol.v2.transport"
	Unknown            ProtocolVersion = "io.choria.protocol.unknown"
	RemoteSigningAgent                 = "aaa_signer"
	RegistrationAgent                  = "registration"
)

func VersionFromJSON added in v0.26.2 

func VersionFromJSON(data []byte) ProtocolVersion

func (ProtocolVersion) String added in v0.26.2 

func (p ProtocolVersion) String() string

func (*ProtocolVersion) UnmarshalJSON added in v0.26.2 

func (p *ProtocolVersion) UnmarshalJSON(data []byte) error

type Reply 

type Reply interface {
	Federable

	SetMessage(message []byte)

	Message() []byte
	RequestID() string
	SenderID() string
	Agent() string
	Time() time.Time
	JSON() ([]byte, error)
	Version() ProtocolVersion
	IsValidJSON(data []byte) error
}

Reply is a core MCollective Reply containing JSON serialized agent payload

type Request 

type Request interface {
	Federable

	SetMessage(message []byte)
	SetCallerID(id string)
	SetCollective(collective string)
	SetAgent(agent string)
	NewFilter() *Filter
	SetFilter(*Filter)
	SetRequestID(id string)
	SetTTL(ttl int)

	Message() []byte
	RequestID() string
	SenderID() string
	CallerID() string
	CallerPublicData() string
	SignerPublicData() string
	Collective() string
	Agent() string
	TTL() int
	Time() time.Time
	Filter() (*Filter, bool)
	JSON() ([]byte, error)
	Version() ProtocolVersion
	IsValidJSON(data []byte) error
}

Request is a core MCollective Request containing JSON serialized agent payload

type SecureReply 

type SecureReply interface {
	SetMessage(reply Reply) error
	Valid() bool
	JSON() ([]byte, error)
	Message() []byte
	Version() ProtocolVersion
	IsValidJSON(data []byte) error
}

SecureReply is a container for a Reply. It's the reply counterpart of a SecureRequest but replies are not signed using cryptographic keys it's only hashed in transport

type SecureRequest 

type SecureRequest interface {
	SetMessage(request Request) error
	SetSigner(signer []byte) error
	Valid() bool
	JSON() ([]byte, error)
	Version() ProtocolVersion
	IsValidJSON(data []byte) error
	Message() []byte
	CallerPublicData() string
}

SecureRequest is a container for the Request. It serializes and signs the payload using the private key so that the message cannot be tampered with in any way once created. Recipients of the message can unpack it and validate it using the certificate of the stated caller

Should a message have been tampered with this validation would fail, this effectively avoids man in the middle attacks and requestor spoofing

type ServerInfoSource added in v0.20.0 

type ServerInfoSource interface {
	Classes() []string
	Facts() json.RawMessage
	Identity() string
	KnownAgents() []string
	DataFuncMap() (ddl.FuncMap, error)
}

type TransportMessage 

type TransportMessage interface {
	Federable

	SetReplyData(reply SecureReply) error
	SetRequestData(request SecureRequest) error

	SetReplyTo(reply string)
	SetSender(sender string)

	ReplyTo() string
	SenderID() string
	SeenBy() [][3]string
	Message() ([]byte, error)

	IsValidJSON(data []byte) error
	JSON() ([]byte, error)
	Version() ProtocolVersion
}

TransportMessage is a container for SecureRequests and SecureReplies it has routing information required to construct the various middleware topic names and such, it's also Federation aware and can track reply to targets, who saw it etc

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.