machines

README

Machines Watchers for Choria Autonomous Agents

This contains an Autonomous Agent Watcher plugin capable of managing the typical /etc/choria/machines directory via Choria Key-Value Store and the archive watcher.

In effect this allows you to Configuration Manage sets of Autonomous Agents on a fleet where you do not have other Configuration Management tools or where you just want to manage these out of band.

Goals

Create an opinionated manager for other machines that will safely and securely set up a server for hosting autonomous agents using the properties of the archive watcher.

If this watcher is used it is one that would be compiled into the Choria binary and configured using KV.

Autonomous Agent Archives

These archives are prepared as per the instructions in the archive watcher with the following hard constraints:

• The checksums file must be SHA256SUMS and must be present
• The tar file must create a directory matching the name exactly, yourmachine-1.2.3.tar.gz must create yourmachine
• Checksums of the SHA256SUMS file and the archive must be specified

Configuring

An Autonomous agent must be created that polls the Key-Value store and then configures the machines type watcher:

watchers:
  - name: desired_state
    type: kv
    interval: 1m
    state_match: [MANAGE]
    properties:
       bucket: MACHINES
       key: machines
       mode: poll
       bucket_prefix: false
    
  - name: manage_machines
    state_match: [MANAGE]
    type: machines
    interval: 1m
    state_matchin:
      - MANAGE
    properties:
      data_item: machines
      purge_unknown: true
      machine_manage_interval: 1m
      public_key: 64031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da

Here we set an optional public_key, when this is set to a ed25519 public key it will verify and only accept data from the data store that has a valid signature signed using the corresponding private key.

A keypair can be created using the signer command:

go run cmd/mms.go keys
 Public Key: 64031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da
Private Key: d8bd4d6392af154e996a18a4ccd5f51931d8e861d42966a677d85fbb598b66d364031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da

The data can now be created:

$ cat machines.json
[
 {
   "name": "facts",
     "source": "https://my.example.net/metadata/metadata-machine-1.0.0.tgz",
     "verify": "SHA256SUMS",
     "verify_checksum": "1e85719c6959eb0f2c8f2166e30ae952ccaef2c286f31868ea1d311d3738a339",
     "checksum": "f11ea2005de97bf309bafac46e77c01925307a26675f44f388d4502d2b9d00bf",
     "match": "has_command('facter')"
 }
]
$ go run cmd/mms.go pack machines.json d8bd4d6392af154e996a18a4ccd5f51931d8e861d42966a677d85fbb598b66d364031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da > spec.json
$ cat spec.json | choria kv put MACHINES machines -
{"machines":"WwogewogICAibmFtZSI6ICJmYWN0cyIsCiAgICAgInNvdXJjZSI6ICJodHRwczovL215LmV4YW1wbGUubmV0L21ldGFkYXRhL21ldGFkYXRhLW1hY2hpbmUtMS4wLjAudGd6IiwKICAgICAidmVyaWZ5IjogIlNIQTI1NlNVTVMiLAogICAgICJ2ZXJpZnlfY2hlY2tzdW0iOiAiMWU4NTcxOWM2OTU5ZWIwZjJjOGYyMTY2ZTMwYWU5NTJjY2FlZjJjMjg2ZjMxODY4ZWExZDMxMWQzNzM4YTMzOSIsCiAgICAgImNoZWNrc3VtIjogImYxMWVhMjAwNWRlOTdiZjMwOWJhZmFjNDZlNzdjMDE5MjUzMDdhMjY2NzVmNDRmMzg4ZDQ1MDJkMmI5ZDAwYmYiLAogICAgICJtYXRjaCI6ICJoYXNfY29tbWFuZCgnZmFjdGVyJykiCiB9Cl0K","signature":"f06d4a1cfe9ac79d26b5e6646fdfa9d845a5506c9a2fe0a71fb8416f6f7edd253a1eb46363c12ca5f6148b19ab1ed9a5f25c89b09b3360a09b7d054bf4b55204"}

After this the machines will be downloaded and maintained. In the pack command above the key is optional so the same command can be used to encode the specification without signing. They key can be read from the environment variable KEY.

Note the has_command('facter') for the matcher key, this is a small expr expression that is run on the node to determine if a specific machine should go on a node. The Key-Value is for the entire connected DC so in order to allow heterogeneous environments machines that should not go on the entire fleet can be limited using matchers.

Function	Description
identity	Regular expression match over the machine identity
has_file	Determines if a regular file is present on the machine
has_directory	Determines if a directory is present on the machine
has_command	Searches PATH for a command, note the PATH choria runs with is quite limited

The expression format is the typical used by Choria for example a match might be identity('^web') && has_command('facter') would do pretty much the right thing.

Compiling Autonomous Agents into Choria

Current main of Choria also supports compiling Autonomous Agents into Choria, if you're really paranoid or strict you would compile the above autonomous agent into Choria and use it to bootstrap others in a trusted manner from a trusted source allowing just the properties you want to be adjusted via Key-Value Store.

In this mode you can even forgo the entire Key-Value integration and compile urls and all checksums right into the binary for the truly paranoid.

package metamgr

import (
	"github.com/choria-io/go-choria/aagent/machine"
	"github.com/choria-io/go-choria/aagent/plugin"
)

func ChoriaPlugin() *plugin.MachinePlugin {
	return plugin.NewMachinePlugin("metamgr", &machine.Machine{
		MachineName: "metamgr",
		InitialState: "MANAGE",
		// rest of the autonomous agent
    })
}

You can now include this file in the user_plugins.yaml and it will be compiled in, see below example. This way you have an unmodifiable way to bootstrap a trusted set of Autonomous Agents onto new servers without needing Configuration Management

Compiling into Choria

Compiling this into Choria is reasonably simple, assuming you already figured out how to compile choria :-)

# packager/user_plugins.yaml
archive_watcher: github.com/choria-io/go-choria/aagent/watchers/archivewatcher
machines_watcher: github.com/choria-io/go-choria/aagent/watchers/machineswatcher
machines_manager: github.com/choria-io/go-choria/aagent/watchers/machineswatcher/manager

Do go generate and recompile, this will include the watcher.

Documentation

Index

• Variables
• func ChoriaPlugin() *plugin.WatcherPlugin
• func New(machine model.Machine, name string, states []string, failEvent string, ...) (any, error)
• type ManagedMachine
• type Properties
• type Specification
• type State
• type StateNotification
  • func (s *StateNotification) CloudEvent() cloudevents.Event
  • func (s *StateNotification) JSON() ([]byte, error)
  • func (s *StateNotification) String() string
• type Watcher
  • func (w *Watcher) CurrentState() any
  • func (w *Watcher) Run(ctx context.Context, wg *sync.WaitGroup)

Variables

var (
	// PublicKey allows a public key to be compiled in to the binary during CI while using a standard
	// compiled in machine.yaml, effectively this is equivalent to setting the public_key property
	PublicKey = ""
)

Functions

func ChoriaPlugin

func ChoriaPlugin() *plugin.WatcherPlugin

func New

func New(machine model.Machine, name string, states []string, failEvent string, successEvent string, interval string, ai time.Duration, rawprop map[string]any) (any, error)

Types

type ManagedMachine

type ManagedMachine struct {
	Name                     string `json:"name" yaml:"name"`
	Source                   string `json:"source" yaml:"source"`
	Username                 string `json:"username" yaml:"username"`
	Password                 string `json:"password" yaml:"password"`
	ContentChecksumsChecksum string `json:"verify_checksum" yaml:"verify_checksum" mapstructure:"verify_checksum"`
	ArchiveChecksum          string `json:"checksum" yaml:"checksum" mapstructure:"checksum"`
	Matcher                  string `json:"match" yaml:"match" mapstructure:"match"`
	Governor                 string `json:"governor" yaml:"governor" mapstructure:"governor"`

	Interval string `json:"-"`
	Target   string `json:"-"`
}

type Properties

type Properties struct {
	// DataItem is the data item key to get ManagedMachines from, typically sourced from Key-Value store
	DataItem string `mapstructure:"data_item"`
	// PurgeUnknown will remove machines not declared in DataItem
	PurgeUnknown bool `mapstructure:"purge_unknown"`
	// MachineManageInterval is the interval that created machines will use to manage their archives
	MachineManageInterval time.Duration
	// PublicKey is the optional ed25519 public key used to sign the specification, when set
	// the specification received will be validated and any invalid specification will be discarded
	PublicKey string `mapstructure:"public_key"`
}

type Specification

type Specification struct {
	Machines  []byte `json:"machines"`
	Signature string `json:"signature,omitempty"`
}

type State

type State int

const (
	Unknown State = iota
	Skipped
	Error
	Updated
	Unchanged
)

type StateNotification

type StateNotification struct {
	event.Event

	PreviousManagedMachines []string `json:"machines"`
	PreviousOutcome         string   `json:"previous_outcome"`
	PreviousRunTime         int64    `json:"previous_run_time"`
}

StateNotification describes the current state of the watcher described by io.choria.machine.watcher.exec.v1.state

func (*StateNotification) CloudEvent

func (s *StateNotification) CloudEvent() cloudevents.Event

CloudEvent creates a CloudEvent from the state notification

func (*StateNotification) JSON

func (s *StateNotification) JSON() ([]byte, error)

JSON creates a JSON representation of the notification

func (*StateNotification) String

func (s *StateNotification) String() string

String is a string representation of the notification suitable for printing

type Watcher

type Watcher struct {
	*watcher.Watcher
	// contains filtered or unexported fields
}

func (*Watcher) CurrentState

func (w *Watcher) CurrentState() any

func (*Watcher) Run

func (w *Watcher) Run(ctx context.Context, wg *sync.WaitGroup)