maestro

README

maestro

Pelion Edge systems management daemon for Pelion Edge OS.

Overview

Maestro is a replacement for a number of typical Linux OS system utilities and management programs, while providing cloud-connected systems management. Maestro is designed specifically for cloud-connected Linux OS embedded computers, with somewhat limited RAM and disk space, where file systems are often flash - and would prefer less writing to the FS over time.

Major tasks maestro provides:

• syslog daemon (replaces syslog-ng, syslogd, and others)
• more advanced logging via the grease-log-client library
• to-the-cloud logging
• periodic system stats to cloud
• config management for apps / container (config file templating, config API)
• process creation & control
• container management - starting / stoppping & installation
• network setup (DHCP, static IP settings, with more to come)
• critical systems control (reboot, remote command execution, etc.)
• watchdog support
• time sync
• initial provisioning of system

Maestro can communicate in two ways:

• locally with other process over its local API
• a 'phone-home' style communication with its 'mothership' - this is WigWag's DCS

Advantages:

• less memory footprint
• less installation on disk
• cloud connectivity
• managment via local API

Maestro communicates to Pelion Cloud over https outbouund. It stores its config locally in a private database, but can also use DeviceDB for storage of applications, network settings, configs and other data when used in conjuction with standard Pelion Cloud services.

If you are locally on a gateway / edge system using maestro, you should explore maestro-shell which will let you interact with maestro directly using the local API.

Developing Maestro

Due to the large list of system services provided by Maestro, the recommended way to develop and test Maestro is in isolation using a virtual machine. Running Maestro on your local dev box, while possible, would create a lot of headaches and running Maestro in Docker is not possible. See the 'Why Vagrant?' section for additional info.

Pre-Requisites

Install the following tools:

• vagrant (version 2.2.6)
• virtualbox (version 6.0)
• git bash (on windows only)

NOTE - For the following instructions to work correctly on Windows 10 you must install the latest version of Git and use Git Bash instead of Windows CLI to issue the commands.

Install Vagrant plugins. vagrant-reload is a plugin that allows vagrant to reboot a VM during provisioning. This is needed to reboot the vagrant VM while setting up required network interfaces.

vagrant plugin install vagrant-reload

On Windows use Git Bash to turn off Windows line conversions to protect the Linux scripts with the following command:

git config --global core.autocrlf false

Why Vagrant?

Maestro uses the vagrant for the following reasons:

1. Reliability - Vagrant guarantees that every user that tries to build maestro gets the same environment and toolset.
2. Network control - Unlike docker, vagrant allows us to specify network interfaces. Specifically, we setup a control and test network, which is explained in the vagrant folder README
3. System testing - Once again a limitation to docker, vagrant allows us to start and stop the maestro daemon and configure the system for complete system testing. Additionally, we can bring in other daemons into the system over time for additional testing (ex. devicedb, devicejs)

For more information as to how vagrant sets up the maestro environment, please read the README in the vagrant folder

Building

Start up the virtual machine. The very first time vagrant up is run, the VM will go through a provisioning phase.

git clone git@github.com:armPelionEdge/maestro.git
cd maestro
vagrant up

On Windows also do the following:

VAGRANT_PREFER_SYSTEM_BIN=1 vagrant ssh -c "chmod 400 ~/.ssh/id_rsa"

Then build maestro and its dependencies. You can run this whenever you desire as long as the VM is online

On Linux or Mac:

vagrant ssh -c "build_maestro"

On Windows:

VAGRANT_PREFER_SYSTEM_BIN=1 vagrant ssh -c "build_maestro"

Running

On Linux or Mac:

vagrant ssh -c "sudo maestro"

On Windows:

VAGRANT_PREFER_SYSTEM_BIN=1 vagrant ssh -c "sudo maestro"

Testing

Unit tests

Example of running a networking test:

vagrant ssh # Log in to VM
cd $MAESTRO_SRC # Go to maestro home dir

cd networking # Open networking tests
go test -v -run DhcpRequest # Run DhcpRequest test

System Tests

On the host machine, run the following commands: Note: Make sure you had built using vagrant up and vagrant ssh -c "build_maestro" before running tests.

cd tests # Go to SysTests folder
npm i # Only run once to download dependencies
npm test # Run mocha test suite

Additional Features/Information

Maestro Configuration

To view details on how to configure Maestro, see Maestro Config

DeviceDB

In order to test additional maestro functionality, the vagrant VM automatically installs and runs DeviceDB Edge and DeviceDB Cloud in the background. See DeviceDB for more information.

To view logs from DeviceDB Edge, run:

vagrant ssh
sudo journalctl -u devicedb_edge

To view logs from DeviceDB Cloud, run:

vagrant ssh
docker ps
docker logs <container-id>"

Where <container-id> is the ID of the DeviceDB Server docker container that was shown in docker ps

Documentation

Index

• Variables
• func AddProcessRoutes(router *httprouter.Router)
• func InitImageManager(scratchPath string, imagePath string) (err error)
• func IsValidDownloadURL(s string) bool
• func ProcessJobOp(DB *storage.MaestroDBInstance, msg maestroSpecs.JobOperation) (joberr *maestroSpecs.APIError, taskid string)
• func ShutdownImageManager()
• type ImageManagerInstance
  • func ImageManagerGetInstance() *ImageManagerInstance
  • func (this *ImageManagerInstance) LookupImage(appname string) (ret *StoredImageEntry, ok bool)
  • func (this *ImageManagerInstance) SubmitTask(task *tasks.MaestroTask) (err error)
  • func (this *ImageManagerInstance) ValidateTask(task *tasks.MaestroTask) (err error)
• type Msg_CreateContainer
• type Msg_EraseImage
• type Msg_JobStartRequest
  • func (this *Msg_JobStartRequest) GetArgs() []string
  • func (this *Msg_JobStartRequest) GetContainerTemplate() string
  • func (this *Msg_JobStartRequest) GetDependsOn() []string
  • func (this *Msg_JobStartRequest) GetEnv() []string
  • func (this *Msg_JobStartRequest) GetExecCmd() string
  • func (this *Msg_JobStartRequest) GetJobName() string
  • func (this *Msg_JobStartRequest) GetMessageForProcess() string
  • func (this *Msg_JobStartRequest) GetPgid() int
  • func (this *Msg_JobStartRequest) GetRestartLimit() uint32
  • func (this *Msg_JobStartRequest) GetRestartOnDependencyFail() bool
  • func (this *Msg_JobStartRequest) GetRestartPause() uint32
  • func (this *Msg_JobStartRequest) IsAutostart() bool
  • func (this *Msg_JobStartRequest) IsDaemonize() bool
  • func (this *Msg_JobStartRequest) IsInheritEnv() bool
  • func (this *Msg_JobStartRequest) IsRestart() bool
• type Msg_SendImage
• type Msg_StartProcess
• type Msg_StopJob
• type Msg_StopProcess
• type Msg_TeardownContainer
• type StoredImageEntry
• type UnixHttpEndpoint
  • func (sink *UnixHttpEndpoint) Init(path string) error
  • func (sink *UnixHttpEndpoint) Start(router *httprouter.Router, wg *sync.WaitGroup) error

Variables

var JobManagerInstance *jobManagerInstance

Functions

func AddProcessRoutes

func AddProcessRoutes(router *httprouter.Router)

func InitImageManager

func InitImageManager(scratchPath string, imagePath string) (err error)

func IsValidDownloadURL

func IsValidDownloadURL(s string) bool

func ProcessJobOp

func ProcessJobOp(DB *storage.MaestroDBInstance, msg maestroSpecs.JobOperation) (joberr *maestroSpecs.APIError, taskid string)

func ShutdownImageManager

func ShutdownImageManager()

Types

type ImageManagerInstance

type ImageManagerInstance struct {
	// contains filtered or unexported fields
}

func ImageManagerGetInstance

func ImageManagerGetInstance() *ImageManagerInstance

func (*ImageManagerInstance) LookupImage

func (this *ImageManagerInstance) LookupImage(appname string) (ret *StoredImageEntry, ok bool)

func (*ImageManagerInstance) SubmitTask

func (this *ImageManagerInstance) SubmitTask(task *tasks.MaestroTask) (err error)

implements TaskHandler interface:

func (*ImageManagerInstance) ValidateTask

func (this *ImageManagerInstance) ValidateTask(task *tasks.MaestroTask) (err error)

type Msg_CreateContainer

type Msg_CreateContainer struct {
}

type Msg_EraseImage

type Msg_EraseImage struct {
}

type Msg_JobStartRequest

type Msg_JobStartRequest struct {
	Job                     string   `json:"job"`
	ContainerTemplate       string   `json:"container_template"`
	Message                 string   `json:"message"`
	NoAutoStart             bool     `json:"no_autostart"`
	Restart                 bool     `json:"restart"`
	RestartOnDependencyFail bool     `json:"restart_on_dependency_fail"`
	RestartLimit            uint32   `json:"restart_limit"`
	RestartPause            uint32   `json:"restart_pause"`
	DependsOn               []string `json:"depends_on"`
	Pgid                    int      `json:"pgid"`
	ExecCmd                 string   `json:"exec_cmd"`
	ExecArgs                []string `json:"exec_args"`
	Env                     []string `json:"env"`
	InheritEnv              bool     `json:"inherit_env"`
	Daemonize               bool     `json:"daemonize"` // create a new SID for this process ?
}

func (*Msg_JobStartRequest) GetArgs

func (this *Msg_JobStartRequest) GetArgs() []string

func (*Msg_JobStartRequest) GetContainerTemplate

func (this *Msg_JobStartRequest) GetContainerTemplate() string

func (*Msg_JobStartRequest) GetDependsOn

func (this *Msg_JobStartRequest) GetDependsOn() []string

func (*Msg_JobStartRequest) GetEnv

func (this *Msg_JobStartRequest) GetEnv() []string

func (*Msg_JobStartRequest) GetExecCmd

func (this *Msg_JobStartRequest) GetExecCmd() string

func (*Msg_JobStartRequest) GetJobName

func (this *Msg_JobStartRequest) GetJobName() string

func (*Msg_JobStartRequest) GetMessageForProcess

func (this *Msg_JobStartRequest) GetMessageForProcess() string

func (*Msg_JobStartRequest) GetPgid

func (this *Msg_JobStartRequest) GetPgid() int

func (*Msg_JobStartRequest) GetRestartLimit

func (this *Msg_JobStartRequest) GetRestartLimit() uint32

func (*Msg_JobStartRequest) GetRestartOnDependencyFail

func (this *Msg_JobStartRequest) GetRestartOnDependencyFail() bool

func (*Msg_JobStartRequest) GetRestartPause

func (this *Msg_JobStartRequest) GetRestartPause() uint32

func (*Msg_JobStartRequest) IsAutostart

func (this *Msg_JobStartRequest) IsAutostart() bool

func (*Msg_JobStartRequest) IsDaemonize

func (this *Msg_JobStartRequest) IsDaemonize() bool

func (*Msg_JobStartRequest) IsInheritEnv

func (this *Msg_JobStartRequest) IsInheritEnv() bool

func (*Msg_JobStartRequest) IsRestart

func (this *Msg_JobStartRequest) IsRestart() bool

type Msg_SendImage

type Msg_SendImage struct {
}

type Msg_StartProcess

type Msg_StartProcess struct {
	Path        string   `json:"path"`
	Arguments   []string `json:"arguments"`
	Environment []string `json:"environment"`
	// enherit the standard environment that
	// maestro was started in?
	InheritEnv  bool   `json:"inheritEnv"`
	ContainerId string `json:"ContainerId"`
	Pgid        int    `json:"pgid"`
	Daemonize   bool   `json:"daemonize"`
}

type Msg_StopJob

type Msg_StopJob struct {
}

type Msg_StopProcess

type Msg_StopProcess struct {
}

type Msg_TeardownContainer

type Msg_TeardownContainer struct {
}

type StoredImageEntry

type StoredImageEntry struct {
	// contains filtered or unexported fields
}

type UnixHttpEndpoint

type UnixHttpEndpoint struct {
	SocketPath string

	WG *sync.WaitGroup
	// contains filtered or unexported fields
}

func (*UnixHttpEndpoint) Init

func (sink *UnixHttpEndpoint) Init(path string) error

func (*UnixHttpEndpoint) Start

func (sink *UnixHttpEndpoint) Start(router *httprouter.Router, wg *sync.WaitGroup) error