maestro

package module
v0.0.0-...-ffa2d3d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 24, 2019 License: Apache-2.0 Imports: 31 Imported by: 0

README

maestro

Pelion Edge systems management daemon for Pelion Edge OS.

Overview

Maestro is a replacement for a number of typical Linux OS system utilities and management programs, while providing cloud-connected systems management. Maestro is designed specifically for cloud-connected Linux OS embedded computers, with somewhat limited RAM and disk space, where file systems are often flash - and would prefer less writing to the FS over time.

Major tasks maestro provides:

  • syslog daemon (replaces syslog-ng, syslogd, and others)
  • more advanced logging via the grease-log-client library
  • to-the-cloud logging
  • periodic system stats to cloud
  • config management for apps / container (config file templating, config API)
  • process creation & control
  • container management - starting / stoppping & installation
  • network setup (DHCP, static IP settings, with more to come)
  • critical systems control (reboot, remote command execution, etc.)
  • watchdog support
  • time sync
  • initial provisioning of system

Maestro can communicate in two ways:

  • locally with other process over its local API
  • a 'phone-home' style communication with its 'mothership' - this is WigWag's DCS

Advantages:

  • less memory footprint
  • less installation on disk
  • cloud connectivity
  • managment via local API

Maestro communicates to Pelion Cloud over https outbouund. It stores its config locally in a private database, but can also use DeviceDB for storage of applications, network settings, configs and other data when used in conjuction with standard Pelion Cloud services.

Interacting

If you are locally on a gateway / edge system using maestro, you should explore maestro-shell which will let you interact with maestro directly using the local API.

Building
prerequisites

Build pre-requisites:

  • golang
  • python
  • gcc

Maestro has a simple internal preprocessor build.sh script which use m4 - make sure m4 is installed on your build system (normally included with gcc)

You will need gcc for compiling maestro's dependencies.

On Ubuntu

sudo apt-get install build-essential python should take care of all this.

If you go not have a Go build environment up, you will need to install golang. Just expand the tar ball and put in in /opt, with sudo mv go /opt

You also will need your golang env vars setup correctly. Here is a script you can just run, assuming your Go workspace is at $HOME/work/gostuff:

setup-go.sh

#!/bin/bash

export GIT_TERMINAL_PROMPT=1
export GOROOT=/opt/go
export GOPATH="$HOME/work/gostuff"
export GOBIN="$HOME/work/gostuff/bin"
export PATH="$PATH:$GOROOT/bin:$GOBIN"

then just run: source ./setup-go.sh

Run your build commands from this sub-shell.

On Arch

  • install go: sudo pacman -S golang

  • Add generated executables into your path (so that you can do stuff like maestro on the command line)

    export PATH="$PATH:$HOME/go/bin"
build instructions

go get github.com/armPelionEdge/maestro

Enter github credentials as needed. Now, where $GOPATH is your go workspace folder, as in the sub-shell above...

cd $GOPATH/src/github.com/armPelionEdge/maestro

Run a script to build dependencies in the /vendor folder:

./build-deps.sh

This will take a bit. You're building a bunch of libs used by maestro, including some native code.

Next build maestro:

cd $GOPATH/src/github.com/armPelionEdge/maestro
DEBUG=1 DEBUG2=1 ./build.sh

Eliminate DEBUG and DEBUG2 env var to generate less debug output.

Running tests

Different subsystems have different test suites. Running these may require root privleges, for instance networking. You also need to run the pre-processor if making changes.

Example, from $GOPATH/src/github.com/armPelionEdge/maestro:

DEBUG=1 DEBUG2=1 ./build.sh && cd networking && sudo -E \
  LD_LIBRARY_PATH=../vendor/github.com/armPelionEdge/greasego/deps/lib go test -v -run DhcpRequest

Notes:

  • The dependencies of greasego (a native library extension to go) are in $GOPATH/src/github.com/armPelionEdge/maestro/vendor/github.com/armPelionEdge/greasego/deps/lib, which requires us to add that directory to the LD_LIBRARY_PATH environment variable so that go can find these libraries. A relative path is used in the example to reduce overall command length.
  • The build command is not strictly required to run tests, when you have already built. It's good practice to include this in the command so that you don't test a stale build.
Running maestro

Maestro requires a config in it's working directory called maestro.config, so create it:

touch maestro.config

Build and run maestro:

DEBUG=1 DEBUG2=1 ./build.sh && LD_LIBRARY_PATH=vendor/github.com/armPelionEdge/greasego/deps/lib maestro

If you get a maestro: command not found error, check that your $GOBIN is part of your $PATH

Other examples

The Docker build file, for djs-soft-relay shows build instruction also, using this exact above method. https://github.com/armPelionEdge/cloud-installer/blob/master/djs-soft-relay/build-wwcontainer.sh#L155

TroubleShooting
  1. [ERROR] Failed to create scratch path directory: mkdir /tmp/maestro/images: permission denied [ERROR] Failed to create scratch path directory: mkdir /var/maestro/images: permission denied

Maestro execution on local host creates directories at /tmp/maestro/images and /var/maestro/images path. Verify that the user running maestro has the proper permissions to access the respective directories.

Documentation

Index

Constants

View Source 
const TIMEOUT = time.Second * 10

Variables

View Source 
var JobManagerInstance *jobManagerInstance

Functions

func AddProcessRoutes 

func AddProcessRoutes(router *httprouter.Router)

func InitImageManager 

func InitImageManager(scratchPath string, imagePath string) (err error)

func IsValidDownloadURL 

func IsValidDownloadURL(s string) bool

func New_logBufferFifo 

func New_logBufferFifo(maxsize uint32) (ret *logBufferFifo)

func ProcessJobOp 

func ProcessJobOp(DB *storage.MaestroDBInstance, msg maestroSpecs.JobOperation) (joberr *maestroSpecs.APIError, taskid string)

func ShutdownImageManager 

func ShutdownImageManager()

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func NewSymphonyClient 

func NewSymphonyClient(url string, clientid string, maxBuffers uint32, heartbeatInterval time.Duration) *Client

maxBuffers: this number represents the amount of stored log buffers we will hold before dropping them. This can be from 1 to [max amount of bytes from greasego callback] In effect, this should be close to the same number as NumBanks is set in the target options for the greasego target

func (*Client) Shutdown 

func (client *Client) Shutdown()

func (*Client) Start 

func (client *Client) Start()

func (*Client) SubmitLogs 

func (client *Client) SubmitLogs(data *greasego.TargetCallbackData, godata []byte)

type ImageManagerInstance 

type ImageManagerInstance struct {
	// contains filtered or unexported fields
}

func ImageManagerGetInstance 

func ImageManagerGetInstance() *ImageManagerInstance

func (*ImageManagerInstance) LookupImage 

func (this *ImageManagerInstance) LookupImage(appname string) (ret *StoredImageEntry, ok bool)

func (*ImageManagerInstance) SubmitTask 

func (this *ImageManagerInstance) SubmitTask(task *tasks.MaestroTask) (err error)

implements TaskHandler interface:

func (*ImageManagerInstance) ValidateTask 

func (this *ImageManagerInstance) ValidateTask(task *tasks.MaestroTask) (err error)

type Msg_CreateContainer 

type Msg_CreateContainer struct {
}

type Msg_EraseImage 

type Msg_EraseImage struct {
}

type Msg_JobStartRequest 

type Msg_JobStartRequest struct {
	Job                     string   `json:"job"`
	ContainerTemplate       string   `json:"container_template"`
	Message                 string   `json:"message"`
	NoAutoStart             bool     `json:"no_autostart"`
	Restart                 bool     `json:"restart"`
	RestartOnDependencyFail bool     `json:"restart_on_dependency_fail"`
	RestartLimit            uint32   `json:"restart_limit"`
	RestartPause            uint32   `json:"restart_pause"`
	DependsOn               []string `json:"depends_on"`
	Pgid                    int      `json:"pgid"`
	ExecCmd                 string   `json:"exec_cmd"`
	ExecArgs                []string `json:"exec_args"`
	Env                     []string `json:"env"`
	InheritEnv              bool     `json:"inherit_env"`
	Daemonize               bool     `json:"daemonize"` // create a new SID for this process ?
}

func (*Msg_JobStartRequest) GetArgs 

func (this *Msg_JobStartRequest) GetArgs() []string

func (*Msg_JobStartRequest) GetContainerTemplate 

func (this *Msg_JobStartRequest) GetContainerTemplate() string

func (*Msg_JobStartRequest) GetDependsOn 

func (this *Msg_JobStartRequest) GetDependsOn() []string

func (*Msg_JobStartRequest) GetEnv 

func (this *Msg_JobStartRequest) GetEnv() []string

func (*Msg_JobStartRequest) GetExecCmd 

func (this *Msg_JobStartRequest) GetExecCmd() string

func (*Msg_JobStartRequest) GetJobName 

func (this *Msg_JobStartRequest) GetJobName() string

func (*Msg_JobStartRequest) GetMessageForProcess 

func (this *Msg_JobStartRequest) GetMessageForProcess() string

func (*Msg_JobStartRequest) GetPgid 

func (this *Msg_JobStartRequest) GetPgid() int

func (*Msg_JobStartRequest) GetRestartLimit 

func (this *Msg_JobStartRequest) GetRestartLimit() uint32

func (*Msg_JobStartRequest) GetRestartOnDependencyFail 

func (this *Msg_JobStartRequest) GetRestartOnDependencyFail() bool

func (*Msg_JobStartRequest) GetRestartPause 

func (this *Msg_JobStartRequest) GetRestartPause() uint32

func (*Msg_JobStartRequest) IsAutostart 

func (this *Msg_JobStartRequest) IsAutostart() bool

func (*Msg_JobStartRequest) IsDaemonize 

func (this *Msg_JobStartRequest) IsDaemonize() bool

func (*Msg_JobStartRequest) IsInheritEnv 

func (this *Msg_JobStartRequest) IsInheritEnv() bool

func (*Msg_JobStartRequest) IsRestart 

func (this *Msg_JobStartRequest) IsRestart() bool

type Msg_SendImage 

type Msg_SendImage struct {
}

type Msg_StartProcess 

type Msg_StartProcess struct {
	Path        string   `json:"path"`
	Arguments   []string `json:"arguments"`
	Environment []string `json:"environment"`
	// enherit the standard environment that
	// maestro was started in?
	InheritEnv  bool   `json:"inheritEnv"`
	ContainerId string `json:"ContainerId"`
	Pgid        int    `json:"pgid"`
	Daemonize   bool   `json:"daemonize"`
}

type Msg_StopJob 

type Msg_StopJob struct {
}

type Msg_StopProcess 

type Msg_StopProcess struct {
}

type Msg_TeardownContainer 

type Msg_TeardownContainer struct {
}

type StoredImageEntry 

type StoredImageEntry struct {
	// contains filtered or unexported fields
}

type UnixHttpEndpoint 

type UnixHttpEndpoint struct {
	SocketPath string

	WG *sync.WaitGroup
	// contains filtered or unexported fields
}

func (*UnixHttpEndpoint) Init 

func (sink *UnixHttpEndpoint) Init(path string) error

func (*UnixHttpEndpoint) Start 

func (sink *UnixHttpEndpoint) Start(router *httprouter.Router, wg *sync.WaitGroup) error

Source Files

View all Source files

Directories

Path Synopsis
createcertfromintermediate
selfsigncert
arp
common
fsonly
rp200
rp200_edge
softRelay
testplatform
wwrelayA10
wwrelayA10_edge
Generated source file.
 Generated source file.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.