beekeeper

package module
v0.0.0-...-49ed281 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2021 License: MIT Imports: 37 Imported by: 1

Documentation

Overview

Package beekeeper is a batteries-included cluster computing library

Index

Constants

View Source 
const (
	// DefaultPort is the default port for Beekeeper servers
	DefaultPort = 2020

	// DefaultScanTime is the scan time to be used by scan functions
	DefaultScanTime = time.Second * 2
)
View Source 
const (
	// Version is the version of the package in semantic notation.
	Version = "v0.3.2"

	// License is the copyright license used in this package.
	License = "MIT"

	// Author is the first and last name of the package's author.
	Author = "Camilo Hernández"
)
View Source 
const (
	// OperationNone nil value for operations
	OperationNone = iota

	// OperationStatus ask a node for a status report
	OperationStatus

	// OperationJobTransfer transfer a job via the Data field
	OperationJobTransfer

	// OperationTransferFailed transfer failed, Data contains the details
	OperationTransferFailed

	// OperationTransferAcknowledge transfer was successful
	OperationTransferAcknowledge

	// OperationJobExecute run the local job
	OperationJobExecute

	// OperationJobResult job ran and the details come in the Data
	OperationJobResult
)
View Source 
const (
	// StatusNone nil value for Status
	StatusNone = iota

	// StatusIDLE node is IDLE
	StatusIDLE

	// StatusWorking node is working on a job
	StatusWorking
)

Variables

View Source 
var ErrMessageTooLarge = errors.New("message too large")

ErrMessageTooLarge is triggered when a message exceeds the size limit set by MaxMessageSize

View Source 
var ErrNodeDisconnected = errors.New("node disconnected")

ErrNodeDisconnected is produced when a node is gets disconnected while executing an operation

View Source 
var ErrTimeout = errors.New("time exceeded")

ErrTimeout is produced by functions called with a timeout when the allocated time is exceeded

View Source 
var WatchdogSleep = time.Second * 15

WatchdogSleep is the time between node pings for the watchdog

Functions

func WrapJob 

func WrapJob(job func(*Task))

WrapJob wraps a job function with input and output parsing to transfer the Result. The provided function must never use STDIO.

Types

type Config 

type Config struct {
	// Name of the node. It defaults to the system's hostname.
	Name string `mapstructure:"name,omitempty"`

	// Debug toggles between verbosity for debugging.
	Debug bool `mapstructure:"debug,omitempty"`

	// Token is a passphrase used to restrict usage of the node. Must match on the receiving node.
	Token string `mapstructure:"token,omitempty"`

	// InboundPort is the port to be used for receiving connections. Defaults to 2020.
	InboundPort int `mapstructure:"inbound_port,omitempty"`

	// OutboundPort is the port assumed to be used by a remote node. It's only used to establish a connection, and
	// afterwards a port is negotiated with the remote node. Defaults to 2020.
	OutboundPort int `mapstructure:"outbound_port,omitempty"`

	// TLSCertificate is used for TLS connections between nodes. If none is given a certificate is created on the first
	// run and reused as needed.
	TLSCertificate []byte

	// TLSPrivateKey is used for TLS connections between nodes. If none is given a key is created on the first
	// run and reused as needed.
	TLSPrivateKey []byte

	// AllowExternal sets whether non-local connections should be accepted. It's heavily encouraged that a whitelist
	// and token is set with this featured turn on. Defaults to false.
	AllowExternal bool `mapstructure:"allow_external,omitempty"`

	// Whitelist contains a list of allowed hosts. If none is provided it's understood that the whitelist is disabled.
	// A wildcard sign (*) can be used.
	Whitelist []string `mapstructure:"whitelist,omitempty"`

	// MaxMessageSize is the size limit in bytes for incoming messages. It defaults to 1.024 MB
	MaxMessageSize uint64 `mapstructure:"max_message_size,omitempty"`

	// DisableCleanup turns off the post-build cleanup
	DisableCleanup bool `mapstructure:"disable_cleanup,omitempty"`

	// DisableConnectionWatchdog disables the connection watchdog, and stops disconnection notifications.
	DisableConnectionWatchdog bool `mapstructure:"disable_connection_watchdog,omitempty"`
}

Config holds the configurations for a node or a primary node.

func NewConfigFromFile 

func NewConfigFromFile(path string) (c Config, err error)

NewConfigFromFile parses a file on the provided path as a Config object. If a field is not set, the default value is assigned.

func NewDefaultConfig 

func NewDefaultConfig() (c Config)

NewDefaultConfig returns a new Config with sensible defaults. It's recommended that NewDefaultConfig be used. for the creation of Config structs.

type Conn 

type Conn struct {
	*tls.Conn
}

Conn represents a TLS connection

type LoadBalancer 

type LoadBalancer struct {
	// contains filtered or unexported fields
}

LoadBalancer contains the data needed to try to select the best node for a task. Should be created using NewLoadBalancer.

func NewLoadBalancer 

func NewLoadBalancer(s *Server, ns Nodes) *LoadBalancer

NewLoadBalancer creates and sets up a LoadBalancer from the given Nodes.

func (*LoadBalancer) Execute 

func (lb *LoadBalancer) Execute(t Task, timeout ...time.Duration) (res Result, err error)

Execute will run a task, selecting the node based on it's workload. If multiple nodes are equally as busy, the LoadBalancer will pick the best performing one, or pick based on a Softmax algorithm for exploration.

type Message 

type Message struct {
	// SentAt timestamp for the Message.
	SentAt time.Time

	// Name the sender's name.
	Name string

	// Operation operation the remote node wishes to execute. It may be nilled with OperationNone.
	Operation Operation

	// Data the body of the message. Contains the payload needed for the execution if the Operation.
	Data []byte

	// Token is used as a passphrase to operate in a multi-node environment.
	Token string

	// Addr is the address of the sender
	Addr *net.TCPAddr

	// RespondOnPort is the port that the sender wishes to be used for the response.
	RespondOnPort int

	// Status represents the current action the node is doing.
	Status Status

	// NodeInfo contains metadata about the sender, like OS and current usage.
	NodeInfo NodeInfo
}

Message is used for node communication. It holds the transferable data as well as some metadata about the node.

type Monitor 

type Monitor struct {
	App         *tview.Application
	Pages       *tview.Pages
	CurrentPage int
	// contains filtered or unexported fields
}

Monitor represents a Beekeeper Monitor.

func NewMonitor 

func NewMonitor() *Monitor

NewMonitor creates and returns a *Monitor struct.

func (*Monitor) NextPage 

func (m *Monitor) NextPage()

NextPage changes the page to the n+1 page.

func (*Monitor) PreviousPage 

func (m *Monitor) PreviousPage()

PreviousPage changes the page to the n-1 page.

func (*Monitor) Render 

func (m *Monitor) Render(ns Nodes)

Render prints the Monitor to the console.

func (*Monitor) Run 

func (m *Monitor) Run(configs ...Config)

Run starts the Monitor, renders it and updates it regularly.

func (*Monitor) Stop 

func (m *Monitor) Stop()

Stop stops the monitor's App and Server.

type Node 

type Node struct {
	Conn   *Conn
	Addr   *net.TCPAddr
	Name   string
	Status Status
	Info   NodeInfo
}

Node represents a node node.

func (Node) Equals 

func (n Node) Equals(w2 Node) bool

Equals compares two workers. The comparison is made using the IP addresses of the nodes.

type NodeInfo 

type NodeInfo struct {
	// CPUTemp is the temperature as measured in the CPU dice when possible. Certain OS can return 0.
	CPUTemp float32

	// Usage is the percentage of usage of the host system in a range from 1 (max) to 0 (min).
	Usage float32

	// OS is the GOOS of the host system.
	OS string
}

NodeInfo holds additional info abut a node.

type Nodes 

type Nodes []Node

Nodes is a Node slice

func (Nodes) PrettyPrint 

func (n Nodes) PrettyPrint(writer ...io.Writer)

PrettyPrint prints a formatted table of workers.

type Operation 

type Operation int

Operation is used to specify a Message's intent to the remote node

func (Operation) String 

func (o Operation) String() string

String returns a string representation of the Operation.

type Request 

type Request struct {
	Msg  Message
	Conn Conn
}

Request represents an incoming Message with its connection

type Result 

type Result struct {
	UUID  string
	Task  Task
	Error string
}

Result holds the details from a job execution.

type Server 

type Server struct {
	// Config hold the configuration data of the server.
	Config Config

	// Status represents the action the server is currently doing.
	Status Status
	// contains filtered or unexported fields
}

Server is a node server, that holds the configuration to be used.

func NewServer 

func NewServer(configs ...Config) *Server

NewServer creates a Server struct using the given config or the default if none is provided.

func (*Server) Connect 

func (s *Server) Connect(ip string, timeout ...time.Duration) (Node, error)

Connect established a TCP over TLS connection with the given address. If no node is reachable an error will be returned. An optional timeout argument can be provided.

func (*Server) DistributeJob 

func (s *Server) DistributeJob(pkgName string, function string, nodes ...Node) error

DistributeJob builds a job and sends a copy to the workers. Will fail if an empty workers list is given.

func (*Server) Execute 

func (s *Server) Execute(n Node, t Task, timeout ...time.Duration) (res Result, err error)

Execute runs a task on the given node and blocks until the task results are retrieved. It will fail if no job is present on the node's systems. An optional timeout parameter can be provided.

func (*Server) ExecuteMany 

func (s *Server) ExecuteMany(n Nodes, t Task, timeout ...time.Duration) ([]Result, error)

ExecuteMany runs a task on the provided Nodes and blocks until a Result is sent back. Optionally a timeout argument can be passed.

func (*Server) Scan 

func (s *Server) Scan(waitTime time.Duration) (Nodes, error)

Scan broadcasts a status Request to all IPs and waits the provided amount for a response.

func (*Server) Start 

func (s *Server) Start() error

Start serves a node and blocks.

func (*Server) Stop 

func (s *Server) Stop()

Stop shutdowns a running server.

type Status 

type Status int

Status represent the status of a node.

func (Status) String 

func (s Status) String() string

String returns a string representation of a Status.

type Task 

type Task struct {
	UUID      string
	Arguments map[string]interface{}
	Returns   map[string]interface{}
	Error     string
}

Task is used to run a job. In order to create a Task use NewTask; not this structure directly.

func NewTask 

func NewTask() Task

NewTask creates a Task, initializes and then returns it.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.