boltzmann

README

Boltzmann

Boltzmann is a distributed lightweight arg orchestrator.

Based on the Scheduler Agent Supervisor Cloud Pattern, Boltzmann is a master-less service used to schedule a batch of arg in a parallel and distributed way.

Depending on the configuration, a Boltzmann node might be stateless or stateful as args states may be stored in a embedded or external database (e.g. Redis).

Worker pools (i.e. a Boltzmann node) are ensured for correctness even in a distributed environment by using leases (i.e. distributed mutex lock) and a small leader election consensus algorithm.

Moreover, Leases are implemented using either a RedLock algorithm or through storage engine's built-in data structure (e.g. etcd leases).

Architecture

Task Scheduler

The Scheduler arranges for the steps that make up the arg to be executed and orchestrates their operation. These steps can be combined into a pipeline or workflow. The Scheduler is responsible for ensuring that the steps in this workflow are performed in the right order.

As each step is performed, the Scheduler records the state of the workflow, such as "step not yet started," "step running," or "step completed." The state information should also include an upper limit of the time allowed for the step to finish, called the complete-by time.

If a step requires access to a remote service or resource, the Scheduler invokes the appropriate Agent, passing it the details of the work to be performed. The Scheduler typically communicates with an Agent using asynchronous request/response messaging.

Agent

The Agent contains logic that encapsulates a call to a remote service, or access to a remote resource referenced by a step in a arg. Each Agent typically wraps calls to a single service or resource, implementing the appropriate error handling and retry logic (subject to a timeout constraint, described later).

Supervisor

The Supervisor monitors the status of the steps in the arg being performed by the Scheduler. It runs periodically (the frequency will be system-specific), and examines the status of steps maintained by the Scheduler. If it detects any that have timed out or failed, it arranges for the appropriate Agent to recover the step or execute the appropriate remedial action (this might involve modifying the status of a step).

Note that the recovery or remedial actions are implemented by the Scheduler and Agents. The Supervisor should simply request that these actions be performed.

Usage

Till this day, there are two ways available to use Boltzmann (which are not mutually exclusive):

• A HTTP REST API (HTTP/1.1).
• A gRCP Streaming API (HTTP/2, multiplexed).

Documentation

Index

• Variables
• type BackgroundProcess
• type ErrItemAlreadyExists
  • func (e ErrItemAlreadyExists) Error() string
• type ErrItemNotFound
  • func (e ErrItemNotFound) Error() string
• type ErrOutOfRange
  • func NewOutOfRangeWithType[T, P any](property string, a, b P) ErrOutOfRange[P]
  • func (e ErrOutOfRange[T]) Error() string
• type Identifiable
• type NoopIdentifiable
  • func (n NoopIdentifiable) GetID() string
• type Registry
• type RepositoryConfig
• type Retryable
  • func (r Retryable) Error() string
• type Task
  • func (t Task) GetID() string

Variables

var GlobalValidator = validator.New()

GlobalValidator a global go-playground/validator instance.

Types

type BackgroundProcess

type BackgroundProcess interface {
	Start(ctx context.Context) error
	Shutdown(ctx context.Context) error
}

type ErrItemAlreadyExists

type ErrItemAlreadyExists struct {
	ResourceName string
	ResourceKey  string
}

ErrItemAlreadyExists indicates if an item write operation failed due existent entity already present in the storage.

func (ErrItemAlreadyExists) Error

func (e ErrItemAlreadyExists) Error() string

type ErrItemNotFound

type ErrItemNotFound struct {
	ResourceName string
	ResourceKey  string
}

ErrItemNotFound indicates if an item lookup operation failed due a non-existent entity in the storage.

func (ErrItemNotFound) Error

func (e ErrItemNotFound) Error() string

type ErrOutOfRange

type ErrOutOfRange[T any] struct {
	PropertyName string
	A, B         T
}

ErrOutOfRange indicates if a property is out of a specified range.

func NewOutOfRangeWithType

func NewOutOfRangeWithType[T, P any](property string, a, b P) ErrOutOfRange[P]

NewOutOfRangeWithType allocates a new ErrOutOfRange using T as base structure of a property. This routine will generate an ErrOutOfRange.PropertyName as `T_string_type.property`.

func (ErrOutOfRange[T]) Error

func (e ErrOutOfRange[T]) Error() string

type Identifiable

type Identifiable[T comparable] interface {
	// GetID generates or retrieves (depending on the concrete implementation) the structure's unique identifier.
	GetID() T
}

Identifiable adheres identification capabilities to a Go structure.

type NoopIdentifiable

type NoopIdentifiable struct {
}

NoopIdentifiable is the no-operation Identifiable struct.

func (NoopIdentifiable) GetID

func (n NoopIdentifiable) GetID() string

type Registry

type Registry[T any] interface {
	// Register saves a component with the given key.
	Register(key string, component T) error
	// Get retrieves a component using its key.
	Get(key string) (T, error)
	// Exists indicates whether a component with the given key exists.
	Exists(key string) error
}

Registry a generic registry to manage a specific set of resources of the system.

NOTE: Concurrency safeness depends on concrete implementations of this interface. DO NOT consider every registry as concurrent-safe.

type RepositoryConfig

type RepositoryConfig struct {
	ItemTTL time.Duration // Time for an item to be preserved in the store (aka. Time-To-Live).
}

RepositoryConfig a basic and general-purposed repository configuration. Extend from this structure (with struct-embedding) if more fields are required for specific scenarios.

type Retryable

type Retryable struct {
	Parent error
}

Retryable wraps a parent error to indicate its execution process is able to get retried.

func (Retryable) Error

func (r Retryable) Error() string

type Task

type Task struct {
	TaskID          int               // Task unique identifier.
	ExecutionPlanID string            // Correlation StateID.
	Driver          string            // Driver type to be used by agent executors.
	ResourceURL     string            // Location of the resource as URL.
	AgentArguments  map[string]string // Arguments to be passed to the execution agent.
	TypeMIME        string            // MIME Type of the payload.
	Payload         any               `json:"-"` // Transient data to be passed to internal components for different utilities.
	EncodedPayload  []byte            // Encoded data to be passed to the execution agent.
}

Task a unit of work to be executed by a Boltzmann agent.

func (Task) GetID

func (t Task) GetID() string