drivers

README

This package is intended as a general purpose container abstraction library. With the same code, you can run on Docker, Rkt, etc.

Documentation

Overview

Package drivers is intended as a general purpose container abstraction library. It abstracts across the differences between different container runtimes (e.g. Docker, Rkt, etc.) and provides utlities and data types that are common across all runtimes.

Docker Driver

The docker driver runs functions as Docker containers.

Mock Driver

The mock driver pretends to run functions but doesn't actually run them. This is for testing only.

Index

• Constants
• func ParseImage(image string) (registry, repo, tag string)
• type Config
• type ContainerTask
• type Cookie
• type Driver
• type RunResult
• type Stat
  • func Decimate(maxSamples int, stats []Stat) []Stat
• type Stats
  • func (s *Stats) Scan(value interface{}) error
  • func (s Stats) Value() (driver.Value, error)
• type WaitResult

Constants

const (
	// task statuses
	StatusRunning   = "running"
	StatusSuccess   = "success"
	StatusError     = "error"
	StatusTimeout   = "timeout"
	StatusKilled    = "killed"
	StatusCancelled = "cancelled"
)

TODO: ensure some type is applied to these statuses.

Functions

func ParseImage

func ParseImage(image string) (registry, repo, tag string)

Types

type Config

type Config struct {
	Docker         string `json:"docker"`
	DockerNetworks string `json:"docker_networks"`
	// TODO CPUShares should likely be on a per container basis
	CPUShares       int64  `json:"cpu_shares"`
	ServerVersion   string `json:"server_version"`
	PreForkPoolSize uint64 `json:"pre_fork_pool_size"`
	PreForkImage    string `json:"pre_fork_image"`
	PreForkCmd      string `json:"pre_fork_cmd"`
	PreForkUseOnce  uint64 `json:"pre_fork_use_once"`
	PreForkNetworks string `json:"pre_fork_networks"`
}

type ContainerTask

type ContainerTask interface {
	// Command returns the command to run within the container.
	Command() string

	// EnvVars returns environment variable key-value pairs.
	EnvVars() map[string]string

	// Input feeds the container with data
	Input() io.Reader

	// The id to assign the container
	Id() string

	// Image returns the runtime specific image to run.
	Image() string

	// Timeout specifies the maximum time a task is allowed to run. Return 0 to let it run forever.
	Timeout() time.Duration

	// Driver will write output log from task execution to these writers. Must be
	// non-nil. Use io.Discard if log is irrelevant.
	Logger() (stdout, stderr io.Writer)

	// WriteStat writes a single Stat, implementation need not be thread safe.
	WriteStat(context.Context, Stat)

	// Volumes returns an array of 2-element tuples indicating storage volume mounts.
	// The first element is the path on the host, and the second element is the
	// path in the container.
	Volumes() [][2]string

	// Memory determines the max amount of RAM given to the container to use.
	// 0 is unlimited.
	Memory() uint64

	// CPUs in milli CPU units
	CPUs() uint64

	// Filesystem size limit for the container, in megabytes.
	FsSize() uint64

	// WorkDir returns the working directory to use for the task. Empty string
	// leaves it unset.
	WorkDir() string

	// Close is used to perform cleanup after task execution.
	// Close should be safe to call multiple times.
	Close()
}

The ContainerTask interface guides container execution across a wide variety of container oriented runtimes.

type Cookie

type Cookie interface {
	// Close should clean up any resources the cookie was using, or was going to use.
	Close(ctx context.Context) error

	// Run should execute task on the implementation.
	// RunResult captures the result of task execution. This means if task
	// execution fails due to a problem in the task, Run() MUST return a valid
	// RunResult and nil as the error. The RunResult's Error() and Status()
	// should be used to indicate failure.
	// If the implementation itself suffers problems (lost of network, out of
	// disk etc.), a nil RunResult and an error message is preferred.
	//
	// Run() MUST monitor the context. task cancellation is indicated by
	// cancelling the context.
	Run(ctx context.Context) (WaitResult, error)

	// Freeze the container to pause running processes
	Freeze(ctx context.Context) error

	// Unfreeze a frozen container to unpause frozen processes
	Unfreeze(ctx context.Context) error
}

A DriverCookie identifies a unique request to run a task.

Clients should always call Close() on a DriverCookie after they are done with it.

type Driver

type Driver interface {
	// Prepare can be used in order to do any preparation that a specific driver
	// may need to do before running the task, and can be useful to put
	// preparation that the task can recover from into (i.e. if pulling an image
	// fails because a registry is down, the task doesn't need to be failed).  It
	// returns a cookie that can be used to execute the task.
	// Callers should Close the cookie regardless of whether they run it.
	//
	// The returned cookie should respect the task's timeout when it is run.
	Prepare(ctx context.Context, task ContainerTask) (Cookie, error)

	// close & shutdown the driver
	Close() error
}

type RunResult

type RunResult interface {
	// Error is an actionable/checkable error from the container, nil if
	// Status() returns "success", otherwise non-nil
	Error() error

	// Status should return the current status of the task.
	// Only valid options are {"error", "success", "timeout", "killed", "cancelled"}.
	Status() string
}

RunResult indicates only the final state of the task.

type Stat

type Stat struct {
	Timestamp strfmt.DateTime   `json:"timestamp"`
	Metrics   map[string]uint64 `json:"metrics"`
}

Stat is a bucket of stats from a driver at a point in time for a certain task.

func Decimate

func Decimate(maxSamples int, stats []Stat) []Stat

Decimate will down sample to a max number of points in a given sample by averaging samples together. i.e. max=240, if we have 240 samples, return them all, if we have 480 samples, every 2 samples average them (and time distance), and return 240 samples. This is relatively naive and if len(in) > max, <= max points will be returned, not necessarily max: length(out) = ceil(length(in)/max) -- feel free to fix this, setting a relatively high max will allow good enough granularity at higher lengths, i.e. for max of 1 hour tasks, sampling every 1s, decimate will return 15s samples if max=240. Large gaps in time between samples (a factor > (last-start)/max) will result in a shorter list being returned to account for lost samples. Decimate will modify the input list for efficiency, it is not copy safe. Input must be sorted by timestamp or this will fail gloriously.

type Stats

type Stats []Stat

Stats is a list of Stat, notably implements sql.Valuer

func (*Stats) Scan

func (s *Stats) Scan(value interface{}) error

implements sql.Scanner

func (Stats) Value

func (s Stats) Value() (driver.Value, error)

implements sql.Valuer, returning a string

type WaitResult

type WaitResult interface {
	// Wait may be called to await the result of a container's execution. If the
	// provided context is canceled and the container does not return first, the
	// resulting status will be 'canceled'. If the provided context times out
	// then the resulting status will be 'timeout'.
	Wait(context.Context) RunResult
}