drivers

README

This package is intended as a general purpose container abstraction library. With the same code, you can run on Docker, Rkt, etc.

Documentation

Index

• Constants
• Variables
• func ParseImage(image string) (registry, repo, tag string)
• type Config
  • func DefaultConfig() Config
• type ContainerTask
• type Cookie
• type Driver
• type Memory
  • func (m *Memory) Unmarshal(s string) error
  • func (m *Memory) UnmarshalJSON(p []byte) error
• type RunResult
• type Stat
  • func Decimate(maxSamples int, stats []Stat) []Stat

Constants

const (
	// task statuses
	StatusRunning   = "running"
	StatusSuccess   = "success"
	StatusError     = "error"
	StatusTimeout   = "timeout"
	StatusKilled    = "killed"
	StatusCancelled = "cancelled"
)

TODO: ensure some type is applied to these statuses.

Variables

var (
	// ErrOutOfMemory for OOM in container engine
	ErrOutOfMemory = userError(errors.New("out of memory error"))
)

Set of acceptable errors coming from container engines to TaskRunner

Functions

func ParseImage

func ParseImage(image string) (registry, repo, tag string)

Types

type Config

type Config struct {
	Docker    string `json:"docker" envconfig:"default=unix:///var/run/docker.sock,DOCKER"`
	Memory    Memory `json:"memory" envconfig:"default=256M,MEMORY_PER_JOB"`
	CPUShares int64  `json:"cpu_shares" envconfig:"default=2,CPU_SHARES"`
}

func DefaultConfig

func DefaultConfig() Config

for tests

type ContainerTask

type ContainerTask interface {
	// Command returns the command to run within the container.
	Command() string
	// EnvVars returns environment variable key-value pairs.
	EnvVars() map[string]string
	// Input feeds the container with data
	Input() io.Reader
	// Labels returns container label key-value pairs.
	Labels() map[string]string
	Id() string
	// Image returns the runtime specific image to run.
	Image() string
	// Timeout specifies the maximum time a task is allowed to run. Return 0 to let it run forever.
	Timeout() time.Duration
	// Driver will write output log from task execution to these writers. Must be
	// non-nil. Use io.Discard if log is irrelevant.
	Logger() (stdout, stderr io.Writer)
	// WriteStat writes a single Stat, implementation need not be thread safe.
	WriteStat(Stat)
	// Volumes returns an array of 2-element tuples indicating storage volume mounts.
	// The first element is the path on the host, and the second element is the
	// path in the container.
	Volumes() [][2]string
	// WorkDir returns the working directory to use for the task. Empty string
	// leaves it unset.
	WorkDir() string

	// Close is used to perform cleanup after task execution.
	// Close should be safe to call multiple times.
	Close()
}

The ContainerTask interface guides task execution across a wide variety of container oriented runtimes. This interface is unstable.

FIXME: This interface is large, and it is currently a little Docker specific.

type Cookie

type Cookie interface {
	io.Closer

	// Run should execute task on the implementation.
	// RunResult captures the result of task execution. This means if task
	// execution fails due to a problem in the task, Run() MUST return a valid
	// RunResult and nil as the error. The RunResult's Error() and Status()
	// should be used to indicate failure.
	// If the implementation itself suffers problems (lost of network, out of
	// disk etc.), a nil RunResult and an error message is preferred.
	//
	// Run() MUST monitor the context. task cancellation is indicated by
	// cancelling the context.
	Run(ctx context.Context) (RunResult, error)
}

A DriverCookie identifies a unique request to run a task.

Clients should always call Close() on a DriverCookie after they are done with it.

type Driver

type Driver interface {
	// Prepare can be used in order to do any preparation that a specific driver
	// may need to do before running the task, and can be useful to put
	// preparation that the task can recover from into (i.e. if pulling an image
	// fails because a registry is down, the task doesn't need to be failed).  It
	// returns a cookie that can be used to execute the task.
	// Callers should Close the cookie regardless of whether they run it.
	//
	// The returned cookie should respect the task's timeout when it is run.
	Prepare(ctx context.Context, task ContainerTask) (Cookie, error)
}

type Memory

type Memory uint64

Allows us to implement custom unmarshaling of JSON and envconfig.

func (*Memory) Unmarshal

func (m *Memory) Unmarshal(s string) error

func (*Memory) UnmarshalJSON

func (m *Memory) UnmarshalJSON(p []byte) error

type RunResult

type RunResult interface {
	// Error is an actionable/checkable error from the container.
	error

	// Status should return the current status of the task.
	// Only valid options are {"error", "success", "timeout", "killed", "cancelled"}.
	Status() string
}

RunResult indicates only the final state of the task.

type Stat

type Stat struct {
	Timestamp time.Time
	Metrics   map[string]uint64
}

Stat is a bucket of stats from a driver at a point in time for a certain task.

func Decimate

func Decimate(maxSamples int, stats []Stat) []Stat

Decimate will down sample to a max number of points in a given sample by averaging samples together. i.e. max=240, if we have 240 samples, return them all, if we have 480 samples, every 2 samples average them (and time distance), and return 240 samples. This is relatively naive and if len(in) > max, <= max points will be returned, not necessarily max: length(out) = ceil(length(in)/max) -- feel free to fix this, setting a relatively high max will allow good enough granularity at higher lengths, i.e. for max of 1 hour tasks, sampling every 1s, decimate will return 15s samples if max=240. Large gaps in time between samples (a factor > (last-start)/max) will result in a shorter list being returned to account for lost samples. Decimate will modify the input list for efficiency, it is not copy safe. Input must be sorted by timestamp or this will fail gloriously.