dispatch

Documentation

Overview

Package dispatch is a helper library for building Arvados container dispatchers.

Index

• Constants
• type DispatchFunc
• type Dispatcher
  • func (d *Dispatcher) Run(ctx context.Context) error
  • func (d *Dispatcher) TrackContainer(uuid string) error
  • func (d *Dispatcher) Unlock(uuid string) error
  • func (d *Dispatcher) UpdateState(uuid string, state arvados.ContainerState) error
• type Logger

Constants

const (
	Queued    = arvados.ContainerStateQueued
	Locked    = arvados.ContainerStateLocked
	Running   = arvados.ContainerStateRunning
	Complete  = arvados.ContainerStateComplete
	Cancelled = arvados.ContainerStateCancelled
)

Types

type DispatchFunc

type DispatchFunc func(*Dispatcher, arvados.Container, <-chan arvados.Container) error

A DispatchFunc executes a container (if the container record is Locked) or resume monitoring an already-running container, and wait until that container exits.

While the container runs, the DispatchFunc should listen for updated container records on the provided channel. When the channel closes, the DispatchFunc should stop the container if it's still running, and return.

The DispatchFunc should not return until the container is finished.

type Dispatcher

type Dispatcher struct {
	Arv *arvadosclient.ArvadosClient

	Logger Logger

	// Batch size for container queries
	BatchSize int

	// Queue polling frequency
	PollPeriod time.Duration

	// Time to wait between successive attempts to run the same container
	MinRetryPeriod time.Duration

	// Func that implements the container lifecycle. Must be set
	// to a non-nil DispatchFunc before calling Run().
	RunContainer DispatchFunc
	// contains filtered or unexported fields
}

Dispatcher struct

func (*Dispatcher) Run

func (d *Dispatcher) Run(ctx context.Context) error

Run watches the API server's queue for containers that are either ready to run and available to lock, or are already locked by this dispatcher's token. When a new one appears, Run calls RunContainer in a new goroutine.

func (*Dispatcher) TrackContainer

func (d *Dispatcher) TrackContainer(uuid string) error

TrackContainer ensures a tracker is running for the given UUID, regardless of the current state of the container (except: if the container is locked by a different dispatcher, a tracker will not be started). If the container is not in Locked or Running state, the new tracker will close down immediately.

This allows the dispatcher to put its own RunContainer func into a cleanup phase (for example, to kill local processes created by a prevous dispatch process that are still running even though the container state is final) without the risk of having multiple goroutines monitoring the same UUID.

func (*Dispatcher) Unlock

func (d *Dispatcher) Unlock(uuid string) error

Unlock makes the unlock API call which updates the state of a container to Queued.

func (*Dispatcher) UpdateState

func (d *Dispatcher) UpdateState(uuid string, state arvados.ContainerState) error

UpdateState makes an API call to change the state of a container.

type Logger

type Logger interface {
	Printf(string, ...interface{})
	Warnf(string, ...interface{})
	Debugf(string, ...interface{})
}