models

package
v1.1.6-dev2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 19, 2023 License: Apache-2.0 Imports: 20 Imported by: 1

Documentation

Index

Constants

View Source
const (
	// JobTypeService represents a long-running job that runs on a desired number of nodes
	// matching the specified constraints.
	JobTypeService = "service"

	// JobTypeDaemon represents a long-running job that runs on all nodes matching the
	// specified constraints.
	JobTypeDaemon = "daemon"

	// JobTypeBatch represents a batch job that runs to completion on the desired number
	// of nodes matching the specified constraints.
	JobTypeBatch = "batch"

	// JobTypeOps represents a batch job that runs to completion on all nodes matching
	// the specified constraints.
	JobTypeOps = "ops"
)
View Source
const (
	EngineNoop   = "noop"
	EngineDocker = "docker"
	EngineWasm   = "wasm"
)
View Source
const (
	StorageSourceNoop           = "noop"
	StorageSourceIPFS           = "ipfs"
	StorageSourceRepoClone      = "repoclone"
	StorageSourceRepoCloneLFS   = "repoCloneLFS"
	StorageSourceURL            = "urlDownload"
	StorageSourceS3             = "s3"
	StorageSourceS3PreSigned    = "s3PreSigned"
	StorageSourceInline         = "inline"
	StorageSourceLocalDirectory = "localDirectory"
)
View Source
const (
	PublisherNoop = "noop"
	PublisherIPFS = "ipfs"
	PublisherS3   = "s3"
)
View Source
const (
	DownloadFilenameStdout   = "stdout"
	DownloadFilenameStderr   = "stderr"
	DownloadFilenameExitCode = "exitCode"
	DownloadCIDsFolderName   = "raw"
	DownloadFolderPerm       = 0755
	DownloadFilePerm         = 0644
)
View Source
const (
	MetaReservedPrefix     = "bacalhau.org/"
	MetaRequesterID        = "bacalhau.org/requester.id"
	MetaRequesterPublicKey = "bacalhau.org/requester.publicKey"
	MetaClientID           = "bacalhau.org/client.id"
)
View Source
const (
	EvalStatusBlocked   = "blocked"
	EvalStatusPending   = "pending"
	EvalStatusComplete  = "complete"
	EvalStatusFailed    = "failed"
	EvalStatusCancelled = "canceled"
)
View Source
const (
	EvalTriggerJobRegister     = "job-register"
	EvalTriggerJobCancel       = "job-cancel"
	EvalTriggerRetryFailedExec = "exec-failure"
	EvalTriggerExecUpdate      = "exec-update"
)
View Source
const (
	// DefaultNamespace is the default namespace.
	DefaultNamespace = "default"
)

Variables

This section is empty.

Functions

func CopySlice

func CopySlice[T Copyable[T]](slice []T) []T

func FromLabelSelectorRequirements

func FromLabelSelectorRequirements(requirements ...*LabelSelectorRequirement) ([]labels.Requirement, error)

func NormalizeSlice

func NormalizeSlice[T Normalizable](slice []T)

func ValidateSlice

func ValidateSlice[T Validatable](slice []T) error

Types

type AllocatedResources

type AllocatedResources struct {
	Tasks map[string]*Resources `json:"Tasks"`
}

AllocatedResources is the set of resources to be used by an execution, which maybe be resources allocated to a single task or a set of tasks in the future.

func (*AllocatedResources) Copy

func (*AllocatedResources) Total

func (a *AllocatedResources) Total() *Resources

Total returns the total resources allocated

type BuildVersionInfo

type BuildVersionInfo struct {
	Major      string    `json:"Major,omitempty" example:"0"`
	Minor      string    `json:"Minor,omitempty" example:"3"`
	GitVersion string    `json:"GitVersion" example:"v0.3.12"`
	GitCommit  string    `json:"GitCommit" example:"d612b63108f2b5ce1ab2b9e02444eb1dac1d922d"`
	BuildDate  time.Time `json:"BuildDate" example:"2022-11-16T14:03:31Z"`
	GOOS       string    `json:"GOOS" example:"linux"`
	GOARCH     string    `json:"GOARCH" example:"amd64"`
}

BuildVersionInfo is the version of a Bacalhau binary (either client or server)

type ComputeNodeInfo

type ComputeNodeInfo struct {
	ExecutionEngines   []string  `json:"ExecutionEngines"`
	Publishers         []string  `json:"Publishers"`
	StorageSources     []string  `json:"StorageSources"`
	MaxCapacity        Resources `json:"MaxCapacity"`
	AvailableCapacity  Resources `json:"AvailableCapacity"`
	MaxJobRequirements Resources `json:"MaxJobRequirements"`
	RunningExecutions  int       `json:"RunningExecutions"`
	EnqueuedExecutions int       `json:"EnqueuedExecutions"`
}

type ComputeNodeInfoProvider

type ComputeNodeInfoProvider interface {
	GetComputeInfo(ctx context.Context) ComputeNodeInfo
}

type Copyable

type Copyable[T any] interface {
	Copy() T
}

Copyable is an interface for types that can be copied

type Evaluation

type Evaluation struct {
	// ID is the unique identifier of the evaluation.
	ID string `json:"ID"`

	// Namespace is the namespace the evaluation is created in
	Namespace string `json:"Namespace"`

	// JobID is the unique identifier of the job.
	JobID string `json:"JobID"`

	// TriggeredBy is the root cause that triggered the evaluation.
	TriggeredBy string `json:"TriggeredBy"`

	// Priority is the priority of the evaluation.
	// e.g. 50 is higher priority than 10, and so will be evaluated first.
	Priority int `json:"Priority"`

	// Type is the type of the job that needs to be evaluated.
	Type string `json:"Type"`

	// Status is the current status of the evaluation.
	Status string `json:"Status"`

	// Comment is to provide additional information about the evaluation.
	Comment string `json:"Comment"`

	// WaitUntil is the time until which the evaluation should be ignored, such as to implement backoff when
	// repeatedly failing to assess a job.
	WaitUntil time.Time `json:"WaitUntil"`

	CreateTime int64 `json:"CreateTime"`
	ModifyTime int64 `json:"ModifyTime"`
}

Evaluation is just to ask the scheduler to reassess if additional job instances must be scheduled or if existing ones must be stopped. It is possible that no action is required if the scheduler sees the desired job state matches the observed state. This allows the triggers (e.g. APIs, Node Manager) to be lightweight and submit evaluation on state changes without having to do complex logic to decide what actions to take.

func (*Evaluation) Copy

func (e *Evaluation) Copy() *Evaluation

func (*Evaluation) ShouldEnqueue

func (e *Evaluation) ShouldEnqueue() bool

ShouldEnqueue checks if a given Evaluation should be enqueued into the evaluation_broker

func (*Evaluation) String

func (e *Evaluation) String() string

func (*Evaluation) TerminalStatus

func (e *Evaluation) TerminalStatus() bool

TerminalStatus returns if the current status is terminal and will no longer transition.

func (*Evaluation) UpdateModifyTime

func (e *Evaluation) UpdateModifyTime()

UpdateModifyTime makes sure that time always moves forward, taking into account that different server clocks can drift apart.

type EvaluationReceipt

type EvaluationReceipt struct {
	Evaluation *Evaluation `json:"Evaluation"`
	// ReceiptHandle is a unique identifier when dequeue an Evaluation from a broker.
	ReceiptHandle string `json:"ReceiptHandle"`
}

EvaluationReceipt is a pair of an Evaluation and its ReceiptHandle.

type Execution

type Execution struct {
	// ID of the execution (UUID)
	ID string `json:"ID"`

	// Namespace is the namespace the execution is created in
	Namespace string `json:"Namespace"`

	// ID of the evaluation that generated this execution
	EvalID string `json:"EvalID"`

	// Name is a logical name of the execution.
	Name string `json:"Name"`

	// NodeID is the node this is being placed on
	NodeID string `json:"NodeID"`

	// Job is the parent job of the task being allocated.
	// This is copied at execution time to avoid issues if the job
	// definition is updated.
	JobID string `json:"JobID"`
	// TODO: evaluate using a copy of the job instead of a pointer
	Job *Job `json:"Job,omitempty"`

	// AllocatedResources is the total resources allocated for the execution tasks.
	AllocatedResources *AllocatedResources `json:"AllocatedResources"`

	// DesiredState of the execution on the compute node
	DesiredState State[ExecutionDesiredStateType] `json:"DesiredState"`

	// ComputeState observed state of the execution on the compute node
	ComputeState State[ExecutionStateType] `json:"ComputeState"`

	// the published results for this execution
	PublishedResult *SpecConfig `json:"PublishedResult"`

	// RunOutput is the output of the run command
	// TODO: evaluate removing this from execution spec in favour of calling `bacalhau logs`
	RunOutput *RunCommandResult `json:"RunOutput"`

	// PreviousExecution is the execution that this execution is replacing
	PreviousExecution string `json:"PreviousExecution"`

	// NextExecution is the execution that this execution is being replaced by
	NextExecution string `json:"NextExecution"`

	// FollowupEvalID captures a follow up evaluation created to handle a failed execution
	// that can be rescheduled in the future
	FollowupEvalID string `json:"FollowupEvalID"`

	// Revision is increment each time the execution is updated.
	Revision uint64 `json:"Revision"`

	// CreateTime is the time the execution has finished scheduling and been
	// verified by the plan applier.
	CreateTime int64 `json:"CreateTime"`
	// ModifyTime is the time the execution was last updated.
	ModifyTime int64 `json:"ModifyTime"`
}

Execution is used to allocate the placement of a task group to a node.

func (*Execution) AllocateResources

func (e *Execution) AllocateResources(taskID string, resources Resources)

AllocateResources allocates resources to a task

func (*Execution) Copy

func (e *Execution) Copy() *Execution

Copy provides a copy of the allocation and deep copies the job

func (*Execution) GetCreateTime added in v1.1.0

func (e *Execution) GetCreateTime() time.Time

GetCreateTime returns the creation time

func (*Execution) GetModifyTime added in v1.1.0

func (e *Execution) GetModifyTime() time.Time

GetModifyTime returns the modify time

func (*Execution) IsDiscarded

func (e *Execution) IsDiscarded() bool

IsDiscarded returns true if the execution has failed, been cancelled or rejected.

func (*Execution) IsTerminalComputeState

func (e *Execution) IsTerminalComputeState() bool

IsTerminalComputeState returns true if the execution observed state is terminal

func (*Execution) IsTerminalDesiredState

func (e *Execution) IsTerminalDesiredState() bool

IsTerminalDesiredState returns true if the execution desired state is terminal

func (*Execution) IsTerminalState

func (e *Execution) IsTerminalState() bool

IsTerminalState returns true if the execution desired of observed state is terminal

func (*Execution) JobNamespacedID

func (e *Execution) JobNamespacedID() NamespacedID

func (*Execution) Normalize

func (e *Execution) Normalize()

Normalize Allocation to ensure fields are initialized to the expectations of this version of Bacalhau. Should be called when restoring persisted Executions or receiving Executions from Bacalhau clients potentially on an older version of Bacalhau.

func (*Execution) String

func (e *Execution) String() string

func (*Execution) TotalAllocatedResources

func (e *Execution) TotalAllocatedResources() *Resources

func (*Execution) Validate

func (e *Execution) Validate() error

Validate is used to check a job for reasonable configuration

type ExecutionDesiredStateType

type ExecutionDesiredStateType int
const (
	ExecutionDesiredStatePending ExecutionDesiredStateType = iota
	ExecutionDesiredStateRunning
	ExecutionDesiredStateStopped
)

func (ExecutionDesiredStateType) String

func (i ExecutionDesiredStateType) String() string

type ExecutionStateType

type ExecutionStateType int

ExecutionStateType The state of an execution. An execution represents a single attempt to execute a job on a node. A compute node can have multiple executions for the same job due to retries, but there can only be a single active execution per node at any given time.

const (
	ExecutionStateUndefined ExecutionStateType = iota
	// ExecutionStateNew The execution has been created, but not pushed to a compute node yet.
	ExecutionStateNew
	// ExecutionStateAskForBid A node has been selected to execute a job, and is being asked to bid on the job.
	ExecutionStateAskForBid
	// ExecutionStateAskForBidAccepted compute node has rejected the ask for bid.
	ExecutionStateAskForBidAccepted
	// ExecutionStateAskForBidRejected compute node has rejected the ask for bid.
	ExecutionStateAskForBidRejected
	// ExecutionStateBidAccepted requester has accepted the bid, and the execution is expected to be running on the compute node.
	ExecutionStateBidAccepted // aka running
	// ExecutionStateBidRejected requester has rejected the bid.
	ExecutionStateBidRejected
	// ExecutionStateCompleted The execution has been completed, and the result has been published.
	ExecutionStateCompleted
	// ExecutionStateFailed The execution has failed.
	ExecutionStateFailed
	// ExecutionStateCancelled The execution has been canceled by the user
	ExecutionStateCancelled
)

TODO: change states to reflect non-bidding scheduling

func (ExecutionStateType) IsUndefined

func (s ExecutionStateType) IsUndefined() bool

IsUndefined returns true if the execution state is undefined

func (ExecutionStateType) String

func (i ExecutionStateType) String() string

type InputSource

type InputSource struct {
	// Source is the source of the artifact to be downloaded, e.g a URL, S3 bucket, etc.
	Source *SpecConfig `json:"Source"`

	// Alias is an optional reference to this input source that can be used for
	// dynamic linking to this input. (e.g. dynamic import in wasm by alias)
	Alias string `json:"Alias"`

	// Target is the path where the artifact should be mounted on
	Target string `json:"Target"`
}

func (*InputSource) Copy

func (a *InputSource) Copy() *InputSource

Copy returns a deep copy of the artifact

func (*InputSource) Normalize

func (a *InputSource) Normalize()

Normalize normalizes the artifact's source and target

func (*InputSource) Validate

func (a *InputSource) Validate() error

Validate validates the artifact's source and target

type Job

type Job struct {
	// ID is a unique identifier assigned to this job.
	// It helps to distinguish jobs with the same name after they have been deleted and re-created.
	//The ID is generated by the server and should not be set directly by the client.
	ID string `json:"ID"`

	// Name is the logical name of the job used to refer to it.
	// Submitting a job with the same name as an existing job will result in an
	// update to the existing job.
	Name string `json:"Name"`

	// Namespace is the namespace this job is running in.
	Namespace string `json:"Namespace"`

	// Type is the type of job this is, e.g. "daemon" or "batch".
	Type string `json:"Type"`

	// Priority defines the scheduling priority of this job.
	Priority int `json:"Priority"`

	// Count is the number of replicas that should be scheduled.
	Count int `json:"Count"`

	// Constraints is a selector which must be true for the compute node to run this job.
	Constraints []*LabelSelectorRequirement `json:"Constraints"`

	// Meta is used to associate arbitrary metadata with this job.
	Meta map[string]string `json:"Meta"`

	// Labels is used to associate arbitrary labels with this job, which can be used
	// for filtering.
	Labels map[string]string `json:"Labels"`

	Tasks []*Task `json:"Tasks"`

	// State is the current state of the job.
	State State[JobStateType] `json:"State"`

	// Version is a per-job monotonically increasing version number that is incremented
	// on each job specification update.
	Version uint64 `json:"Version"`

	// Revision is a per-job monotonically increasing revision number that is incremented
	// on each update to the job's state or specification
	Revision uint64 `json:"Revision"`

	CreateTime int64 `json:"CreateTime"`
	ModifyTime int64 `json:"ModifyTime"`
}

func (*Job) AllStorageTypes

func (j *Job) AllStorageTypes() []string

AllStorageTypes returns keys of all storage types required by the job

func (*Job) Copy

func (j *Job) Copy() *Job

Copy returns a deep copy of the Job. It is expected that callers use recover. This job can panic if the deep copy failed as it uses reflection.

func (*Job) GetCreateTime

func (j *Job) GetCreateTime() time.Time

GetCreateTime returns the creation time

func (*Job) GetModifyTime

func (j *Job) GetModifyTime() time.Time

GetModifyTime returns the modify time

func (*Job) IsLongRunning added in v1.1.0

func (j *Job) IsLongRunning() bool

IsLongRunning returns true if the job is long running

func (*Job) IsTerminal

func (j *Job) IsTerminal() bool

IsTerminal returns true if the job is in a terminal state

func (*Job) NamespacedID

func (j *Job) NamespacedID() NamespacedID

NamespacedID returns the namespaced id useful for logging

func (*Job) Normalize

func (j *Job) Normalize()

Normalize is used to canonicalize fields in the Job. This should be called when registering a Job.

func (*Job) SanitizeSubmission added in v1.1.0

func (j *Job) SanitizeSubmission() (warnings []string)

SanitizeSubmission is used to sanitize a job for reasonable configuration when it is submitted.

func (*Job) String

func (j *Job) String() string

func (*Job) Task

func (j *Job) Task() *Task

Task returns the job task TODO: remove this once we have multiple tasks per job

func (*Job) Validate

func (j *Job) Validate() error

Validate is used to check a job for reasonable configuration

func (*Job) ValidateSubmission

func (j *Job) ValidateSubmission() error

ValidateSubmission is used to check a job for reasonable configuration when it is submitted. It is a subset of Validate that does not check fields with defaults, such as job ID

type JobHistory

type JobHistory struct {
	Type           JobHistoryType                   `json:"Type"`
	JobID          string                           `json:"JobID"`
	NodeID         string                           `json:"NodeID,omitempty"`
	ExecutionID    string                           `json:"ExecutionID,omitempty"`
	JobState       *StateChange[JobStateType]       `json:"JobState,omitempty"`
	ExecutionState *StateChange[ExecutionStateType] `json:"ExecutionState,omitempty"`
	NewRevision    uint64                           `json:"NewRevision"`
	Comment        string                           `json:"Comment,omitempty"`
	Time           time.Time                        `json:"Time"`
}

JobHistory represents a single event in the history of a job. An event can be at the job level, or execution (node) level.

{Job,Event}State fields will only be present if the Type field is of the matching type.

type JobHistoryType

type JobHistoryType int
const (
	JobHistoryTypeUndefined JobHistoryType = iota
	JobHistoryTypeJobLevel
	JobHistoryTypeExecutionLevel
)

func (JobHistoryType) MarshalText

func (s JobHistoryType) MarshalText() ([]byte, error)

func (JobHistoryType) String

func (i JobHistoryType) String() string

func (*JobHistoryType) UnmarshalText

func (s *JobHistoryType) UnmarshalText(text []byte) (err error)

type JobStateType

type JobStateType int
const (
	JobStateTypeUndefined JobStateType = iota

	// JobStateTypePending is the state of a job that has been submitted but not
	// yet scheduled.
	JobStateTypePending

	// JobStateTypeRunning is the state of a job that has been scheduled, with at
	// least one active execution.
	JobStateTypeRunning

	// JobStateTypeCompleted is the state of a job that has successfully completed.
	// Only valid for batch jobs.
	JobStateTypeCompleted

	// JobStateTypeFailed is the state of a job that has failed.
	JobStateTypeFailed

	// JobStateTypeStopped is the state of a job that has been stopped by the user.
	JobStateTypeStopped
)

func JobStateTypes added in v1.1.0

func JobStateTypes() []JobStateType

func (JobStateType) IsUndefined

func (s JobStateType) IsUndefined() bool

IsUndefined returns true if the job state is undefined

func (JobStateType) MarshalText added in v1.1.0

func (s JobStateType) MarshalText() ([]byte, error)

func (JobStateType) String

func (i JobStateType) String() string

func (*JobStateType) UnmarshalText added in v1.1.0

func (s *JobStateType) UnmarshalText(text []byte) (err error)

type LabelSelectorRequirement

type LabelSelectorRequirement struct {
	// key is the label key that the selector applies to.
	Key string `json:"Key"`
	// operator represents a key's relationship to a set of values.
	// Valid operators are In, NotIn, Exists and DoesNotExist.
	Operator selection.Operator `json:"Operator"`
	// values is an array of string values. If the operator is In or NotIn,
	// the values array must be non-empty. If the operator is Exists or DoesNotExist,
	// the values array must be empty. This array is replaced during a strategic
	Values []string `json:"Values,omitempty"`
}

LabelSelectorRequirement A selector that contains values, a key, and an operator that relates the key and values. These are based on labels library from kubernetes package. While we use labels.Requirement to represent the label selector requirements in the command line arguments as the library supports multiple parsing formats, and we also use it when matching selectors to labels as that's what the library expects, labels.Requirements are not serializable, so we need to convert them to LabelSelectorRequirements.

func ToLabelSelectorRequirements

func ToLabelSelectorRequirements(requirements ...labels.Requirement) []LabelSelectorRequirement

func (*LabelSelectorRequirement) Copy

func (*LabelSelectorRequirement) String

func (r *LabelSelectorRequirement) String() string

func (*LabelSelectorRequirement) Validate

func (r *LabelSelectorRequirement) Validate() error

type NamespacedID

type NamespacedID struct {
	ID        string `json:"ID"`
	Namespace string `json:"Namespace"`
}

NamespacedID is a tuple of an ID and a namespace

func NewNamespacedID

func NewNamespacedID(id, ns string) NamespacedID

NewNamespacedID returns a new namespaced ID given the ID and namespace

func (NamespacedID) String

func (n NamespacedID) String() string

type Network

type Network int
const (
	// NetworkNone specifies that the job does not require networking.
	NetworkNone Network = iota

	// NetworkFull specifies that the job requires unfiltered raw IP networking.
	NetworkFull

	// NetworkHTTP specifies that the job requires HTTP networking to certain domains.
	//
	// The model is: the job specifier submits a job with the domain(s) it will
	// need to communicate with, the compute provider uses this to make some
	// decision about the risk of the job and bids accordingly, and then at run
	// time the traffic is limited to only the domain(s) specified.
	//
	// As a command, something like:
	//
	//  bacalhau docker run —network=http —domain=crates.io —domain=github.com -i ipfs://Qmy1234myd4t4,dst=/code rust/compile
	//
	// The “risk” for the compute provider is that the job does something that
	// violates its terms, the terms of its hosting provider or ISP, or even the
	// law in its jurisdiction (e.g. accessing and spreading illegal content,
	// performing cyberattacks). So the same sort of risk as operating a Tor
	// exit node.
	//
	// The risk for the job specifier is that we are operating in an environment
	// they are paying for, so there is an incentive to hijack that environment
	// (e.g. via a compromised package download that runs a crypto miner on
	// install, and uses up all the paid-for job time). Having the traffic
	// enforced to only domains specified makes those sorts of attacks much
	// trickier and less valuable.
	//
	// The compute provider might well enforce its limits by other means, but
	// having the domains specified up front allows it to skip bidding on jobs
	// it knows will fail in its executor. So this is hopefully a better UX for
	// job specifiers who can have their job picked up only by someone who will
	// run it successfully.
	NetworkHTTP
)

func ParseNetwork

func ParseNetwork(s string) (Network, error)

func (Network) MarshalText

func (n Network) MarshalText() ([]byte, error)

func (Network) String

func (i Network) String() string

func (*Network) UnmarshalText

func (n *Network) UnmarshalText(text []byte) (err error)

type NetworkConfig

type NetworkConfig struct {
	Type    Network  `json:"Type"`
	Domains []string `json:"Domains,omitempty"`
}

func (*NetworkConfig) Copy

func (n *NetworkConfig) Copy() *NetworkConfig

func (*NetworkConfig) Disabled

func (n *NetworkConfig) Disabled() bool

Disabled returns whether network connections should be completely disabled according to this config.

func (*NetworkConfig) DomainSet

func (n *NetworkConfig) DomainSet() []string

DomainSet returns the "unique set" of domains from the network config. Domains listed multiple times and any subdomain that is also matched by a wildcard is removed.

This is something of an implementation detail – it matches the behavior expected by our Docker HTTP gateway, which complains and/or fails to start if these requirements are not met.

func (*NetworkConfig) Normalize

func (n *NetworkConfig) Normalize()

Normalize ensures that the network config is in a consistent state.

func (*NetworkConfig) Validate

func (n *NetworkConfig) Validate() (err error)

Validate returns an error if any of the fields do not pass validation, or nil otherwise.

type NetworkConfigBuilder

type NetworkConfigBuilder struct {
	// contains filtered or unexported fields
}

func NewNetworkConfigBuilder

func NewNetworkConfigBuilder() *NetworkConfigBuilder

func NewNetworkConfigBuilderFromNetwork

func NewNetworkConfigBuilderFromNetwork(network *NetworkConfig) *NetworkConfigBuilder

func (*NetworkConfigBuilder) Build

func (b *NetworkConfigBuilder) Build() (*NetworkConfig, error)

func (*NetworkConfigBuilder) BuildOrDie

func (b *NetworkConfigBuilder) BuildOrDie() *NetworkConfig

BuildOrDie is a helper that wraps Build and panics on error.

func (*NetworkConfigBuilder) Domains

func (b *NetworkConfigBuilder) Domains(domains ...string) *NetworkConfigBuilder

func (*NetworkConfigBuilder) Type

type NodeInfo

type NodeInfo struct {
	PeerInfo        peer.AddrInfo     `json:"PeerInfo"`
	NodeType        NodeType          `json:"NodeType"`
	Labels          map[string]string `json:"Labels"`
	ComputeNodeInfo *ComputeNodeInfo  `json:"ComputeNodeInfo,omitempty" yaml:",omitempty"`
	BacalhauVersion BuildVersionInfo  `json:"BacalhauVersion"`
}

func (NodeInfo) ID added in v1.1.4

func (n NodeInfo) ID() string

ID returns the node ID

func (NodeInfo) IsComputeNode

func (n NodeInfo) IsComputeNode() bool

IsComputeNode returns true if the node is a compute node

type NodeInfoProvider

type NodeInfoProvider interface {
	GetNodeInfo(ctx context.Context) NodeInfo
}

type NodeType

type NodeType int
const (
	NodeTypeRequester NodeType
	NodeTypeCompute
)

func ParseNodeType

func ParseNodeType(s string) (NodeType, error)

func (NodeType) MarshalText added in v1.1.0

func (e NodeType) MarshalText() ([]byte, error)

func (NodeType) String

func (i NodeType) String() string

func (*NodeType) UnmarshalText added in v1.1.0

func (e *NodeType) UnmarshalText(text []byte) (err error)

type Normalizable

type Normalizable interface {
	Normalize()
}

Normalizable is an interface for types that can be normalized (e.g. empty maps are converted to nil)

type Plan

type Plan struct {
	EvalID      string `json:"EvalID"`
	EvalReceipt string `json:"EvalReceipt"`
	// TODO: passing the evalID should be enough once we persist evaluations
	Eval     *Evaluation `json:"Eval,omitempty"`
	Priority int         `json:"Priority"`

	Job *Job `json:"Job,omitempty"`

	DesiredJobState JobStateType `json:"DesiredJobState,omitempty"`
	Comment         string       `json:"Comment,omitempty"`

	// NewExecutions holds the executions to be created.
	NewExecutions []*Execution `json:"NewExecutions,omitempty"`

	UpdatedExecutions map[string]*PlanExecutionDesiredUpdate `json:"UpdatedExecutions,omitempty"`
}

Plan holds actions as a result of processing an evaluation by the scheduler.

func NewPlan

func NewPlan(eval *Evaluation, job *Job) *Plan

NewPlan creates a new Plan instance.

func (*Plan) AppendApprovedExecution

func (p *Plan) AppendApprovedExecution(execution *Execution)

AppendApprovedExecution marks an execution as accepted and ready to be started.

func (*Plan) AppendExecution

func (p *Plan) AppendExecution(execution *Execution)

AppendExecution appends the execution to the plan executions.

func (*Plan) AppendStoppedExecution

func (p *Plan) AppendStoppedExecution(execution *Execution, comment string)

AppendStoppedExecution marks an execution to be stopped.

func (*Plan) MarkJobCompleted

func (p *Plan) MarkJobCompleted()

func (*Plan) MarkJobFailed

func (p *Plan) MarkJobFailed(comment string)

func (*Plan) MarkJobRunningIfEligible added in v1.1.3

func (p *Plan) MarkJobRunningIfEligible()

MarkJobRunningIfEligible updates the job state to "Running" under certain conditions.

type PlanExecutionDesiredUpdate

type PlanExecutionDesiredUpdate struct {
	Execution    *Execution                `json:"Execution"`
	DesiredState ExecutionDesiredStateType `json:"DesiredState"`
	Comment      string                    `json:"Comment,omitempty"`
}

type Resources

type Resources struct {
	// CPU units
	CPU float64 `json:"CPU,omitempty"`
	// Memory in bytes
	Memory uint64 `json:"Memory,omitempty"`
	// Disk in bytes
	Disk uint64 `json:"Disk,omitempty"`
	// GPU units
	GPU uint64 `json:"GPU,omitempty"`
}

func (*Resources) Add

func (r *Resources) Add(other Resources) *Resources

Add returns the sum of the resources

func (*Resources) Copy

func (r *Resources) Copy() *Resources

Copy returns a deep copy of the resources

func (*Resources) IsZero

func (r *Resources) IsZero() bool

func (*Resources) LessThan

func (r *Resources) LessThan(other Resources) bool

func (*Resources) LessThanEq

func (r *Resources) LessThanEq(other Resources) bool

func (*Resources) Max

func (r *Resources) Max(other Resources) *Resources

func (*Resources) Merge

func (r *Resources) Merge(other Resources) *Resources

Merge merges the resources, preferring the current resources

func (*Resources) String

func (r *Resources) String() string

return string representation of ResourceUsageData

func (*Resources) Sub

func (r *Resources) Sub(other Resources) *Resources

func (*Resources) Validate

func (r *Resources) Validate() error

Validate returns an error if the resources are invalid

type ResourcesConfig

type ResourcesConfig struct {
	// CPU https://github.com/BTBurke/k8sresource string
	CPU string `json:"CPU,omitempty"`
	// Memory github.com/c2h5oh/datasize string
	Memory string `json:"Memory,omitempty"`
	// Memory github.com/c2h5oh/datasize string
	Disk string `json:"Disk,omitempty"`
	GPU  string `json:"GPU,omitempty"`
}

func (*ResourcesConfig) Copy

func (r *ResourcesConfig) Copy() *ResourcesConfig

Copy returns a deep copy of the resources

func (*ResourcesConfig) Normalize

func (r *ResourcesConfig) Normalize()

Normalize normalizes the resources

func (*ResourcesConfig) ToResources

func (r *ResourcesConfig) ToResources() (*Resources, error)

ToResources converts the resources config to resources

func (*ResourcesConfig) Validate

func (r *ResourcesConfig) Validate() error

Validate returns an error if the resources are invalid

type ResourcesConfigBuilder

type ResourcesConfigBuilder struct {
	// contains filtered or unexported fields
}

func NewResourcesConfigBuilder

func NewResourcesConfigBuilder() *ResourcesConfigBuilder

func (*ResourcesConfigBuilder) Build

func (*ResourcesConfigBuilder) BuildOrDie

func (r *ResourcesConfigBuilder) BuildOrDie() *ResourcesConfig

BuildOrDie is the same as Build, but panics if an error occurs

func (*ResourcesConfigBuilder) CPU

func (*ResourcesConfigBuilder) Disk

func (*ResourcesConfigBuilder) GPU

func (*ResourcesConfigBuilder) Memory

type ResultPath

type ResultPath struct {
	// Name
	Name string `json:"Name"`
	// The path to the file/dir
	Path string `json:"Path"`
}

func (*ResultPath) Copy

func (p *ResultPath) Copy() *ResultPath

Copy returns a copy of the path

func (*ResultPath) Normalize

func (p *ResultPath) Normalize()

Normalize normalizes the path to a canonical form

func (*ResultPath) Validate

func (p *ResultPath) Validate() error

Validate validates the path

type RunCommandResult

type RunCommandResult struct {
	// stdout of the run. Yaml provided for `describe` output
	STDOUT string `json:"Stdout"`

	// bool describing if stdout was truncated
	StdoutTruncated bool `json:"StdoutTruncated"`

	// stderr of the run.
	STDERR string `json:"stderr"`

	// bool describing if stderr was truncated
	StderrTruncated bool `json:"StderrTruncated"`

	// exit code of the run.
	ExitCode int `json:"ExitCode"`

	// Runner error
	ErrorMsg string `json:"ErrorMsg"`
}

func NewRunCommandResult

func NewRunCommandResult() *RunCommandResult

type SpecConfig

type SpecConfig struct {
	// Type of the config
	Type string `json:"Type"`

	// Params is a map of the config params
	Params map[string]interface{} `json:"Params,omitempty"`
}

func NewSpecConfig

func NewSpecConfig(t string) *SpecConfig

NewSpecConfig returns a new spec config

func (*SpecConfig) Copy

func (s *SpecConfig) Copy() *SpecConfig

Copy returns a shallow copy of the spec config TODO: implement deep copy if the value is a nested map, slice or Copyable

func (*SpecConfig) IsEmpty added in v1.1.0

func (s *SpecConfig) IsEmpty() bool

IsEmpty returns true if the spec config is empty

func (*SpecConfig) IsType

func (s *SpecConfig) IsType(t string) bool

IsType returns true if the current SpecConfig

func (*SpecConfig) Normalize

func (s *SpecConfig) Normalize()

func (*SpecConfig) Validate

func (s *SpecConfig) Validate() error

func (*SpecConfig) ValidateAllowBlank added in v1.1.0

func (s *SpecConfig) ValidateAllowBlank() error

ValidateAllowBlank is the same as Validate but allows blank types. This is useful for when you want to validate a spec config that is optional.

func (*SpecConfig) WithParam

func (s *SpecConfig) WithParam(key string, value interface{}) *SpecConfig

WithParam adds a param to the spec config

type State

type State[T any] struct {
	// StateType is the current state of the object.
	StateType T `json:"StateType"`

	// Message is a human readable message describing the state.
	Message string `json:"Message,omitempty"`
}

State is a generic struct for representing the state of an object, with an optional human readable message.

func NewExecutionDesiredState

func NewExecutionDesiredState(stateType ExecutionDesiredStateType) State[ExecutionDesiredStateType]

NewExecutionDesiredState returns a new ExecutionDesiredStateType with the specified state type

func NewExecutionState

func NewExecutionState(stateType ExecutionStateType) State[ExecutionStateType]

NewExecutionState returns a new ExecutionState with the specified state type

func NewJobState

func NewJobState(stateType JobStateType) State[JobStateType]

NewJobState returns a new JobState with the specified state type

func (State[T]) WithMessage

func (s State[T]) WithMessage(message string) State[T]

WithMessage returns a new State with the specified message.

type StateChange

type StateChange[StateType any] struct {
	Previous StateType `json:"Previous,omitempty"`
	New      StateType `json:"New,omitempty"`
}

StateChange represents a change in state of one of the state types.

type Task

type Task struct {
	// Name of the task
	Name string `json:"Name"`

	Engine *SpecConfig `json:"Engine"`

	Publisher *SpecConfig `json:"Publisher"`

	// Map of environment variables to be used by the driver
	Env map[string]string `json:"Env,omitempty"`

	// Meta is used to associate arbitrary metadata with this task.
	Meta map[string]string `json:"Meta,omitempty"`

	// InputSources is a list of remote artifacts to be downloaded before running the task
	// and mounted into the task.
	InputSources []*InputSource `json:"InputSources,omitempty"`

	// ResultPaths is a list of task volumes to be included in the task's published result
	ResultPaths []*ResultPath `json:"ResultPaths,omitempty"`

	// ResourcesConfig is the resources needed by this task
	ResourcesConfig *ResourcesConfig `json:"Resources,omitempty"`

	Network *NetworkConfig `json:"Network,omitempty"`

	Timeouts *TimeoutConfig `json:"Timeouts,omitempty"`
}

func (*Task) AllStorageTypes

func (t *Task) AllStorageTypes() []string

func (*Task) Copy

func (t *Task) Copy() *Task

func (*Task) Normalize

func (t *Task) Normalize()

func (*Task) ToBuilder

func (t *Task) ToBuilder() *TaskBuilder

ToBuilder returns a new task builder with the same values as the task

func (*Task) Validate

func (t *Task) Validate() error

Validate is used to check a job for reasonable configuration

func (*Task) ValidateSubmission added in v1.1.0

func (t *Task) ValidateSubmission() error

ValidateSubmission is used to check a task for reasonable configuration when it is submitted. It is a subset of Validate that does not check fields with defaults, such as timeouts and resources.

type TaskBuilder

type TaskBuilder struct {
	// contains filtered or unexported fields
}

func NewTaskBuilder

func NewTaskBuilder() *TaskBuilder

func NewTaskBuilderFromTask

func NewTaskBuilderFromTask(task *Task) *TaskBuilder

func (*TaskBuilder) Build

func (b *TaskBuilder) Build() (*Task, error)

func (*TaskBuilder) BuildOrDie

func (b *TaskBuilder) BuildOrDie() *Task

BuildOrDie is the same as Build, but panics if an error occurs

func (*TaskBuilder) Engine

func (b *TaskBuilder) Engine(engine *SpecConfig) *TaskBuilder

func (*TaskBuilder) InputSources

func (b *TaskBuilder) InputSources(inputSources ...*InputSource) *TaskBuilder

func (*TaskBuilder) Name

func (b *TaskBuilder) Name(name string) *TaskBuilder

func (*TaskBuilder) Network

func (b *TaskBuilder) Network(network *NetworkConfig) *TaskBuilder

func (*TaskBuilder) Publisher

func (b *TaskBuilder) Publisher(publisher *SpecConfig) *TaskBuilder

func (*TaskBuilder) ResourcesConfig

func (b *TaskBuilder) ResourcesConfig(resourcesConfig *ResourcesConfig) *TaskBuilder

func (*TaskBuilder) ResultPaths

func (b *TaskBuilder) ResultPaths(resultPaths ...*ResultPath) *TaskBuilder

func (*TaskBuilder) Timeouts

func (b *TaskBuilder) Timeouts(timeouts *TimeoutConfig) *TaskBuilder

type TimeoutConfig

type TimeoutConfig struct {
	// ExecutionTimeout is the maximum amount of time a task is allowed to run in seconds.
	// Zero means no timeout, such as for a daemon task.
	ExecutionTimeout int64 `json:"ExecutionTimeout,omitempty"`
}

TimeoutConfig is the configuration for timeout related settings, such as execution and shutdown timeouts.

func (*TimeoutConfig) Copy

func (t *TimeoutConfig) Copy() *TimeoutConfig

Copy returns a deep copy of the timeout config.

func (*TimeoutConfig) GetExecutionTimeout

func (t *TimeoutConfig) GetExecutionTimeout() time.Duration

GetExecutionTimeout returns the execution timeout duration

func (*TimeoutConfig) Validate

func (t *TimeoutConfig) Validate() error

type Validatable

type Validatable interface {
	Validate() error
}

Validatable is an interface for types that can be validated

Directories

Path Synopsis
migration

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL