compute

package
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 16, 2024 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Overview

Package compute is a generated GoMock package.

Index

Constants

View Source 
const (
	EventObjectExecutionUpsert = "ExecutionUpsert"
	EventObjectExecutionEvent  = "ExecutionEvent"
)

Watcher event object types

View Source 
const (
	EventTopicExecution            models.EventTopic = "Execution"
	EventTopicExecutionScanning    models.EventTopic = "Exec Scanning"
	EventTopicExecutionDownloading models.EventTopic = "Downloading Inputs"
	EventTopicExecutionPreparing   models.EventTopic = "Preparing Environment"
	EventTopicExecutionRunning     models.EventTopic = "Running Execution"
	EventTopicExecutionPublishing  models.EventTopic = "Publishing Results"
	EventTopicRestart              models.EventTopic = "Restart"
)
View Source 
const StorageDirectoryPerms = 0o755

Variables

This section is empty.

Functions

func ExecCompletedEvent added in v1.5.2 

func ExecCompletedEvent() *models.Event

func ExecFailedDueToNodeRestartEvent added in v1.5.2 

func ExecFailedDueToNodeRestartEvent() *models.Event

ExecFailedDueToNodeRestartEvent returns an event indicating that the execution failed due to a node restart

func ExecRunningEvent added in v1.5.2 

func ExecRunningEvent() *models.Event

Types

type BaseEndpoint 

type BaseEndpoint struct {
	// contains filtered or unexported fields
}

Base implementation of Endpoint

func NewBaseEndpoint 

func NewBaseEndpoint(params BaseEndpointParams) BaseEndpoint

func (BaseEndpoint) AskForBid 

func (s BaseEndpoint) AskForBid(
	ctx context.Context, request legacy.AskForBidRequest) (legacy.AskForBidResponse, error)

func (BaseEndpoint) BidAccepted 

func (s BaseEndpoint) BidAccepted(
	ctx context.Context, request legacy.BidAcceptedRequest) (legacy.BidAcceptedResponse, error)

func (BaseEndpoint) BidRejected 

func (s BaseEndpoint) BidRejected(
	ctx context.Context, request legacy.BidRejectedRequest) (legacy.BidRejectedResponse, error)

func (BaseEndpoint) CancelExecution 

func (s BaseEndpoint) CancelExecution(
	ctx context.Context, request legacy.CancelExecutionRequest) (legacy.CancelExecutionResponse, error)

type BaseEndpointParams 

type BaseEndpointParams struct {
	ExecutionStore store.ExecutionStore
}

type BaseExecutor 

type BaseExecutor struct {
	ID string

	Storages storage.StorageProvider
	// contains filtered or unexported fields
}

BaseExecutor is the base implementation for backend service. All operations are executed asynchronously, and a callback is used to notify the caller of the result.

func NewBaseExecutor 

func NewBaseExecutor(params BaseExecutorParams) *BaseExecutor

func (*BaseExecutor) Cancel 

func (e *BaseExecutor) Cancel(ctx context.Context, execution *models.Execution) error

Cancel the execution.

func (*BaseExecutor) Run 

func (e *BaseExecutor) Run(ctx context.Context, execution *models.Execution) (err error)

Run the execution after it has been accepted, and propose a result to the requester to be verified.

func (*BaseExecutor) Start added in v1.0.4 

func (e *BaseExecutor) Start(ctx context.Context, execution *models.Execution) *StartResult

func (*BaseExecutor) Wait added in v1.0.4 

func (e *BaseExecutor) Wait(ctx context.Context, execution *models.Execution) (*models.RunCommandResult, error)

type BaseExecutorParams 

type BaseExecutorParams struct {
	ID                     string
	Store                  store.ExecutionStore
	Storages               storage.StorageProvider
	StorageDirectory       string
	Executors              executor.ExecProvider
	ResultsPath            ResultsPath
	Publishers             publisher.PublisherProvider
	FailureInjectionConfig models.FailureInjectionConfig
}

type Bidder added in v0.3.26 

type Bidder struct {
	// contains filtered or unexported fields
}

func NewBidder added in v0.3.26 

func NewBidder(params BidderParams) Bidder

func (Bidder) RunBidding added in v0.3.26 

func (b Bidder) RunBidding(ctx context.Context, execution *models.Execution) error

type BidderParams added in v0.3.26 

type BidderParams struct {
	SemanticStrategy []bidstrategy.SemanticBidStrategy
	ResourceStrategy []bidstrategy.ResourceBidStrategy
	UsageCalculator  capacity.UsageCalculator
	Store            store.ExecutionStore
}

type Callback 

type Callback interface {
	OnBidComplete(ctx context.Context, result legacy.BidResult)
	OnRunComplete(ctx context.Context, result legacy.RunResult)
	OnComputeFailure(ctx context.Context, err legacy.ComputeError)
}

Callback Callbacks are used to notify the caller of the result of a job execution.

type Endpoint 

type Endpoint interface {
	// AskForBid asks for a bid for a given job, which will assign executionID to the job and return a bid
	// is interested in bidding on.
	AskForBid(context.Context, legacy.AskForBidRequest) (legacy.AskForBidResponse, error)
	// BidAccepted accepts a bid for a given executionID, which will trigger executing the job in the backend.
	// The execution can be synchronous or asynchronous, depending on the backend implementation.
	BidAccepted(context.Context, legacy.BidAcceptedRequest) (legacy.BidAcceptedResponse, error)
	// BidRejected rejects a bid for a given executionID.
	BidRejected(context.Context, legacy.BidRejectedRequest) (legacy.BidRejectedResponse, error)
	// CancelExecution cancels a job for a given executionID.
	CancelExecution(context.Context, legacy.CancelExecutionRequest) (legacy.CancelExecutionResponse, error)
}

Endpoint is the frontend and entry point to the compute node. Requesters, whether through API, CLI or other means, do interact with the frontend service to submit jobs, ask for bids, accept or reject bids, etc.

type ErrExecTimeout added in v1.5.0 

type ErrExecTimeout struct {
	ExecutionTimeout time.Duration
}

ErrExecTimeout is an error that is returned when an execution times out.

func NewErrExecTimeout added in v1.5.0 

func NewErrExecTimeout(executionTimeout time.Duration) ErrExecTimeout

func (ErrExecTimeout) Details added in v1.5.0 

func (e ErrExecTimeout) Details() map[string]string

func (ErrExecTimeout) Error added in v1.5.0 

func (e ErrExecTimeout) Error() string

func (ErrExecTimeout) Retryable added in v1.5.0 

func (e ErrExecTimeout) Retryable() bool

type Executor 

type Executor interface {
	// Run triggers the execution of a job.
	Run(ctx context.Context, execution *models.Execution) error
	// Cancel cancels the execution of a job.
	Cancel(ctx context.Context, execution *models.Execution) error
}

Executor Backend service that is responsible for running and publishing executions. Implementations can be synchronous or asynchronous by using Callbacks.

type ExecutorBuffer 

type ExecutorBuffer struct {
	ID string
	// contains filtered or unexported fields
}

ExecutorBuffer is a backend.Executor implementation that buffers executions locally until enough capacity is available to be able to run them. The buffer accepts a delegate backend.Executor that will be used to run the jobs. The buffer is implemented as a FIFO queue, where the order of the executions is determined by the order in which they were enqueued. However, an execution with high resource usage requirements might be skipped if there are newer jobs with lower resource usage requirements that can be executed immediately. This is done to improve utilization of compute nodes, though it might result in starvation and should be re-evaluated in the future.

func NewExecutorBuffer 

func NewExecutorBuffer(params ExecutorBufferParams) *ExecutorBuffer

func (*ExecutorBuffer) Cancel 

func (s *ExecutorBuffer) Cancel(_ context.Context, execution *models.Execution) error

func (*ExecutorBuffer) EnqueuedExecutionsCount added in v1.0.4 

func (s *ExecutorBuffer) EnqueuedExecutionsCount() int

EnqueuedExecutionsCount return number of items enqueued

func (*ExecutorBuffer) Run 

func (s *ExecutorBuffer) Run(ctx context.Context, execution *models.Execution) error

Run enqueues the execution and tries to run it if there is enough capacity.

func (*ExecutorBuffer) RunningExecutions 

func (s *ExecutorBuffer) RunningExecutions() []*models.Execution

RunningExecutions return list of running executions

type ExecutorBufferParams 

type ExecutorBufferParams struct {
	ID                     string
	DelegateExecutor       Executor
	Store                  store.ExecutionStore
	RunningCapacityTracker capacity.Tracker
	EnqueuedUsageTracker   capacity.UsageTracker
}

type InputCleanupFn added in v1.0.4 

type InputCleanupFn = func(context.Context) error

InputCleanupFn is a function type that defines the contract for cleaning up resources associated with input volume data after the job execution has either completed or failed to start. The function is expected to take a context.Context as an argument, which can be used for timeout and cancellation signals. It returns an error if the cleanup operation fails.

For example, an InputCleanupFn might be responsible for deallocating storage used for input volumes, or deleting temporary input files that were created as part of the job's execution. The nature of it operation depends on the storage provided by `storageProvider` and input sources of the jobs associated tasks. For the case of a wasm job its input and entry module storage volumes should be removed via the method after the jobs execution reaches a terminal state.

func PrepareRunArguments added in v1.0.4 

func PrepareRunArguments(
	ctx context.Context,
	storageProvider storage.StorageProvider,
	storageDirectory string,
	execution *models.Execution,
	resultsDir string,
) (*executor.RunCommandRequest, InputCleanupFn, error)

type MessageHandler added in v1.5.2 

type MessageHandler struct {
	// contains filtered or unexported fields
}

func NewMessageHandler added in v1.5.2 

func NewMessageHandler(executionStore store.ExecutionStore) *MessageHandler

func (*MessageHandler) HandleMessage added in v1.5.2 

func (m *MessageHandler) HandleMessage(ctx context.Context, message *envelope.Message) error

HandleMessage handles incoming messages

func (*MessageHandler) ShouldProcess added in v1.5.2 

func (m *MessageHandler) ShouldProcess(ctx context.Context, message *envelope.Message) bool

type MockCallback added in v1.0.0 

type MockCallback struct {
	// contains filtered or unexported fields
}

MockCallback is a mock of Callback interface.

func NewMockCallback added in v1.0.4 

func NewMockCallback(ctrl *gomock.Controller) *MockCallback

NewMockCallback creates a new mock instance.

func (*MockCallback) EXPECT added in v1.0.4 

func (m *MockCallback) EXPECT() *MockCallbackMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockCallback) OnBidComplete added in v1.0.0 

func (m *MockCallback) OnBidComplete(ctx context.Context, result legacy.BidResult)

OnBidComplete mocks base method.

func (*MockCallback) OnComputeFailure added in v1.0.0 

func (m *MockCallback) OnComputeFailure(ctx context.Context, err legacy.ComputeError)

OnComputeFailure mocks base method.

func (*MockCallback) OnRunComplete added in v1.0.0 

func (m *MockCallback) OnRunComplete(ctx context.Context, result legacy.RunResult)

OnRunComplete mocks base method.

type MockCallbackMockRecorder added in v1.0.4 

type MockCallbackMockRecorder struct {
	// contains filtered or unexported fields
}

MockCallbackMockRecorder is the mock recorder for MockCallback.

func (*MockCallbackMockRecorder) OnBidComplete added in v1.0.4 

func (mr *MockCallbackMockRecorder) OnBidComplete(ctx, result interface{}) *gomock.Call

OnBidComplete indicates an expected call of OnBidComplete.

func (*MockCallbackMockRecorder) OnComputeFailure added in v1.0.4 

func (mr *MockCallbackMockRecorder) OnComputeFailure(ctx, err interface{}) *gomock.Call

OnComputeFailure indicates an expected call of OnComputeFailure.

func (*MockCallbackMockRecorder) OnRunComplete added in v1.0.4 

func (mr *MockCallbackMockRecorder) OnRunComplete(ctx, result interface{}) *gomock.Call

OnRunComplete indicates an expected call of OnRunComplete.

type MockEndpoint added in v1.0.4 

type MockEndpoint struct {
	// contains filtered or unexported fields
}

MockEndpoint is a mock of Endpoint interface.

func NewMockEndpoint added in v1.0.4 

func NewMockEndpoint(ctrl *gomock.Controller) *MockEndpoint

NewMockEndpoint creates a new mock instance.

func (*MockEndpoint) AskForBid added in v1.0.4 

func (m *MockEndpoint) AskForBid(arg0 context.Context, arg1 legacy.AskForBidRequest) (legacy.AskForBidResponse, error)

AskForBid mocks base method.

func (*MockEndpoint) BidAccepted added in v1.0.4 

func (m *MockEndpoint) BidAccepted(arg0 context.Context, arg1 legacy.BidAcceptedRequest) (legacy.BidAcceptedResponse, error)

BidAccepted mocks base method.

func (*MockEndpoint) BidRejected added in v1.0.4 

func (m *MockEndpoint) BidRejected(arg0 context.Context, arg1 legacy.BidRejectedRequest) (legacy.BidRejectedResponse, error)

BidRejected mocks base method.

func (*MockEndpoint) CancelExecution added in v1.0.4 

func (m *MockEndpoint) CancelExecution(arg0 context.Context, arg1 legacy.CancelExecutionRequest) (legacy.CancelExecutionResponse, error)

CancelExecution mocks base method.

func (*MockEndpoint) EXPECT added in v1.0.4 

func (m *MockEndpoint) EXPECT() *MockEndpointMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

type MockEndpointMockRecorder added in v1.0.4 

type MockEndpointMockRecorder struct {
	// contains filtered or unexported fields
}

MockEndpointMockRecorder is the mock recorder for MockEndpoint.

func (*MockEndpointMockRecorder) AskForBid added in v1.0.4 

func (mr *MockEndpointMockRecorder) AskForBid(arg0, arg1 interface{}) *gomock.Call

AskForBid indicates an expected call of AskForBid.

func (*MockEndpointMockRecorder) BidAccepted added in v1.0.4 

func (mr *MockEndpointMockRecorder) BidAccepted(arg0, arg1 interface{}) *gomock.Call

BidAccepted indicates an expected call of BidAccepted.

func (*MockEndpointMockRecorder) BidRejected added in v1.0.4 

func (mr *MockEndpointMockRecorder) BidRejected(arg0, arg1 interface{}) *gomock.Call

BidRejected indicates an expected call of BidRejected.

func (*MockEndpointMockRecorder) CancelExecution added in v1.0.4 

func (mr *MockEndpointMockRecorder) CancelExecution(arg0, arg1 interface{}) *gomock.Call

CancelExecution indicates an expected call of CancelExecution.

type MockExecutor added in v1.0.4 

type MockExecutor struct {
	// contains filtered or unexported fields
}

MockExecutor is a mock of Executor interface.

func NewMockExecutor added in v1.0.4 

func NewMockExecutor(ctrl *gomock.Controller) *MockExecutor

NewMockExecutor creates a new mock instance.

func (*MockExecutor) Cancel added in v1.0.4 

func (m *MockExecutor) Cancel(ctx context.Context, execution *models.Execution) error

Cancel mocks base method.

func (*MockExecutor) EXPECT added in v1.0.4 

func (m *MockExecutor) EXPECT() *MockExecutorMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockExecutor) Run added in v1.0.4 

func (m *MockExecutor) Run(ctx context.Context, execution *models.Execution) error

Run mocks base method.

type MockExecutorMockRecorder added in v1.0.4 

type MockExecutorMockRecorder struct {
	// contains filtered or unexported fields
}

MockExecutorMockRecorder is the mock recorder for MockExecutor.

func (*MockExecutorMockRecorder) Cancel added in v1.0.4 

func (mr *MockExecutorMockRecorder) Cancel(ctx, execution interface{}) *gomock.Call

Cancel indicates an expected call of Cancel.

func (*MockExecutorMockRecorder) Run added in v1.0.4 

func (mr *MockExecutorMockRecorder) Run(ctx, execution interface{}) *gomock.Call

Run indicates an expected call of Run.

type NodeInfoDecorator added in v1.2.1 

type NodeInfoDecorator struct {
	// contains filtered or unexported fields
}

func NewNodeInfoDecorator added in v1.2.1 

func NewNodeInfoDecorator(params NodeInfoDecoratorParams) *NodeInfoDecorator

func (*NodeInfoDecorator) DecorateNodeInfo added in v1.2.1 

func (n *NodeInfoDecorator) DecorateNodeInfo(ctx context.Context, nodeInfo models.NodeInfo) models.NodeInfo

type NodeInfoDecoratorParams added in v1.2.1 

type NodeInfoDecoratorParams struct {
	Executors              executor.ExecProvider
	Publisher              publisher.PublisherProvider
	Storages               storage.StorageProvider
	RunningCapacityTracker capacity.Tracker
	QueueCapacityTracker   capacity.UsageTracker
	ExecutorBuffer         *ExecutorBuffer
	MaxJobRequirements     models.Resources
}

type ResultsPath added in v1.0.4 

type ResultsPath struct {
	// where do we copy the results from jobs temporarily?
	ResultsDir string
}

func NewResultsPath added in v1.0.4 

func NewResultsPath() (*ResultsPath, error)

func (*ResultsPath) Close added in v1.0.4 

func (results *ResultsPath) Close() error

func (*ResultsPath) EnsureResultsDir added in v1.0.4 

func (results *ResultsPath) EnsureResultsDir(executionID string) (string, error)

EnsureResultsDir ensures that the results directory exists.

func (*ResultsPath) PrepareResultsDir added in v1.0.4 

func (results *ResultsPath) PrepareResultsDir(executionID string) (string, error)

PrepareResultsDir creates a temporary directory to store the results of a job execution.

type StartResult added in v1.0.4 

type StartResult struct {
	Err error
	// contains filtered or unexported fields
}

func (*StartResult) Cleanup added in v1.0.4 

func (r *StartResult) Cleanup(ctx context.Context) error

type Startup added in v1.0.4 

type Startup struct {
	// contains filtered or unexported fields
}

func NewStartup added in v1.0.4 

func NewStartup(execStore store.ExecutionStore, execBuffer Executor) *Startup

func (*Startup) Execute added in v1.0.4 

func (s *Startup) Execute(ctx context.Context) error

Execute is used by the compute node to perform startup tasks that should happen before the node takes part in the rest of the network. This might be executions setup, or cleaning previous inputs etc.

Source Files

View all Source files

Directories

Path Synopsis
disk
system
system/gpu
Package store is a generated GoMock package.
 Package store is a generated GoMock package.
boltdb
resolver
test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.