scheduler

package
v0.0.0-...-cfef81d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 20, 2025 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type InMemoryBuildQueue 

type InMemoryBuildQueue struct {
	// contains filtered or unexported fields
}

InMemoryBuildQueue implements a BuildQueue that can distribute requests through the Remote Worker protocol to worker processes. All of the state of the build queue (i.e., list of queued execution requests and list of workers) is kept in memory.

func NewInMemoryBuildQueue 

func NewInMemoryBuildQueue(clock clock.Clock, uuidGenerator util.UUIDGenerator, randomNumberGenerator random.SingleThreadedGenerator, configuration *InMemoryBuildQueueConfiguration, actionRouter routing.ActionRouter) *InMemoryBuildQueue

NewInMemoryBuildQueue creates a new InMemoryBuildQueue that is in the initial state. It does not have any queues, workers or queued execution requests. All of these are created by sending it RPCs.

func (*InMemoryBuildQueue) AddDrain 

func (bq *InMemoryBuildQueue) AddDrain(ctx context.Context, request *buildqueuestate_pb.AddOrRemoveDrainRequest) (*emptypb.Empty, error)

AddDrain inserts a new drain into the list of drains currently tracked by the platform queue.

func (*InMemoryBuildQueue) Execute 

func (bq *InMemoryBuildQueue) Execute(in *remoteexecution_pb.ExecuteRequest, out remoteexecution_pb.Execution_ExecuteServer) error

Execute an action by scheduling it in the build queue. This call blocks until the action is completed.

func (*InMemoryBuildQueue) GetOperation 

func (bq *InMemoryBuildQueue) GetOperation(ctx context.Context, request *buildqueuestate_pb.GetOperationRequest) (*buildqueuestate_pb.GetOperationResponse, error)

GetOperation returns detailed information about a single operation identified by name.

func (*InMemoryBuildQueue) KillOperations 

func (bq *InMemoryBuildQueue) KillOperations(ctx context.Context, request *buildqueuestate_pb.KillOperationsRequest) (*emptypb.Empty, error)

KillOperations requests that one or more operations that are currently QUEUED or EXECUTING are moved the COMPLETED stage immediately. The next time any worker associated with the operation contacts the scheduler, it is requested to stop executing the operation.

func (*InMemoryBuildQueue) ListDrains 

func (bq *InMemoryBuildQueue) ListDrains(ctx context.Context, request *buildqueuestate_pb.ListDrainsRequest) (*buildqueuestate_pb.ListDrainsResponse, error)

ListDrains returns a list of all the drains that are present within a given platform queue.

func (*InMemoryBuildQueue) ListInvocationChildren 

func (bq *InMemoryBuildQueue) ListInvocationChildren(ctx context.Context, request *buildqueuestate_pb.ListInvocationChildrenRequest) (*buildqueuestate_pb.ListInvocationChildrenResponse, error)

ListInvocationChildren returns properties of all client invocations for which one or more operations are either queued or executing within a given platform queue.

When justQueuedInvocations is false, entries for invocations are returned even if they have no queued operations; only ones that are being executed right now. Entries will be sorted by invocation ID.

When justQueuedInvocations is true, entries for invocations are returned only if they have queued operations. Entries will be sorted by priority at which operations are scheduled.

func (*InMemoryBuildQueue) ListOperations 

func (bq *InMemoryBuildQueue) ListOperations(ctx context.Context, request *buildqueuestate_pb.ListOperationsRequest) (*buildqueuestate_pb.ListOperationsResponse, error)

ListOperations returns detailed information about all of the operations tracked by the InMemoryBuildQueue.

func (*InMemoryBuildQueue) ListPlatformQueues 

func (bq *InMemoryBuildQueue) ListPlatformQueues(ctx context.Context, request *emptypb.Empty) (*buildqueuestate_pb.ListPlatformQueuesResponse, error)

ListPlatformQueues returns a list of all platform queues currently managed by the scheduler.

func (*InMemoryBuildQueue) ListQueuedOperations 

func (bq *InMemoryBuildQueue) ListQueuedOperations(ctx context.Context, request *buildqueuestate_pb.ListQueuedOperationsRequest) (*buildqueuestate_pb.ListQueuedOperationsResponse, error)

ListQueuedOperations returns properties of all queued operations contained for a given invocation within a platform queue.

func (*InMemoryBuildQueue) ListWorkers 

func (bq *InMemoryBuildQueue) ListWorkers(ctx context.Context, request *buildqueuestate_pb.ListWorkersRequest) (*buildqueuestate_pb.ListWorkersResponse, error)

ListWorkers returns basic properties of all workers for a given platform queue.

func (*InMemoryBuildQueue) RegisterPredeclaredPlatformQueue 

func (bq *InMemoryBuildQueue) RegisterPredeclaredPlatformQueue(publicKeys []*ecdh.PublicKey, workerInvocationStickinessLimits []time.Duration, maximumQueuedBackgroundLearningOperations int, backgroundLearningOperationPriority int32, sizeClasses []uint32) error

RegisterPredeclaredPlatformQueue adds a platform queue to InMemoryBuildQueue that remains present, regardless of whether workers appear.

The main purpose of this method is to create platform queues that are capable of using multiple size classes, as a maximum size class and initialsizeclass.Analyzer can be provided for specifying how operations are assigned to size classes.

func (*InMemoryBuildQueue) RemoveDrain 

func (bq *InMemoryBuildQueue) RemoveDrain(ctx context.Context, request *buildqueuestate_pb.AddOrRemoveDrainRequest) (*emptypb.Empty, error)

RemoveDrain removes a drain from the list of drains currently tracked by the platform queue.

func (*InMemoryBuildQueue) Synchronize 

func (bq *InMemoryBuildQueue) Synchronize(ctx context.Context, request *remoteworker_pb.SynchronizeRequest) (*remoteworker_pb.SynchronizeResponse, error)

Synchronize the state of a worker with the scheduler. This call is used by a worker to report the completion of an operation and to request more work.

func (*InMemoryBuildQueue) TerminateWorkers 

func (bq *InMemoryBuildQueue) TerminateWorkers(ctx context.Context, request *buildqueuestate_pb.TerminateWorkersRequest) (*emptypb.Empty, error)

TerminateWorkers can be used to indicate that workers are going to be terminated in the nearby future. This function will block until any operations running on the workers complete, thereby allowing the workers to be terminated without interrupting operations.

func (*InMemoryBuildQueue) WaitExecution 

func (bq *InMemoryBuildQueue) WaitExecution(in *remoteexecution_pb.WaitExecutionRequest, out remoteexecution_pb.Execution_WaitExecutionServer) error

WaitExecution attaches to an existing operation that was created by Execute(). This call can be used by the client to reattach to an operation in case of network failure.

type InMemoryBuildQueueConfiguration 

type InMemoryBuildQueueConfiguration struct {
	// ExecutionUpdateInterval specifies how frequently Execute()
	// and WaitExecution() should stream updates for a task to the
	// client.
	ExecutionUpdateInterval time.Duration

	// OperationWithNoWaitersTimeout specifies how long an operation
	// may remain registered without having a single client calling
	// Execute() or WaitExecution() on it.
	OperationWithNoWaitersTimeout time.Duration

	// PlatformQueueWithNoWorkersTimeout specifies how long a
	// platform may remain registered by InMemoryBuildQueue when no
	// Synchronize() calls are received for any workers.
	PlatformQueueWithNoWorkersTimeout time.Duration

	// BusyWorkerSynchronizationInterval specifies how frequently
	// busy workers should be instructed to report their state, even
	// if no changes to their running state occurred.
	BusyWorkerSynchronizationInterval time.Duration

	// GetIdleWorkerSynchronizationInterval returns the maximum
	// amount of time a synchronization performed by a worker
	// against the scheduler may block. Once this amount of time is
	// passed, the worker is instructed to resynchronize, as a form
	// of health checking.
	//
	// Implementations may add jitter to this value to ensure
	// synchronization requests get smeared out over time.
	GetIdleWorkerSynchronizationInterval func() time.Duration

	// WorkerTaskRetryCount specifies how many times a worker may
	// redundantly request that a single task is started. By
	// limiting this, we can prevent a single task from
	// crash-looping a worker indefinitely.
	WorkerTaskRetryCount int

	// WorkerWithNoSynchronizationsTimeout specifies how long a
	// worker may remain registered by InMemoryBuildQueue when no
	// Synchronize() calls are received.
	WorkerWithNoSynchronizationsTimeout time.Duration

	// VerificationPrivateKeyRefreshInterval specifies the interval
	// at which the private key used to obtain the shared secret for
	// computing verification_zeros is refreshed. This ensures that
	// accidental disclosure of verification_zeros does not permit a
	// malicious worker to associate itself with a platform
	// indefinitely.
	VerificationPrivateKeyRefreshInterval time.Duration
}

InMemoryBuildQueueConfiguration contains all the tunable settings of the InMemoryBuildQueue.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.