executor

README

executor

• Project README
• Release/Build Status
• Changelog
• License
• Contribution Guidelines
• Code of Conduct
• Secure Coding Guidelines

Table of Contents

1. Description
2. Structure and Organisation
3. Class Diagram
4. Functionality
5. Data Types
6. Testing
7. Proposed Functionality/Requirements
8. References

Specification

Description

The executor package is responsible for executing the jobs received by the device management service (DMS). It provides an unified interface to run various executors such as docker, firecracker etc

Structure and Organisation

Here is quick overview of the contents of this pacakge:

• README: Current file which is aimed towards developers who wish to use and modify the executor functionality.

• init: This file initializes a logger instance for the executor package.

• types: This file contains the interfaces that other packages in the DMS call to utilise functionality offered by the executor package.

• docker: This folder contains the implementation of docker executor.

• firecracker: This folder contains the implementation of firecracker executor.

Class Diagram

Source

executor class diagram

Rendered from source file

!$rootUrlGitlab = "https://gitlab.com/nunet/device-management-service/-/raw/main"
!$packageRelativePath = "/executor"
!$packageUrlGitlab = $rootUrlGitlab + $packageRelativePath
 
!include $packageUrlGitlab/specs/class_diagram.puml

Functionality

The main functionality offered by the executor package is defined via the Executor interface.

type Executor interface {
	// Start initiates an execution for the given ExecutionRequest.
	// It returns an error if the execution already exists and is in a started or terminal state.
	// Implementations may also return other errors based on resource limitations or internal faults.
	Start(ctx context.Context, request *types.ExecutionRequest) error

	// Run initiates and waits for the completion of an execution for the given ExecutionRequest.
	// It returns a ExecutionResult and an error if any part of the operation fails.
	// Specifically, it will return an error if the execution already exists and is in a started or terminal state.
	Run(ctx context.Context, request *types.ExecutionRequest) (*types.ExecutionResult, error)

	// Wait monitors the completion of an execution identified by its executionID.
	// It returns two channels:
	// 1. A channel that emits the execution result once the task is complete.
	// 2. An error channel that relays any issues encountered, such as when the
	//    execution is non-existent or has already concluded.
	Wait(ctx context.Context, executionID string) (<-chan *types.ExecutionResult, <-chan error)

	// Cancel attempts to cancel an ongoing execution identified by its executionID.
	// Returns an error if the execution does not exist or is already in a terminal state.
	Cancel(ctx context.Context, executionID string) error

	// GetLogStream provides a stream of output for an ongoing or completed execution identified by its executionID.
	// The 'Tail' flag indicates whether to exclude hstorical data or not.
	// The 'follow' flag indicates whether the stream should continue to send data as it is produced.
	// Returns an io.ReadCloser to read the output stream and an error if the operation fails.
	// Specifically, it will return an error if the execution does not exist.
	GetLogStream(ctx context.Context, executionID string) (io.ReadCloser, error)
}

Its methods are explained below:

Start

• signature: Start(ctx context.Context, request executor.ExecutionRequest) -> error
  
• input #1: Go context
  
• input #2: executor.ExecutionRequest
  
• output: error

Start function takes a Go context object and a executor.ExecutionRequest type as input. It returns an error if the execution already exists and is in a started or terminal state. Implementations may also return other errors based on resource limitations or internal faults.

Run

• signature: Run(ctx context.Context, request executor.ExecutionRequest) -> (executor.ExecutionResult, error)
  
• input #1: Go context
  
• input #2: executor.ExecutionRequest
  
• output (success): executor.ExecutionResult
  
• output (error): error

Run initiates and waits for the completion of an execution for the given Execution Request. It returns a executor.ExecutionResult and an error if any part of the operation fails. Specifically, it will return an error if the execution already exists and is in a started or terminal state.

Wait

• signature: Wait(ctx context.Context, executionID string) -> (<-chan executor.ExecutionResult, <-chan error)
  
• input #1: Go context
  
• input #2: executor.ExecutionRequest.ExecutionID
  
• output #1: Channel that returns executor.ExecutionResult
  
• output #2: Channel that returns error

Wait monitors the completion of an execution identified by its executionID. It returns two channels:

1. A channel that emits the execution result once the task is complete;
2. An error channel that relays any issues encountered, such as when the execution is non-existent or has already concluded.

Cancel

• signature: Cancel(ctx context.Context, executionID string) -> error
  
• input #1: Go context
  
• input #2: executor.ExecutionRequest.ExecutionID
  
• output: error

Cancel attempts to terminate an ongoing execution identified by its executionID. It returns an error if the execution does not exist or is already in a terminal state.

GetLogStream

• signature: GetLogStream(ctx context.Context, request executor.LogStreamRequest, executionID string) -> (io.ReadCloser, error)
  
• input #1: Go context
  
• input #2: executor.LogStreamRequest
  
• input #3: executor.ExecutionRequest.ExecutionID
  
• output #1: io.ReadCloser
  
• output #2: error

GetLogStream provides a stream of output for an ongoing or completed execution identified by its executionID. There are two flags that can be used to modify the functionality:

• The Tail flag indicates whether to exclude historical data or not.
• The follow flag indicates whether the stream should continue to send data as it is produced.

It returns an io.ReadCloser object to read the output stream and an error if the operation fails. Specifically, it will return an error if the execution does not exist.

Data Types

• types.ExecutionRequest: This is the input that executor receives to initiate a job execution.

• types.ExecutionResult: This contains the result of the job execution.

• executor.LogStreamRequest: This contains input parameters sent to the executor to get job execution logs.

// LogStreamRequest is the request object for streaming logs from an execution
type LogStreamRequest struct {
	// ID of the job
	JobID string

	// ID of the execution
	ExecutionID string

	// Tail the logs
	Tail bool

	// Follow the logs
	Follow bool
}

• types.SpecConfig: This allows arbitrary configuration/parameters as needed during implementation of specific executor.

• types.Resources: This contains resources to be used for execution.

• storage.StorageVolume: This contains parameters of storage volume used during execution.

Testing

Unit tests are defined in subpackages which implement the interface defined in this package.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of executor package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

• executor package implementation

References

Documentation

Index

• type Executor

Types

type Executor

type Executor interface {
	// GetID returns the unique identifier for the executor.
	GetID() string
	// Start initiates an execution for the given ExecutionRequest.
	// It returns an error if the execution already exists and is in a started or terminal state.
	// Implementations may also return other errors based on resource limitations or internal faults.
	Start(ctx context.Context, request *types.ExecutionRequest) error

	// Run initiates and waits for the completion of an execution for the given ExecutionRequest.
	// It returns a ExecutionResult and an error if any part of the operation fails.
	// Specifically, it will return an error if the execution already exists and is in a started or terminal state.
	Run(ctx context.Context, request *types.ExecutionRequest) (*types.ExecutionResult, error)

	// Pause attempts to pause an ongoing execution identified by its executionID.
	// Returns an error if the execution does not exist or is already in a terminal state.
	Pause(ctx context.Context, executionID string) error

	// Resume attempts to resume a paused execution identified by its executionID.
	// Returns an error if the execution does not exist, is not paused, or is already in a terminal state.
	Resume(ctx context.Context, executionID string) error

	// Wait monitors the completion of an execution identified by its executionID.
	// It returns two channels:
	// 1. A channel that emits the execution result once the task is complete.
	// 2. An error channel that relays any issues encountered, such as when the
	//    execution is non-existent or has already concluded.
	Wait(ctx context.Context, executionID string) (<-chan *types.ExecutionResult, <-chan error)

	// Cancel attempts to cancel an ongoing execution identified by its executionID.
	// Returns an error if the execution does not exist or is already in a terminal state.
	Cancel(ctx context.Context, executionID string) error

	// GetLogStream provides a stream of output for an ongoing or completed execution identified by its executionID.
	// The 'Tail' flag indicates whether to exclude hstorical data or not.
	// The 'follow' flag indicates whether the stream should continue to send data as it is produced.
	// Returns an io.ReadCloser to read the output stream and an error if the operation fails.
	// Specifically, it will return an error if the execution does not exist.
	GetLogStream(ctx context.Context, request types.LogStreamRequest) (io.ReadCloser, error)

	// List returns a slice of ExecutionListItem containing information about current executions.
	// This includes the execution ID and whether it's currently running.
	List() []types.ExecutionListItem

	// GetStatus returns the status of the execution identified by its executionID.
	// It returns the status of the execution and an error if the execution is not found or status is unknown.
	GetStatus(ctx context.Context, executionID string) (types.ExecutionStatus, error)

	// WaitForStatus waits for the execution to reach a specific status.
	// It returns an error if the execution is not found or the status is unknown.
	WaitForStatus(ctx context.Context, executionID string, status types.ExecutionStatus, timeout *time.Duration) error

	// Exec executes a command in a container and returns the exit code, output, and an error if the operation fails.
	Exec(ctx context.Context, containerID string, command []string) (int, string, error)
}

Executor serves as an execution manager for running jobs on a specific backend, such as a Docker daemon. It provides a comprehensive set of methods to initiate, monitor, terminate, and retrieve output streams for executions.