grader

package
v1.9.68 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 12, 2023 License: BSD-3-Clause Imports: 30 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// QueuePriorityHigh represents a QueuePriority with high priority.
	QueuePriorityHigh = QueuePriority(0)
	// QueuePriorityNormal represents a QueuePriority with normal priority.
	QueuePriorityNormal = QueuePriority(1)
	// QueuePriorityLow represents a QueuePriority with low priority.
	QueuePriorityLow = QueuePriority(2)
	// QueuePriorityEphemeral represents a QueuePriority with the lowest
	// priority. This also does not persist the results in the filesystem.
	QueuePriorityEphemeral = QueuePriority(3)

	// QueueCount represents the total number of queues in the grader.
	QueueCount = 4

	// DefaultQueueName is the default queue name.
	DefaultQueueName = "default"

	// QueueEventTypeManagerAdded represents when a run is added to the QueueManager.
	QueueEventTypeManagerAdded QueueEventType = iota

	// QueueEventTypeManagerRemoved represents when a run is removed from the QueueManager.
	QueueEventTypeManagerRemoved

	// QueueEventTypeQueueAdded represents when a run is added to a Queue.
	QueueEventTypeQueueAdded

	// QueueEventTypeQueueRemoved represents when a run is removed from a Qeuue.
	QueueEventTypeQueueRemoved

	// QueueEventTypeRetried represents when a run is retried.
	QueueEventTypeRetried

	// QueueEventTypeAbandoned represents when a run is abandoned due to too many retries.
	QueueEventTypeAbandoned
)

Variables

This section is empty.

Functions

func CreateArchiveFromGit added in v1.1.17 

func CreateArchiveFromGit(
	archivePath string,
	gitserverURL string,
	gitserverAuthorization string,
	problemName string,
	inputHash string,
) (int64, error)

CreateArchiveFromGit creates an archive that can be sent to a Runner as an Input from a git repository.

func GetLibinteractiveVersion added in v1.1.0 

func GetLibinteractiveVersion() (string, error)

GetLibinteractiveVersion returns the version of the installed libinteractive jar.

func GetRepositoryPath added in v1.1.17 

func GetRepositoryPath(
	root string,
	problemName string,
) string

GetRepositoryPath returns the path of a problem repository.

func IsProblemSlow added in v1.9.4 

func IsProblemSlow(
	gitserverURL string,
	gitserverAuthorization string,
	problemName string,
	inputHash string,
) (bool, error)

IsProblemSlow returns whether the problem at that particular commit is slow. It uses a global cache to avoid having to ask this question for every single problem.

func NewCachedInputFactory added in v1.1.0 

func NewCachedInputFactory(inputPath string) common.CachedInputFactory

NewCachedInputFactory returns a new CachedInputFactory.

func NewInputFactory added in v1.1.0 

func NewInputFactory(
	problemName string,
	config *common.Config,
) common.InputFactory

NewInputFactory returns a new InputFactory for the specified problem name and configuration.

Types

type ArtifactManager added in v1.9.9 

type ArtifactManager struct {
	Submissions SubmissionsArtifacts
	// contains filtered or unexported fields
}

ArtifactManager is an abstraction around the filesystem. All writes will go to both the filesystem and (if it's set) S3, and reads will be attempted against the filesystem first and then S3 as fallback.

func NewArtifactManager added in v1.9.9 

func NewArtifactManager(s3c *s3.S3) *ArtifactManager

NewArtifactManager returns a new ArtifactManager.

func (*ArtifactManager) Grader added in v1.9.10 

func (a *ArtifactManager) Grader(ctx *common.Context, runID int64) Artifacts

Grader returns a wrapper for the grader artifacts.

type Artifacts added in v1.9.10 

type Artifacts interface {
	// Get returns a io.ReadCloser with the contents of the artifact.
	Get(ctx *common.Context, filename string) (io.ReadCloser, error)
	// Put atomically creates an artifact with the contents as specified by the
	// reader.
	Put(ctx *common.Context, filename string, r io.Reader) error
	// Clean cleans the local filesystem's contents. No attempt to clean the S3
	// artifacts is done.
	Clean() error
}

Artifacts is an interface to interact with grader artifacts.

type CachedInputFactory added in v1.1.0 

type CachedInputFactory struct {
	// contains filtered or unexported fields
}

A CachedInputFactory is a grader-specific CachedInputFactory. It reads all its inputs from the filesystem, and validates that they have not been accidentally corrupted by comparing the input against its hash.

func (*CachedInputFactory) GetInputHash added in v1.1.0 

func (factory *CachedInputFactory) GetInputHash(
	dirname string,
	info os.FileInfo,
) (hash string, ok bool)

GetInputHash returns the hash of the current InputFactory.

func (*CachedInputFactory) NewInput added in v1.1.0 

func (factory *CachedInputFactory) NewInput(
	hash string,
	mgr *common.InputManager,
) common.Input

NewInput returns an Input with the provided hash.

type Context 

type Context struct {
	common.Context
	QueueManager          *QueueManager
	InflightMonitor       *InflightMonitor
	InputManager          *common.InputManager
	LibinteractiveVersion string
}

A Context holds the state of the Grader.

func NewContext 

func NewContext(reader io.Reader) (*Context, error)

NewContext returns a new Context where the configuration is read in a JSON format from the supplied io.Reader.

func (*Context) Close 

func (ctx *Context) Close()

Close releases all resources owned by the context.

func (*Context) Wrap added in v1.9.1 

func (ctx *Context) Wrap(c context.Context) *Context

Wrap returns a new Context with the applied context.

type EphemeralRunManager added in v1.1.0 

type EphemeralRunManager struct {
	sync.Mutex
	// contains filtered or unexported fields
}

EphemeralRunManager handles a queue of recently-submitted ephemeral runs. This has a fixed maximum size with a last-in, first-out eviction policy.

func NewEphemeralRunManager added in v1.1.0 

func NewEphemeralRunManager(ctx *Context) *EphemeralRunManager

NewEphemeralRunManager returns a new EphemeralRunManager.

func (*EphemeralRunManager) Commit added in v1.1.0 

func (mgr *EphemeralRunManager) Commit(runInfo *RunInfo) error

Commit adds the files produced by the Run into the FIFO cache, potentially evicting older runs in the process.

func (*EphemeralRunManager) Get added in v1.1.0 

func (mgr *EphemeralRunManager) Get(token string) (string, bool)

Get returns the filesystem path for a previous ephemeral run, and whether or not it actually exists.

func (*EphemeralRunManager) Initialize added in v1.1.0 

func (mgr *EphemeralRunManager) Initialize() error

Initialize goes through the past ephemeral runs in order to add them to the FIFO cache.

func (*EphemeralRunManager) SetEphemeral added in v1.1.0 

func (mgr *EphemeralRunManager) SetEphemeral(runInfo *RunInfo) (string, error)

SetEphemeral makes a RunContext ephemeral. It does this by setting its runtime path to be in the ephemeral subdirectory.

func (*EphemeralRunManager) String added in v1.1.0 

func (mgr *EphemeralRunManager) String() string

type EphemeralRunRequest added in v1.1.0 

type EphemeralRunRequest struct {
	Source   string               `json:"source"`
	Language string               `json:"language"`
	Input    *common.LiteralInput `json:"input"`
}

A EphemeralRunRequest represents a client's request to run some code.

type InflightMonitor 

type InflightMonitor struct {
	sync.Mutex
	// contains filtered or unexported fields
}

InflightMonitor manages all in-flight Runs (Runs that have been picked up by a runner) and tracks their state in case the runner becomes unresponsive.

func NewInflightMonitor 

func NewInflightMonitor() *InflightMonitor

NewInflightMonitor returns a new InflightMonitor.

func (*InflightMonitor) Add 

func (monitor *InflightMonitor) Add(
	runCtx *RunContext,
	runner string,
) *InflightRun

Add creates an InflightRun wrapper for the specified RunContext, adds it to the InflightMonitor, and monitors it for timeouts. A RunContext can be later accesssed through its attempt ID.

func (*InflightMonitor) Get 

func (monitor *InflightMonitor) Get(attemptID uint64) (*RunContext, <-chan struct{}, bool)

Get returns the RunContext associated with the specified attempt ID.

func (*InflightMonitor) GetRunData 

func (monitor *InflightMonitor) GetRunData() []*RunData

GetRunData returns the list of in-flight run information.

func (*InflightMonitor) MarshalJSON 

func (monitor *InflightMonitor) MarshalJSON() ([]byte, error)

MarshalJSON returns a JSON representation of the InflightMonitor.

func (*InflightMonitor) Remove 

func (monitor *InflightMonitor) Remove(attemptID uint64)

Remove removes the specified attempt ID from the in-flight runs and signals the RunContext for completion.

type InflightRun 

type InflightRun struct {
	// contains filtered or unexported fields
}

InflightRun is a wrapper around a RunContext when it is handed off a queue and a runner has been assigned to it.

type Input added in v1.1.0 

type Input struct {
	// contains filtered or unexported fields
}

Input is a common.Input generated from a git repository that is then stored in a .tar.gz file that can be sent to a runner.

func (*Input) Delete added in v1.1.0 

func (input *Input) Delete() error

func (*Input) Persist added in v1.1.0 

func (input *Input) Persist() error

Persist writes the Input to disk and stores its hash.

func (*Input) Release added in v1.1.20 

func (input *Input) Release()

func (*Input) Transmit added in v1.1.0 

func (input *Input) Transmit(w http.ResponseWriter) error

Transmit sends a serialized version of the Input to the runner. It sends a .tar.gz file with the Content-SHA1 header with the hexadecimal representation of its SHA-1 hash.

func (*Input) Verify added in v1.1.0 

func (input *Input) Verify() error

type InputFactory added in v1.1.0 

type InputFactory struct {
	// contains filtered or unexported fields
}

InputFactory is a common.InputFactory that can store specific versions of a problem's git repository into a .tar.gz file that can be easily shipped to runners.

func (*InputFactory) NewInput added in v1.1.0 

func (factory *InputFactory) NewInput(
	hash string,
	mgr *common.InputManager,
) common.Input

NewInput creates a new Input that is identified by the supplied hash.

type Queue 

type Queue struct {
	Name string
	// contains filtered or unexported fields
}

Queue represents a RunContext queue with three discrete priorities.

func (*Queue) AddRun 

func (queue *Queue) AddRun(
	ctx *common.Context,
	runInfo *RunInfo,
	inputRef *common.InputRef,
) error

AddRun adds a new RunContext to the current Queue.

func (*Queue) AddWaitableRun added in v1.1.39 

func (queue *Queue) AddWaitableRun(
	ctx *common.Context,
	runInfo *RunInfo,
	inputRef *common.InputRef,
) (*RunWaitHandle, error)

AddWaitableRun adds a new RunContext to the current Queue, and returns a RunWaitHandle so the caller can wait for the run state changes. If the desired queue is full, there is no retry mechanism and the run is immediately given up.

func (*Queue) GetRun 

func (queue *Queue) GetRun(
	runner string,
	monitor *InflightMonitor,
	closeNotifier <-chan bool,
) (*RunContext, <-chan struct{}, bool)

GetRun dequeues a RunContext from the queue and adds it to the global InflightMonitor. This function will block if there are no RunContext objects in the queue.

type QueueEvent added in v1.1.0 

type QueueEvent struct {
	Delta    time.Duration
	Priority QueuePriority
	Type     QueueEventType
}

QueueEvent represents an event that happens from the QueueManager's perspective.

type QueueEventType added in v1.1.0 

type QueueEventType int

QueueEventType represents the type of event that just occurred.

type QueueInfo 

type QueueInfo struct {
	Lengths [QueueCount]int
}

QueueInfo has information about one queue.

type QueueManager 

type QueueManager struct {
	sync.Mutex
	PostProcessor *RunPostProcessor
	// contains filtered or unexported fields
}

QueueManager is an expvar-friendly manager for Queues.

func NewQueueManager 

func NewQueueManager(channelLength int, graderRuntimePath string) *QueueManager

NewQueueManager creates a new QueueManager.

func (*QueueManager) Add 

func (manager *QueueManager) Add(name string) *Queue

Add creates a new queue or fetches a previously created queue with the specified name and returns it.

func (*QueueManager) AddEvent added in v1.1.0 

func (manager *QueueManager) AddEvent(event *QueueEvent)

AddEvent adds an event and broadcasts it to all the listeners.

func (*QueueManager) AddEventListener added in v1.1.0 

func (manager *QueueManager) AddEventListener(c chan<- *QueueEvent)

AddEventListener registers a listener for event notifications.

func (*QueueManager) Close added in v1.1.0 

func (manager *QueueManager) Close()

Close terminates the event listener goroutine.

func (*QueueManager) Get 

func (manager *QueueManager) Get(name string) (*Queue, error)

Get gets the queue with the specified name.

func (*QueueManager) GetQueueInfo 

func (manager *QueueManager) GetQueueInfo() map[string]QueueInfo

GetQueueInfo returns the length of all the queues.

func (*QueueManager) MarshalJSON 

func (manager *QueueManager) MarshalJSON() ([]byte, error)

MarshalJSON returns a JSON representation of the queue lengths for reporting purposes.

type QueuePriority added in v1.1.0 

type QueuePriority int

QueuePriority represents the relative priority of a queue with respect with other queues. All the runs in a higher priority queue will be run before those in a lower priority queue.

type RunContext 

type RunContext struct {
	*common.Context
	RunInfo *RunInfo
	// contains filtered or unexported fields
}

RunContext is a wrapper around a RunInfo. This is used when a Run is sitting on a Queue on the grader.

func (*RunContext) Close 

func (runCtx *RunContext) Close()

Close finalizes the run, stores its results in the filesystem, and releases any resources associated with the RunContext.

func (*RunContext) Debug 

func (runCtx *RunContext) Debug() error

Debug marks a RunContext as being for debug. This causes some additional logging and in C/C++ it enables AddressSanitizer. Use with caution, since ASan needs a relaxed sandboxing profile.

func (*RunContext) Requeue 

func (runCtx *RunContext) Requeue(lastAttempt bool) bool

Requeue adds a RunContext back to the Queue from where it came from, if it has any retries left. It always adds the RunContext to the highest-priority queue.

func (*RunContext) String 

func (runCtx *RunContext) String() string

type RunData 

type RunData struct {
	AttemptID    uint64
	ID           int64
	GUID         string
	Queue        string
	AttemptsLeft int
	Runner       string
	Time         int64
	Elapsed      int64
}

RunData represents the data of a single run.

type RunInfo 

type RunInfo struct {
	ID           int64
	SubmissionID int64
	GUID         string
	Contest      *string
	Problemset   *int64
	Run          *common.Run
	Result       runner.RunResult
	Artifacts    Artifacts
	Priority     QueuePriority
	PenaltyType  string
	ScoreMode    string

	CreationTime time.Time
	QueueTime    time.Time
}

RunInfo holds the necessary data of a Run, even after the RunContext is gone.

func NewRunInfo added in v1.1.38 

func NewRunInfo() *RunInfo

NewRunInfo returns an empty RunInfo.

type RunPostProcessor 

type RunPostProcessor struct {
	// contains filtered or unexported fields
}

A RunPostProcessor broadcasts the events of runs that have been finished to all registered listeners.

func NewRunPostProcessor 

func NewRunPostProcessor() *RunPostProcessor

NewRunPostProcessor returns a new RunPostProcessor.

func (*RunPostProcessor) AddListener 

func (postProcessor *RunPostProcessor) AddListener(c chan<- *RunInfo)

AddListener adds a channel that will be notified for every Run that has finished.

func (*RunPostProcessor) Close 

func (postProcessor *RunPostProcessor) Close()

Close notifies the RunPostProcessor goroutine that there is no more work to be done.

func (*RunPostProcessor) PostProcess 

func (postProcessor *RunPostProcessor) PostProcess(run *RunInfo)

PostProcess queues the provided run for post-processing. All the registered listeners will be notified about this run.

type RunWaitHandle added in v1.1.38 

type RunWaitHandle struct {
	// contains filtered or unexported fields
}

RunWaitHandle allows waiting on the run to change state.

func (*RunWaitHandle) Ready added in v1.1.38 

func (h *RunWaitHandle) Ready() <-chan struct{}

Ready returns a channel that will be closed when the Run is ready.

func (*RunWaitHandle) Running added in v1.1.38 

func (h *RunWaitHandle) Running() <-chan struct{}

Running returns a channel that will be closed when the Run is picked up by a runner.

type SubmissionsArtifacts added in v1.9.9 

type SubmissionsArtifacts struct {
	// contains filtered or unexported fields
}

SubmissionsArtifacts is an object that allows interacting with submissions.

func (*SubmissionsArtifacts) GetSource added in v1.9.9 

func (a *SubmissionsArtifacts) GetSource(ctx *common.Context, guid string) (string, error)

GetSource returns the source of a submission, identified by its guid.

func (*SubmissionsArtifacts) PutSource added in v1.9.9 

func (a *SubmissionsArtifacts) PutSource(ctx *common.Context, guid string, r io.Reader) error

PutSource writes the source of the submission to the filesystem (and maybe to S3).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.