ugrade

README

UGrade

Simple competitive programming contest platform using contestant PC as grader

Motivation

To organize competitive programming competition we usually deploy several grader to handle contestant submissions. Sometimes the number of contestant is so huge that our grader cannot handle the submissions. Why don't we use contestant machine as grader instead?

TC Generator (Temporary Design)

Every problem should have jury solution, checker and testcase generator. Testcase generator basically just a normal program that generate input files for grading. Testcase case generator should generate the same input file in every machine and every time (like pure function).

Grading system will ask testcase generator how many input files they should generates:

int main(int argc, char** argv) {
    if (argc < 2) return -1;
    
    string action = argv[1];
    if (action == "sample_count")
        cout<<sample_count()<<endl; // tells grading system how many sample input will be generated
    
    if (action == "testcase_count")
        cout<<testcase_count()<<endl;  // tells grading system how many testcase input will be generated
    
    return 0;
}

Grading system then run testcase generator N times, depending how many testcase input they provided:

if (action == "sample") {
    unsigned int sample_id;
    sscanf(argv[2], "%d", &sample_id);
    
    // generate sample input `sample_id`. Use stdout to output the testcase.
    sample(sample_id);
} else if (action == "testcase") {
    unsigned int tc_id;
    sscanf(argv[2], "%d", &tc_id);
    
    // generate testcase input `tc_id`. Use stdout to output the testcase.
    testcase(tc_id);
}

License

MIT

Documentation

Index

• Constants
• Variables
• type Client
• type Command
• type Consumer
• type FSBind
• type JobResult
• type JobSolver
• type JobSpec
• type LanguageListItem
• type LanguageListResult
• type ProblemListItem
• type ProblemListResult
• type Sandbox
• type SignInRequest
• type SignInResult
• type SourceCode
• type SubmissionListItem
• type SubmissionListResult
• type SubmitRequest
• type SubmitResult
• type Usage
• type Verdict

Constants

const (
	// VerdictCE stands for Compilation Error.
	VerdictCE = Verdict("CE")

	// VerdictIE stands for Internal Error.
	VerdictIE = Verdict("IE")

	// VerdictRTE stands for Run Time Error.
	VerdictRTE = Verdict("RTE")

	// VerdictMLE stands for Memory Limit Exceeded.
	VerdictMLE = Verdict("MLE")

	// VerdictTLE stands for Time Limit Exceeded.
	VerdictTLE = Verdict("TLE")

	// VerdictWA stands for Wrong Answer.
	VerdictWA = Verdict("WA")

	// VerdictPENDING indicating that job is not graded yet.
	VerdictPENDING = Verdict("PENDING")

	// VerdictAC stands for Accepted.
	VerdictAC = Verdict("AC")
)

Variables

var ErrCompilationError = xerrors.New("compile error")

ErrCompilationError indicates program source code cannot compiled.

var ErrInternalError = xerrors.New("internal error")

ErrInternalError indicates process cannot executed inside sandbox.

var ErrMemoryLimitExceeded = xerrors.New("memory limit exceeded")

ErrMemoryLimitExceeded indicates process inside sandbox use too much memory.

var ErrNoSuchJob = xerrors.New("no such job")

ErrNoSuchJob indicating that currently no active job to be done.

var ErrRuntimeError = xerrors.New("runtime error")

ErrRuntimeError indicates process exited wit non zero return value.

var ErrTimeLimitExceeded = xerrors.New("time limit exceeded")

ErrTimeLimitExceeded indicates process inside sandbox use too much cpu or wall clock time.

Types

type Client

type Client interface {
	Submit(ctx context.Context, request SubmitRequest) (*SubmitResult, error)
	Submissions(ctx context.Context) (*SubmissionListResult, error)
	SignOut(ctx context.Context) error
	SignIn(ctx context.Context, request SignInRequest) (*SignInResult, error)
	Problems(ctx context.Context) (*ProblemListResult, error)
	Languages(ctx context.Context) (*LanguageListResult, error)
}

Client represent ugrade client to interact with ugrade server.

type Command

type Command struct {
	// ImagePath contains path to image (compressed tar.xz file) in host filesystem.
	ImagePath string

	// Path contains path to executable file to execute in sandboxed filesystem.
	Path string

	// Args contains arguments to passed to the process.
	Args []string

	// Bind mounts host filesystem into sandboxed filesystem.
	Binds []FSBind

	// Path to file to be used as stdin for sandboxed process.
	// If empty, stdin will derived from parent standard input.
	Stdin  string
	Stdout string
	Stderr string

	// Working directory of process inside sandbox.
	Dir string

	// TimeLimit indicates maximum allowed cpu time of process to use.
	// Proces will killed when running longer than this limit.
	TimeLimit time.Duration

	// WallTimeLimit indicates maximum allowed time of process including CPU and IO.
	WallTimeLimit time.Duration

	// MemoryLimit indicates maximum allowed memory in bytes allocation to be used by process.
	// Process will killed when allocating memory more than this limit.
	MemoryLimit uint64

	// MemoryThrottle will cause process to use no more than this value, but not killed when using more than this.
	// When memory allocation is too high, the process will be throttled.
	MemoryThrottle uint64

	// FileSize will cause process to exit when trying to generate file with size exceeding this value.
	FileSize uint64

	// OpenFile limit number of open file of process.
	OpenFile uint64

	// NProc limit number of process that process can create.
	NProc uint64

	// Limit stack size of process.
	StackSize uint64
}

Command contain information to execute.

type Consumer

type Consumer interface {
	Consume(ctx context.Context, token string) error
}

Consumer consume job from server and send result

type FSBind

type FSBind struct {
	Host    string
	Sandbox string
}

FSBind represent binding filesystem from host to sandbox.

type JobResult

type JobResult struct {
	Verdict Verdict
}

JobResult represent job result generated by worker.

type JobSolver

type JobSolver interface {
	Solve(ctx context.Context, spec JobSpec) (*JobResult, error)
}

JobSolver solve job and gives result.

type JobSpec

type JobSpec struct {
	// TCGen represent testcase generator used for generating testcase input files.
	TCGen SourceCode

	// Solution represent jury solution.
	Solution SourceCode

	// Checker represent checker program to check correctness of contestant submissions by
	// comparing to jury outputs.
	Checker SourceCode

	// Submission represent contestant program solution.
	Submission SourceCode

	// TimeLimit represent maximum time allowed for solution to be executed.
	TimeLimit time.Duration

	// OutputLimit represent maximum output generated by programs.
	OutputLimit uint64

	// MemoryLimit represent maximum allowed memory used by program.
	MemoryLimit uint64

	// Tolerance represent tolerance factor of problem.
	Tolerance float64
}

JobSpec represent job specification from ugrade server.

type LanguageListItem

type LanguageListItem struct {
	ID         string
	Name       string
	Extensions []string
}

LanguageListItem represent single language when running `lang ls` command.

type LanguageListResult

type LanguageListResult struct {
	Languages []LanguageListItem
}

LanguageListResult represent result of calling `lang ls` command.

type ProblemListItem

type ProblemListItem struct {
	ID       string
	ShortID  string
	Name     string
	Disabled bool
}

ProblemListItem represent single problem when running `problem ls` command.

type ProblemListResult

type ProblemListResult struct {
	Problems []ProblemListItem
}

ProblemListResult represent result of calling `problem ls` command.

type Sandbox

type Sandbox interface {
	Execute(ctx context.Context, command Command) (Usage, error)
}

Sandbox execute ugsbox

type SignInRequest

type SignInRequest struct {
	ContestShortID string
	Email          string
	Password       string
}

SignInRequest represent input for sign in command

type SignInResult

type SignInResult struct {
	UserID      string
	UserName    string
	ContestID   string
	ContestName string
}

SignInResult represent result of sign in command

type SourceCode

type SourceCode struct {
	// Path contains absolute path to source code file.
	Path string

	// Language contains language id of source code program.
	Language string
}

SourceCode represent program's source code.

type SubmissionListItem

type SubmissionListItem struct {
	ID           string
	ProblemName  string
	LanguageName string
	IssuerName   string
	Verdict      string
	IssuedAt     string
}

SubmissionListItem represent single submission when running `submission ls` command.

type SubmissionListResult

type SubmissionListResult struct {
	Submissions []SubmissionListItem
}

SubmissionListResult represent result of calling `submission ls` command.

type SubmitRequest

type SubmitRequest struct {
	LanguageID string
	ProblemID  string
	SourceCode string
}

SubmitRequest represent input for submit command.

type SubmitResult

type SubmitResult struct {
	ID string
}

SubmitResult represent result of submit command.

type Usage

type Usage struct {
	// Memory represent memory consumption of process.
	Memory uint64

	// CPU represent CPU time used by process.
	CPU time.Duration

	// WallTime repsent wall clock time of process in the system.
	WallTime time.Duration
}

Usage represent resource used by process inside sandbox.

type Verdict

type Verdict string

Verdict represent verdict of submission.