Documentation
¶
Overview ¶
Library containing utilities exclusively for the master process
Index ¶
Constants ¶
const ( CodeSuccess int = 0 // Success code CodeTraceLimit int = 77 // The traces are overflown CodeOom int = 137 // When the process exits on OOM CodeFatal int = 14 // When the process could not start CodeCantRunCommand int = 15 // When the controller could not run the command )
List of the possible errors returned by the prover as an exit code. There are other possibilities but the one listed here are the one of interests. Some of these codes are generated by the executor and not the child process itself. These includes, CodeFatal, CodeTooManyRetries and CodeCantRunCommand.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Executor ¶
type Executor struct {
Config *config.Config
// Logger specific to the executor
Logger *logrus.Entry
}
The executor is responsible for running the commands specified by the jobs
func NewExecutor ¶
type FsWatcher ¶
type FsWatcher struct {
// Unique ID of the container. Used to identify the owner of a locked file
LocalID string
// List of jobs that we are currently matching
JobToWatch []JobDefinition
// Suffix to append to the end of a file
InProgress string
// Logger specific to the file watcher
Logger *logrus.Entry
}
FsWatcher is a struct who will watch the filesystem and return files as if it were a message queue.
func NewFsWatcher ¶
type Job ¶
type Job struct {
// Configuration parameters relative to the job
Def *JobDefinition
// Original name of the file when it was found
OriginalFile string
// Name of the locked file. If this value is set, it means that the job
// was successfully locked.
LockedFile string
// Height of the file in the priority queue
Start int
End int
VersionExecutionTracer string
VersionStateManager string
VersionCompressor string
// The hex string of the content hash
ContentHash string
}
Defines a job to be processed. Jobs are parsed from filenames using regexp.
func NewJob ¶
func NewJob(jdef *JobDefinition, filename string) (j *Job, err error)
Parse a filename into a Job. Returns an error if the file does not corresponds to the specified template of the job type.
func (*Job) DeferToLargeFile ¶
Returns the name of the input file modified so that it is retried in large mode. It fails if the job's definition does not provide a suffix to retry in large mode. This is still unexpected because the configuration validation ensures that if there is an exit code amenable to defer the job to a larger machine, then the suffix must be set. If the status code of the prover is zero it will return an error.
func (*Job) InProgressPath ¶
Returns the full path to the inprogress file
func (*Job) ResponseFile ¶
Returns the name of the output file for the job
func (*Job) Score ¶
Returns the score of a JOB. The score is obtained as 100*job.Stop + P, where P is 1 if the job is an execution job, 2 if the job is a compression job and 3 if the job is an aggregation job. The lower the score the higher will be the priority of the job. The 100 value is chosen to make the score easy to mentally compute.
type JobDefinition ¶
type JobDefinition struct {
// Parameters for the job definition provided by the user
RequestsRootDir string
// Name of the job
Name string
// The regexp to use to match input files. For instance,
//
// `^\d+-\d+-etv0.1.2-stv\d.\d.\d-getZkProof.json$`
//
// Will tell the controller to accept any version of the state-manager
// but to only accept execution trace. The regexp should always start "^"
// and end with "$" otherwise you are going to match in-progress files.
//
InputFileRegexp *regexp2.Regexp
// Template to use to generate the output file. The template should have the
// form of a go template. For instance,
//
// `{{.From}}-{{.To}}-pv{{.Version}}-stv{{.Stv}}-etv{{.Etv}}-zkProof.json`
//
OutputFileTmpl *template.Template
// Priority at which this type of job should be processed. The lower the
// more of a priority.
//
// Typically 0 for execution, 1 for compression and 2 for aggregation.
//
Priority int
// The associated compiled regexp, this saves on recompiling the regexps
// everytime we want to use them. If a field is not needed, it can be left
// at zero.
ParamsRegexp struct {
Start *regexp2.Regexp
End *regexp2.Regexp
Stv *regexp2.Regexp
Etv *regexp2.Regexp
Cv *regexp2.Regexp
ContentHash *regexp2.Regexp
}
// Regexp of the failure code so that we can trim it if we want to retry.
FailureSuffix *regexp2.Regexp
}
JobDefinition represents a collection of static parameters allowing to define a job.
func AggregatedDefinition ¶
func AggregatedDefinition(conf *config.Config) JobDefinition
Definition of an execution prover job.
func CompressionDefinition ¶
func CompressionDefinition(conf *config.Config) JobDefinition
Definition of an execution prover job.
func ExecutionDefinition ¶
func ExecutionDefinition(conf *config.Config) JobDefinition
Definition of an execution prover job. The function panics on any error since it is called at start up.
type OutputFileRessouce ¶
type OutputFileRessouce struct {
Job
}
OutputFileRessouce collects all the data needed to fill the output template file.
Source Files
Directories