cron

package
v0.9.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 20, 2020 License: GPL-3.0 Imports: 18 Imported by: 0

Documentation

Overview

Package cron schedules dataset and shell script updates

Index

Constants

This section is empty.

Variables

View Source 
var (

	// DefaultCheckInterval is the frequency cron will check all stored jobs
	// for scheduled updates without any additional configuration. Qri recommends
	// not running updates more than once an hour for performance and storage
	// consumption reasons, making a check every minute a reasonable default
	DefaultCheckInterval = time.Minute
)
View Source 
var ErrUnreachable = fmt.Errorf("cannot establish a connection to the server")

ErrUnreachable defines errors where the server cannot be reached TODO (b5): consider moving this to qfs

Functions

This section is empty.

Types

type Cron 

type Cron struct {
	// contains filtered or unexported fields
}

Cron coordinates the scheduling of running jobs at specified periodicities (intervals) with a provided job runner function

func NewCron 

func NewCron(schedule, log JobStore, factory RunJobFactory) *Cron

NewCron creates a Cron with the default check interval

func NewCronInterval 

func NewCronInterval(schedule, log JobStore, factory RunJobFactory, checkInterval time.Duration) *Cron

NewCronInterval creates a Cron with a custom check interval

func (*Cron) Job 

func (c *Cron) Job(ctx context.Context, name string) (*Job, error)

Job proxies to the schedule store for reading a job by name

func (*Cron) ListJobs 

func (c *Cron) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs proxies to the schedule store for reading jobs

func (*Cron) ListLogs 

func (c *Cron) ListLogs(ctx context.Context, offset, limit int) ([]*Job, error)

ListLogs returns a list of jobs that have been executed

func (*Cron) Log 

func (c *Cron) Log(ctx context.Context, logName string) (*Job, error)

Log gives a specific Job by logged job name

func (*Cron) LogFile 

func (c *Cron) LogFile(ctx context.Context, logName string) (io.ReadCloser, error)

LogFile returns a reader for a file at the given name

func (*Cron) Schedule 

func (c *Cron) Schedule(ctx context.Context, job *Job) error

Schedule adds a job to the cron scheduler

func (*Cron) ServeHTTP 

func (c *Cron) ServeHTTP(addr string) error

ServeHTTP spins up an HTTP server at the specified address

func (*Cron) Start 

func (c *Cron) Start(ctx context.Context) error

Start initiates the check loop, looking for updates to execute once at every iteration of the configured check interval. Start blocks until the passed context completes

func (*Cron) Unschedule 

func (c *Cron) Unschedule(ctx context.Context, name string) error

Unschedule removes a job from the cron scheduler, cancelling any future job executions

type DatasetOptions 

type DatasetOptions struct {
	Title     string
	Message   string
	Recall    string
	BodyPath  string
	FilePaths []string

	Publish             bool
	Strict              bool
	Force               bool
	ConvertFormatToPrev bool
	ShouldRender        bool

	Config  map[string]string
	Secrets map[string]string
}

DatasetOptions encapsulates options passed to `qri save` TODO (b5) - we should contribute flexbuffer support for golang & remove this entirely

func (*DatasetOptions) MarshalFlatbuffer 

func (o *DatasetOptions) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes to a builder

func (*DatasetOptions) UnmarshalFlatbuffer 

func (o *DatasetOptions) UnmarshalFlatbuffer(fbo *cronfb.DatasetOptions)

UnmarshalFlatbuffer reads flatbuffer data into DatasetOptions

type FlatbufferJobStore 

type FlatbufferJobStore struct {
	// contains filtered or unexported fields
}

FlatbufferJobStore is a jobstore implementation that saves to a file of flatbuffer bytes. FlatbufferJobStore is safe for concurrent use

func (*FlatbufferJobStore) CreateLogFile 

func (s *FlatbufferJobStore) CreateLogFile(j *Job) (f io.WriteCloser, path string, err error)

CreateLogFile creates a log file in the specified logs directory

func (*FlatbufferJobStore) DeleteJob 

func (s *FlatbufferJobStore) DeleteJob(ctx context.Context, name string) error

DeleteJob removes a job from the store by name. deleting a non-existent job won't return an error

func (*FlatbufferJobStore) Destroy 

func (s *FlatbufferJobStore) Destroy() error

Destroy removes the path entirely

func (*FlatbufferJobStore) Job 

func (s *FlatbufferJobStore) Job(ctx context.Context, name string) (*Job, error)

Job gets job details from the store by name

func (*FlatbufferJobStore) ListJobs 

func (s *FlatbufferJobStore) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs lists jobs currently in the store

func (*FlatbufferJobStore) PutJob 

func (s *FlatbufferJobStore) PutJob(ctx context.Context, job *Job) error

PutJob places a job in the store. If the job name matches the name of a job that already exists, it will be overwritten with the new job

func (*FlatbufferJobStore) PutJobs 

func (s *FlatbufferJobStore) PutJobs(ctx context.Context, add ...*Job) error

PutJobs places one or more jobs in the store. Putting a job who's name already exists must overwrite the previous job, making all job names unique

type HTTPClient 

type HTTPClient struct {
	Addr string
}

HTTPClient implements the Scheduler interface over HTTP, talking to a Cron HTTPServer

func (HTTPClient) Job 

func (c HTTPClient) Job(ctx context.Context, name string) (*Job, error)

Job gets a job by querying an HTTP server

func (HTTPClient) ListJobs 

func (c HTTPClient) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs jobs by querying an HTTP server

func (HTTPClient) ListLogs 

func (c HTTPClient) ListLogs(ctx context.Context, offset, limit int) ([]*Job, error)

ListLogs gives a log of executed jobs

func (HTTPClient) Log 

func (c HTTPClient) Log(ctx context.Context, logName string) (*Job, error)

Log returns a single executed job by job.LogName

func (HTTPClient) LogFile 

func (c HTTPClient) LogFile(ctx context.Context, logName string) (io.ReadCloser, error)

LogFile returns a reader for a file at the given name

func (HTTPClient) Ping 

func (c HTTPClient) Ping() error

Ping confirms client can dial the server, if a connection cannot be established at all, Ping will return ErrUnreachable

func (HTTPClient) Schedule 

func (c HTTPClient) Schedule(ctx context.Context, job *Job) error

Schedule adds a job to the cron scheduler via an HTTP request

func (HTTPClient) Unschedule 

func (c HTTPClient) Unschedule(ctx context.Context, name string) error

Unschedule removes a job from scheduling

type Job 

type Job struct {
	Name         string                    `json:"name"`
	Alias        string                    `json:"alias"`
	Type         JobType                   `json:"type"`
	Periodicity  iso8601.RepeatingInterval `json:"periodicity"`
	PrevRunStart time.Time                 `json:"lastRunStart,omitempty"`

	RunNumber   int64     `json:"runNumber,omitempty"`
	RunStart    time.Time `json:"runStart,omitempty"`
	RunStop     time.Time `json:"runStop,omitempty"`
	RunError    string    `json:"runError,omitempty"`
	LogFilePath string    `json:"logFilePath,omitempty"`

	RepoPath string `json:"repoPath,omitempty"`

	Options Options `json:"options,omitempty"`
}

Job represents a "cron job" that can be scheduled for repeated execution at a specified Periodicity (time interval)

a Job struct has one of three "run" states, which describe it's position in the execution lifecycle: * unexected: job.RunStart.IsZero() && job.RunStop.IsZero() * executing: !job.RunStart.IsZero() && job.RunStop.IsZero() * completed: !job.RunStart.IsZero() && !job.RunStop.IsZero()

func (*Job) Copy 

func (job *Job) Copy() *Job

Copy creates a copy of a job

func (*Job) FlatbufferBytes 

func (job *Job) FlatbufferBytes() []byte

FlatbufferBytes formats a job as a flatbuffer byte slice

func (*Job) LogName 

func (job *Job) LogName() string

LogName returns a canonical name string for a job that's executed and saved to a logging system

func (*Job) MarshalFlatbuffer 

func (job *Job) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes a job to a builder

func (*Job) NextExec 

func (job *Job) NextExec() time.Time

NextExec returns the next time execution horizon. If job periodicity is improperly configured, the returned time will be zero

func (*Job) UnmarshalFlatbuffer 

func (job *Job) UnmarshalFlatbuffer(j *cronfb.Job) error

UnmarshalFlatbuffer decodes a job from a flatbuffer

func (*Job) Validate 

func (job *Job) Validate() error

Validate confirms a Job contains valid details for scheduling

type JobStore 

type JobStore interface {
	// JobStores must implement the ReadJobs interface for fetching stored jobs
	ReadJobs
	// PutJob places one or more jobs in the store. Putting a job who's name
	// already exists must overwrite the previous job, making all job names unique
	PutJobs(context.Context, ...*Job) error
	// PutJob places a job in the store. Putting a job who's name already exists
	// must overwrite the previous job, making all job names unique
	PutJob(context.Context, *Job) error
	// DeleteJob removes a job from the store
	DeleteJob(ctx context.Context, name string) error
}

JobStore handles the persistence of Job details. JobStore implementations must be safe for concurrent use

func NewFlatbufferJobStore 

func NewFlatbufferJobStore(path string) JobStore

NewFlatbufferJobStore creates a job store that persists to a file

type JobType 

type JobType string

JobType is a type for distinguishing between two different kinds of jobs JobType should be used as a shorthand for defining how to execute a job 

const (
	// JTDataset indicates a job that runs "qri update" on a dataset specified
	// by Job Name. The job periodicity is determined by the specified dataset's
	// Meta.AccrualPeriodicity field. LastRun should closely match the datasets's
	// latest Commit.Timestamp value
	JTDataset JobType = "dataset"
	// JTShellScript represents a shell script to be run locally, which might
	// update one or more datasets. A non-zero exit code from shell script
	// indicates the job failed to execute properly
	JTShellScript JobType = "shell"
)

func (JobType) Enum 

func (jt JobType) Enum() int8

Enum returns the enumerated representation of a JobType

type LogFileCreator 

type LogFileCreator interface {
	// CreateLogFile returns a file to write output to
	CreateLogFile(job *Job) (f io.WriteCloser, path string, err error)
}

LogFileCreator is an interface for generating log files to write to, JobStores should implement this interface

type MemJobStore 

type MemJobStore struct {
	// contains filtered or unexported fields
}

MemJobStore is an in-memory implementation of the JobStore interface Jobs stored in MemJobStore can be persisted for the duration of a process at the longest. MemJobStore is safe for concurrent use

func (*MemJobStore) DeleteJob 

func (s *MemJobStore) DeleteJob(ctx context.Context, name string) error

DeleteJob removes a job from the store by name. deleting a non-existent job won't return an error

func (*MemJobStore) Job 

func (s *MemJobStore) Job(ctx context.Context, name string) (*Job, error)

Job gets job details from the store by name

func (*MemJobStore) ListJobs 

func (s *MemJobStore) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs lists jobs currently in the store

func (*MemJobStore) PutJob 

func (s *MemJobStore) PutJob(ctx context.Context, job *Job) error

PutJob places a job in the store. If the job name matches the name of a job that already exists, it will be overwritten with the new job

func (*MemJobStore) PutJobs 

func (s *MemJobStore) PutJobs(ctx context.Context, js ...*Job) error

PutJobs places one or more jobs in the store. Putting a job whose name already exists must overwrite the previous job, making all job names unique

type Options 

type Options interface {
	MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT
}

Options is an interface for job options

type ReadJobs 

type ReadJobs interface {
	// ListJobs should return the set of jobs sorted in reverse-chronological order
	// (newest first order) of the last time they were run. When two LastRun times
	// are equal, Jobs should alpha sort the names
	// passing a limit and offset of 0 must return the entire list of stored jobs
	ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)
	// Job gets a job by it's name. All job names in a set must be unique. It's
	// the job of the set backing ReadJobs functions to enforce uniqueness
	Job(ctx context.Context, name string) (*Job, error)
}

ReadJobs are functions for fetching a set of jobs. ReadJobs defines canoncial behavior for listing & fetching jobs

type RunJobFactory 

type RunJobFactory func(ctx context.Context) (runner RunJobFunc)

RunJobFactory is a function that returns a runner

type RunJobFunc 

type RunJobFunc func(ctx context.Context, streams ioes.IOStreams, job *Job) error

RunJobFunc is a function for executing a job. Cron takes care of scheduling job execution, and delegates the work of executing a job to a RunJobFunc implementation.

type Scheduler 

type Scheduler interface {
	// ListJobs lists currently scheduled jobs
	ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)
	// Job gets a single scheduled job by name
	Job(ctx context.Context, name string) (*Job, error)

	// Schedule adds a job to the scheduler for execution once every period
	Schedule(ctx context.Context, job *Job) error
	// Unschedule removes a job from the scheduler
	Unschedule(ctx context.Context, name string) error

	// ListLogs gives a log of executed jobs
	ListLogs(ctx context.Context, offset, limit int) ([]*Job, error)
	// Log returns a single executed job by job.LogName
	Log(ctx context.Context, logName string) (*Job, error)
	// JobLogFile returns a reader for a file at the given name
	LogFile(ctx context.Context, logName string) (io.ReadCloser, error)
}

Scheduler is the generic interface for the Cron Scheduler, it's implemented by both Cron and HTTPClient for easier RPC communication

type ShellScriptOptions 

type ShellScriptOptions struct {
}

ShellScriptOptions encapsulates options for running a shell script cron job

func (*ShellScriptOptions) MarshalFlatbuffer 

func (o *ShellScriptOptions) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes to a builder

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.