Documentation ¶
Overview ¶
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner.
Index ¶
- func BuildTempWorkerBinary(ctx context.Context, opts CompileOpts) (string, error)
- func Execute(ctx context.Context, p *pipepb.Pipeline, endpoint string, opt *JobOptions, ...) (*universalPipelineResult, error)
- func IsWorkerCompatibleBinary() (string, bool)
- func Prepare(ctx context.Context, client jobpb.JobServiceClient, p *pipepb.Pipeline, ...) (id, endpoint, stagingToken string, err error)
- func Stage(ctx context.Context, id, endpoint, binary, st string) (retrievalToken string, err error)
- func StageViaLegacyAPI(ctx context.Context, cc *grpc.ClientConn, binary, st string) (retrievalToken string, err error)
- func StageViaPortableAPI(ctx context.Context, cc *grpc.ClientConn, binary, st string) (retErr error)
- func Submit(ctx context.Context, client jobpb.JobServiceClient, id, token string) (string, error)
- func UpdateGoEnvironmentWorker(worker string, p *pipepb.Pipeline) error
- func WaitForCompletion(ctx context.Context, client jobpb.JobServiceClient, jobID string) error
- type CompileOpts
- type JobOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildTempWorkerBinary ¶
func BuildTempWorkerBinary(ctx context.Context, opts CompileOpts) (string, error)
BuildTempWorkerBinary creates a local worker binary in the tmp directory for linux/amd64. Caller responsible for deleting the binary.
func Execute ¶
func Execute(ctx context.Context, p *pipepb.Pipeline, endpoint string, opt *JobOptions, async bool) (*universalPipelineResult, error)
Execute executes a pipeline on the universal runner serving the given endpoint. Convenience function.
func IsWorkerCompatibleBinary ¶
IsWorkerCompatibleBinary returns the path to itself and true if running a linux-amd64 binary that can directly be used as a worker binary.
func Prepare ¶
func Prepare(ctx context.Context, client jobpb.JobServiceClient, p *pipepb.Pipeline, opt *JobOptions) (id, endpoint, stagingToken string, err error)
Prepare prepares a job to the given job service. It returns the preparation id artifact staging endpoint, and staging token if successful.
func Stage ¶
Stage stages the worker binary and any additional files to the given artifact staging endpoint. It returns the retrieval token if successful.
func StageViaLegacyAPI ¶
func StageViaLegacyAPI(ctx context.Context, cc *grpc.ClientConn, binary, st string) (retrievalToken string, err error)
StageViaLegacyAPI is a beam internal function for uploading artifacts to the staging service via the legacy API.
It will be unexported at a later time.
func StageViaPortableAPI ¶
func StageViaPortableAPI(ctx context.Context, cc *grpc.ClientConn, binary, st string) (retErr error)
StageViaPortableAPI is a beam internal function for uploading artifacts to the staging service via the portable API.
It will be unexported at a later time.
func UpdateGoEnvironmentWorker ¶
UpdateGoEnvironmentWorker sets the worker artifact payload in the default environment.
func WaitForCompletion ¶
WaitForCompletion monitors the given job until completion. It logs any messages and state changes received.
Types ¶
type CompileOpts ¶
type CompileOpts struct {
OS, Arch string
}
CompileOpts are additional options for dynamic compiles of the local code for development purposes. Production runs should build the worker binary separately for the target environment. See https://beam.apache.org/documentation/sdks/go-cross-compilation/ for details.
type JobOptions ¶
type JobOptions struct { // Name is the job name. Name string // Experiments are additional experiments. Experiments []string // Worker is the worker binary override. Worker string // RetainDocker is an option to pass to the runner indicating the docker containers should be cached. RetainDocker bool // Indicates a limit on parallelism the runner should impose. Parallelism int // Loopback indicates this job is running in loopback mode and will reconnect to the local process. Loopback bool }
JobOptions capture the various options for submitting jobs to universal runners.