pool

Documentation

Overview

Package pool facilitates a external worker service, as an alternate mode for the standard Beam container.

This is predeominantly to serve as a process spawner within a given container VM for an arbitrary number of jobs, instead of for a single worker instance.

Workers will be spawned as executed OS processes.

Index

• type Process
  • func New(ctx context.Context, port int, containerExecutable string) (*Process, error)
  • func (s *Process) ServeAndWait() error
  • func (s *Process) StartWorker(_ context.Context, req *fnpb.StartWorkerRequest) (*fnpb.StartWorkerResponse, error)
  • func (s *Process) Stop(ctx context.Context) error
  • func (s *Process) StopWorker(_ context.Context, req *fnpb.StopWorkerRequest) (*fnpb.StopWorkerResponse, error)

Types

type Process

type Process struct {
	fnpb.UnimplementedBeamFnExternalWorkerPoolServer
	// contains filtered or unexported fields
}

Process implements fnpb.BeamFnExternalWorkerPoolServer, by starting external processes.

func New

func New(ctx context.Context, port int, containerExecutable string) (*Process, error)

New initializes a process based ExternalWorkerService, at the given port.

func (*Process) ServeAndWait

func (s *Process) ServeAndWait() error

ServeAndWait starts the ExternalWorkerService and blocks until exit.

func (*Process) StartWorker

func (s *Process) StartWorker(_ context.Context, req *fnpb.StartWorkerRequest) (*fnpb.StartWorkerResponse, error)

StartWorker initializes a new worker harness, implementing BeamFnExternalWorkerPoolServer.StartWorker.

func (*Process) Stop

func (s *Process) Stop(ctx context.Context) error

Stop terminates the service and stops all workers.

func (*Process) StopWorker

func (s *Process) StopWorker(_ context.Context, req *fnpb.StopWorkerRequest) (*fnpb.StopWorkerResponse, error)

StopWorker terminates a worker harness, implementing BeamFnExternalWorkerPoolServer.StopWorker.