rbe

package
v0.0.0-...-864d242 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 3, 2024 License: Apache-2.0 Imports: 37 Imported by: 0

Documentation

Overview

Package rbe implements communication with RBE APIs.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Dial

func Dial(ctx context.Context, count int) ([]grpc.ClientConnInterface, error)

Dial dials RBE backend with proper authentication.

Returns multiple identical clients each representing a separate HTTP2 connection.

func EnqueueCancel

func EnqueueCancel(ctx context.Context, tr *model.TaskRequest, ttr *model.TaskToRun) error

EnqueueCancel enqueues a `rbe-cancel` task to cancel RBE reservation of a task.

func NewInternalsClient

func NewInternalsClient(ctx context.Context, projectID string) (internalspb.InternalsClient, error)

NewInternalsClient constructs an RPC client to call Swarming internals service.

Types

type CreateBotSessionRequest

type CreateBotSessionRequest struct {
	// Session is a serialized Swarming Bot Session proto.
	Session []byte `json:"session"`

	// PollToken is a token produced by Python server in `/bot/poll`. Required.
	//
	// This token encodes configuration of the bot maintained by the Python
	// Swarming server.
	//
	// TODO: To be removed.
	PollToken []byte `json:"poll_token"`

	// SessionToken is a session token of a previous session if recreating it.
	//
	// Optional. See the corresponding field in UpdateBotSessionRequest.
	//
	// TODO: To be removed.
	SessionToken []byte `json:"session_token,omitempty"`

	// BotVersion identifies the bot software. It is reported to RBE as is.
	BotVersion string `json:"bot_version,omitempty"`

	// WorkerProperties are passed to RBE as worker properties.
	WorkerProperties *WorkerProperties `json:"worker_properties,omitempty"`
}

CreateBotSessionRequest is a body of `/bot/rbe/session/create` request.

func (*CreateBotSessionRequest) ExtractDebugRequest

func (r *CreateBotSessionRequest) ExtractDebugRequest() any

func (*CreateBotSessionRequest) ExtractPollToken

func (r *CreateBotSessionRequest) ExtractPollToken() []byte

func (*CreateBotSessionRequest) ExtractSession

func (r *CreateBotSessionRequest) ExtractSession() []byte

func (*CreateBotSessionRequest) ExtractSessionToken

func (r *CreateBotSessionRequest) ExtractSessionToken() []byte

type CreateBotSessionResponse

type CreateBotSessionResponse struct {
	// Session is a serialized Swarming Bot Session proto.
	//
	// It is derived from the session in the request, except it has RBE session
	// info populated now.
	Session []byte `json:"session"`

	// SessionToken is a freshly produced session token.
	//
	// It encodes the RBE bot session ID and bot configuration provided via the
	// poll token.
	//
	// The session token is needed to call `/bot/rbe/session/update`. This call
	// also will periodically refresh it.
	//
	// TODO: To be removed.
	SessionToken []byte `json:"session_token"`

	// SessionID is an RBE bot session ID as encoded in the token.
	//
	// Primarily for the bot debug log.
	SessionID string `json:"session_id"`
}

CreateBotSessionResponse is a body of `/bot/rbe/session/create` response.

type Lease

type Lease struct {
	// ID is the unique reservation ID treated as an opaque string. Required.
	ID string `json:"id"`

	// State is a lease state as stringy remoteworkers.LeaseState enum. Required.
	//
	// Possible values:
	//   * PENDING
	//   * ACTIVE
	//   * COMPLETED
	//   * CANCELLED
	State string `json:"state"`

	// Payload is the reservation payload.
	//
	// Note it is serialized using regular JSON rules, i.e. fields are in
	// "snake_case".
	Payload *internalspb.TaskPayload `json:"payload,omitempty"`

	// Result is the execution result.
	//
	// Note it is serialized using regular JSON rules, i.e. fields are in
	// "snake_case".
	Result *internalspb.TaskResult `json:"result,omitempty"`
}

Lease is a JSON representation of a relevant subset of remoteworkers.Lease.

type ReservationServer

type ReservationServer struct {
	// contains filtered or unexported fields
}

ReservationServer is responsible for creating and canceling RBE reservations.

func NewReservationServer

func NewReservationServer(ctx context.Context, cc grpc.ClientConnInterface, internals internalspb.InternalsClient, serverVersion string) *ReservationServer

NewReservationServer creates a new reservation server given an RBE client connection.

func (*ReservationServer) ExpireSliceBasedOnReservation

func (s *ReservationServer) ExpireSliceBasedOnReservation(ctx context.Context, reservationName string) error

ExpireSliceBasedOnReservation checks the reservation status by calling the Reservations API and invokes ExpireSlice if the reservation is dead.

It is ultimately invoked from a PubSub push handler when handling notifications from the RBE scheduler.

`reservationName` is a full reservation name, including the project and RBE instance IDs: `projects/.../instances/.../reservations/...`.

TODO: Make private once old PubSub handler is gone.

func (*ReservationServer) RegisterPSHandlers

func (s *ReservationServer) RegisterPSHandlers(disp *pubsub.Dispatcher)

RegisterPSHandlers registers PubSub handlers.

func (*ReservationServer) RegisterTQTasks

func (s *ReservationServer) RegisterTQTasks(disp *tq.Dispatcher)

RegisterTQTasks registers task queue handlers.

type SessionServer

type SessionServer struct {
	// contains filtered or unexported fields
}

SessionServer serves handlers for creating and updating RBE bot sessions.

func NewSessionServer

func NewSessionServer(ctx context.Context, cc []grpc.ClientConnInterface, hmacSecret *hmactoken.Secret, backendVer string) *SessionServer

NewSessionServer creates a new session server given an RBE client connection.

func (*SessionServer) CreateBotSession

func (srv *SessionServer) CreateBotSession(ctx context.Context, body *CreateBotSessionRequest, r *botsrv.Request) (botsrv.Response, error)

CreateBotSession is an RPC handler that creates a new bot session.

func (*SessionServer) UpdateBotSession

func (srv *SessionServer) UpdateBotSession(ctx context.Context, body *UpdateBotSessionRequest, r *botsrv.Request) (botsrv.Response, error)

UpdateBotSession is an RPC handler that updates a bot session.

type UpdateBotSessionRequest

type UpdateBotSessionRequest struct {
	// Session is a serialized Swarming Bot Session proto.
	//
	// It should have an RBE session info inside, as populated by
	// `/bot/rbe/session/create`.
	Session []byte `json:"session"`

	// SessionToken is a token returned by the previous API call. Required.
	//
	// This token is initially returned by `/bot/rbe/session/create` and then
	// refreshed with every `/bot/rbe/session/update` call.
	//
	// TODO: To be removed.
	SessionToken []byte `json:"session_token"`

	// PollToken is a token produced by Python server in `/bot/poll`.
	//
	// It is optional and present only in the outer bot poll loop, when the bot
	// polls both Python Swarming server (to get new configs) and Swarming RBE
	// server (to get new tasks).
	//
	// Internals of this token will be copied into the session token returned in
	// the response to this call.
	//
	// TODO: To be removed.
	PollToken []byte `json:"poll_token,omitempty"`

	// BotVersion identifies the bot software. It is reported to RBE as is.
	BotVersion string `json:"bot_version,omitempty"`

	// WorkerProperties are passed to RBE as worker properties.
	WorkerProperties *WorkerProperties `json:"worker_properties,omitempty"`

	// The intended bot session status as stringy remoteworkers.BotStatus enum.
	//
	// Possible values:
	//   * OK
	//   * UNHEALTHY
	//   * HOST_REBOOTING
	//   * BOT_TERMINATING
	//   * INITIALIZING
	//   * MAINTENANCE
	Status string `json:"status"`

	// Nonblocking is true if the bot doesn't want to block waiting for new
	// leases to appear.
	Nonblocking bool `json:"nonblocking"`

	// The lease the bot is currently working or have just finished working on.
	//
	// Allowed lease states here are:
	//   * ACTIVE: the bot is still working on the lease.
	//   * COMPLETED: the bot has finished working on the lease. Result field
	//     should be populated. This state is also used to report the bot is done
	//     working on a canceled lease.
	//
	// Payload field is always ignored.
	Lease *Lease `json:"lease,omitempty"`
}

UpdateBotSessionRequest is a body of `/bot/rbe/session/update` request.

If PollToken is present, it will be used to refresh the state stored in the session token.

func (*UpdateBotSessionRequest) ExtractDebugRequest

func (r *UpdateBotSessionRequest) ExtractDebugRequest() any

func (*UpdateBotSessionRequest) ExtractPollToken

func (r *UpdateBotSessionRequest) ExtractPollToken() []byte

func (*UpdateBotSessionRequest) ExtractSession

func (r *UpdateBotSessionRequest) ExtractSession() []byte

func (*UpdateBotSessionRequest) ExtractSessionToken

func (r *UpdateBotSessionRequest) ExtractSessionToken() []byte

type UpdateBotSessionResponse

type UpdateBotSessionResponse struct {
	// Session is a serialized Swarming Bot Session proto.
	//
	// It is derived from the session in the request, except it has its expiration
	// time bumped.
	Session []byte `json:"session"`

	// SessionToken is a refreshed session token, if available.
	//
	// It carries the same RBE bot session ID inside as the incoming token. The
	// bot must use it in the next `/bot/rbe/session/update` request.
	//
	// If the incoming token has expired already, this field will be empty, since
	// it is not possible to refresh an expired token.
	//
	// TODO: To be removed.
	SessionToken []byte `json:"session_token,omitempty"`

	// The session status as seen by the server, as remoteworkers.BotStatus enum.
	//
	// Possible values:
	//   * OK: if the session is healthy.
	//   * BOT_TERMINATING: if the session has expired.
	Status string `json:"status"`

	// The lease the bot should be working on or should cancel now, if any.
	//
	// Possible lease states here:
	//   * PENDING: the bot should start working on this new lease. It has Payload
	//     field populated. Can only happen in reply to a bot reporting no lease
	//     or a completed lease.
	//   * ACTIVE: the bot should keep working on the lease it just reported. Can
	//     only happen in reply to a bot reporting an active lease. Payload is not
	//     populate (the bot should know it already).
	//   * CANCELLED: the bot should stop working on the lease it just reported.
	//     Once the bot is done working on the lease, it should update the session
	//     again, marking the lease as COMPLETED. Payload is not populated.
	//
	// If the bot was stuck for a while and the RBE canceled the lease as lost,
	// this field will be unpopulated, even if the bot reported an active lease.
	// The bot should give up on the current lease ASAP, without even reporting
	// its result back (because the server gave up on it already anyway).
	Lease *Lease `json:"lease,omitempty"`
}

UpdateBotSessionResponse is a body of `/bot/rbe/session/update` response.

type WorkerProperties

type WorkerProperties struct {
	// PoolID will be used as `rbePoolID` bot session property.
	PoolID string `json:"pool_id"`
	// PoolVersion will be used as `rbePoolVersion` bot session property.
	PoolVersion string `json:"pool_version"`
}

WorkerProperties are RBE worker properties unrelated to actual scheduling.

They aren't validated by Swarming and just passed along to RBE. The RBE bots obtain them via some external mechanism (e.g. the GCE metadata server).

They are optional and currently used only on bots managed by RBE Worker Provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL