botapi

package
v0.0.0-...-ae4aa15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 28, 2025 License: Apache-2.0 Imports: 35 Imported by: 0

Documentation

Overview

Package botapi implements core Bot API handlers.

Handlers related to RBE are in the "rbe" package for now.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BotAPIServer 

type BotAPIServer struct {
	// contains filtered or unexported fields
}

BotAPIServer implements core Bot API handlers.

Handlers are implement in individual Go files. They are all installed into the server router in main.go.

func NewBotAPIServer 

func NewBotAPIServer(cfg *cfg.Provider, secret *hmactoken.Secret, project, version string) *BotAPIServer

NewBotAPIServer constructs a new BotAPIServer.

func (*BotAPIServer) BotCode 

func (srv *BotAPIServer) BotCode(c *router.Context)

BotCode serves the bot archive blob or an HTTP redirect to it.

Used to bootstrap bots and by the self-updating bots.

Uses optional "Version" route parameter. Its value is either a concrete bot archive digest to fetch, or an empty string (in which case the handler will serve a redirect to the current stable bot version).

Attempts are made to utilize GAE's edge cache by setting the corresponding headers.

func (*BotAPIServer) Claim 

func (srv *BotAPIServer) Claim(ctx context.Context, body *ClaimRequest, r *botsrv.Request) (botsrv.Response, error)

Claim implements the handler that claims pending tasks.

This transactionally and idempotently assigns the task to the calling bot or instructs the bot to skip the task if it is no longer pending. On success it returns detailed description of the task to let the bot know what it should be executing.

Called by bots after they get a lease from the RBE.

func (*BotAPIServer) Event 

func (srv *BotAPIServer) Event(ctx context.Context, body *EventRequest, r *botsrv.Request) (botsrv.Response, error)

Event implements the handler that logs events sent by the bot.

func (*BotAPIServer) Handshake 

func (srv *BotAPIServer) Handshake(ctx context.Context, body *HandshakeRequest, _ *botsrv.Request) (botsrv.Response, error)

Handshake implements the bot handshake handler.

It is the first ever request sent by the bot. It registers the bot in the datastore and establishes a new bot session.

func (*BotAPIServer) IDToken 

func (srv *BotAPIServer) IDToken(ctx context.Context, body *TokenRequest, r *botsrv.Request) (botsrv.Response, error)

IDToken mints ID tokens to be used inside the task.

See OAuthToken for details.

func (*BotAPIServer) OAuthToken 

func (srv *BotAPIServer) OAuthToken(ctx context.Context, body *TokenRequest, r *botsrv.Request) (botsrv.Response, error)

OAuthToken mints OAuth tokens to be used inside the task.

There are two flavors of service accounts the bot may use:

  • "system": this account is associated directly with the bot (in bots.cfg), and can be used at any time (when running a task or not).
  • "task": this account is associated with the task currently executing on the bot, and may be used only when bot is actually running this task.

The returned token is expected to be alive for at least ~5 min, but can live longer (but no longer than ~1h). In general the client should assume the token is short-lived.

Multiple bots may share the exact same access token if their configuration match (the token is cached by Swarming for performance reasons).

Besides the token, the response also contains the actual service account email (if it is really configured), or two special strings in place of the email:

  • "none" if the bot is not configured to use service accounts at all.
  • "bot" if the bot should use tokens produced by bot_config.py hook.

Returns following errors:

  • INVALID_ARGUMENT on a bad request or if the service account is somehow misconfigured.
  • PERMISSION_DENIED if the caller is not allowed to use the service account.
  • INTERNAL on retriable transient errors.

func (*BotAPIServer) Poll 

func (srv *BotAPIServer) Poll(ctx context.Context, body *UnimplementedRequest, r *botsrv.Request) (botsrv.Response, error)

Poll implements the handler that tell the bot what to do next.

TODO: Doc, implement.

func (*BotAPIServer) TaskError 

func (srv *BotAPIServer) TaskError(ctx context.Context, body *UnimplementedRequest, r *botsrv.Request) (botsrv.Response, error)

TaskError implements the handler that collects internal task errors.

Uses optional "TaskID" route parameter with the task being worked on by the bot.

TODO: Doc, implement.

func (*BotAPIServer) TaskUpdate 

func (srv *BotAPIServer) TaskUpdate(ctx context.Context, body *UnimplementedRequest, r *botsrv.Request) (botsrv.Response, error)

TaskUpdate implements handler that collects task state updates.

Uses optional "TaskID" route parameter with the task being worked on by the bot.

TODO: Doc, implement.

type BotGroupCfg 

type BotGroupCfg struct {
	// Dimensions is the server-assigned dimensions from bots.cfg.
	Dimensions map[string][]string `json:"dimensions"`
}

BotGroupCfg is derived from the server's bots.cfg and sent to the bot.

type BotRBEParams 

type BotRBEParams struct {
	// Instance if the full RBE instance name the bot should be using.
	Instance string `json:"instance"`
	// HybridMode, if true, indicates to use RBE and native scheduler together.
	HybridMode bool `json:"hybrid_mode"`
	// Sleep is how long to sleep (in seconds) before next Swarming check in.
	Sleep float64 `json:"sleep"`
	// PollToken is a legacy unused field preserved for compatibility.
	PollToken string `json:"poll_token"`
}

BotRBEParams is sent to the bot by the handshake and poll handlers.

Describes how the bot should be interacting with RBE API.

type ClaimCommand 

type ClaimCommand string

ClaimCommand instructs the bot what to do after it calls /bot/claim. 

const (
	// ClaimSkip means the bot should skip this task and poll for another one.
	ClaimSkip ClaimCommand = "skip"
	// ClaimRun means the bot should start running the task.
	ClaimRun ClaimCommand = "run"
	// ClaimTerminate means the bot process should gracefully terminate.
	ClaimTerminate ClaimCommand = "terminate"
)

type ClaimRequest 

type ClaimRequest struct {
	// Session is a serialized Swarming Bot Session proto.
	Session []byte `json:"session"`

	// State is (mostly) arbitrary JSON dict with various properties of the bot.
	//
	// This field is used by the bot to opportunistically report its state changes
	// when running tasks back-to-back (since it doesn't call /bot/poll in that
	// case and has no other way to report the state).
	//
	// Optional.
	State botstate.Dict `json:"state,omitempty"`

	// ClaimID is an opaque string used to make this request idempotent.
	//
	// Generated by the bot (usually derived from RBE's lease ID, but it is not
	// a requirement).
	//
	// Required.
	ClaimID string `json:"claim_id"`

	// TaskID is the TaskResultSummary packed key of the task to claim.
	//
	// The bot takes it from the RBE task payload and passes it to /bot/claim as
	// is, see swarming.internals.rbe.TaskPayload proto message.
	//
	// Required.
	TaskID string `json:"task_id"`

	// TaskToRunShard is an entity class shard index for claimed TaskToRun.
	//
	// The bot takes it from the RBE task payload and passes it to /bot/claim as
	// is, see swarming.internals.rbe.TaskPayload proto message.
	//
	// Required.
	TaskToRunShard int `json:"task_to_run_shard"`

	// TaskToRunID identifies TaskToRun to claim.
	//
	// The bot takes it from the RBE task payload and passes it to /bot/claim as
	// is, see swarming.internals.rbe.TaskPayload proto message.
	//
	// Required.
	TaskToRunID int64 `json:"task_to_run_id"`
}

ClaimRequest is sent by the bot.

func (*ClaimRequest) ExtractDebugRequest 

func (r *ClaimRequest) ExtractDebugRequest() any

func (*ClaimRequest) ExtractSession 

func (r *ClaimRequest) ExtractSession() []byte

type ClaimResponse 

type ClaimResponse struct {
	// Cmd instructs the bot what to do next.
	Cmd ClaimCommand `json:"cmd"`

	// Session is a serialized bot session proto.
	//
	// If not empty, contains the refreshed session.
	Session []byte `json:"session,omitempty"`

	// Manifest is full details about the task to execute.
	//
	// Present only for ClaimRun command.
	Manifest *TaskManifest `json:"manifest,omitempty"`

	// TaskID is TaskRunResult ID of the task to run (if any).
	//
	// This is populated for both ClaimRun and ClaimTerminate.
	TaskID string `json:"task_id,omitempty"`
}

ClaimResponse is returned by the server.

type EventRequest 

type EventRequest struct {
	// Session is a serialized Swarming Bot Session proto.
	Session []byte `json:"session"`

	// RequestUUID is used to skip reporting duplicate events on retries.
	//
	// Generated by the client (usually an UUID4 string). Optional.
	RequestUUID string `json:"request_uuid,omitempty"`

	// Event is the kind of the event the bot is reporting.
	//
	// Required. Must be in the list of allowed events.
	Event model.BotEventType `json:"event"`

	// Message is an optional arbitrary text message associated with the event.
	//
	// Will show up in the UI and when listing bot events in the API. Not
	// interpreted by the server in any way.
	Message string `json:"message,omitempty"`

	// State is (mostly) arbitrary JSON dict with various properties of the bot.
	//
	// Optional. If set, will be used to see if the bot should be quarantined or
	// put into maintenance. If not set, the current bot state in the datastore
	// won't be affected.
	State botstate.Dict `json:"state,omitempty"`

	// Version is the bot's own version, if known.
	//
	// Optional. If set, ends up reported together with the event.
	Version string `json:"version,omitempty"`
}

EventRequest is sent by the bot.

func (*EventRequest) ExtractDebugRequest 

func (r *EventRequest) ExtractDebugRequest() any

func (*EventRequest) ExtractSession 

func (r *EventRequest) ExtractSession() []byte

type EventResponse 

type EventResponse struct {
}

EventResponse is returned by the server.

type HandshakeRequest 

type HandshakeRequest struct {
	// Dimensions are the initial dimensions collected by the bot.
	//
	// They are initial in a sense that they are collected by the base version
	// of the bot code, without using custom hooks script yet. In many cases,
	// these dimensions are very different from what the bot would end up using
	// after loading the hooks. For that reason these dimensions aren't actually
	// written to the datastore (to avoid the bot "flapping" between two sets of
	// dimensions when it restart and performs the handshake). They are still
	// used to check if the bot should be quarantined, and may be used in debug
	// logs.
	//
	// At least `id` dimension must be set. It is the bot ID.
	Dimensions map[string][]string `json:"dimensions"`

	// State is (mostly) arbitrary JSON dict with various properties of the bot.
	//
	// Values here are not indexed and they do not affect how tasks are scheduled
	// on the bot. The server is still aware of some keys and checks them to
	// decide how to handle the bot calls.
	State botstate.Dict `json:"state,omitempty"`

	// Version is the bot's own version.
	//
	// It is a digest of the running bot archive. Here it is used FYI only.
	Version string `json:"version"`

	// SessionID is an ID of a new session the bot is opening.
	//
	// If the bot has already opened this session (i.e. this is a retry), this
	// session will be reused.
	//
	// May be absent in calls from very old bots. An auto-generated ID will be
	// used instead in that case. Such bots will be asked to self update on the
	// very first poll call.
	SessionID string `json:"session_id,omitempty"`
}

HandshakeRequest is sent by the bot.

func (*HandshakeRequest) ExtractDebugRequest 

func (r *HandshakeRequest) ExtractDebugRequest() any

func (*HandshakeRequest) ExtractSession 

func (r *HandshakeRequest) ExtractSession() []byte

type HandshakeResponse 

type HandshakeResponse struct {
	// Session is a serialized bot session proto.
	Session []byte `json:"session"`
	// BotVersion is the bot version the bot should be running, as FYI to the bot.
	BotVersion string `json:"bot_version"`
	// ServerVersion is the current server version, as FYI to the bot.
	ServerVersion string `json:"server_version"`
	// BotConfig is the body of the custom hooks script, if any.
	BotConfig string `json:"bot_config,omitempty"`
	// BotConfigName is the name of the custom hooks script or "bot_config.py".
	BotConfigName string `json:"bot_config_name"`
	// BotConfigRev is the revision of the main bot_config.py (not the hooks script).
	BotConfigRev string `json:"bot_config_rev"`
	// BotGroupCfg is derived from the server's bots.cfg.
	BotGroupCfg BotGroupCfg `json:"bot_group_cfg"`
	// RBE defines how the bot should be interacting with RBE API.
	RBE *BotRBEParams `json:"rbe,omitempty"`
}

HandshakeResponse is returned by the server.

type TaskCache 

type TaskCache struct {
	// Name is a logical cache name.
	Name string `json:"name"`
	// Path is where to mount it relative to the task root directory.
	Path string `json:"path"`
	// Hint is cache size hint (as a decimal string) or "-1" if unknown.
	Hint string `json:"hint"`
}

TaskCache describes one named cache requested by a task.

type TaskManifest 

type TaskManifest struct {
	// TaskID is TaskRunResult ID of the task to run.
	TaskID string `json:"task_id"`

	// Caches is a list of named caches requested by the task.
	Caches []TaskCache `json:"caches,omitempty"`
	// CIPDInput are CIPD packages that the bot should fetch.
	CIPDInput *model.CIPDInput `json:"cipd_input,omitempty"`
	// Command is the actual command line the bot should execute.
	Command []string `json:"command"`
	// Containment describes the task process containment (not implemented).
	Containment *model.Containment `json:"containment,omitempty"`
	// Dimensions are task's dimension requirements.
	Dimensions model.TaskDimensions `json:"dimensions"`
	// Env is the environment variables to set.
	Env model.Env `json:"env,omitempty"`
	// EnvPrefixes is values to prepend to environment variables.
	EnvPrefixes model.EnvPrefixes `json:"env_prefixes,omitempty"`
	// GracePeriodSecs is how long to wait for the task to gracefully terminate.
	GracePeriodSecs int64 `json:"grace_period"`
	// HardTimeoutSecs is how long to allow the task to run.
	HardTimeoutSecs int64 `json:"hard_timeout"`
	// IOTimeoutSecs is how long to allow no stdout before the task is terminated.
	IOTimeoutSecs int64 `json:"io_timeout"`
	// SecretBytes are the task secret, if any.
	SecretBytes []byte `json:"secret_bytes,omitempty"`
	// CASInputRoot is what CAS files to fetch.
	CASInputRoot *model.CASDigest `json:"cas_input_root,omitempty"`
	// Outputs are list of extra outputs to upload to RBE-CAS as task results.
	Outputs []string `json:"outputs,omitempty"`
	// Realm is the task's security realm.
	Realm *TaskRealm `json:"realm,omitempty"`
	// RelativeCwd is the directory to set as current when running the task.
	RelativeCwd string `json:"relative_cwd,omitempty"`
	// ResultDB is parameters of the ResultDB invocation for the task.
	ResultDB *TaskResultDB `json:"resultdb,omitempty"`
	// ServiceAccounts describe what service account a task can use.
	ServiceAccounts struct {
		System TaskServiceAccount `json:"system"`
		Task   TaskServiceAccount `json:"task"`
	} `json:"service_accounts"`

	// BotID is the ID of the calling bot.
	BotID string `json:"bot_id"`
	// BotDimensions are dimensions of the calling bot.
	BotDimensions map[string][]string `json:"bot_dimensions"`
	// BotAuthenticatedAs is how the server authenticated the bot.
	BotAuthenticatedAs identity.Identity `json:"bot_authenticated_as"`
}

TaskManifest describes to the bot what it should execute.

This is based to the task runner process.

type TaskRealm 

type TaskRealm struct {
	// Name is the name of the realm.
	Name string `json:"name,omitempty"`
}

TaskRealm defines the task's security realm.

It is put into LUCI_CONTEXT["realm"].

type TaskResultDB 

type TaskResultDB struct {
	// Hostname is the hostname of ResultDB service to use.
	Hostname string `json:"hostname"`
	// CurrentInvocation is the ResultDB invocation to use.
	CurrentInvocation struct {
		// Name is the invocation name.
		Name string `json:"name"`
		// UpdateToken is the invocation's update token.
		UpdateToken string `json:"update_token"`
	} `json:"current_invocation"`
}

TaskResultDB is parameters of the ResultDB invocation for the task.

It is put into LUCI_CONTEXT["resultdb"].

type TaskServiceAccount 

type TaskServiceAccount struct {
	// ServiceAccount is 'none', 'bot' or an email.
	//
	// Bot interprets 'none' and 'bot' locally. When it sees something else, it
	// uses /bot/oauth_token or /bot/id_token API endpoints to grab tokens through
	// the server.
	ServiceAccount string `json:"service_account"`
}

TaskServiceAccount describe what service account a task can use.

type TokenRequest 

type TokenRequest struct {
	// Session is a serialized Swarming Bot Session proto.
	Session []byte `json:"session"`

	// AccountID is either "system" or "task".
	//
	// Defines what sort of service account to mint a token for.
	AccountID string `json:"account_id"`

	// TaskID is the task ID that the bot thinks it is running.
	//
	// Required when minting tokens for "task" account. Will be checked against
	// the task ID associated with the bot in the datastore.
	TaskID string `json:"task_id,omitempty"`

	// Scopes is a list of OAuth scopes to mint a service account token with.
	//
	// Used only by OAuthToken handler. Must be unset in IDToken handler.
	Scopes []string `json:"scopes,omitempty"`

	// Audience is an audience to put into a minted ID token.
	//
	// Used only by IDToken handler. Must be unset in OAuthToken handler.
	Audience string `json:"audience,omitempty"`
}

TokenRequest is sent by the bot.

func (*TokenRequest) ExtractDebugRequest 

func (r *TokenRequest) ExtractDebugRequest() any

func (*TokenRequest) ExtractSession 

func (r *TokenRequest) ExtractSession() []byte

type TokenResponse 

type TokenResponse struct {
	// ServiceAccount is a service account the token is from.
	//
	// This is either an email, or literals "bot" (if the bot is configured to use
	// its own service account as a system or task account) or "none" (if
	// there's no service account assigned at all).
	ServiceAccount string `json:"service_account"`

	// AccessToken is the minted OAuth access token.
	//
	// Returned only by OAuthToken handler when actually using an assigned service
	// account (i.e. not "bot" or "none"). Unset in IDToken handler response.
	AccessToken string `json:"access_token,omitempty"`

	// IDToken is the minted ID token.
	//
	// Returned only by IDToken handler when actually using an assigned service
	// account (i.e. not "bot" or "none"). Unset in OAuthToken handler response.
	IDToken string `json:"id_token,omitempty"`

	// Expiry is token's Unix expiration timestamp in seconds.
	//
	// Returned only when actually using an assigned service account (i.e. not
	// "bot" or "none").
	Expiry int64 `json:"expiry,omitempty"`
}

TokenResponse is returned by the server.

type UnimplementedRequest 

type UnimplementedRequest struct{}

UnimplementedRequest is used as a placeholder in unimplemented handlers.

func (*UnimplementedRequest) ExtractDebugRequest 

func (r *UnimplementedRequest) ExtractDebugRequest() any

func (*UnimplementedRequest) ExtractSession 

func (r *UnimplementedRequest) ExtractSession() []byte

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.