buildlet

package
v0.0.0-...-4a87e05 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 20, 2024 License: BSD-3-Clause Imports: 41 Imported by: 40

README

Go Reference

golang.org/x/build/buildlet

Package buildlet contains client tools for working with a buildlet server.

Documentation

Overview

Package buildlet contains client tools for working with a buildlet server.

Index

Constants

View Source 
const (
	// GomoteCreateStreamVersion is the gomote protocol version at which JSON streamed responses started.
	GomoteCreateStreamVersion = "20191119"

	// GomoteCreateMinVersion is the oldest "gomote create" protocol version that's still supported.
	GomoteCreateMinVersion = "20160922"
)

Variables

View Source 
var ErrClosed = errors.New("buildlet: Client closed")
View Source 
var ErrQuotaExceeded = errors.New("quota exceeded")

ErrQuotaExceeded matches errors.Is when VM creation fails with a quota error. Currently, it only supports GCE quota errors.

View Source 
var ErrTimeout = errors.New("buildlet: timeout waiting for command to complete")

ErrTimeout is a sentinel error that represents that waiting for a command to complete has exceeded the given timeout.

View Source 
var GCEGate func()

GCEGate optionally specifies a function to run before any GCE API call. It's intended to be used to bound QPS rate to GCE.

View Source 
var NoKeyPair = KeyPair{}

NoKeyPair is used by the coordinator to speak http directly to buildlets, inside their firewall, without TLS.

Functions

func RegisterFlags 

func RegisterFlags()

RegisterFlags registers "user" and "staging" flags that control the behavior of NewCoordinatorClientFromFlags. These are used by remote client commands like gomote.

Types

type Client 

type Client interface {
	RemoteClient
	ConnectSSH(user, authorizedPubKey string) (net.Conn, error)
	IPPort() string
	InstanceName() string
	IsBroken() bool
	MarkBroken()
	Name() string
	ProxyRoundTripper() http.RoundTripper
	SetDescription(v string)
	SetDialer(dialer func(context.Context) (net.Conn, error))
	SetHTTPClient(httpClient *http.Client)
	SetInstanceName(v string)
	SetName(name string)
	SetOnHeartbeatFailure(fn func())
	Status(ctx context.Context) (Status, error)
	String() string
	URL() string
}

Client is an interface that represent the methods exposed by client. The fake buildlet client should be used instead of client when testing things that use the client interface. This includes a number of coordinator-internal details; users outside the coordinator should use RemoteClient.

func NewClient 

func NewClient(ipPort string, kp KeyPair) Client

NewClient returns a *client that will manipulate ipPort, authenticated using the provided keypair.

This constructor returns immediately without testing the host or auth.

func StartNewVM 

func StartNewVM(creds *google.Credentials, buildEnv *buildenv.Environment, instName, hostType string, opts VMOpts) (Client, error)

StartNewVM boots a new VM on GCE and returns a buildlet client configured to speak to it.

type CoordinatorClient 

type CoordinatorClient struct {
	// Auth specifies how to authenticate to the coordinator.
	Auth UserPass

	// Instance optionally specifies the build coordinator to connect
	// to. If zero, the production coordinator is used.
	Instance build.CoordinatorInstance
	// contains filtered or unexported fields
}

A CoordinatorClient makes calls to the build coordinator.

func NewCoordinatorClientFromFlags 

func NewCoordinatorClientFromFlags() (*CoordinatorClient, error)

NewCoordinatorClientFromFlags constructs a CoordinatorClient for the current user.

func (*CoordinatorClient) CreateBuildlet 

func (cc *CoordinatorClient) CreateBuildlet(builderType string) (RemoteClient, error)

CreateBuildlet creates a new buildlet of the given builder type on cc.

This takes a builderType (instead of a hostType), but the returned buildlet can be used as any builder that has the same underlying buildlet type. For instance, a linux-amd64 buildlet can act as either linux-amd64 or linux-386-387.

It may expire at any time. To release it, call Client.Close.

func (*CoordinatorClient) CreateBuildletWithStatus 

func (cc *CoordinatorClient) CreateBuildletWithStatus(builderType string, status func(types.BuildletWaitStatus)) (RemoteClient, error)

CreateBuildletWithStatus is like CreateBuildlet but accepts an optional status callback.

func (*CoordinatorClient) NamedBuildlet 

func (cc *CoordinatorClient) NamedBuildlet(name string) (RemoteClient, error)

NamedBuildlet returns a buildlet client for the named remote buildlet. Names are not validated. Use Client.Status to check whether the client works.

func (*CoordinatorClient) RemoteBuildlets 

func (cc *CoordinatorClient) RemoteBuildlets() ([]RemoteBuildlet, error)

type DirEntry 

type DirEntry struct {
	// Line is of the form "drw-rw-rw\t<name>" and then if a regular file,
	// also "\t<size>\t<modtime>". in either case, without trailing newline.
	// TODO: break into parsed fields?
	Line string
}

DirEntry is the information about a file on a buildlet.

func (DirEntry) Digest 

func (de DirEntry) Digest() string

Digest returns the SHA-1 digest of the file, such as "da39a3ee5e6b4b0d3255bfef95601890afd80709". It returns the empty string if the digest isn't included.

func (DirEntry) IsDir 

func (de DirEntry) IsDir() bool

IsDir reports whether de describes a directory. That is, it tests for the os.ModeDir bit being set in de.Perm().

func (DirEntry) Name 

func (de DirEntry) Name() string

Name returns the relative path to the file, such as "src/net/http/" or "src/net/http/jar.go".

func (DirEntry) Perm 

func (de DirEntry) Perm() string

Perm returns the permission bits in string form, such as "-rw-r--r--" or "drwxr-xr-x".

func (DirEntry) String 

func (de DirEntry) String() string

type EC2Client 

type EC2Client struct {
	// contains filtered or unexported fields
}

EC2Client is the client used to create buildlets on EC2.

func NewEC2Client 

func NewEC2Client(client *cloud.AWSClient) *EC2Client

NewEC2Client creates a new EC2Client.

func (*EC2Client) StartNewVM 

func (c *EC2Client) StartNewVM(ctx context.Context, buildEnv *buildenv.Environment, hconf *dashboard.HostConfig, vmName, hostType string, opts *VMOpts) (Client, error)

StartNewVM boots a new VM on EC2, waits until the client is accepting connections on the configured port and returns a buildlet client configured communicate with it.

type ExecOpts 

type ExecOpts struct {
	// Output is the output of stdout and stderr.
	// If nil, the output is discarded.
	Output io.Writer

	// Dir is the directory from which to execute the command,
	// as an absolute or relative path using the buildlet's native
	// path separator, or a slash-separated relative path.
	// If relative, it is relative to the buildlet's work directory.
	//
	// Dir is optional. If not specified, it defaults to the directory of
	// the command, or the work directory if SystemLevel is set.
	Dir string

	// Args are the arguments to pass to the cmd given to Client.Exec.
	Args []string

	// ExtraEnv are KEY=VALUE pairs to append to the buildlet
	// process's environment.
	ExtraEnv []string

	// Path, if non-nil, specifies the PATH variable of the executed process's
	// environment. Each path in the list should use separators native to the
	// buildlet's platform, and a non-nil empty list clears the path.
	//
	// The following expansions apply:
	//   - the string "$PATH" expands to any existing PATH element(s)
	//   - the substring "$WORKDIR" expands to buildlet's temp workdir
	//
	// After expansion, the list is joined with an OS-specific list
	// separator and supplied to the executed process as its PATH
	// environment variable.
	Path []string

	// SystemLevel controls whether the command is expected to be found outside of
	// the buildlet's environment.
	SystemLevel bool

	// Debug, if true, instructs to the buildlet to print extra debug
	// info to the output before the command begins executing.
	Debug bool

	// OnStartExec is an optional hook that runs after the 200 OK
	// response from the buildlet, but before the output begins
	// writing to Output.
	OnStartExec func()
}

ExecOpts are options for a remote command invocation.

type FakeClient 

type FakeClient struct {
	// contains filtered or unexported fields
}

FakeClient is a fake buildlet client used for testing. Not all functions are implemented.

func (*FakeClient) Close 

func (fc *FakeClient) Close() error

Close is a fake client closer.

func (*FakeClient) ConnectSSH 

func (fc *FakeClient) ConnectSSH(user, authorizedPubKey string) (net.Conn, error)

ConnectSSH connects to a fake SSH server.

func (*FakeClient) Exec 

func (fc *FakeClient) Exec(ctx context.Context, cmd string, opts ExecOpts) (remoteErr, execErr error)

Exec fakes the execution.

func (*FakeClient) GetTar 

func (fc *FakeClient) GetTar(ctx context.Context, dir string) (io.ReadCloser, error)

GetTar gives a vake tar zipped directory.

func (*FakeClient) IPPort 

func (fc *FakeClient) IPPort() string

IPPort provides a fake ip and port pair.

func (*FakeClient) InstanceName 

func (fc *FakeClient) InstanceName() string

InstanceName gives the fake instance name.

func (*FakeClient) IsBroken 

func (fc *FakeClient) IsBroken() bool

IsBroken returns a fake broken response.

func (*FakeClient) ListDir 

func (fc *FakeClient) ListDir(ctx context.Context, dir string, opts ListDirOpts, fn func(DirEntry)) error

ListDir lists a directory on a fake buildlet.

func (*FakeClient) MarkBroken 

func (fc *FakeClient) MarkBroken()

MarkBroken marks the fake client as broken.

func (*FakeClient) Name 

func (fc *FakeClient) Name() string

Name is the name of the fake client.

func (*FakeClient) ProxyRoundTripper 

func (fc *FakeClient) ProxyRoundTripper() http.RoundTripper

ProxyRoundTripper provides a fake proxy.

func (*FakeClient) ProxyTCP 

func (fc *FakeClient) ProxyTCP(port int) (io.ReadWriteCloser, error)

ProxyTCP provides a fake proxy.

func (*FakeClient) Put 

func (fc *FakeClient) Put(ctx context.Context, r io.Reader, path string, mode os.FileMode) error

Put places a file on a fake buildlet.

func (*FakeClient) PutTar 

func (fc *FakeClient) PutTar(ctx context.Context, r io.Reader, dir string) error

PutTar fakes putting a tar zipped file on a buildldet.

func (*FakeClient) PutTarFromURL 

func (fc *FakeClient) PutTarFromURL(ctx context.Context, tarURL, dir string) error

PutTarFromURL fakes putting a tar zipped file on a builelt.

func (*FakeClient) RemoteName 

func (fc *FakeClient) RemoteName() string

RemoteName gives the remote name of the fake buildlet.

func (*FakeClient) RemoveAll 

func (fc *FakeClient) RemoveAll(ctx context.Context, paths ...string) error

RemoveAll deletes the provided paths, relative to the work directory for a fake buildlet.

func (*FakeClient) SetDescription 

func (fc *FakeClient) SetDescription(v string)

SetDescription sets the description on a fake client.

func (*FakeClient) SetDialer 

func (fc *FakeClient) SetDialer(dialer func(context.Context) (net.Conn, error))

SetDialer sets the function that creates a new connection to the fake buildlet.

func (*FakeClient) SetHTTPClient 

func (fc *FakeClient) SetHTTPClient(httpClient *http.Client)

SetHTTPClient sets the HTTP client on a fake client.

func (*FakeClient) SetInstanceName 

func (fc *FakeClient) SetInstanceName(v string)

SetInstanceName sets the GCE or EC2 instance name on a fake client.

func (*FakeClient) SetName 

func (fc *FakeClient) SetName(name string)

SetName sets the name on a fake client.

func (*FakeClient) SetOnHeartbeatFailure 

func (fc *FakeClient) SetOnHeartbeatFailure(fn func())

SetOnHeartbeatFailure sets a function to be called when heartbeats against this fake buildlet fail.

func (*FakeClient) Status 

func (fc *FakeClient) Status(ctx context.Context) (Status, error)

Status provides a status on the fake client.

func (*FakeClient) String 

func (fc *FakeClient) String() string

String provides a fake string representation of the client.

func (*FakeClient) URL 

func (fc *FakeClient) URL() string

URL is the URL for a fake buildlet.

func (*FakeClient) WorkDir 

func (fc *FakeClient) WorkDir(ctx context.Context) (string, error)

WorkDir is the working directory for the fake buildlet.

type GCEError 

type GCEError struct {
	OpErrors []*compute.OperationErrorErrors
}

func (*GCEError) Error 

func (q *GCEError) Error() string

func (*GCEError) Is 

func (q *GCEError) Is(target error) bool

type GRPCCoordinatorClient 

type GRPCCoordinatorClient struct {
	Client protos.GomoteServiceClient
}

func (*GRPCCoordinatorClient) CreateBuildlet 

func (c *GRPCCoordinatorClient) CreateBuildlet(ctx context.Context, builderType string) (RemoteClient, error)

func (*GRPCCoordinatorClient) CreateBuildletWithStatus 

func (c *GRPCCoordinatorClient) CreateBuildletWithStatus(ctx context.Context, builderType string, status func(types.BuildletWaitStatus)) (RemoteClient, error)

type KeyPair 

type KeyPair struct {
	CertPEM string
	KeyPEM  string
}

KeyPair is the TLS public certificate PEM file and its associated private key PEM file that a builder will use for its HTTPS server. The zero value means no HTTPs, which is used by the coordinator for machines running within a firewall.

func NewKeyPair 

func NewKeyPair() (KeyPair, error)

func (KeyPair) IsZero 

func (kp KeyPair) IsZero() bool

func (KeyPair) Password 

func (kp KeyPair) Password() string

Password returns the SHA1 of the KeyPEM. This is used as the HTTP Basic Auth password.

type ListDirOpts 

type ListDirOpts struct {
	// Recursive controls whether the directory is listed
	// recursively.
	Recursive bool

	// Skip are the directories to skip, relative to the directory
	// passed to ListDir. Each item should contain only forward
	// slashes and not start or end in slashes.
	Skip []string

	// Digest controls whether the SHA-1 digests of regular files
	// are returned.
	Digest bool
}

ListDirOpts are options for Client.ListDir.

type RemoteBuildlet 

type RemoteBuildlet struct {
	HostType    string // "host-linux-bullseye"
	BuilderType string // "linux-386-387"
	Name        string // "buildlet-adg-openbsd-386-2"
	Created     time.Time
	Expires     time.Time
}

type RemoteClient 

type RemoteClient interface {
	Close() error
	Exec(ctx context.Context, cmd string, opts ExecOpts) (remoteErr, execErr error)
	GetTar(ctx context.Context, dir string) (io.ReadCloser, error)
	ListDir(ctx context.Context, dir string, opts ListDirOpts, fn func(DirEntry)) error
	Put(ctx context.Context, r io.Reader, path string, mode os.FileMode) error
	PutTar(ctx context.Context, r io.Reader, dir string) error
	PutTarFromURL(ctx context.Context, tarURL, dir string) error
	ProxyTCP(port int) (io.ReadWriteCloser, error)
	RemoteName() string
	RemoveAll(ctx context.Context, paths ...string) error
	WorkDir(ctx context.Context) (string, error)
}

RemoteClient is a subset of methods that can be used by a gomote client.

type Status 

type Status struct {
	Version int // buildlet version, coordinator rejects value that is too old (see minBuildletVersion).
}

Status provides status information about the buildlet.

A coordinator can use the provided information to decide what, if anything, to do with a buildlet.

type UserPass 

type UserPass struct {
	Username string // "user-$USER"
	Password string // buildlet key
}

type VM 

type VM struct {
	// Name is the name of the GCE VM instance.
	// For example, it's of the form "mote-bradfitz-plan9-386-foo",
	// and not "plan9-386-foo".
	Name   string
	IPPort string
	TLS    KeyPair
	Type   string // buildlet type
}

type VMOpts 

type VMOpts struct {
	// Zone is the GCE zone to create the VM in.
	// Optional; defaults to provided build environment's zone.
	Zone string

	// ProjectID is the GCE project ID (e.g. "foo-bar-123", not
	// the numeric ID).
	// Optional; defaults to provided build environment's project ID ("name").
	ProjectID string

	// TLS optionally specifies the TLS keypair to use.
	// If zero, http without auth is used.
	TLS KeyPair

	// Optional description of the VM.
	Description string

	// Optional metadata to put on the instance.
	Meta map[string]string

	// DeleteIn optionally specifies a duration at which
	// to delete the VM.
	// If zero, a short default is used (not enough for longtest builders).
	// Negative means no deletion timeout.
	DeleteIn time.Duration

	// OnInstanceRequested optionally specifies a hook to run synchronously
	// after the computeService.Instances.Insert call, but before
	// waiting for its operation to proceed.
	OnInstanceRequested func()

	// OnInstanceCreated optionally specifies a hook to run synchronously
	// after the instance operation succeeds.
	OnInstanceCreated func()

	// OnInstanceCreated optionally specifies a hook to run synchronously
	// after the computeService.Instances.Get call.
	// Only valid for GCE resources.
	OnGotInstanceInfo func(*compute.Instance)

	// OnInstanceCreated optionally specifies a hook to run synchronously
	// after the EC2 instance information is retrieved.
	// Only valid for EC2 resources.
	OnGotEC2InstanceInfo func(*cloud.Instance)

	// OnBeginBuildletProbe optionally specifies a hook to run synchronously
	// before StartNewVM tries to hit buildletURL to see if it's up yet.
	OnBeginBuildletProbe func(buildletURL string)

	// OnEndBuildletProbe optionally specifies a hook to run synchronously
	// after StartNewVM tries to hit the buildlet's URL to see if it's up.
	// The hook parameters are the return values from http.Get.
	OnEndBuildletProbe func(*http.Response, error)

	// SkipEndpointVerification does not verify that the builder is listening
	// on port 80 or 443 before creating a buildlet client.
	SkipEndpointVerification bool

	// UseIAPTunnel uses an IAP tunnel to connect to buildlets on GCP.
	UseIAPTunnel bool

	// DiskSizeGB specifies the size of the boot disk in base-2 GB. The default
	// disk size is used if unset.
	// Only valid for GCE resources.
	DiskSizeGB int64
}

VMOpts control how new VMs are started.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.