buildlet

package
v0.0.0-...-16694d9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 4, 2021 License: BSD-3-Clause Imports: 39 Imported by: 0

README

Go Reference

golang.org/x/build/buildlet

Package buildlet contains client tools for working with a buildlet server.

Documentation

Overview

Package buildlet contains client tools for working with a buildlet server.

Index

Constants

View Source 
const (
	// GomoteCreateStreamVersion is the gomote protocol version at which JSON streamed responses started.
	GomoteCreateStreamVersion = "20191119"

	// GomoteCreateMinVersion is the oldest "gomote create" protocol version that's still supported.
	GomoteCreateMinVersion = "20160922"
)

Variables

View Source 
var (
	// TODO(evanbrown): resource requirements should be
	// defined per-builder in dashboard/builders.go
	BuildletCPU      = api.MustParse("2")         // 2 Cores
	BuildletCPULimit = api.MustParse("8")         // 8 Cores
	BuildletMemory   = api.MustParse("4000000Ki") // 4,000,000Ki RAM
)
View Source 
var ErrClosed = errors.New("buildlet: Client closed")
View Source 
var ErrTimeout = errors.New("buildlet: timeout waiting for command to complete")
View Source 
var GCEGate func()

GCEGate optionally specifies a function to run before any GCE API call. It's intended to be used to bound QPS rate to GCE.

View Source 
var NoKeyPair = KeyPair{}

NoKeyPair is used by the coordinator to speak http directly to buildlets, inside their firewall, without TLS.

Functions

func DestroyVM 

func DestroyVM(ts oauth2.TokenSource, proj, zone, instance string) error

DestroyVM sends a request to delete a VM. Actual VM description is currently (2015-01-19) very slow for no good reason. This function returns once it's been requested, not when it's done.

func RegisterFlags 

func RegisterFlags()

RegisterFlags registers "user" and "staging" flags that control the behavior of NewCoordinatorClientFromFlags. These are used by remote client commands like gomote.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

A Client interacts with a single buildlet.

func NewClient 

func NewClient(ipPort string, kp KeyPair) *Client

NewClient returns a *Client that will manipulate ipPort, authenticated using the provided keypair.

This constructor returns immediately without testing the host or auth.

func StartNewVM 

func StartNewVM(creds *google.Credentials, buildEnv *buildenv.Environment, instName, hostType string, opts VMOpts) (*Client, error)

StartNewVM boots a new VM on GCE and returns a buildlet client configured to speak to it.

func StartPod 

func StartPod(ctx context.Context, kubeClient *kubernetes.Client, podName, hostType string, opts PodOpts) (*Client, error)

StartPod creates a new pod on a Kubernetes cluster and returns a buildlet client configured to speak to it.

func (*Client) Close 

func (c *Client) Close() error

Closes destroys and closes down the buildlet, destroying all state immediately.

func (*Client) ConnectSSH 

func (c *Client) ConnectSSH(user, authorizedPubKey string) (net.Conn, error)

ConnectSSH opens an SSH connection to the buildlet for the given username. The authorizedPubKey must be a line from an ~/.ssh/authorized_keys file and correspond to the private key to be used to communicate over the net.Conn.

func (*Client) DestroyVM 

func (c *Client) DestroyVM(ts oauth2.TokenSource, proj, zone, instance string) error

DestroyVM shuts down the buildlet and destroys the VM instance.

func (*Client) Exec 

func (c *Client) Exec(ctx context.Context, cmd string, opts ExecOpts) (remoteErr, execErr error)

Exec runs cmd on the buildlet.

Two errors are returned: one is whether the command succeeded remotely (remoteErr), and the second (execErr) is whether there were system errors preventing the command from being started or seen to completition. If execErr is non-nil, the remoteErr is meaningless.

If the context's deadline is exceeded, the returned execErr is ErrTimeout.

func (*Client) GCEInstanceName 

func (c *Client) GCEInstanceName() string

GCEInstanceName gets an instance name for GCE buildlets. This value differs from the buildlet name used in the CLI and web interface. For non-GCE buildlets, this will return an empty string.

func (*Client) GetTar 

func (c *Client) GetTar(ctx context.Context, dir string) (io.ReadCloser, error)

GetTar returns a .tar.gz stream of the given directory, relative to the buildlet's work dir. The provided dir may be empty to get everything.

func (*Client) IPPort 

func (c *Client) IPPort() string

func (*Client) IsBroken 

func (c *Client) IsBroken() bool

IsBroken reports whether this client is broken in some way.

func (*Client) ListDir 

func (c *Client) ListDir(ctx context.Context, dir string, opts ListDirOpts, fn func(DirEntry)) error

ListDir lists the contents of a directory. The fn callback is run for each entry. The directory dir itself is not included.

func (*Client) MarkBroken 

func (c *Client) MarkBroken()

MarkBroken marks this client as broken in some way.

func (*Client) Name 

func (c *Client) Name() string

Name returns the name of this buildlet. It returns the first non-empty string from the name given to SetName, its remote buildlet name, its ip:port, or "(unnamed-buildlet)" in the case where ip:port is empty because there's a custom dialer.

func (*Client) ProxyRoundTripper 

func (c *Client) ProxyRoundTripper() http.RoundTripper

ProxyRoundTripper returns a RoundTripper that sends HTTP requests directly through to the underlying buildlet, adding auth and X-Buildlet-Proxy headers as necessary. This is really only intended for use by the coordinator.

func (*Client) ProxyTCP 

func (c *Client) ProxyTCP(port int) (io.ReadWriteCloser, error)

ProxyTCP connects to the given port on the remote buildlet. The buildlet client must currently be a gomote client (RemoteName != "") and the target type must be a VM type running on GCE. This was primarily created for RDP to Windows machines, but it might get reused for other purposes in the future.

func (*Client) Put 

func (c *Client) Put(ctx context.Context, r io.Reader, path string, mode os.FileMode) error

Put writes the provided file to path (relative to workdir) and sets mode.

func (*Client) PutTar 

func (c *Client) PutTar(ctx context.Context, r io.Reader, dir string) error

PutTar writes files to the remote buildlet, rooted at the relative directory dir. If dir is empty, they're placed at the root of the buildlet's work directory. The dir is created if necessary. The Reader must be of a tar.gz file.

func (*Client) PutTarFromURL 

func (c *Client) PutTarFromURL(ctx context.Context, tarURL, dir string) error

PutTarFromURL tells the buildlet to download the tar.gz file from tarURL and write it to dir, a relative directory from the workdir. If dir is empty, they're placed at the root of the buildlet's work directory. The dir is created if necessary. The url must be of a tar.gz file.

func (*Client) RemoteName 

func (c *Client) RemoteName() string

RemoteName returns the name of this client's buildlet on the coordinator. If this buildlet isn't a remote buildlet created via gomote, this returns the empty string.

func (*Client) RemoveAll 

func (c *Client) RemoveAll(ctx context.Context, paths ...string) error

RemoveAll deletes the provided paths, relative to the work directory.

func (*Client) SetDescription 

func (c *Client) SetDescription(v string)

SetDescription sets a short description of where the buildlet connection came from. This is used by the build coordinator status page, mostly for debugging.

func (*Client) SetDialer 

func (c *Client) SetDialer(dialer func(context.Context) (net.Conn, error))

SetDialer sets the function that creates a new connection to the buildlet. By default, net.Dialer.DialContext is used. SetDialer has effect only when TLS isn't used.

TODO(bradfitz): this is only used for ssh connections to buildlets, which previously required the client to do its own net.Dial + upgrade request. But now that the net/http client supports read/write bodies for protocol upgrades, we could change how ssh works and delete this.

func (*Client) SetGCEInstanceName 

func (c *Client) SetGCEInstanceName(v string)

SetGCEInstanceName sets an instance name for GCE buildlets. This value differs from the buildlet name used in the CLI and web interface.

func (*Client) SetHTTPClient 

func (c *Client) SetHTTPClient(httpClient *http.Client)

SetHTTPClient replaces the underlying HTTP client. It should only be called before the Client is used.

func (*Client) SetName 

func (c *Client) SetName(name string)

func (*Client) SetOnHeartbeatFailure 

func (c *Client) SetOnHeartbeatFailure(fn func())

SetOnHeartbeatFailure sets a function to be called when heartbeats against this builder fail, or when the client is destroyed with Close. The function fn is never called more than once. SetOnHeartbeatFailure must be set before any use of the buildlet.

func (*Client) SetReleaseMode 

func (c *Client) SetReleaseMode(v bool)

SetReleaseMode sets whether this client is being used in "release mode", to prepare the final binaries to be shipped to users. All this mode does for now is disable pargzip multi-stream gzip files. See golang.org/issue/19052.

SetReleaseMode must be set before using the client.

func (*Client) Status 

func (c *Client) Status(ctx context.Context) (Status, error)

Status returns an Status value describing this buildlet.

func (*Client) String 

func (c *Client) String() string

func (*Client) URL 

func (c *Client) URL() string

URL returns the buildlet's URL prefix, without a trailing slash.

func (*Client) WorkDir 

func (c *Client) WorkDir(ctx context.Context) (string, error)

WorkDir returns the absolute path to the buildlet work directory.

type CoordinatorClient 

type CoordinatorClient struct {
	// Auth specifies how to authenticate to the coordinator.
	Auth UserPass

	// Instance optionally specifies the build coordinator to connect
	// to. If zero, the production coordinator is used.
	Instance build.CoordinatorInstance
	// contains filtered or unexported fields
}

A CoordinatorClient makes calls to the build coordinator.

func NewCoordinatorClientFromFlags 

func NewCoordinatorClientFromFlags() (*CoordinatorClient, error)

NewCoordinatorClientFromFlags constructs a CoordinatorClient for the current user.

func (*CoordinatorClient) CreateBuildlet 

func (cc *CoordinatorClient) CreateBuildlet(builderType string) (*Client, error)

CreateBuildlet creates a new buildlet of the given builder type on cc.

This takes a builderType (instead of a hostType), but the returned buildlet can be used as any builder that has the same underlying buildlet type. For instance, a linux-amd64 buildlet can act as either linux-amd64 or linux-386-387.

It may expire at any time. To release it, call Client.Close.

func (*CoordinatorClient) CreateBuildletWithStatus 

func (cc *CoordinatorClient) CreateBuildletWithStatus(builderType string, status func(types.BuildletWaitStatus)) (*Client, error)

CreateBuildletWithStatus is like CreateBuildlet but accepts an optional status callback.

func (*CoordinatorClient) NamedBuildlet 

func (cc *CoordinatorClient) NamedBuildlet(name string) (*Client, error)

NamedBuildlet returns a buildlet client for the named remote buildlet. Names are not validated. Use Client.Status to check whether the client works.

func (*CoordinatorClient) RemoteBuildlets 

func (cc *CoordinatorClient) RemoteBuildlets() ([]RemoteBuildlet, error)

type DirEntry 

type DirEntry struct {
	// contains filtered or unexported fields
}

DirEntry is the information about a file on a buildlet.

func (DirEntry) Digest 

func (de DirEntry) Digest() string

Digest returns the SHA-1 digest of the file, such as "da39a3ee5e6b4b0d3255bfef95601890afd80709". It returns the empty string if the digest isn't included.

func (DirEntry) IsDir 

func (de DirEntry) IsDir() bool

IsDir reports whether de describes a directory. That is, it tests for the os.ModeDir bit being set in de.Perm().

func (DirEntry) Name 

func (de DirEntry) Name() string

Name returns the relative path to the file, such as "src/net/http/" or "src/net/http/jar.go".

func (DirEntry) Perm 

func (de DirEntry) Perm() string

Perm returns the permission bits in string form, such as "-rw-r--r--" or "drwxr-xr-x".

func (DirEntry) String 

func (de DirEntry) String() string

type EC2Client 

type EC2Client struct {
	// contains filtered or unexported fields
}

EC2Client is the client used to create buildlets on EC2.

func NewEC2Client 

func NewEC2Client(client *cloud.AWSClient) *EC2Client

NewEC2Client creates a new EC2Client.

func (*EC2Client) StartNewVM 

func (c *EC2Client) StartNewVM(ctx context.Context, buildEnv *buildenv.Environment, hconf *dashboard.HostConfig, vmName, hostType string, opts *VMOpts) (*Client, error)

StartNewVM boots a new VM on EC2, waits until the client is accepting connections on the configured port and returns a buildlet client configured communicate with it.

type ExecOpts 

type ExecOpts struct {
	// Output is the output of stdout and stderr.
	// If nil, the output is discarded.
	Output io.Writer

	// Dir is the directory from which to execute the command.
	// It is optional. If not specified, it defaults to the directory of
	// the command, or the work directory if SystemLevel is set.
	Dir string

	// Args are the arguments to pass to the cmd given to Client.Exec.
	Args []string

	// ExtraEnv are KEY=VALUE pairs to append to the buildlet
	// process's environment.
	ExtraEnv []string

	// Path, if non-nil, specifies the PATH variable of the executed
	// process's environment. A non-nil empty list clears the path.
	// The following expansions apply:
	//   - the string "$PATH" expands to any existing PATH element(s)
	//   - the substring "$WORKDIR" expands to buildlet's temp workdir
	// After expansions, the list is joined with an OS-specific list
	// separator and supplied to the executed process as its PATH
	// environment variable.
	Path []string

	// SystemLevel controls whether the command is run outside of
	// the buildlet's environment.
	SystemLevel bool

	// Debug, if true, instructs to the buildlet to print extra debug
	// info to the output before the command begins executing.
	Debug bool

	// OnStartExec is an optional hook that runs after the 200 OK
	// response from the buildlet, but before the output begins
	// writing to Output.
	OnStartExec func()
}

ExecOpts are options for a remote command invocation.

type KeyPair 

type KeyPair struct {
	CertPEM string
	KeyPEM  string
}

KeyPair is the TLS public certificate PEM file and its associated private key PEM file that a builder will use for its HTTPS server. The zero value means no HTTPs, which is used by the coordinator for machines running within a firewall.

func NewKeyPair 

func NewKeyPair() (KeyPair, error)

func (KeyPair) IsZero 

func (kp KeyPair) IsZero() bool

func (KeyPair) Password 

func (kp KeyPair) Password() string

Password returns the SHA1 of the KeyPEM. This is used as the HTTP Basic Auth password.

type ListDirOpts 

type ListDirOpts struct {
	// Recursive controls whether the directory is listed
	// recursively.
	Recursive bool

	// Skip are the directories to skip, relative to the directory
	// passed to ListDir. Each item should contain only forward
	// slashes and not start or end in slashes.
	Skip []string

	// Digest controls whether the SHA-1 digests of regular files
	// are returned.
	Digest bool
}

ListDirOpts are options for Client.ListDir.

type PodOpts 

type PodOpts struct {
	// ProjectID is the GCE project ID. Required.
	ProjectID string

	// ImageRegistry specifies the Docker registry Kubernetes
	// will use to create the pod. Required.
	ImageRegistry string

	// TLS optionally specifies the TLS keypair to use.
	// If zero, http without auth is used.
	TLS KeyPair

	// Description optionally describes the pod.
	Description string

	// Labels optionally specify key=value strings that Kubernetes
	// can use to filter and group pods.
	Labels map[string]string

	// DeleteIn optionally specifies a duration at which
	// to delete the pod.
	DeleteIn time.Duration

	// OnPodCreating optionally specifies a hook to run synchronously
	// after the pod create request has been made, but before the create
	// has succeeded.
	OnPodCreating func()

	// OnPodCreated optionally specifies a hook to run synchronously
	// after the pod create request succeeds.
	OnPodCreated func()

	// OnGotPodInfo optionally specifies a hook to run synchronously
	// after the pod Get call.
	OnGotPodInfo func()
}

PodOpts control how new pods are started.

type RemoteBuildlet 

type RemoteBuildlet struct {
	HostType    string // "host-linux-jessie"
	BuilderType string // "linux-386-387"
	Name        string // "buildlet-adg-openbsd-386-2"
	Created     time.Time
	Expires     time.Time
}

type Status 

type Status struct {
	Version int // buildlet version, coordinator rejects any value less than 1.
}

Status provides status information about the buildlet.

A coordinator can use the provided information to decide what, if anything, to do with a buildlet.

type UserPass 

type UserPass struct {
	Username string // "user-$USER"
	Password string // buildlet key
}

type VM 

type VM struct {
	// Name is the name of the GCE VM instance.
	// For example, it's of the form "mote-bradfitz-plan9-386-foo",
	// and not "plan9-386-foo".
	Name   string
	IPPort string
	TLS    KeyPair
	Type   string // buildlet type
}

func ListVMs 

func ListVMs(ts oauth2.TokenSource, proj, zone string) ([]VM, error)

ListVMs lists all VMs.

type VMOpts 

type VMOpts struct {
	// Zone is the GCE zone to create the VM in.
	// Optional; defaults to provided build environment's zone.
	Zone string

	// ProjectID is the GCE project ID (e.g. "foo-bar-123", not
	// the numeric ID).
	// Optional; defaults to provided build environment's project ID ("name").
	ProjectID string

	// TLS optionally specifies the TLS keypair to use.
	// If zero, http without auth is used.
	TLS KeyPair

	// Optional description of the VM.
	Description string

	// Optional metadata to put on the instance.
	Meta map[string]string

	// DeleteIn optionally specifies a duration at which
	// to delete the VM.
	// If zero, a reasonable default is used.
	// Negative means no deletion timeout.
	DeleteIn time.Duration

	// OnInstanceRequested optionally specifies a hook to run synchronously
	// after the computeService.Instances.Insert call, but before
	// waiting for its operation to proceed.
	OnInstanceRequested func()

	// OnInstanceCreated optionally specifies a hook to run synchronously
	// after the instance operation succeeds.
	OnInstanceCreated func()

	// OnInstanceCreated optionally specifies a hook to run synchronously
	// after the computeService.Instances.Get call.
	// Only valid for GCE resources.
	OnGotInstanceInfo func(*compute.Instance)

	// OnInstanceCreated optionally specifies a hook to run synchronously
	// after the EC2 instance information is retrieved.
	// Only valid for EC2 resources.
	OnGotEC2InstanceInfo func(*cloud.Instance)

	// OnBeginBuildletProbe optionally specifies a hook to run synchronously
	// before StartNewVM tries to hit buildletURL to see if it's up yet.
	OnBeginBuildletProbe func(buildletURL string)

	// OnEndBuildletProbe optionally specifies a hook to run synchronously
	// after StartNewVM tries to hit the buildlet's URL to see if it's up.
	// The hook parameters are the return values from http.Get.
	OnEndBuildletProbe func(*http.Response, error)

	// SkipEndpointVerification does not verify that the builder is listening
	// on port 80 or 443 before creating a buildlet client.
	SkipEndpointVerification bool
}

VMOpts control how new VMs are started.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.