buildenv

README

golang.org/x/build/buildenv

Package buildenv contains definitions for the environments the Go build system can run in.

Documentation

Overview

Package buildenv contains definitions for the environments the Go build system can run in.

Index

• Variables
• func CheckUserCredentials()
• func RegisterFlags()
• func RegisterStagingFlag()
• type Environment
  • func ByProjectID(projectID string) *Environment
  • func FromFlags() *Environment
  • func (e Environment) ComputePrefix() string
  • func (e Environment) Credentials(ctx context.Context) (*google.Credentials, error)
  • func (e Environment) DashBase() string
  • func (e Environment) RandomVMZone() string
  • func (e Environment) Region() string
  • func (e Environment) SnapshotURL(builderType, rev string) string
• type KubeConfig

Variables

var Development = &Environment{
	GoProjectName: "golang-org",
	IsProd:        false,
	StaticIP:      "127.0.0.1",
}

var Production = &Environment{
	ProjectName:           "symbolic-datum-552",
	ProjectNumber:         872405196845,
	GoProjectName:         "golang-org",
	IsProd:                true,
	ControlZone:           "us-central1-f",
	VMZones:               []string{"us-central1-a", "us-central1-b", "us-central1-c", "us-central1-f"},
	StaticIP:              "107.178.219.46",
	MachineType:           "n1-standard-4",
	PreferContainersOnCOS: true,
	KubeBuild: KubeConfig{
		MinNodes:    2,
		MaxNodes:    2,
		Name:        "buildlets",
		MachineType: "n1-standard-4",
	},
	KubeTools: KubeConfig{
		MinNodes:    4,
		MaxNodes:    4,
		Name:        "go",
		MachineType: "n1-standard-4",
	},
	DashURL:             "https://build.golang.org/",
	PerfDataURL:         "https://perfdata.golang.org",
	CoordinatorName:     "farmer",
	BuildletBucket:      "go-builder-data",
	LogBucket:           "go-build-log",
	SnapBucket:          "go-build-snap",
	AutoCertCacheBucket: "farmer-golang-org-autocert-cache",
	COSServiceAccount:   "linux-cos-builders@symbolic-datum-552.iam.gserviceaccount.com",
	AWSSecurityGroup:    "go-builders",
	AWSRegion:           "us-east-2",
}

Production defines the environment that the coordinator and build infrastructure is deployed to for production usage at build.golang.org.

var Staging = &Environment{
	ProjectName:           "go-dashboard-dev",
	ProjectNumber:         302018677728,
	GoProjectName:         "go-dashboard-dev",
	IsProd:                true,
	ControlZone:           "us-central1-f",
	VMZones:               []string{"us-central1-a", "us-central1-b", "us-central1-c", "us-central1-f"},
	StaticIP:              "104.154.113.235",
	MachineType:           "n1-standard-1",
	PreferContainersOnCOS: true,
	KubeBuild: KubeConfig{
		MinNodes:    1,
		MaxNodes:    1,
		Name:        "buildlets",
		MachineType: "n1-standard-4",
	},
	KubeTools: KubeConfig{
		MinNodes:    3,
		MaxNodes:    3,
		Name:        "go",
		MachineType: "n1-standard-4",
	},
	DashURL:           "https://go-dashboard-dev.appspot.com/",
	PerfDataURL:       "https://perfdata.golang.org",
	CoordinatorName:   "farmer",
	BuildletBucket:    "dev-go-builder-data",
	LogBucket:         "dev-go-build-log",
	SnapBucket:        "dev-go-build-snap",
	COSServiceAccount: "linux-cos-builders@go-dashboard-dev.iam.gserviceaccount.com",
	AWSSecurityGroup:  "staging-go-builders",
	AWSRegion:         "us-east-1",
}

Staging defines the environment that the coordinator and build infrastructure is deployed to before it is released to production. For local dev, override the project with the program's flag to set a custom project.

Functions

func CheckUserCredentials

func CheckUserCredentials()

CheckUserCredentials warns if the gcloud Application Default Credentials file doesn't exist and says how to log in properly.

func RegisterFlags

func RegisterFlags()

RegisterFlags registers the "staging" and "localdev" flags.

func RegisterStagingFlag

func RegisterStagingFlag()

RegisterStagingFlag registers the "staging" flag.

Types

type Environment

type Environment struct {
	// The GCP project name that the build infrastructure will be provisioned in.
	// This field may be overridden as necessary without impacting other fields.
	ProjectName string

	// ProjectNumber is the GCP build infrastructure project's number, as visible
	// in the admin console. This is used for things such as constructing the
	// "email" of the default service account.
	ProjectNumber int64

	// The GCP project name for the Go project, where build status is stored.
	// This field may be overridden as necessary without impacting other fields.
	GoProjectName string

	// The IsProd flag indicates whether production functionality should be
	// enabled. When true, GCE and Kubernetes builders are enabled and the
	// coordinator serves on 443. Otherwise, GCE and Kubernetes builders are
	// disabled and the coordinator serves on 8119.
	IsProd bool

	// ControlZone is the GCE zone that the coordinator instance and Kubernetes cluster
	// will run in. This field may be overridden as necessary without impacting
	// other fields.
	ControlZone string

	// VMZones are the GCE zones that the VMs will be deployed to. These
	// GCE zones will be periodically cleaned by deleting old VMs. The zones
	// should all exist within a single region.
	VMZones []string

	// StaticIP is the public, static IP address that will be attached to the
	// coordinator instance. The zero value means the address will be looked
	// up by name. This field is optional.
	StaticIP string

	// MachineType is the GCE machine type to use for the coordinator.
	MachineType string

	// KubeBuild is the Kubernetes config for the buildlet cluster.
	KubeBuild KubeConfig
	// KubeTools is the Kubernetes config for the tools cluster.
	KubeTools KubeConfig

	// PreferContainersOnCOS controls whether we do most builds on
	// Google's Container-Optimized OS Linux image running on a VM
	// rather than using Kubernetes for builds. This does not
	// affect cross-compiled builds just running make.bash. Those
	// still use Kubernetes for now.
	// See https://golang.org/issue/25108.
	PreferContainersOnCOS bool

	// DashURL is the base URL of the build dashboard, ending in a slash.
	DashURL string

	// PerfDataURL is the base URL of the benchmark storage server.
	PerfDataURL string

	// CoordinatorName is the hostname of the coordinator instance.
	CoordinatorName string

	// BuildletBucket is the GCS bucket that stores buildlet binaries.
	// TODO: rename. this is not just for buildlets; also for bootstrap.
	BuildletBucket string

	// LogBucket is the GCS bucket to which logs are written.
	LogBucket string

	// SnapBucket is the GCS bucket to which snapshots of
	// completed builds (after make.bash, before tests) are
	// written.
	SnapBucket string

	// MaxBuilds is the maximum number of concurrent builds that
	// can run. Zero means unlimited. This is typically only used
	// in a development or staging environment.
	MaxBuilds int

	// AutoCertCacheBucket is the GCS bucket to use for the
	// golang.org/x/crypto/acme/autocert (LetsEncrypt) cache.
	// If empty, LetsEncrypt isn't used.
	AutoCertCacheBucket string

	// COSServiceAccount (Container Optimized OS) is the service
	// account that will be assigned to a VM instance that hosts
	// a container when the instance is created.
	COSServiceAccount string

	// AWSSecurityGroup is the security group name that any VM instance
	// created on EC2 should contain. These security groups are
	// collections of firewall rules to be applied to the VM.
	AWSSecurityGroup string

	// AWSRegion is the region where AWS resources are deployed.
	AWSRegion string
}

Environment describes the configuration of the infrastructure for a coordinator and its buildlet resources running on Google Cloud Platform. Staging and Production are the two common build environments.

func ByProjectID

func ByProjectID(projectID string) *Environment

ByProjectID returns an Environment for the specified project ID. It is currently limited to the symbolic-datum-552 and go-dashboard-dev projects. ByProjectID will panic if the project ID is not known.

func FromFlags

func FromFlags() *Environment

FromFlags returns the build environment specified from flags, as registered by RegisterFlags or RegisterStagingFlag. By default it returns the production environment.

func (Environment) ComputePrefix

func (e Environment) ComputePrefix() string

ComputePrefix returns the URI prefix for Compute Engine resources in a project.

func (Environment) Credentials

func (e Environment) Credentials(ctx context.Context) (*google.Credentials, error)

Credentials returns the credentials required to access the GCP environment with the necessary scopes.

func (Environment) DashBase

func (e Environment) DashBase() string

DashBase returns the base URL of the build dashboard, ending in a slash.

func (Environment) RandomVMZone

func (e Environment) RandomVMZone() string

RandomVMZone returns a randomly selected zone from the zones in VMZones. The Zone value will be returned if VMZones is not set.

func (Environment) Region

func (e Environment) Region() string

Region returns the GCE region, derived from its zone.

func (Environment) SnapshotURL

func (e Environment) SnapshotURL(builderType, rev string) string

SnapshotURL returns the absolute URL of the .tar.gz containing a built Go tree for the builderType and Go rev (40 character Git commit hash). The tarball is suitable for passing to (*buildlet.Client).PutTarFromURL.

type KubeConfig

type KubeConfig struct {
	// MinNodes is the minimum number of nodes in the Kubernetes cluster.
	// The autoscaler will ensure that at least this many nodes is always
	// running despite any scale-down decision.
	MinNodes int64

	// MaxNodes is the maximum number of nodes that the autoscaler can
	// provision in the Kubernetes cluster.
	// If MaxNodes is 0, Kubernetes is not used.
	MaxNodes int64

	// MachineType is the GCE machine type to use for the Kubernetes cluster nodes.
	MachineType string

	// Name is the name of the Kubernetes cluster that will be created.
	Name string
}

KubeConfig describes the configuration of a Kubernetes cluster.