launch

package
v0.0.0-...-5fb8a3f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 30, 2024 License: Apache-2.0 Imports: 46 Imported by: 0

Documentation

Overview

cluster builds on the launch package and implements launching Metropolis nodes and clusters in a virtualized environment using qemu. It's kept in a separate package as it depends on a Metropolis node image, which might not be required for some use of the launch library.

Index

Constants

View Source 
const SOCKSPort uint16 = 1080

Variables

View Source 
var (
	// InsecurePrivateKey is a ED25519 key that can be used during development as
	// a fixed Owner Key when bootstrapping new clusters.
	InsecurePrivateKey = ed25519.NewKeyFromSeed([]byte(
		"\xb5\xcf\x35\x0f\xbf\xfb\xea\xfa\xa0\xf0\x29\x9d\xfa\xf7\xca\x6f" +
			"\xa2\xc2\xc7\x87\xd7\x03\x3e\xb2\x11\x4f\x36\xe0\x22\x73\x4f\x87"))
	// InsecurePublicKey is the ED25519 public key corresponding to
	// InsecurePrivateKey.
	InsecurePublicKey = InsecurePrivateKey.Public().(ed25519.PublicKey)
	// InsecureClusterBootstrap is a ClusterBootstrap message to be used within
	// NodeParameters when bootstrapping a development cluster that should be owned
	// by the InsecurePrivateKey.
	InsecureClusterBootstrap = &apb.NodeParameters_ClusterBootstrap{
		OwnerPublicKey: InsecurePublicKey,
	}
)
View Source 
var ClusterPorts = []uint16{

	uint16(node.CuratorServicePort),
	uint16(node.DebugServicePort),
	uint16(node.KubernetesAPIPort),
	uint16(node.KubernetesAPIWrappedPort),

	SOCKSPort,
}

ClusterPorts contains all ports handled by Nanoswitch.

View Source 
var NodePorts = []node.Port{
	node.ConsensusPort,

	node.CuratorServicePort,
	node.DebugServicePort,

	node.KubernetesAPIPort,
	node.KubernetesAPIWrappedPort,
	node.CuratorServicePort,
	node.DebuggerPort,
	node.MetricsPort,
}

NodePorts is the list of ports a fully operational Metropolis node listens on

Functions

func LaunchNode 

func LaunchNode(ctx context.Context, ld, sd string, tpmFactory *TPMFactory, options *NodeOptions, doneC chan error) error

LaunchNode launches a single Metropolis node instance with the given options. The instance runs mostly paravirtualized but with some emulated hardware similar to how a cloud provider might set up its VMs. The disk is fully writable, and the changes are kept across reboots and shutdowns. ld and sd point to the launch directory and the socket directory, holding the nodes' state files (storage, tpm state, firmware state), and UNIX socket files (swtpm <-> QEMU interplay) respectively. The directories must exist before LaunchNode is called. LaunchNode will update options.Runtime and options.Mac if either are not initialized.

func NewSerialFileLogger 

func NewSerialFileLogger(p string) (io.ReadWriter, error)

Types

type Cluster 

type Cluster struct {
	// Owner is the TLS Certificate of the owner of the test cluster. This can be
	// used to authenticate further clients to the running cluster.
	Owner tls.Certificate
	// Ports is the PortMap used to access the first nodes' services (defined in
	// ClusterPorts) and the SOCKS proxy (at SOCKSPort).
	Ports launch.PortMap

	// Nodes is a map from Node ID to its runtime information.
	Nodes map[string]*NodeInCluster
	// NodeIDs is a list of node IDs that are backing this cluster, in order of
	// creation.
	NodeIDs []string

	// CACertificate is the cluster's CA certificate.
	CACertificate *x509.Certificate

	// SOCKSDialer is used by DialNode to establish connections to nodes via the
	// SOCKS server ran by nanoswitch.
	SOCKSDialer proxy.Dialer
	// contains filtered or unexported fields
}

Cluster is the running Metropolis cluster launched using the LaunchCluster function.

func LaunchCluster 

func LaunchCluster(ctx context.Context, opts ClusterOptions) (*Cluster, error)

LaunchCluster launches a cluster of Metropolis node VMs together with a Nanoswitch instance to network them all together.

The given context will be used to run all qemu instances in the cluster, and canceling the context or calling Close() will terminate them.

func (*Cluster) AllNodesHealthy 

func (c *Cluster) AllNodesHealthy(ctx context.Context) error

AllNodesHealthy returns nil if all the nodes in the cluster are seemingly healthy.

func (*Cluster) ApproveNode 

func (c *Cluster) ApproveNode(ctx context.Context, id string) error

ApproveNode approves a node by ID, waiting for it to become UP.

func (*Cluster) Close 

func (c *Cluster) Close() error

Close cancels the running clusters' context and waits for all virtualized nodes to stop. It returns an error if stopping the nodes failed, or one of the nodes failed to fully start in the first place.

func (*Cluster) ConnectOptions 

func (c *Cluster) ConnectOptions() *metroctl.ConnectOptions

ConnectOptions returns metroctl.ConnectOptions that describe connectivity to the launched cluster.

func (*Cluster) CuratorClient 

func (c *Cluster) CuratorClient() (*grpc.ClientConn, error)

CuratorClient returns an authenticated owner connection to a Curator instance within Cluster c, or nil together with an error.

func (*Cluster) DialNode 

func (c *Cluster) DialNode(_ context.Context, addr string) (net.Conn, error)

DialNode is a grpc.WithContextDialer compatible dialer which dials nodes by their ID. This is performed by connecting to the cluster nanoswitch via its SOCKS proxy, and using the cluster node list for name resolution.

For example: 

grpc.Dial("metropolis-deadbeef:1234", grpc.WithContextDialer(c.DialNode))

func (*Cluster) GetKubeClientSet 

func (c *Cluster) GetKubeClientSet() (kubernetes.Interface, error)

GetKubeClientSet gets a Kubernetes client set accessing the Metropolis Kubernetes authenticating proxy using the cluster owner identity. It currently has access to everything (i.e. the cluster-admin role) via the owner-admin binding.

func (*Cluster) KubernetesControllerNodeAddresses 

func (c *Cluster) KubernetesControllerNodeAddresses(ctx context.Context) ([]string, error)

KubernetesControllerNodeAddresses returns the list of IP addresses of nodes which are currently Kubernetes controllers, ie. run an apiserver. This list might be empty if no node is currently configured with the 'KubernetesController' node.

func (*Cluster) MakeConsensusMember 

func (c *Cluster) MakeConsensusMember(ctx context.Context, id string) error

MakeConsensusMember adds the ConsensusMember role to a node by ID.

func (*Cluster) MakeKubernetesController 

func (c *Cluster) MakeKubernetesController(ctx context.Context, id string) error

MakeKubernetesController adds the KubernetesController role to a node by ID.

func (*Cluster) MakeKubernetesWorker 

func (c *Cluster) MakeKubernetesWorker(ctx context.Context, id string) error

MakeKubernetesWorker adds the KubernetesWorker role to a node by ID.

func (*Cluster) MakeMetroctlWrapper 

func (c *Cluster) MakeMetroctlWrapper() (string, error)

MakeMetroctlWrapper builds and returns the path to a shell script which calls metroctl (from //metropolis/cli/metroctl, which must be included as a data dependency of the built target) with all the required flags to connect to the launched cluster.

func (*Cluster) MetroctlFlags 

func (c *Cluster) MetroctlFlags() string

MetroctlFlags return stringified flags to pass to a metroctl binary to connect to the launched cluster.

func (*Cluster) RebootNode 

func (c *Cluster) RebootNode(ctx context.Context, idx int) error

RebootNode reboots the cluster member node matching the given index, and waits for it to rejoin the cluster. It will use the given context ctx to run cluster API requests, whereas the resulting QEMU process will be created using the cluster's context c.ctxT. The nodes are indexed starting at 0.

func (*Cluster) ShutdownNode 

func (c *Cluster) ShutdownNode(idx int) error

ShutdownNode performs an ungraceful shutdown (i.e. power off) of the node given by idx. If the node is already shut down, this is a no-op.

func (*Cluster) StartNode 

func (c *Cluster) StartNode(idx int) error

StartNode performs a power on of the node given by idx. If the node is already running, this is a no-op.

type ClusterOptions 

type ClusterOptions struct {
	// The number of nodes this cluster should be started with.
	NumNodes int

	// Node are default options of all nodes.
	Node NodeOptions

	// If true, node logs will be saved to individual files instead of being printed
	// out to stderr. The path of these files will be still printed to stdout.
	//
	// The files will be located within the launch directory inside TEST_TMPDIR (or
	// the default tempdir location, if not set).
	NodeLogsToFiles bool

	// LeaveNodesNew, if set, will leave all non-bootstrap nodes in NEW, without
	// bootstrapping them. The nodes' address information in Cluster.Nodes will be
	// incomplete.
	LeaveNodesNew bool

	// Optional local registry which will be made available to the cluster to
	// pull images from. This is a more efficient alternative to preseeding all
	// images used for testing.
	LocalRegistry *localregistry.Server

	// InitialClusterConfiguration will be passed to the first node when creating the
	// cluster, and defines some basic properties of the cluster. If not specified,
	// the cluster will default to defaults as defined in
	// metropolis.proto.api.NodeParameters.
	InitialClusterConfiguration *cpb.ClusterConfiguration
}

ClusterOptions contains all options for launching a Metropolis cluster.

type NodeInCluster 

type NodeInCluster struct {
	// ID of the node, which can be used to dial this node's services via DialNode.
	ID     string
	Pubkey []byte
	// Address of the node on the network ran by nanoswitch. Not reachable from the
	// host unless dialed via DialNode or via the nanoswitch SOCKS proxy (reachable
	// on Cluster.Ports[SOCKSPort]).
	ManagementAddress string
}

NodeInCluster represents information about a node that's part of a Cluster.

type NodeOptions 

type NodeOptions struct {
	// Name is a human-readable identifier to be used in debug output.
	Name string

	// CPUs is the number of virtual CPUs of the VM.
	CPUs int

	// ThreadsPerCPU is the number of threads per CPU. This is multiplied by
	// CPUs to get the total number of threads.
	ThreadsPerCPU int

	// MemoryMiB is the RAM size in MiB of the VM.
	MemoryMiB int

	// DiskBytes contains the size of the root disk in bytes or zero if the
	// unmodified image size is used.
	DiskBytes uint64

	// Ports contains the port mapping where to expose the internal ports of the VM to
	// the host. See IdentityPortMap() and ConflictFreePortMap(). Ignored when
	// ConnectToSocket is set.
	Ports launch.PortMap

	// If set to true, reboots are honored. Otherwise, all reboots exit the Launch()
	// command. Metropolis nodes generally restart on almost all errors, so unless you
	// want to test reboot behavior this should be false.
	AllowReboot bool

	// By default, the VM is connected to the Host via SLIRP. If ConnectToSocket is
	// set, it is instead connected to the given file descriptor/socket. If this is
	// set, all port maps from the Ports option are ignored. Intended for networking
	// this instance together with others for running more complex network
	// configurations.
	ConnectToSocket *os.File

	// When PcapDump is set, all traffic is dumped to a pcap file in the
	// runtime directory (e.g. "net0.pcap" for the first interface).
	PcapDump bool

	// SerialPort is an io.ReadWriter over which you can communicate with the serial
	// port of the machine. It can be set to an existing file descriptor (like
	// os.Stdout/os.Stderr) or any Go structure implementing this interface.
	SerialPort io.ReadWriter

	// NodeParameters is passed into the VM and subsequently used for bootstrapping or
	// registering into a cluster.
	NodeParameters *apb.NodeParameters

	// Mac is the node's MAC address.
	Mac *net.HardwareAddr

	// Runtime keeps the node's QEMU runtime state.
	Runtime *NodeRuntime

	// RunVNC starts a VNC socket for troubleshooting/testing console code. Note:
	// this will not work in tests, as those use a built-in qemu which does not
	// implement a VGA device.
	RunVNC bool
}

NodeOptions contains all options that can be passed to Launch()

type NodeRuntime 

type NodeRuntime struct {

	// CtxC is the QEMU context's cancellation function.
	CtxC context.CancelFunc
	// contains filtered or unexported fields
}

NodeRuntime keeps the node's QEMU runtime options.

type TPMFactory 

type TPMFactory struct {
	// contains filtered or unexported fields
}

A TPMFactory manufactures virtual TPMs using swtpm.

A factory has an assigned state directory into which it will write per-factory data (like CA certificates and keys). Each manufactured TPM also has a state directory, which is first generated on manufacturing, and then passed to an swtpm instance.

func NewTPMFactory 

func NewTPMFactory(stateDir string) (*TPMFactory, error)

NewTPMFactory creates a new TPM factory at a given state path. The state path is a directory used to persist TPM factory data. It will be created if needed, and can be reused across TPM factories (but not used in parallel).

func (*TPMFactory) Manufacture 

func (f *TPMFactory) Manufacture(ctx context.Context, path string, platform *TPMPlatform) error

Manufacture builds a new TPM for a given platform at a path. The path points to a directory that will be created if it doens't exist yet, and can be passed to swtpm to actually emulate the created TPM.

type TPMPlatform 

type TPMPlatform struct {
	Manufacturer string
	Version      string
	Model        string
}

A TPMPlatform defines a platform that a TPM is part of. This will usually be some kind of device, in this case a virtual device.

Source Files

View all Source files

Directories

Path Synopsis
cli
launch
launch-cluster

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.