placemat

package module
v1.5.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 29, 2020 License: MIT Imports: 36 Imported by: 0

README

GitHub release CircleCI GoDoc Go Report Card

Placemat

Placemat is a tool to simulate data center networks and servers using rkt Pods, QEMU/KVM virtual machines, and Linux networking stacks. Placemat can simulate virtually any kind of network topologies to help tests and experiments for software usually used in data centers.

Features

  • No daemons

    Placemat is a single binary executable. It just builds networks and virtual machines when it starts, and destroys them when it terminates. This simplicity makes placemat great for a continuous testing tool.

  • Declarative YAML

    Networks, virtual machines, and other kind of resources are defined in YAML files in a declarative fashion. Users need not mind the order of creation and/or destruction of resources.

  • Virtual BMC for IPMI power management

    Power on/off/reset of VMs can be done by IPMI commands. See virtual BMC for details.

  • Automation

    Placemat supports cloud-init and ignition to automate virtual machine initialization. Files on the host machine can be exported to guests as a VVFAT drive. QEMU disk images can be downloaded from remote HTTP servers.

    All of these help implementation of fully-automated tests.

  • UEFI

    Not only traditional BIOS, but placemat VMs can be booted in UEFI mode if OVMF is available.

Usage

This project provides these commands:

  • placemat is the main tool to build networks and virtual machines.
  • pmctl is a utility tool to control VMs and Pods.
placemat command

placemat reads all YAML files specified in command-line arguments, then creates resources defined in YAML. To destroy, just kill the process (by sending a signal or Control-C).

$ placemat [OPTIONS] YAML [YAML ...]

Options:
  -bmc-cert string
        Certificate file for BMC HTTPS servers.
  -bmc-key string
        Key file for BMC HTTPS servers.
  -cache-dir string
        directory for cache data
  -data-dir string
        directory to store data (default "/var/scratch/placemat")
  -debug
        show QEMU's and Pod's stdout and stderr
  -enable-virtfs
        enable VirtFS to share files between guest and host OS.
  -force
        force run with removal of garbage
  -graphic
        run QEMU with graphical console
  -listen-addr string
        listen address (default "127.0.0.1:10808")
  -run-dir string
        run directory (default "/tmp")
  -shared-dir string
        shared directory (default "/mnt/placemat")

If -cache-dir is not specified, the default will be /home/${SUDO_USER}/placemat_data if sudo is used for placemat. If sudo is not used, cache directory will be the same as -data-dir. -force is used for forced run. Remaining garbage, for example virtual networks, mounts, socket files will be removed.

pmctl command

pmctl is a command line tool to control VMs, Pods and Networks.

See pmctl

Getting started

Prerequisites

For Ubuntu or Debian, you can install them as follows:

$ sudo apt-get update
$ sudo apt-get install qemu-system-x86 qemu-utils ovmf picocom socat cloud-utils

As to rkt, obtain a deb (or rpm) package then install it as follows:

$ wget https://github.com/rkt/rkt/releases/download/v1.30.0/rkt_1.30.0-1_amd64.deb
$ sudo dpkg -i rkt_1.30.0-1_amd64.deb
Install placemat

You can choose go get or debian package for installation.

Install placemat and pmctl:

$ go get -u github.com/cybozu-go/placemat/pkg/placemat
$ go get -u github.com/cybozu-go/placemat/pkg/pmctl

or

$ wget https://github.com/cybozu-go/placemat/releases/download/v${VERSION}/placemat_${VERSION}_amd64.deb
$ sudo dpkg -i placemat_${VERSION}_amd64.deb
Run examples

See examples how to write YAML files.

To launch placemat from YAML files, run it with sudo as follows:

$ sudo $GOPATH/bin/placemat cluster.yml

To connect to a serial console of a VM, use pmctl node enter:

$ sudo $GOPATH/bin/pmctl node enter VM

This will launch picocom. To exit, type Ctrl-Q, then Ctrl-X.

Specification

See specifications under docs directory.

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CleanupNetworks 

func CleanupNetworks(r *Runtime, c *Cluster)

CleanupNetworks removes all remaining network resources.

func CleanupNodes 

func CleanupNodes(r *Runtime, nodes []*Node)

CleanupNodes cleans files created at runtime for QEMU.

func CleanupPods added in v1.1.0 

func CleanupPods(r *Runtime, pods []*Pod)

CleanupPods cleans file created at runtime for rkt.

Types

type Cluster 

type Cluster struct {
	Networks    []*Network
	Images      []*Image
	DataFolders []*DataFolder
	Nodes       []*Node
	Pods        []*Pod
	// contains filtered or unexported fields
}

Cluster is a set of resources in a virtual data center.

func ReadYaml 

func ReadYaml(r *bufio.Reader) (*Cluster, error)

ReadYaml reads a yaml file and constructs Cluster

func (*Cluster) Append 

func (c *Cluster) Append(other *Cluster) *Cluster

Append appends another cluster into the receiver.

func (*Cluster) Cleanup 

func (c *Cluster) Cleanup(r *Runtime) error

Cleanup remaining resources

func (*Cluster) GetDataFolder 

func (c *Cluster) GetDataFolder(name string) (*DataFolder, error)

GetDataFolder looks up the data folder by name. It returns non-nil error if the named folder is not found.

func (*Cluster) GetImage 

func (c *Cluster) GetImage(name string) (*Image, error)

GetImage looks up the image by name. It returns non-nil error if the named image is not found.

func (*Cluster) GetNetwork 

func (c *Cluster) GetNetwork(name string) (*Network, error)

GetNetwork looks up the network by name. It returns non-nil error if the named network is not found.

func (*Cluster) GetNode 

func (c *Cluster) GetNode(name string) (*Node, error)

GetNode looks up the node by name. It returns non-nil error if the named node is not found.

func (*Cluster) GetPod 

func (c *Cluster) GetPod(name string) (*Pod, error)

GetPod looks up the pod by name. It returns non-nil error if the named pod is not found.

func (*Cluster) Resolve 

func (c *Cluster) Resolve() error

Resolve resolves inter-resource references and checks unique constraints.

func (*Cluster) Start 

func (c *Cluster) Start(ctx context.Context, r *Runtime) error

Start constructs the virtual data center with given resources. It stop when ctx is cancelled.

type DataFolder 

type DataFolder struct {
	*DataFolderSpec
	// contains filtered or unexported fields
}

DataFolder represents a data folder configuration

func NewDataFolder 

func NewDataFolder(spec *DataFolderSpec) (*DataFolder, error)

NewDataFolder creates DataFolder from DataFolderSpec.

func (*DataFolder) Path 

func (d *DataFolder) Path() string

Path returns the filesystem path to the directory having folder contents.

func (*DataFolder) Prepare 

func (d *DataFolder) Prepare(ctx context.Context, baseDir string, c *cache) error

Prepare copies or downloads necessary files to prepare folder contents.

type DataFolderFileSpec 

type DataFolderFileSpec struct {
	Name string `json:"name"`
	URL  string `json:"url,omitempty"`
	File string `json:"file,omitempty"`
}

DataFolderFileSpec represents a DataFolder's File definition in YAML

type DataFolderSpec 

type DataFolderSpec struct {
	Kind  string               `json:"kind"`
	Name  string               `json:"name"`
	Dir   string               `json:"dir,omitempty"`
	Files []DataFolderFileSpec `json:"files,omitempty"`
}

DataFolderSpec represents a DataFolder definition in YAML

type Decompressor 

type Decompressor interface {
	Decompress(io.Reader) (io.Reader, error)
}

Decompressor defines an interface to decompress data from io.Reader.

func NewDecompressor 

func NewDecompressor(format string) (Decompressor, error)

NewDecompressor returns a Decompressor for "format". If format is not supported, this returns a non-nil error.

type Image 

type Image struct {
	*ImageSpec
	// contains filtered or unexported fields
}

Image represents an image configuration

func NewImage 

func NewImage(spec *ImageSpec) (*Image, error)

NewImage creates *Image from spec.

func (*Image) Path 

func (i *Image) Path() string

Path returns the filesystem path to the image file.

func (*Image) Prepare 

func (i *Image) Prepare(ctx context.Context, c *cache) error

Prepare downloads the image if it is not in the cache.

type ImageSpec 

type ImageSpec struct {
	Kind              string `json:"kind"`
	Name              string `json:"name"`
	URL               string `json:"url,omitempty"`
	File              string `json:"file,omitempty"`
	CompressionMethod string `json:"compression,omitempty"`
}

ImageSpec represents an Image specification in YAML.

type Network 

type Network struct {
	*NetworkSpec
	// contains filtered or unexported fields
}

Network represents a network configuration

func NewNetwork 

func NewNetwork(spec *NetworkSpec) (*Network, error)

NewNetwork creates *Network from spec.

func (*Network) Create 

func (n *Network) Create(ng *nameGenerator) error

Create creates a virtual L2 switch using Linux bridge.

func (*Network) CreateTap 

func (n *Network) CreateTap() (string, error)

CreateTap add a tap device to the bridge and return the tap device name.

func (*Network) CreateVeth 

func (n *Network) CreateVeth() (string, string, error)

CreateVeth creates a veth pair and add one of the pair to the bridge. It returns both names of the pair.

func (*Network) Destroy 

func (n *Network) Destroy() error

Destroy deletes all created tap and veth devices, then the bridge.

type NetworkSpec 

type NetworkSpec struct {
	Kind    string `json:"kind"`
	Name    string `json:"name"`
	Type    string `json:"type"`
	UseNAT  bool   `json:"use-nat"`
	Address string `json:"address,omitempty"`
}

NetworkSpec represents a Network specification in YAML

type NetworkType 

type NetworkType int

NetworkType represents a network type. 

const (
	NetworkInternal NetworkType = iota
	NetworkExternal
	NetworkBMC
)

Network types.

type Node 

type Node struct {
	*NodeSpec
	// contains filtered or unexported fields
}

Node represents a virtual machine.

func NewNode 

func NewNode(spec *NodeSpec) (*Node, error)

NewNode creates a Node from spec.

func (*Node) Resolve 

func (n *Node) Resolve(c *Cluster) error

Resolve resolves references to other resources in the cluster.

func (*Node) Start 

func (n *Node) Start(ctx context.Context, r *Runtime, nodeCh chan<- bmcInfo) (*NodeVM, error)

Start starts the Node as a QEMU process. This will not wait the process termination; instead, it returns the process information.

func (*Node) StartSWTPM added in v1.3.6 

func (n *Node) StartSWTPM(ctx context.Context, r *Runtime) error

StartSWTPM starts swtpm with software TPM socket

type NodeSpec 

type NodeSpec struct {
	Kind         string           `json:"kind"`
	Name         string           `json:"name"`
	Interfaces   []string         `json:"interfaces,omitempty"`
	Volumes      []NodeVolumeSpec `json:"volumes,omitempty"`
	IgnitionFile string           `json:"ignition,omitempty"`
	CPU          int              `json:"cpu,omitempty"`
	Memory       string           `json:"memory,omitempty"`
	UEFI         bool             `json:"uefi,omitempty"`
	TPM          bool             `json:"tpm,omitempty"`
	SMBIOS       SMBIOSConfig     `json:"smbios,omitempty"`
}

NodeSpec represents a Node specification in YAML

type NodeVM 

type NodeVM struct {
	// contains filtered or unexported fields
}

NodeVM holds resources to manage and monitor a QEMU process.

func (*NodeVM) IsRunning 

func (n *NodeVM) IsRunning() bool

IsRunning returns true if the VM is running.

func (*NodeVM) ListSnapshots added in v1.3.1 

func (n *NodeVM) ListSnapshots(ctx context.Context, node *Node) (string, error)

ListSnapshots returns all available snapshots of VM

func (*NodeVM) LoadVM added in v1.3.0 

func (n *NodeVM) LoadVM(ctx context.Context, node *Node, tag string) error

LoadVM loads a snapshot of the VM. To load a snapshot, localds and vvfat devices have to be detached. NOTE: virtio block device does not support hot add. After loading snapshot, you will no longer access mounted block device other than rootfs. 

https://github.com/ceph/qemu-kvm/blob/de4eb6c5347e40b02dbe72cda18b58654ad11242/hw/pci-hotplug.c#L143

func (*NodeVM) PowerOff 

func (n *NodeVM) PowerOff() error

PowerOff turns off the power of the VM.

func (*NodeVM) PowerOn 

func (n *NodeVM) PowerOn() error

PowerOn turns on the power of the VM.

func (*NodeVM) ResumeVM added in v1.3.4 

func (n *NodeVM) ResumeVM(ctx context.Context) error

ResumeVM resumes VM

func (*NodeVM) SaveVM added in v1.3.0 

func (n *NodeVM) SaveVM(ctx context.Context, node *Node, tag string) error

SaveVM saves a snapshot of the VM. To save a snapshot, localds and vvfat devices have to be detached. NOTE: virtio block device does not support hot add. After saving snapshot, you will no longer access mounted block device other than rootfs. 

https://github.com/ceph/qemu-kvm/blob/de4eb6c5347e40b02dbe72cda18b58654ad11242/hw/pci-hotplug.c#L143

type NodeVolume 

type NodeVolume interface {
	Kind() string
	Name() string
	Resolve(*Cluster) error
	Create(context.Context, string) ([]string, error)
}

NodeVolume defines the interface for Node volumes.

func NewImageVolume 

func NewImageVolume(name string, cache string, imageName string, cow bool) NodeVolume

NewImageVolume creates a volume for type "image".

func NewLVVolume added in v1.5.0 

func NewLVVolume(name string, cache string, size, vg string) NodeVolume

NewLVVolume creates a volume for type "lv".

func NewLocalDSVolume 

func NewLocalDSVolume(name, cache string, u, n string) NodeVolume

NewLocalDSVolume creates a volume for type "localds".

func NewRawVolume 

func NewRawVolume(name string, cache string, size, format string) NodeVolume

NewRawVolume creates a volume for type "raw".

func NewVVFATVolume 

func NewVVFATVolume(name string, folderName string) NodeVolume

NewVVFATVolume creates a volume for type "vvfat".

type NodeVolumeSpec 

type NodeVolumeSpec struct {
	Kind          string `json:"kind"`
	Name          string `json:"name"`
	Image         string `json:"image,omitempty"`
	UserData      string `json:"user-data,omitempty"`
	NetworkConfig string `json:"network-config,omitempty"`
	Size          string `json:"size,omitempty"`
	Folder        string `json:"folder,omitempty"`
	CopyOnWrite   bool   `json:"copy-on-write,omitempty"`
	Cache         string `json:"cache,omitempty"`
	Format        string `json:"format,omitempty"`
	VG            string `json:"vg,omitempty"`
}

NodeVolumeSpec represents a Node's Volume specification in YAML

type Pod 

type Pod struct {
	*PodSpec
	// contains filtered or unexported fields
}

Pod represents a pod resource.

func NewPod 

func NewPod(spec *PodSpec) (*Pod, error)

NewPod creates a Pod from spec.

func (*Pod) Prepare 

func (p *Pod) Prepare(ctx context.Context) error

Prepare fetches container images to run Pod.

func (*Pod) Resolve 

func (p *Pod) Resolve(c *Cluster) error

Resolve resolves references to other resources in the cluster.

func (*Pod) Start 

func (p *Pod) Start(ctx context.Context, r *Runtime) error

Start starts the Pod using rkt. It does not return until the process finishes or ctx is cancelled.

type PodAppMountSpec 

type PodAppMountSpec struct {
	Volume string `json:"volume"`
	Target string `json:"target"`
}

PodAppMountSpec represents a App's Mount definition in YAML

type PodAppSpec 

type PodAppSpec struct {
	Name           string            `json:"name"`
	Image          string            `json:"image"`
	ReadOnlyRootfs bool              `json:"readonly-rootfs"`
	User           string            `json:"user,omitempty"`
	Group          string            `json:"group,omitempty"`
	Exec           string            `json:"exec,omitempty"`
	Args           []string          `json:"args,omitempty"`
	Env            map[string]string `json:"env,omitempty"`
	CapsRetain     []string          `json:"caps-retain,omitempty"`
	Mount          []PodAppMountSpec `json:"mount,omitempty"`
}

PodAppSpec represents a Pod's App definition in YAML

type PodInterfaceSpec 

type PodInterfaceSpec struct {
	Network   string   `json:"network"`
	Addresses []string `json:"addresses,omitempty"`
}

PodInterfaceSpec represents a Pod's Interface definition in YAML

type PodSpec 

type PodSpec struct {
	Kind        string             `json:"kind"`
	Name        string             `json:"name"`
	InitScripts []string           `json:"init-scripts,omitempty"`
	Interfaces  []PodInterfaceSpec `json:"interfaces,omitempty"`
	Volumes     []*PodVolumeSpec   `json:"volumes,omitempty"`
	Apps        []*PodAppSpec      `json:"apps"`
}

PodSpec represents a Pod specification in YAML

type PodVolume 

type PodVolume interface {
	// Name returns the volume name.
	Name() string
	// Resolve resolves references in the volume definition.
	Resolve(*Cluster) error
	// Spec returns a command-line argument for the volume.
	Spec() string
}

PodVolume is an interface of a volume for Pod.

func NewPodVolume 

func NewPodVolume(spec *PodVolumeSpec) (PodVolume, error)

NewPodVolume makes a PodVolume, or returns an error.

type PodVolumeSpec 

type PodVolumeSpec struct {
	Name     string `json:"name"`
	Kind     string `json:"kind"`
	Folder   string `json:"folder,omitempty"`
	ReadOnly bool   `json:"readonly"`
	Mode     string `json:"mode,omitempty"`
	UID      string `json:"uid,omitempty"`
	GID      string `json:"gid,omitempty"`
}

PodVolumeSpec represents a Pod's Volume definition in YAML

type Runtime 

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime contains the runtime information to run Cluster.

func NewRuntime 

func NewRuntime(force, graphic, enableVirtFS bool, runDir, dataDir, cacheDir, sharedDir, listenAddr, bmcCert, bmcKey string) (*Runtime, error)

NewRuntime initializes a new Runtime.

type SMBIOSConfig 

type SMBIOSConfig struct {
	Manufacturer string `json:"manufacturer,omitempty"`
	Product      string `json:"product,omitempty"`
	Serial       string `json:"serial,omitempty"`
}

SMBIOSConfig represents a Node's SMBIOS definition in YAML

type Server added in v1.1.0 

type Server struct {
	// contains filtered or unexported fields
}

Server is the API Server of placemat.

func NewServer added in v1.1.0 

func NewServer(cluster *Cluster, vms map[string]*NodeVM, r *Runtime) *Server

NewServer creates a new Server instance.

func (Server) ServeHTTP added in v1.1.0 

func (s Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

Handler implements http.Handler

Source Files

View all Source files

Directories

Path Synopsis
pkg
placemat
pmctl
pmctl/cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.