sabakan

package module
v2.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 18, 2019 License: MIT Imports: 12 Imported by: 4

README

GitHub release CircleCI GoDoc Go Report Card

Sabakan

sabakan architecture

Sabakan is a versatile network boot server designed for large on-premise data centers. Currently, it is made only for CoreOS Container Linux.

Project Status: GA (General Availability)

Features

  • High availability

    High availability of sabakan is just as easy as running multiple sabakan servers.

    Sabakan data are stored and shared in etcd. For example, DHCP lease information are shared between sabakan instances to avoid conflicts.

  • Machine inventory with IPAM (IP address management)

    Sabakan keeps an inventory of machines in a data center. Their IP addresses are automatically assigned by sabakan.

  • DHCP service

    Sabakan provides DHCP service that supports UEFI HTTP Boot and iPXE HTTP Boot. It also supports DHCP relay request to make DHCP service highly available.

  • HTTP service (network file server)

    Sabakan provides HTTP service for network boot clients. Users can upload any kind of files other than OS images to sabakan. Clients can download them to initialize the system after boot.

  • Template system for Ignition

    Ignition is a boot provisioning system for CoreOS Container Linux. Ignition configuration is not friendly for operators as it is written in a plain JSON.

    Sabakan provides a friendly and super versatile template system for Ignition configurations. For each client machine, sabakan renders Ignition configuration from templates.

  • Life-cycle management

    Machines in the inventory has a life-cycle status. The status can be changed through REST API. Users can build an automatic status controller to mark machines as unhealthy, unreachable, retiring, or retired.

  • Disk encryption support

    To help implementing full disk encryption on client machines, sabakan accepts and stores encrypted disk encryption keys. The key can be downloaded in the next boot to decrypt disks.

    sabakan-cryptsetup is a tool for clients to encrypt disks; the tool generates a disk encryption key, encrypts it, and sends the encrypted key to sabakan. In the next boot, it downloads the encrypted key from sabakan, decrypts it, then uses it to decrypt disks.

  • Audit logs

    To track problems and life-cycle events, sabakan keeps operation logs within its etcd storage.

Programs

This repository contains these programs:

  • sabakan: the network service to manage servers.
  • sabactl: CLI tool for sabakan.
  • sabakan-cryptsetup: a utility to encrypt a block device using dm-crypt.

To see their usage, run them with -h option.

Documentation

docs directory contains tutorials and specifications.

Read getting started first.

Examples

mtest/ directory contains a set of utilities to setup sabakan on Ubuntu virtual machines.

testadata/ directory contains a sample Ignition template.

An example of production usage can be found in github.com/cybozu-go/neco. The repository bootstraps a full data center system using etcd, vault, sabakan, and many other tools.

Usage

Run sabakan with docker

# create directory to store OS images
$ sudo mkdir -p /var/lib/sabakan

# -advertise-url is the canonical URL of this sabakan.
$ docker run -d --read-only --cap-drop ALL --cap-add NET_BIND_SERVICE \
    --network host --name sabakan \
    --mount type=bind,source=/var/lib/sabakan,target=/var/lib/sabakan \
    quay.io/cybozu/sabakan:2.2 \
    -etcd-endpoints http://foo.bar:2379,http://zot.bar:2379 \
    -advertise-url http://12.34.56.78:10080

License

Sabakan is licensed under MIT license.

Documentation

Index

Constants

View Source 
const (
	AuditAssets   = AuditCategory("assets")
	AuditCrypts   = AuditCategory("crypts")
	AuditDHCP     = AuditCategory("dhcp")
	AuditIgnition = AuditCategory("ignition")
	AuditImage    = AuditCategory("image")
	AuditIPAM     = AuditCategory("ipam")
	AuditIPXE     = AuditCategory("ipxe")
	AuditMachines = AuditCategory("machines")
)

Audit categories.

View Source 
const (
	AuditKeyUser = AuditContextKey("user")
	AuditKeyIP   = AuditContextKey("ip")
	AuditKeyHost = AuditContextKey("host")
)

Audit context keys. Values must be string.

View Source 
const (
	Ignition2_2 = IgnitionVersion("2.2")
	Ignition2_3 = IgnitionVersion("2.3")
)

Supported ignition versions

View Source 
const (
	// MaxImages is the maximum number of images that an index can hold.
	MaxImages = 5

	// ImageKernelFilename is a filename appear in TAR archive of an image.
	ImageKernelFilename = "kernel"

	// ImageInitrdFilename is a filename appear in TAR archive of an image.
	ImageInitrdFilename = "initrd.gz"
)
View Source 
const (
	StateUninitialized = MachineState("uninitialized")
	StateHealthy       = MachineState("healthy")
	StateUnhealthy     = MachineState("unhealthy")
	StateUnreachable   = MachineState("unreachable")
	StateUpdating      = MachineState("updating")
	StateRetiring      = MachineState("retiring")
	StateRetired       = MachineState("retired")
)

Machine state definitions.

View Source 
const DefaultLeaseDuration = 60 * time.Minute

DefaultLeaseDuration is 60 minutes.

View Source 
const SchemaVersion = "3"

SchemaVersion is the schema version

View Source 
const Version = "2.2.2"

Version is sabakan version

Variables

View Source 
var ErrBadRequest = errors.New("bad request")

ErrBadRequest is a special err for models. A model should return this when the request is bad

View Source 
var ErrConflicted = errors.New("key conflicted")

ErrConflicted is a special error for models. A model should return this when it fails to update a resource due to conflicts.

View Source 
var ErrNotFound = errors.New("not found")

ErrNotFound is a special err for models. A model should return this when it cannot find a resource by a specified key.

Functions

func IsValidBmcType 

func IsValidBmcType(bmcType string) bool

IsValidBmcType returns true if role is valid as BMC type

func IsValidIgnitionID 

func IsValidIgnitionID(id string) bool

IsValidIgnitionID returns true if id is valid as ignition ID

func IsValidImageID 

func IsValidImageID(id string) bool

IsValidImageID returns true if id is valid as an image ID.

func IsValidImageOS 

func IsValidImageOS(os string) bool

IsValidImageOS returns true if id is valid as OS.

func IsValidKernelParams 

func IsValidKernelParams(s string) bool

IsValidKernelParams returns true if s is valid as an kernel params

func IsValidLabelName 

func IsValidLabelName(name string) bool

IsValidLabelName returns true if label name is valid This is the same as the validation for Kubernetes label names.

func IsValidLabelValue 

func IsValidLabelValue(value string) bool

IsValidLabelValue returns true if label value is valid This is the same as the validation for Kubernetes label values.

func IsValidRole 

func IsValidRole(role string) bool

IsValidRole returns true if role is valid as machine role

Types

type Asset 

type Asset struct {
	Name        string            `json:"name"`
	ID          int               `json:"id,string"`
	ContentType string            `json:"content-type"`
	Date        time.Time         `json:"date"`
	Sha256      string            `json:"sha256"`
	URLs        []string          `json:"urls"`
	Exists      bool              `json:"exists"`
	Options     map[string]string `json:"options"`
}

Asset represents an asset.

type AssetHandler 

type AssetHandler interface {
	ServeContent(asset *Asset, content io.ReadSeeker)
	Redirect(url string)
}

AssetHandler is an interface for AssetModel.Get

type AssetModel 

type AssetModel interface {
	GetIndex(ctx context.Context) ([]string, error)
	GetInfo(ctx context.Context, name string) (*Asset, error)
	Put(ctx context.Context, name, contentType string, csum []byte, options map[string]string, r io.Reader) (*AssetStatus, error)
	Get(ctx context.Context, name string, h AssetHandler) error
	Delete(ctx context.Context, name string) error
}

AssetModel is an interface to manage assets.

type AssetStatus 

type AssetStatus struct {
	Status int `json:"status"`
	ID     int `json:"id,string"`
}

AssetStatus is the status of an asset.

type AuditCategory 

type AuditCategory string

AuditCategory is the type of audit categories.

type AuditContextKey 

type AuditContextKey string

AuditContextKey is the type of context keys for audit.

type AuditLog 

type AuditLog struct {
	Timestamp time.Time     `json:"ts"`
	Revision  int64         `json:"rev,string"`
	User      string        `json:"user"`
	IP        string        `json:"ip"`
	Host      string        `json:"host"`
	Category  AuditCategory `json:"category"`
	Instance  string        `json:"instance"`
	Action    string        `json:"action"`
	Detail    string        `json:"detail"`
}

AuditLog represents an audit log entry.

func NewAuditLog 

func NewAuditLog(ctx context.Context, ts time.Time, rev int64, cat AuditCategory,
	instance, action, detail string) *AuditLog

NewAuditLog creates an audit log entry and initializes it.

type BMCInfo 

type BMCInfo struct {
	IPv4 NICConfig `json:"ipv4"`
}

BMCInfo represents BMC NIC configuration information.

type DHCPConfig 

type DHCPConfig struct {
	LeaseMinutes uint     `json:"lease-minutes"`
	DNSServers   []string `json:"dns-servers,omitempty"`

	// obsoleted fields
	GatewayOffset uint `json:"gateway-offset"`
}

DHCPConfig is a set of DHCP configurations.

func (*DHCPConfig) LeaseDuration 

func (c *DHCPConfig) LeaseDuration() time.Duration

LeaseDuration returns lease duration for IP addreses.

func (*DHCPConfig) Validate 

func (c *DHCPConfig) Validate() error

Validate validates configurations

type DHCPModel 

type DHCPModel interface {
	PutConfig(ctx context.Context, config *DHCPConfig) error
	GetConfig() (*DHCPConfig, error)
	Lease(ctx context.Context, ifaddr net.IP, mac net.HardwareAddr) (net.IP, error)
	Renew(ctx context.Context, ciaddr net.IP, mac net.HardwareAddr) error
	Release(ctx context.Context, ciaddr net.IP, mac net.HardwareAddr) error
	Decline(ctx context.Context, ciaddr net.IP, mac net.HardwareAddr) error
}

DHCPModel is an interface for DHCPConfig.

type HealthModel 

type HealthModel interface {
	GetHealth(ctx context.Context) error
}

HealthModel is an interface for etcd health status

type IPAMConfig 

type IPAMConfig struct {
	MaxNodesInRack    uint   `json:"max-nodes-in-rack"`
	NodeIPv4Pool      string `json:"node-ipv4-pool"`
	NodeIPv4Offset    string `json:"node-ipv4-offset,omitempty"`
	NodeRangeSize     uint   `json:"node-ipv4-range-size"`
	NodeRangeMask     uint   `json:"node-ipv4-range-mask"`
	NodeIPPerNode     uint   `json:"node-ip-per-node"`
	NodeIndexOffset   uint   `json:"node-index-offset"`
	NodeGatewayOffset uint   `json:"node-gateway-offset"`

	BMCIPv4Pool      string `json:"bmc-ipv4-pool"`
	BMCIPv4Offset    string `json:"bmc-ipv4-offset,omitempty"`
	BMCRangeSize     uint   `json:"bmc-ipv4-range-size"`
	BMCRangeMask     uint   `json:"bmc-ipv4-range-mask"`
	BMCGatewayOffset uint   `json:"bmc-ipv4-gateway-offset"`
}

IPAMConfig is a set of IPAM configurations.

func (*IPAMConfig) GatewayAddress 

func (c *IPAMConfig) GatewayAddress(addr *net.IPNet) *net.IPNet

GatewayAddress returns a gateway address for the given node address

func (*IPAMConfig) GenerateIP 

func (c *IPAMConfig) GenerateIP(mc *Machine)

GenerateIP generates IP addresses for a machine. Generated IP addresses are stored in mc.

func (*IPAMConfig) LeaseRange 

func (c *IPAMConfig) LeaseRange(ifaddr net.IP) *LeaseRange

LeaseRange returns a LeaseRange for the interface that receives DHCP requests. If no range can be assigned, this returns nil.

func (*IPAMConfig) Validate 

func (c *IPAMConfig) Validate() error

Validate validates configurations

type IPAMModel 

type IPAMModel interface {
	PutConfig(ctx context.Context, config *IPAMConfig) error
	GetConfig() (*IPAMConfig, error)
}

IPAMModel is an interface for IPAMConfig.

type IgnitionModel 

type IgnitionModel interface {
	PutTemplate(ctx context.Context, role, id string, tmpl *IgnitionTemplate) error
	GetTemplateIDs(ctx context.Context, role string) ([]string, error)
	GetTemplate(ctx context.Context, role string, id string) (*IgnitionTemplate, error)
	DeleteTemplate(ctx context.Context, role string, id string) error
}

IgnitionModel is an interface for ignition template.

type IgnitionTemplate 

type IgnitionTemplate struct {
	Version  IgnitionVersion        `json:"version"`
	Template json.RawMessage        `json:"template"`
	Metadata map[string]interface{} `json:"meta"`
}

IgnitionTemplate represents an ignition template. The exact type of Template is determined by Version.

type IgnitionVersion 

type IgnitionVersion string

IgnitionVersion represents the specification version of Ignition.

type Image 

type Image struct {
	ID     string    `json:"id"`
	Date   time.Time `json:"date"`
	URLs   []string  `json:"urls"`
	Exists bool      `json:"exists"`
}

Image represents a set of image files for iPXE boot.

type ImageIndex 

type ImageIndex []*Image

ImageIndex is a list of *Image.

func (ImageIndex) Append 

func (i ImageIndex) Append(img *Image) (ImageIndex, []string)

Append appends a new *Image to the index.

If the index has MaxImages images, the oldest image will be discarded. ID of discarded images are returned in the second return value.

func (ImageIndex) Find 

func (i ImageIndex) Find(id string) *Image

Find an image whose ID is id.

If no image can be found, this returns nil.

func (ImageIndex) Remove 

func (i ImageIndex) Remove(id string) ImageIndex

Remove removes an image entry from the index.

type ImageModel 

type ImageModel interface {
	// These are for /api/v1/images
	GetIndex(ctx context.Context, os string) (ImageIndex, error)
	Upload(ctx context.Context, os, id string, r io.Reader) error
	Download(ctx context.Context, os, id string, out io.Writer) error
	Delete(ctx context.Context, os, id string) error

	// This is for /api/v1/boot/OS/{kernel,initrd.gz}
	// Calling f will serve the content to the HTTP client.
	ServeFile(ctx context.Context, os, filename string,
		f func(modtime time.Time, content io.ReadSeeker)) error
}

ImageModel is an interface to manage boot images.

type KernelParams 

type KernelParams string

KernelParams is a kernel parameters.

type KernelParamsModel 

type KernelParamsModel interface {
	PutParams(ctx context.Context, os string, params string) error
	GetParams(ctx context.Context, os string) (string, error)
}

KernelParamsModel is an interface for kernel parameters.

type LeaseRange 

type LeaseRange struct {
	BeginAddress net.IP
	Count        int
	// contains filtered or unexported fields
}

LeaseRange is a range of IP addresses for DHCP lease.

func (*LeaseRange) IP 

func (l *LeaseRange) IP(n int) net.IP

IP returns n-th IP address in the range.

func (*LeaseRange) Key 

func (l *LeaseRange) Key() string

Key return key string.

type LogModel 

type LogModel interface {
	Dump(ctx context.Context, since, until time.Time, w io.Writer) error
}

LogModel is an interface for audit logs.

type Machine 

type Machine struct {
	Spec   MachineSpec   `json:"spec"`
	Status MachineStatus `json:"status"`
	Info   MachineInfo   `json:"info"`
}

Machine represents a server hardware.

func NewMachine 

func NewMachine(spec MachineSpec) *Machine

NewMachine creates a new machine instance.

func (*Machine) AddLabels 

func (m *Machine) AddLabels(labels map[string]string)

AddLabels adds labels to Machine by merging maps.

func (*Machine) DeleteLabel 

func (m *Machine) DeleteLabel(label string) error

DeleteLabel deletes label from Machine.

func (*Machine) SetState 

func (m *Machine) SetState(ms MachineState) error

SetState sets the state of the machine.

type MachineBMC 

type MachineBMC struct {
	IPv4 string `json:"ipv4"`
	IPv6 string `json:"ipv6"`
	Type string `json:"type"`
}

MachineBMC is a bmc interface struct for Machine

type MachineInfo 

type MachineInfo struct {
	Network NetworkInfo `json:"network"`
	BMC     BMCInfo     `json:"bmc"`
}

MachineInfo is a set of associated information of a Machine.

type MachineModel 

type MachineModel interface {
	Register(ctx context.Context, machines []*Machine) error
	Get(ctx context.Context, serial string) (*Machine, error)
	SetState(ctx context.Context, serial string, state MachineState) error
	AddLabels(ctx context.Context, serial string, labels map[string]string) error
	DeleteLabel(ctx context.Context, serial string, label string) error
	SetRetireDate(ctx context.Context, serial string, date time.Time) error
	Query(ctx context.Context, query Query) ([]*Machine, error)
	Delete(ctx context.Context, serial string) error
}

MachineModel is an interface for machine database.

type MachineSpec 

type MachineSpec struct {
	Serial       string            `json:"serial"`
	Labels       map[string]string `json:"labels"`
	Rack         uint              `json:"rack"`
	IndexInRack  uint              `json:"index-in-rack"`
	Role         string            `json:"role"`
	IPv4         []string          `json:"ipv4"`
	IPv6         []string          `json:"ipv6"`
	RegisterDate time.Time         `json:"register-date"`
	RetireDate   time.Time         `json:"retire-date"`
	BMC          MachineBMC        `json:"bmc"`
}

MachineSpec is a set of attributes to define a machine.

type MachineState 

type MachineState string

MachineState represents a machine's state.

func (MachineState) IsValid 

func (ms MachineState) IsValid() bool

IsValid returns true only if the MachineState is pre-defined.

func (MachineState) String 

func (ms MachineState) String() string

String implements fmt.Stringer interface.

type MachineStatus 

type MachineStatus struct {
	Timestamp time.Time    `json:"timestamp"`
	Duration  float64      `json:"duration"`
	State     MachineState `json:"state"`
}

MachineStatus represents the status of a machine.

type Model 

type Model struct {
	Runner
	Storage      StorageModel
	Machine      MachineModel
	IPAM         IPAMModel
	DHCP         DHCPModel
	Image        ImageModel
	Asset        AssetModel
	Ignition     IgnitionModel
	Log          LogModel
	KernelParams KernelParamsModel
	Health       HealthModel
	Schema       SchemaModel
}

Model is a struct that consists of sub-models.

type NICConfig 

type NICConfig struct {
	Address  string `json:"address"`
	Netmask  string `json:"netmask"`
	MaskBits int    `json:"maskbits"`
	Gateway  string `json:"gateway"`
}

NICConfig represents NIC configuration information.

type NetworkInfo 

type NetworkInfo struct {
	IPv4 []NICConfig `json:"ipv4"`
}

NetworkInfo represents NIC configurations.

type Query 

type Query map[string]string

Query is an URL query

func (Query) BMCType 

func (q Query) BMCType() string

BMCType returns value of bmc-type in the query

func (Query) IPv4 

func (q Query) IPv4() string

IPv4 returns value of ipv4 in the query

func (Query) IPv6 

func (q Query) IPv6() string

IPv6 returns value of ipv6 in the query

func (Query) IsEmpty 

func (q Query) IsEmpty() bool

IsEmpty returns true if query is empty or no values are presented

func (Query) Labels 

func (q Query) Labels() []string

Labels return label's key and value combined with '='

func (Query) Match 

func (q Query) Match(m *Machine) bool

Match returns true if all non-empty fields matches Machine

func (Query) Rack 

func (q Query) Rack() string

Rack returns value of rack in the query

func (Query) Role 

func (q Query) Role() string

Role returns value of role in the query

func (Query) Serial 

func (q Query) Serial() string

Serial returns value of serial in the query

func (Query) State 

func (q Query) State() string

State returns value of state the query

type Runner 

type Runner interface {
	Run(ctx context.Context, ch chan<- struct{}) error
}

Runner is an interface to run the underlying goroutines.

The caller must pass a channel as follows. Receiving a value from the channel effectively guarantees that the driver gets ready. 

ch := make(chan struct{})
well.Go(func(ctx context.Context) error {
    driver.Run(ctx, ch)
})
<-ch

type SchemaModel 

type SchemaModel interface {
	Version(ctx context.Context) (string, error)
	Upgrade(ctx context.Context) error
}

SchemaModel is an interface for schema versioning.

type StorageModel 

type StorageModel interface {
	GetEncryptionKey(ctx context.Context, serial string, diskByPath string) ([]byte, error)
	PutEncryptionKey(ctx context.Context, serial string, diskByPath string, key []byte) error
	DeleteEncryptionKeys(ctx context.Context, serial string) ([]string, error)
}

StorageModel is an interface for disk encryption keys.

Source Files

View all Source files

Directories

Path Synopsis
Package e2e implements end-to-end test for sabakan.
 Package e2e implements end-to-end test for sabakan.
models
etcd
Package etcd implements sabakan model on etcd.
 Package etcd implements sabakan model on etcd.
mock
Package mock implements mockup sabakan model for testing.
 Package mock implements mockup sabakan model for testing.
pkg
sabactl
sabactl/cmd
sabakan
sabakan-cryptsetup
sabakan-cryptsetup/cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.