pkg

package
v0.4.5-1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 18, 2020 License: Apache-2.0 Imports: 15 Imported by: 2

Documentation

Index

Constants

View Source 
const (
	ZDBModeUser = "user"
	ZDBModeSeq  = "seq"
)

Enumeration of the modes 0-db can operate in

Variables

View Source 
var (
	//DefaultMountOptions has sane values for mount
	DefaultMountOptions = MountOptions{
		ReadOnly: false,
		Limit:    256,
		Type:     SSDDevice,
	}

	//ReadOnlyMountOptions shortcut for readonly mount options
	ReadOnlyMountOptions = MountOptions{
		ReadOnly: true,
	}
)
View Source 
var (
	NetworkSchemaV1 = versioned.MustParse("1.0.0")
	NetworkSchemaV2 = versioned.MustParse("2.0.0")
	// NetworkSchemaLatestVersion network object latest version
	NetworkSchemaLatestVersion = NetworkSchemaV2
)

Version of the network workloads keep in cache by networkd

View Source 
var (

	// DefaultPolicy value
	DefaultPolicy = StoragePolicy{
		Raid: Single,
	}

	// NullPolicy does not create pools
	NullPolicy = StoragePolicy{}
)

Functions

This section is empty.

Types

type Allocation 

type Allocation struct {
	VolumeID   string
	VolumePath string
}

Allocation is returned when calling the ZDB allocate. it contains the volume ID and the volume path that has the namespace allocated

type BrokenDevice 

type BrokenDevice struct {
	// Path to allow identification of the disk
	Path string
	// Err returned which lead to the disk being marked as faulty
	Err error
}

BrokenDevice is a disk which is somehow not fully functional. Storage keeps track of disks which have failed at some point, so they are not used, and to be able to later report this to other daemons.

type BrokenPool 

type BrokenPool struct {
	// Label of the broken pool
	Label string
	// Err returned by the action which let to the pool being marked as broken
	Err error
}

BrokenPool contains info about a malfunctioning storage pool

type CPUTimesStat 

type CPUTimesStat []TimesStat

CPUTimesStat alias for []TimesStat required by zbus

type Container 

type Container struct {
	// Name of container
	Name string
	// path to the rootfs of the container
	RootFS string
	// Env env variables to container in format {'KEY=VALUE', 'KEY2=VALUE2'}
	Env []string
	// WorkingDir of the entrypoint command
	WorkingDir string
	// Network network info for container
	Network NetworkInfo
	// Mounts extra mounts for container
	Mounts []MountInfo
	// Entrypoint the process to start inside the container
	Entrypoint string
	// Interactivity enable Core X as PID 1 on the container
	Interactive bool
	// CPU count limit
	CPU uint
	// Memory limit in bytes
	Memory uint64
	// Logs backends
	Logs []logger.Logs
	// Stats container metrics backend
	Stats []stats.Stats
}

Container creation info

type ContainerID 

type ContainerID string

ContainerID type

type ContainerModule 

type ContainerModule interface {
	// Run creates and starts a container on the node. It also auto
	// starts command defined by `entrypoint` inside the container
	// ns: tenant namespace
	// data: Container info
	Run(ns string, data Container) (ContainerID, error)

	// ListNS list the name of all the container namespaces
	ListNS() ([]string, error)

	// List all the existing container IDs from a certain namespace ns
	// if ns is empty, then the container IDs from all existing namespaces will be return
	List(ns string) ([]ContainerID, error)

	// Inspect, return information about the container, given its container id
	Inspect(ns string, id ContainerID) (Container, error)
	Delete(ns string, id ContainerID) error
}

ContainerModule defines rpc interface to containerd

type ContainerNetworkConfig added in v0.4.0 

type ContainerNetworkConfig struct {
	IPs         []string
	PublicIP6   bool
	YggdrasilIP bool
}

ContainerNetworkConfig defines how to construct the network namespace of a container

type DeviceType 

type DeviceType string

DeviceType is the actual type of hardware that the storage device runs on, i.e. SSD or HDD 

const (
	SSDDevice DeviceType = "ssd"
	HDDDevice DeviceType = "hdd"
)

Known device types

type DiskIOCountersStat 

type DiskIOCountersStat struct {
	disk.IOCountersStat
	Time time.Time
}

DiskIOCountersStat struct

func (*DiskIOCountersStat) String 

func (s *DiskIOCountersStat) String() string

type DisksIOCountersStat 

type DisksIOCountersStat map[string]DiskIOCountersStat

DisksIOCountersStat alias for map[string]IOCountersStat required by zbus

type ErrInvalidDeviceType added in v0.3.4 

type ErrInvalidDeviceType struct {
	DeviceType DeviceType
}

ErrInvalidDeviceType raised when trying to allocate space on unsupported device type

func (ErrInvalidDeviceType) Error added in v0.3.4 

func (e ErrInvalidDeviceType) Error() string

type ErrNotEnoughSpace 

type ErrNotEnoughSpace struct {
	DeviceType DeviceType
}

ErrNotEnoughSpace indicates that there is not enough space in a pool of the requested type to create the filesystem

func (ErrNotEnoughSpace) Error 

func (e ErrNotEnoughSpace) Error() string

type FarmID 

type FarmID uint64

FarmID is the identification of a farm

type Filesystem added in v0.4.9 

type Filesystem struct {
	// Filesystem ID
	ID int
	// Path of the Filesystem
	Path string
	// Usage reports the current usage of the Filesystem
	Usage Usage
	// Name of the Filesystem
	Name string
	// FsType of the Filesystem
	FsType string
	// DiskType of the Filesystem
	DiskType DeviceType
}

Filesystem represents a storage space that can be used as a filesystem

type Flister 

type Flister interface {
	// Mount mounts an flist located at url using the 0-db located at storage.
	// MountOptions, can be nil, in that case falls to default, other wise
	// use the provided values.
	// Returns the path in the filesystem where the flist is mounted or an error
	Mount(url string, storage string, opts MountOptions) (path string, err error)

	// Mount mounts an flist located at url using the 0-db located at storage.
	// MountOptions, can be nil, in that case falls to default, other wise
	// use the provided values.
	// Name is a unique name to identify this mount. The flist can later be unmounted
	// with the same name. It is up to the caller to ensure `name` is unique.
	// Returns the path in the filesystem where the flist is mounted or an error
	NamedMount(name string, url string, storage string, ots MountOptions) (path string, err error)

	// Umount the flist mounted at path
	Umount(path string) error

	// NamedUmount unmounts the flist mounted via the NamedMount call, with the same name
	NamedUmount(path string) error

	// HashFromRootPath returns flist hash from a running g8ufs mounted with NamedMount
	HashFromRootPath(name string) (string, error)

	// FlistHash returns md5 of flist if available (requesting the hub)
	FlistHash(url string) (string, error)
}

Flister is the interface for the flist module

type HostMonitor 

type HostMonitor interface {
	Uptime(ctx context.Context) <-chan time.Duration
}

HostMonitor interface (provided by monitord)

type Identifier 

type Identifier interface {
	Identity() string
}

Identifier is the interface that defines how an object can be used as an identity

type IdentityManager 

type IdentityManager interface {
	// NodeID returns the node id (public key)
	NodeID() StrIdentifier

	// FarmID return the farm id this node is part of. this is usually a configuration
	// that the node is booted with. An error is returned if the farmer id is not configured
	FarmID() (FarmID, error)

	// Sign signs the message with privateKey and returns a signature.
	Sign(message []byte) ([]byte, error)

	// Verify reports whether sig is a valid signature of message by publicKey.
	Verify(message, sig []byte) error

	// Encrypt encrypts message with the public key of the node
	Encrypt(message []byte) ([]byte, error)

	// Decrypt decrypts message with the private of the node
	Decrypt(message []byte) ([]byte, error)

	// EncryptECDH aes encrypt msg using a shared key derived from private key of the node and public key of the other party using Elliptic curve Diffie Helman algorithm
	// the nonce if prepended to the encrypted message
	EncryptECDH(msg []byte, publicKey []byte) ([]byte, error)

	// DecryptECDH decrypt aes encrypted msg using a shared key derived from private key of the node and public key of the other party using Elliptic curve Diffie Helman algorithm
	DecryptECDH(msg []byte, publicKey []byte) ([]byte, error)

	// PrivateKey sends the keypair
	PrivateKey() []byte
}

IdentityManager interface.

type Member 

type Member struct {
	Namespace   string
	IPv6        net.IP
	IPv4        net.IP
	YggdrasilIP net.IP
}

Member holds information about a the network namespace of a container

type MountInfo 

type MountInfo struct {
	Source string // source of the mount point on the host
	Target string // target of mount inside the container
}

MountInfo defines a mount point

type MountOptions 

type MountOptions struct {
	// ReadOnly
	ReadOnly bool
	// Limit size of read-write layer in Mib
	Limit uint64
	// Type of disk to use
	Type DeviceType
}

MountOptions struct

type NetID 

type NetID string

NetID is a type defining the ID of a network

type NetResource 

type NetResource struct {
	Name string
	//unique id inside the reservation is an autoincrement (USE AS NET_ID)
	NetID NetID `json:"net_id"`
	// IP range of the network, must be an IPv4 /16
	NetworkIPRange types.IPNet `json:"ip_range"`

	NodeID string `json:"node_id"`
	// IPV4 subnet for this network resource
	Subnet types.IPNet `json:"subnet"`

	WGPrivateKey string `json:"wg_private_key"`
	WGPublicKey  string `json:"wg_public_key"`
	WGListenPort uint16 `json:"wg_listen_port"`

	Peers []Peer `json:"peers"`
}

NetResource is the description of a part of a network local to a specific node

func (*NetResource) Valid added in v0.4.0 

func (nr *NetResource) Valid() error

Valid checks if the network resource is valid.

type NetlinkAddress 

type NetlinkAddress netlink.Addr

NetlinkAddress alias

type NetlinkAddresses 

type NetlinkAddresses []NetlinkAddress

NetlinkAddresses alias for [][]NetlinkAddress

type Network 

type Network struct {
	Name string `json:"name"`
	//unique id inside the reservation is an autoincrement (USE AS NET_ID)
	NetID NetID `json:"net_id"`
	// IP range of the network, must be an IPv4 /16
	IPRange types.IPNet `json:"ip_range"`

	NetResources []NetResource `json:"net_resources"`
}

Network represent the description if a user private network

type NetworkInfo 

type NetworkInfo struct {

	// Containers don't need to know about anything about bridges,
	// IPs, wireguards since this is all is only known by the network
	// resource which is out of the scope of this module
	Namespace string
}

NetworkInfo defines a network configuration for a container

type Networker 

type Networker interface {
	// Ready return nil is networkd is ready to operate
	// This function is used by other deamon to test if networkd is done booting
	Ready() error

	// Create a new network resource
	CreateNR(NetResource) (string, error)
	// Delete a network resource
	DeleteNR(NetResource) error

	// Join a network (with network id) will create a new isolated namespace
	// that is hooked to the network bridge with a veth pair, and assign it a
	// new IP from the network resource range. The method return the new namespace
	// name.
	// The member name specifies the name of the member, and must be unique
	// The NetID is the network id to join
	Join(networkdID NetID, containerID string, cfg ContainerNetworkConfig) (join Member, err error)
	// Leave delete a container nameapce created by Join
	Leave(networkdID NetID, containerID string) (err error)

	// ZDBPrepare creates a network namespace with a macvlan interface into it
	// to allow the 0-db container to be publicly accessible
	// it retusn the name of the network namespace created
	// hw is an optional hardware address that will be set on the new interface
	ZDBPrepare(hw net.HardwareAddr) (string, error)

	// ZDBDestroy is the opposite of ZDPrepare, it makes sure network setup done
	// for zdb is rewind. ns param is the namespace return by the ZDBPrepare
	ZDBDestroy(ns string) error

	// SetupTap sets up a tap device in the network namespace for the networkID. It is hooked
	// to the network bridge. The name of the tap interface is returned
	SetupTap(networkID NetID) (string, error)

	// TapExists checks if the tap device exists already
	TapExists(networkID NetID) (bool, error)

	// RemoveTap removes the tap device from the network namespace
	// of the networkID
	RemoveTap(networkID NetID) error

	// GetSubnet of the network with the given ID on the local node
	GetSubnet(networkID NetID) (net.IPNet, error)

	// GetDefaultGwIP returns the IP(v4) of the default gateway inside the network
	// resource identified by the network ID on the local node
	GetDefaultGwIP(networkID NetID) (net.IP, error)

	// Addrs return the IP addresses of interface
	// if the interface is in a network namespace netns needs to be not empty
	Addrs(iface string, netns string) ([]net.IP, error)

	// ZOSAddresses monitoring streams for ZOS bridge IPs
	ZOSAddresses(ctx context.Context) <-chan NetlinkAddresses

	// DMZAddresses monitoring streams for dmz public interface
	DMZAddresses(ctx context.Context) <-chan NetlinkAddresses

	// YggAddresses monitoring streams for yggdrasil interface
	YggAddresses(ctx context.Context) <-chan NetlinkAddresses

	PublicAddresses(ctx context.Context) <-chan NetlinkAddresses
}

Networker is the interface for the network module

type NicIOCounterStat 

type NicIOCounterStat struct {
	net.IOCountersStat
	RateOut uint64
	RateIn  uint64
}

NicIOCounterStat counter for a nic

type NicsIOCounterStat 

type NicsIOCounterStat []NicIOCounterStat

NicsIOCounterStat alias for []NicIOCounterStat

type Peer 

type Peer struct {
	// IPV4 subnet of the network resource of the peer
	Subnet types.IPNet `json:"subnet"`

	WGPublicKey string        `json:"wg_public_key"`
	AllowedIPs  []types.IPNet `json:"allowed_ips"`
	Endpoint    string        `json:"endpoint"`
}

Peer is the description of a peer of a NetResource

func (*Peer) Valid added in v0.4.0 

func (p *Peer) Valid() error

Valid checks if peer is valid

type PoolStats 

type PoolStats struct {
	disk.UsageStat
	// Counters IO counter for each pool device
	Counters map[string]disk.IOCountersStat `json:"counters"`
}

PoolStats is pool statistics reported by storaged

type PoolsStats 

type PoolsStats map[string]PoolStats

PoolsStats alias for map[string]PoolStats

type Provision added in v0.4.9 

type Provision interface {
	Counters(ctx context.Context) <-chan ProvisionCounters
	DecommissionCached(id string, reason string) error
}

Provision interface

type ProvisionCounters 

type ProvisionCounters struct {
	Container int64 `json:"container"`
	Volume    int64 `jons:"volume"`
	Network   int64 `json:"network"`
	ZDB       int64 `json:"zdb"`
	VM        int64 `json:"vm"`
	Debug     int64 `json:"debug"`
}

ProvisionCounters struct

type RaidProfile 

type RaidProfile string

RaidProfile type 

const (
	// Single profile
	Single RaidProfile = "single"
	// Raid0 profile
	Raid0 RaidProfile = "raid0"
	// Raid1 profile
	Raid1 RaidProfile = "raid1"
	// Raid10 profile
	Raid10 RaidProfile = "raid10"
)

func (RaidProfile) Validate 

func (p RaidProfile) Validate() error

Validate make sure profile is correct

type StorageModule 

type StorageModule interface {
	VolumeAllocater
	ZDBAllocater

	// Total gives the total amount of storage available for a device type
	Total(kind DeviceType) (uint64, error)
	// BrokenPools lists the broken storage pools that have been detected
	BrokenPools() []BrokenPool
	// BrokenDevices lists the broken devices that have been detected
	BrokenDevices() []BrokenDevice

	//Monitor returns stats stream about pools
	Monitor(ctx context.Context) <-chan PoolsStats
}

StorageModule defines the api for storage

type StoragePolicy 

type StoragePolicy struct {
	// Raid profile for this policy
	Raid RaidProfile
	// Number of disks to use in a single pool
	// note that, the disks count must be valid for
	// the chosen raid profile.
	Disks uint8

	// Only create this amount of storage pools. Default to 0 -> unlimited.
	// The spared disks can later be used in automatic repair if a physical
	// disk got corrupt or bad.
	// Note that if it's set to 0 (unlimited), some disks might be spared anyway
	// in case the number of disks required in the policy doesn't add up to pools
	// for example, a pool of 2s on a machine with 5 disks.
	MaxPools uint8
}

StoragePolicy describes the pool creation policy

type StrIdentifier 

type StrIdentifier string

StrIdentifier is a helper type that implement the Identifier interface on top of simple string

func (StrIdentifier) Identity 

func (s StrIdentifier) Identity() string

Identity implements the Identifier interface

type SystemMonitor 

type SystemMonitor interface {
	Memory(ctx context.Context) <-chan VirtualMemoryStat
	CPU(ctx context.Context) <-chan CPUTimesStat
	Disks(ctx context.Context) <-chan DisksIOCountersStat
	Nics(ctx context.Context) <-chan NicsIOCounterStat
}

SystemMonitor interface (provided by monitord)

type TimesStat 

type TimesStat struct {
	cpu.TimesStat
	Percent float64
	Time    time.Time
}

TimesStat struct

func (*TimesStat) String 

func (s *TimesStat) String() string

type Usage added in v0.4.9 

type Usage struct {
	Size uint64
	Used uint64
}

Usage struct

type VDisk 

type VDisk struct {
	// Path to disk
	Path string
	// Size in bytes
	Size int64
}

VDisk info returned by a call to inspect

func (*VDisk) Name added in v0.4.9 

func (d *VDisk) Name() string

Name returns the Name part of the disk path

type VDiskModule 

type VDiskModule interface {
	// AllocateDisk with given id and size, return path to virtual disk
	Allocate(id string, size int64) (string, error)
	// DeallocateVDisk removes a virtual disk
	Deallocate(id string) error
	// Exists checks if disk with that ID already allocated
	Exists(id string) bool
	// Inspect return info about the disk
	Inspect(id string) (VDisk, error)
	// List lists all the available vdisks
	List() ([]VDisk, error)
}

VDiskModule interface

type VM 

type VM struct {
	// virtual machine name, or ID
	Name string
	// CPU is number of cores assigned to the VM
	CPU uint8
	// Memory size in Mib
	Memory int64
	// Network is network info
	Network VMNetworkInfo
	// KernelImage path to uncompressed linux kernel ELF
	KernelImage string
	// InitrdImage (optiona) path to initrd disk
	InitrdImage string
	// KernelArgs to override the default kernel arguments. (default: "ro console=ttyS0 noapic reboot=k panic=1 pci=off nomodules")
	KernelArgs string
	// Disks are a list of disks that are going to
	// be auto allocated on the provided storage path
	Disks []VMDisk
}

VM config structure

func (*VM) Validate 

func (vm *VM) Validate() error

Validate vm data

type VMDisk 

type VMDisk struct {
	// Size is disk size in Mib
	Path     string
	ReadOnly bool
	Root     bool
}

VMDisk specifies vm disk params

type VMInfo 

type VMInfo struct {
	// Flag for enabling/disabling Hyperthreading
	// Required: true
	HtEnabled bool

	// Memory size of VM
	// Required: true
	Memory int64

	// Number of vCPUs (either 1 or an even number)
	CPU int64
}

VMInfo returned by the inspect method

type VMModule 

type VMModule interface {
	Run(vm VM) error
	Inspect(name string) (VMInfo, error)
	Delete(name string) error
	Exists(id string) bool
}

VMModule defines the virtual machine module interface

type VMNetworkInfo 

type VMNetworkInfo struct {
	// Tap device name
	Tap string
	// Mac address of the device
	MAC string
	// Address of the device in the form of cidr
	AddressCIDR net.IPNet
	// Gateway gateway address
	GatewayIP net.IP
	// Nameservers dns servers
	Nameservers []net.IP
}

VMNetworkInfo structure

type VersionMonitor 

type VersionMonitor interface {
	Version(ctx context.Context) <-chan semver.Version
}

VersionMonitor interface (provided by identityd)

type VirtualMemoryStat 

type VirtualMemoryStat mem.VirtualMemoryStat

VirtualMemoryStat struct

type VolumeAllocater 

type VolumeAllocater interface {
	// CreateFilesystem creates a filesystem with a given size. The filesystem
	// is mounted, and the path to the mountpoint is returned. The filesystem
	// is only attempted to be created in a pool of the given type. If no
	// more space is available in such a pool, `ErrNotEnoughSpace` is returned.
	// It is up to the caller to handle such a situation and decide if he wants
	// to try again on a different devicetype
	CreateFilesystem(name string, size uint64, poolType DeviceType) (Filesystem, error)

	// ReleaseFilesystem signals that the named filesystem is no longer needed.
	// The filesystem will be unmounted and subsequently removed.
	// All data contained in the filesystem will be lost, and the
	// space which has been reserved for this filesystem will be reclaimed.
	ReleaseFilesystem(name string) error

	// ListFilesystems return all the filesystem managed by storeaged present on the nodes
	// this can be an expensive call on server with a lot of disk, don't use it in a
	// intensive loop
	// Special filesystem like internal cache and vdisk are not return by this function
	// to access them use the GetCacheFS or GetVdiskFS
	ListFilesystems() ([]Filesystem, error)

	// Path return the filesystem named name
	// if no filesystem with this name exists, an error is returned
	Path(name string) (Filesystem, error)

	// GetCacheFS return the special filesystem used by 0-OS to store internal state and flist cache
	GetCacheFS() (Filesystem, error)
}

VolumeAllocater is the zbus interface of the storage module responsible for volume allocation

type ZDBAllocater 

type ZDBAllocater interface {
	// Allocate is responsible to make sure the subvolume used by a 0-db as enough storage capacity
	// of specified size, type and mode
	// it returns the volume ID and its path or an error if it couldn't allocate enough storage
	// Note: if allocation already exists with the namespace name, the current allocation is returned
	// so no need to call Find before calling allocate
	Allocate(namespace string, diskType DeviceType, size uint64, mode ZDBMode) (Allocation, error)

	// Find searches the system for the current allocation for the namespace
	// Return error = "not found" if no allocation exists.
	Find(namespace string) (allocation Allocation, err error)
}

ZDBAllocater is the zbus interface of the storage module responsible for 0-db allocation

type ZDBMode 

type ZDBMode string

ZDBMode is the enumeration of the modes 0-db can operate in

type ZDBNamespace 

type ZDBNamespace struct {
	ID       string
	DiskType DeviceType
	Size     uint64
	Mode     ZDBMode
	Password string
	Port     int // Listen port of the 0-db owning the namespace
}

ZDBNamespace is a 0-db namespace

Source Files

View all Source files

Directories

Path Synopsis
dmi
smartctl
logger
stats
Package gedis implements a client for Gedis (https://github.com/threefoldtech/digitalmeX/tree/master/docs/Gedis)
 Package gedis implements a client for Gedis (https://github.com/threefoldtech/digitalmeX/tree/master/docs/Gedis)
bootstrap
bridge
dhcp
ifaceutil
latency
macvlan
namespace
ndmz
nft
nr
portm
portm/backend
tuntap
types
wireguard
yggdrasil
yggdrasil/api
Package provision exposes the Engine type.
 Package provision exposes the Engine type.
common
Package common hold logic that is used by both the provision and primitive package it purpose is mainly to avoid circular dependencies
 Package common hold logic that is used by both the provision and primitive package it purpose is mainly to avoid circular dependencies
explorer
primitives
primitives/cache
filesystem
zdbpool
Package zdb implements a client to 0-db: https://github.com/threefoldtech/0-DB
 Package zdb implements a client to 0-db: https://github.com/threefoldtech/0-DB
Package zinit exposes function to interat with zinit service life cyle management
 Package zinit exposes function to interat with zinit service life cyle management

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.