Documentation ¶
Index ¶
- Constants
- Variables
- type Allocation
- type BrokenDevice
- type BrokenPool
- type Container
- type ContainerID
- type ContainerModule
- type ContainerNetworkConfig
- type DeviceType
- type DiskIOCountersStat
- type DisksIOCountersStat
- type ErrInvalidDeviceType
- type ErrNotEnoughSpace
- type FarmID
- type Filesystem
- type Flister
- type HostMonitor
- type Identifier
- type IdentityManager
- type Member
- type MountInfo
- type MountOptions
- type NetID
- type NetResource
- type NetlinkAddress
- type NetlinkAddresses
- type Network
- type NetworkInfo
- type Networker
- type NicIOCounterStat
- type NicsIOCounterStat
- type Peer
- type PoolStats
- type PoolsStats
- type Provision
- type ProvisionCounters
- type RaidProfile
- type StorageModule
- type StoragePolicy
- type StrIdentifier
- type SystemMonitor
- type TimesStat
- type Usage
- type VDisk
- type VDiskModule
- type VM
- type VMDisk
- type VMIface
- type VMInfo
- type VMModule
- type VMNetworkInfo
- type VersionMonitor
- type VirtualMemoryStat
- type VolumeAllocater
- type ZDBAllocater
- type ZDBMode
- type ZDBNamespace
Constants ¶
const ( ZDBModeUser = "user" ZDBModeSeq = "seq" )
Enumeration of the modes 0-db can operate in
Variables ¶
var ( //DefaultMountOptions has sane values for mount DefaultMountOptions = MountOptions{ ReadOnly: false, Limit: 256, Type: SSDDevice, } //ReadOnlyMountOptions shortcut for readonly mount options ReadOnlyMountOptions = MountOptions{ ReadOnly: true, } )
var ( NetworkSchemaV1 = versioned.MustParse("1.0.0") NetworkSchemaV2 = versioned.MustParse("2.0.0") // NetworkSchemaLatestVersion network object latest version NetworkSchemaLatestVersion = NetworkSchemaV2 )
Version of the network workloads keep in cache by networkd
var ( // DefaultPolicy value DefaultPolicy = StoragePolicy{ Raid: Single, } // NullPolicy does not create pools NullPolicy = StoragePolicy{} )
Functions ¶
This section is empty.
Types ¶
type Allocation ¶
Allocation is returned when calling the ZDB allocate. it contains the volume ID and the volume path that has the namespace allocated
type BrokenDevice ¶
type BrokenDevice struct { // Path to allow identification of the disk Path string // Err returned which lead to the disk being marked as faulty Err error }
BrokenDevice is a disk which is somehow not fully functional. Storage keeps track of disks which have failed at some point, so they are not used, and to be able to later report this to other daemons.
type BrokenPool ¶
type BrokenPool struct { // Label of the broken pool Label string // Err returned by the action which let to the pool being marked as broken Err error }
BrokenPool contains info about a malfunctioning storage pool
type Container ¶
type Container struct { // Name of container Name string // path to the rootfs of the container RootFS string // Env env variables to container in format {'KEY=VALUE', 'KEY2=VALUE2'} Env []string // WorkingDir of the entrypoint command WorkingDir string // Network network info for container Network NetworkInfo // Mounts extra mounts for container Mounts []MountInfo // Entrypoint the process to start inside the container Entrypoint string // Interactivity enable Core X as PID 1 on the container Interactive bool // CPU count limit CPU uint // Memory limit in bytes Memory uint64 // Logs backends Logs []logger.Logs // Stats container metrics backend Stats []stats.Stats // Elevated privileges (to use fuse inside) Elevated bool }
Container creation info
type ContainerModule ¶
type ContainerModule interface { // Run creates and starts a container on the node. It also auto // starts command defined by `entrypoint` inside the container // ns: tenant namespace // data: Container info Run(ns string, data Container) (ContainerID, error) // ListNS list the name of all the container namespaces ListNS() ([]string, error) // List all the existing container IDs from a certain namespace ns // if ns is empty, then the container IDs from all existing namespaces will be return List(ns string) ([]ContainerID, error) // Inspect, return information about the container, given its container id Inspect(ns string, id ContainerID) (Container, error) Delete(ns string, id ContainerID) error }
ContainerModule defines rpc interface to containerd
type ContainerNetworkConfig ¶ added in v0.4.0
ContainerNetworkConfig defines how to construct the network namespace of a container
type DeviceType ¶
type DeviceType string
DeviceType is the actual type of hardware that the storage device runs on, i.e. SSD or HDD
const ( SSDDevice DeviceType = "ssd" HDDDevice DeviceType = "hdd" )
Known device types
type DiskIOCountersStat ¶
type DiskIOCountersStat struct { disk.IOCountersStat Time time.Time }
DiskIOCountersStat struct
func (*DiskIOCountersStat) String ¶
func (s *DiskIOCountersStat) String() string
type DisksIOCountersStat ¶
type DisksIOCountersStat map[string]DiskIOCountersStat
DisksIOCountersStat alias for map[string]IOCountersStat required by zbus
type ErrInvalidDeviceType ¶ added in v0.3.4
type ErrInvalidDeviceType struct {
DeviceType DeviceType
}
ErrInvalidDeviceType raised when trying to allocate space on unsupported device type
func (ErrInvalidDeviceType) Error ¶ added in v0.3.4
func (e ErrInvalidDeviceType) Error() string
type ErrNotEnoughSpace ¶
type ErrNotEnoughSpace struct {
DeviceType DeviceType
}
ErrNotEnoughSpace indicates that there is not enough space in a pool of the requested type to create the filesystem
func (ErrNotEnoughSpace) Error ¶
func (e ErrNotEnoughSpace) Error() string
type Filesystem ¶ added in v0.4.9
type Filesystem struct { // Filesystem ID ID int // Path of the Filesystem Path string // Usage reports the current usage of the Filesystem Usage Usage // Name of the Filesystem Name string // FsType of the Filesystem FsType string // DiskType of the Filesystem DiskType DeviceType }
Filesystem represents a storage space that can be used as a filesystem
type Flister ¶
type Flister interface { // Mount mounts an flist located at url using the 0-db located at storage. // MountOptions, can be nil, in that case falls to default, other wise // use the provided values. // Returns the path in the filesystem where the flist is mounted or an error Mount(url string, storage string, opts MountOptions) (path string, err error) // Mount mounts an flist located at url using the 0-db located at storage. // MountOptions, can be nil, in that case falls to default, other wise // use the provided values. // Name is a unique name to identify this mount. The flist can later be unmounted // with the same name. It is up to the caller to ensure `name` is unique. // Returns the path in the filesystem where the flist is mounted or an error NamedMount(name string, url string, storage string, ots MountOptions) (path string, err error) // Umount the flist mounted at path Umount(path string) error // NamedUmount unmounts the flist mounted via the NamedMount call, with the same name NamedUmount(path string) error // HashFromRootPath returns flist hash from a running g8ufs mounted with NamedMount HashFromRootPath(name string) (string, error) // FlistHash returns md5 of flist if available (requesting the hub) FlistHash(url string) (string, error) }
Flister is the interface for the flist module
type HostMonitor ¶
HostMonitor interface (provided by monitord)
type Identifier ¶
type Identifier interface {
Identity() string
}
Identifier is the interface that defines how an object can be used as an identity
type IdentityManager ¶
type IdentityManager interface { // NodeID returns the node id (public key) NodeID() StrIdentifier // FarmID return the farm id this node is part of. this is usually a configuration // that the node is booted with. An error is returned if the farmer id is not configured FarmID() (FarmID, error) // Sign signs the message with privateKey and returns a signature. Sign(message []byte) ([]byte, error) // Verify reports whether sig is a valid signature of message by publicKey. Verify(message, sig []byte) error // Encrypt encrypts message with the public key of the node Encrypt(message []byte) ([]byte, error) // Decrypt decrypts message with the private of the node Decrypt(message []byte) ([]byte, error) // EncryptECDH aes encrypt msg using a shared key derived from private key of the node and public key of the other party using Elliptic curve Diffie Helman algorithm // the nonce if prepended to the encrypted message EncryptECDH(msg []byte, publicKey []byte) ([]byte, error) // DecryptECDH decrypt aes encrypted msg using a shared key derived from private key of the node and public key of the other party using Elliptic curve Diffie Helman algorithm DecryptECDH(msg []byte, publicKey []byte) ([]byte, error) // PrivateKey sends the keypair PrivateKey() []byte }
IdentityManager interface.
type MountInfo ¶
type MountInfo struct { Source string // source of the mount point on the host Target string // target of mount inside the container }
MountInfo defines a mount point
type MountOptions ¶
type MountOptions struct { // ReadOnly ReadOnly bool // Limit size of read-write layer in Mib Limit uint64 // Type of disk to use Type DeviceType }
MountOptions struct
type NetResource ¶
type NetResource struct { Name string //unique id inside the reservation is an autoincrement (USE AS NET_ID) NetID NetID `json:"net_id"` // IP range of the network, must be an IPv4 /16 NetworkIPRange types.IPNet `json:"ip_range"` NodeID string `json:"node_id"` // IPV4 subnet for this network resource Subnet types.IPNet `json:"subnet"` WGPrivateKey string `json:"wg_private_key"` WGPublicKey string `json:"wg_public_key"` WGListenPort uint16 `json:"wg_listen_port"` Peers []Peer `json:"peers"` }
NetResource is the description of a part of a network local to a specific node
func (*NetResource) Valid ¶ added in v0.4.0
func (nr *NetResource) Valid() error
Valid checks if the network resource is valid.
type NetlinkAddresses ¶
NetlinkAddresses alias for [][]NetlinkAddress
type Network ¶
type Network struct { Name string `json:"name"` //unique id inside the reservation is an autoincrement (USE AS NET_ID) NetID NetID `json:"net_id"` // IP range of the network, must be an IPv4 /16 IPRange types.IPNet `json:"ip_range"` NetResources []NetResource `json:"net_resources"` }
Network represent the description if a user private network
type NetworkInfo ¶
type NetworkInfo struct { // Containers don't need to know about anything about bridges, // IPs, wireguards since this is all is only known by the network // resource which is out of the scope of this module Namespace string }
NetworkInfo defines a network configuration for a container
type Networker ¶
type Networker interface { // Ready return nil is networkd is ready to operate // This function is used by other deamon to test if networkd is done booting Ready() error // Create a new network resource CreateNR(NetResource) (string, error) // Delete a network resource DeleteNR(NetResource) error // Join a network (with network id) will create a new isolated namespace // that is hooked to the network bridge with a veth pair, and assign it a // new IP from the network resource range. The method return the new namespace // name. // The member name specifies the name of the member, and must be unique // The NetID is the network id to join Join(networkdID NetID, containerID string, cfg ContainerNetworkConfig) (join Member, err error) // Leave delete a container nameapce created by Join Leave(networkdID NetID, containerID string) (err error) // ZDBPrepare creates a network namespace with a macvlan interface into it // to allow the 0-db container to be publicly accessible // it retusn the name of the network namespace created // hw is an optional hardware address that will be set on the new interface ZDBPrepare(hw net.HardwareAddr) (string, error) // ZDBDestroy is the opposite of ZDPrepare, it makes sure network setup done // for zdb is rewind. ns param is the namespace return by the ZDBPrepare ZDBDestroy(ns string) error // SetupTap sets up a tap device in the network namespace for the networkID. It is hooked // to the network bridge. The name of the tap interface is returned SetupTap(networkID NetID) (string, error) // TapExists checks if the tap device exists already TapExists(networkID NetID) (bool, error) // RemoveTap removes the tap device from the network namespace // of the networkID RemoveTap(networkID NetID) error // PublicIPv4Support enabled on this node for reservations PublicIPv4Support() bool // SetupPubTap sets up a tap device in the host namespace for the public ip // reservation id. It is hooked to the public bridge. The name of the tap // interface is returned SetupPubTap(PubIPReservationID string) (string, error) // PubTapExists checks if the tap device for the public network exists already PubTapExists(PubIPReservationID string) (bool, error) // RemovePubTap removes the public tap device from the host namespace RemovePubTap(PubIPReservationID string) error // DisconnectPubTap disconnects the public tap from the network. The interface // itself is not removed and will need to be cleaned up later DisconnectPubTap(PubIPReservationID string) error // GetSubnet of the network with the given ID on the local node GetSubnet(networkID NetID) (net.IPNet, error) // GetNet returns the full network range of the network GetNet(networkID NetID) (net.IPNet, error) // GetPublicIPv6Subnet returns the IPv6 prefix op the public subnet of the host GetPublicIPv6Subnet() (net.IPNet, error) // GetDefaultGwIP returns the IPs of the default gateways inside the network // resource identified by the network ID on the local node, for IPv4 and IPv6 // respectively GetDefaultGwIP(networkID NetID) (net.IP, net.IP, error) // GetIPv6From4 generates an IPv6 address from a given IPv4 address in a NR GetIPv6From4(networkID NetID, ip net.IP) (net.IPNet, error) // Addrs return the IP addresses of interface // if the interface is in a network namespace netns needs to be not empty Addrs(iface string, netns string) ([]net.IP, error) // ZOSAddresses monitoring streams for ZOS bridge IPs ZOSAddresses(ctx context.Context) <-chan NetlinkAddresses // DMZAddresses monitoring streams for dmz public interface DMZAddresses(ctx context.Context) <-chan NetlinkAddresses // YggAddresses monitoring streams for yggdrasil interface YggAddresses(ctx context.Context) <-chan NetlinkAddresses PublicAddresses(ctx context.Context) <-chan NetlinkAddresses }
Networker is the interface for the network module
type NicIOCounterStat ¶
type NicIOCounterStat struct { net.IOCountersStat RateOut uint64 RateIn uint64 }
NicIOCounterStat counter for a nic
type NicsIOCounterStat ¶
type NicsIOCounterStat []NicIOCounterStat
NicsIOCounterStat alias for []NicIOCounterStat
type Peer ¶
type Peer struct { // IPV4 subnet of the network resource of the peer Subnet types.IPNet `json:"subnet"` WGPublicKey string `json:"wg_public_key"` AllowedIPs []types.IPNet `json:"allowed_ips"` Endpoint string `json:"endpoint"` }
Peer is the description of a peer of a NetResource
type PoolStats ¶
type PoolStats struct { disk.UsageStat // Counters IO counter for each pool device Counters map[string]disk.IOCountersStat `json:"counters"` }
PoolStats is pool statistics reported by storaged
type Provision ¶ added in v0.4.9
type Provision interface { Counters(ctx context.Context) <-chan ProvisionCounters DecommissionCached(id string, reason string) error }
Provision interface
type ProvisionCounters ¶
type ProvisionCounters struct { Container int64 `json:"container"` Volume int64 `jons:"volume"` Network int64 `json:"network"` ZDB int64 `json:"zdb"` VM int64 `json:"vm"` Debug int64 `json:"debug"` }
ProvisionCounters struct
type RaidProfile ¶
type RaidProfile string
RaidProfile type
const ( // Single profile Single RaidProfile = "single" // Raid0 profile Raid0 RaidProfile = "raid0" // Raid1 profile Raid1 RaidProfile = "raid1" // Raid10 profile Raid10 RaidProfile = "raid10" )
func (RaidProfile) Validate ¶
func (p RaidProfile) Validate() error
Validate make sure profile is correct
type StorageModule ¶
type StorageModule interface { VolumeAllocater ZDBAllocater // Total gives the total amount of storage available for a device type Total(kind DeviceType) (uint64, error) // BrokenPools lists the broken storage pools that have been detected BrokenPools() []BrokenPool // BrokenDevices lists the broken devices that have been detected BrokenDevices() []BrokenDevice //Monitor returns stats stream about pools Monitor(ctx context.Context) <-chan PoolsStats }
StorageModule defines the api for storage
type StoragePolicy ¶
type StoragePolicy struct { // Raid profile for this policy Raid RaidProfile // Number of disks to use in a single pool // note that, the disks count must be valid for // the chosen raid profile. Disks uint8 // Only create this amount of storage pools. Default to 0 -> unlimited. // The spared disks can later be used in automatic repair if a physical // disk got corrupt or bad. // Note that if it's set to 0 (unlimited), some disks might be spared anyway // in case the number of disks required in the policy doesn't add up to pools // for example, a pool of 2s on a machine with 5 disks. MaxPools uint8 }
StoragePolicy describes the pool creation policy
type StrIdentifier ¶
type StrIdentifier string
StrIdentifier is a helper type that implement the Identifier interface on top of simple string
func (StrIdentifier) Identity ¶
func (s StrIdentifier) Identity() string
Identity implements the Identifier interface
type SystemMonitor ¶
type SystemMonitor interface { Memory(ctx context.Context) <-chan VirtualMemoryStat CPU(ctx context.Context) <-chan TimesStat Disks(ctx context.Context) <-chan DisksIOCountersStat Nics(ctx context.Context) <-chan NicsIOCounterStat }
SystemMonitor interface (provided by monitord)
type VDiskModule ¶
type VDiskModule interface { // AllocateDisk with given id and size, return path to virtual disk Allocate(id string, size int64) (string, error) // DeallocateVDisk removes a virtual disk Deallocate(id string) error // Exists checks if disk with that ID already allocated Exists(id string) bool // Inspect return info about the disk Inspect(id string) (VDisk, error) // List lists all the available vdisks List() ([]VDisk, error) }
VDiskModule interface
type VM ¶
type VM struct { // virtual machine name, or ID Name string // CPU is number of cores assigned to the VM CPU uint8 // Memory size in Mib Memory int64 // Network is network info Network VMNetworkInfo // KernelImage path to uncompressed linux kernel ELF KernelImage string // InitrdImage (optiona) path to initrd disk InitrdImage string // KernelArgs to override the default kernel arguments. (default: "ro console=ttyS0 noapic reboot=k panic=1 pci=off nomodules") KernelArgs string // Disks are a list of disks that are going to // be auto allocated on the provided storage path Disks []VMDisk // If this flag is set, the VM module will not auto start // this machine hence, also no auto clean up when it exits // it's up to the caller to check for the machine status // and do clean up (module.Delete(vm)) when needed NoKeepAlive bool }
VM config structure
type VMIface ¶ added in v0.4.9
type VMIface struct { // Tap device name Tap string // Mac address of the device MAC string // Address of the device in the form of cidr for ipv4 IP4AddressCIDR net.IPNet // Gateway address for ipv4 IP4GatewayIP net.IP // Full subnet for the IP4 resource. This allows configuration of networking for // non local subnets (i.e. NR on other nodes). // Does not need to be set for public ifaces IP4Net net.IPNet // Address of the device in the form of cidr for ipv6 IP6AddressCIDR net.IPNet // Gateway address for ipv6 IP6GatewayIP net.IP // Private or public network Public bool }
VMIface structure
type VMInfo ¶
type VMInfo struct { // Flag for enabling/disabling Hyperthreading // Required: true HtEnabled bool // Memory size of VM // Required: true Memory int64 // Number of vCPUs (either 1 or an even number) CPU int64 }
VMInfo returned by the inspect method
type VMModule ¶
type VMModule interface { Run(vm VM) error Inspect(name string) (VMInfo, error) Delete(name string) error Exists(name string) bool Logs(name string) (string, error) List() ([]string, error) }
VMModule defines the virtual machine module interface
type VMNetworkInfo ¶
type VMNetworkInfo struct { // Interfaces for the vm network Ifaces []VMIface // Nameservers dns servers Nameservers []net.IP // NewStyle version of the network. This specifically differentiates between // the old way of statically setting the IP via the `IP` kernel parameter, // or using a boot script. The new way allows multiple interfaces and more // configuration NewStyle bool }
VMNetworkInfo structure
type VersionMonitor ¶
VersionMonitor interface (provided by identityd)
type VolumeAllocater ¶
type VolumeAllocater interface { // CreateFilesystem creates a filesystem with a given size. The filesystem // is mounted, and the path to the mountpoint is returned. The filesystem // is only attempted to be created in a pool of the given type. If no // more space is available in such a pool, `ErrNotEnoughSpace` is returned. // It is up to the caller to handle such a situation and decide if he wants // to try again on a different devicetype CreateFilesystem(name string, size uint64, poolType DeviceType) (Filesystem, error) // ReleaseFilesystem signals that the named filesystem is no longer needed. // The filesystem will be unmounted and subsequently removed. // All data contained in the filesystem will be lost, and the // space which has been reserved for this filesystem will be reclaimed. ReleaseFilesystem(name string) error // ListFilesystems return all the filesystem managed by storeaged present on the nodes // this can be an expensive call on server with a lot of disk, don't use it in a // intensive loop // Special filesystem like internal cache and vdisk are not return by this function // to access them use the GetCacheFS or GetVdiskFS ListFilesystems() ([]Filesystem, error) // Path return the filesystem named name // if no filesystem with this name exists, an error is returned Path(name string) (Filesystem, error) // GetCacheFS return the special filesystem used by 0-OS to store internal state and flist cache GetCacheFS() (Filesystem, error) }
VolumeAllocater is the zbus interface of the storage module responsible for volume allocation
type ZDBAllocater ¶
type ZDBAllocater interface { // Allocate is responsible to make sure the subvolume used by a 0-db as enough storage capacity // of specified size, type and mode // it returns the volume ID and its path or an error if it couldn't allocate enough storage // Note: if allocation already exists with the namespace name, the current allocation is returned // so no need to call Find before calling allocate Allocate(namespace string, diskType DeviceType, size uint64, mode ZDBMode) (Allocation, error) // Find searches the system for the current allocation for the namespace // Return error = "not found" if no allocation exists. Find(namespace string) (allocation Allocation, err error) }
ZDBAllocater is the zbus interface of the storage module responsible for 0-db allocation
type ZDBNamespace ¶
type ZDBNamespace struct { ID string DiskType DeviceType Size uint64 Mode ZDBMode Password string Port int // Listen port of the 0-db owning the namespace }
ZDBNamespace is a 0-db namespace
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package gedis implements a client for Gedis (https://github.com/threefoldtech/digitalmeX/tree/master/docs/Gedis)
|
Package gedis implements a client for Gedis (https://github.com/threefoldtech/digitalmeX/tree/master/docs/Gedis) |
options
Package options abstract setting common networking sys flags on the selected namespaces
|
Package options abstract setting common networking sys flags on the selected namespaces |
Package provision exposes the Engine type.
|
Package provision exposes the Engine type. |
common
Package common hold logic that is used by both the provision and primitive package it purpose is mainly to avoid circular dependencies
|
Package common hold logic that is used by both the provision and primitive package it purpose is mainly to avoid circular dependencies |
Package zdb implements a client to 0-db: https://github.com/threefoldtech/0-DB
|
Package zdb implements a client to 0-db: https://github.com/threefoldtech/0-DB |
Package zinit exposes function to interat with zinit service life cyle management
|
Package zinit exposes function to interat with zinit service life cyle management |