kamatera

package
v0.0.0-...-7b44715 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 3, 2024 License: Apache-2.0 Imports: 28 Imported by: 0

README

Cluster Autoscaler for Kamatera

The cluster autoscaler for Kamatera scales nodes in a Kamatera cluster.

Kamatera Kubernetes

Kamatera supports Kubernetes clusters using our Rancher app or by creating a self-managed cluster directly on Kamatera compute servers, the autoscaler supports both methods.

Cluster Autoscaler Node Groups

An autoscaler node group is composed of multiple Kamatera servers with the same server configuration. All servers belonging to a node group are identified by Kamatera server tags k8sca-CLUSTER_NAME, k8scang-NODEGROUP_NAME. The cluster and node groups must be specified in the autoscaler cloud configuration file.

Deployment

Copy examples/deployment.yaml and modify the configuration as needed, see below regarding the required configuration values and format. When the configuraiont is ready, deploy it to your cluster e.g. using kubectl apply -f deployment.yaml.

Configuration

The cluster autoscaler only considers the cluster and node groups defined in the configuration file.

You can see an example of the cloud config file at examples/deployment.yaml,

Important Note: The cluster and node group names must be 15 characters or less.

it is an INI file with the following fields:

Key Value Mandatory Default
global/kamatera-api-client-id Kamatera API Client ID yes none
global/kamatera-api-secret Kamatera API Secret yes none
global/cluster-name max 15 characters: english letters, numbers, dash, underscore, space, dot: distinct string used to set the cluster server tag yes none
global/default-min-size default minimum size of a node group (must be > 0) no 1
global/default-max-size default maximum size of a node group no 254
global/default-<SERVER_CONFIG_KEY> replace <SERVER_CONFIG_KEY> with the relevant configuration key see below see below
nodegroup "name" max 15 characters: english letters, numbers, dash, underscore, space, dot: distinct string within the cluster used to set the node group server tag yes none
nodegroup "name"/min-size minimum size for a specific node group no global/defaut-min-size
nodegroup "name"/max-size maximum size for a specific node group no global/defaut-min-size
nodegroup "name"/<SERVER_CONFIG_KEY> replace <SERVER_CONFIG_KEY> with the relevant configuration key no global/default-<SERVER_CONFIG_KEY>
Server configuration keys

Following are the supported server configuration keys:

Key Value Mandatory Default
name-prefix Prefix for all created server names no none
password Server root password no none
ssh-key Public SSH key to add to the server authorized keys no none
datacenter Datacenter ID yes none
image Image ID or name yes none
cpu CPU type and size identifier yes none
ram RAM size in MB yes none
disk Disk specifications - see below for details yes none
dailybackup boolean - set to true to enable daily backups no false
managed boolean - set to true to enable managed services no false
network Network specifications - see below for details yes none
billingcycle "hourly" or "monthly" no "hourly"
monthlypackage For monthly billing only - the monthly network package to use no none
script-base64 base64 encoded server initialization script, must be provided to connect the server to the cluster, see below for details no none
Disk specifications

Server disks are specified using an array of strings which are the same as the cloudcli --disk argument as specified in cloudcli server create. For multiple disks, include the configuration multiple times, example:

[global]
; default for all node groups: single 100gb disk
default-disk = "size=100"

[nodegroup "ng1"]
; this node group will use the default

[nodegroup "ng2"]
; override the default and use 2 disks
disk = "size=100"
disk = "size=200"
Network specifications

Networks are specified using an array of strings which are the same as the cloudcli --network argument as specified in cloudcli server create. For multiple networks, include the configuration multiple times, example:

[global]
; default for all node groups: single public network with auto-assigned ip
default-network = "name=wan,ip=auto"

[nodegroup "ng1"]
; this node group will use the default

[nodegroup "ng2"]
; override the default and attach 2 networks - 1 public and 1 private
network = "name=wan,ip=auto"
network = "name=lan-12345-abcde,ip=auto"
Server Initialization Script

This script is required so that the server will connect to the relevant cluster. The specific script depends on how you create and manage the cluster.

See below for some common configurations, but the exact script may need to be modified depending on your requirements and server image.

The script needs to be provided as a base64 encoded string. You can encode your script using the following command: cat script.sh | base64 -w0.

Kamatera Rancher Server Initialization Script

Using Kamatera Rancher you need to get the command to join a server to the cluster. This is available from the following URL: https://rancher.domain/v3/clusterregistrationtokens. The relevant command is available under data[].nodeCommand, if you have a single cluster, it will be the first one. If you have multiple cluster you will have to locate the relevant cluster from the array using clusterId. The command will look like this:

sudo docker run -d --privileged --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run  rancher/rancher-agent:v2.6.4 --server https://rancher.domain --token aaa --ca-checksum bbb

You can replace this command in the example script at examples/server-init-rancher.sh.txt

Kubeadm Initialization Script

The example script at examples/server-init-kubeadm.sh.txt can be used as a base for writing your own script to join the server to your cluster.

Development

Make sure you are inside the cluster-autoscaler path of the autoscaler repository.

Run tests:

go test -v k8s.io/autoscaler/cluster-autoscaler/cloudprovider/kamatera

Setup a Kamatera cluster, you can use this guide

Get the cluster kubeconfig and set in local file and set in the KUBECONFIG environment variable. Make sure you are connected to the cluster using kubectl get nodes. Create a cloud config file according to the above documentation and set it's path in CLOUD_CONFIG_FILE env var.

Build the binary and run it:

make build &&\
./cluster-autoscaler-amd64 --cloud-config $CLOUD_CONFIG_FILE --cloud-provider kamatera --kubeconfig $KUBECONFIG -v2

Build the docker image:

make container

Tag and push it to a Docker registry

docker tag staging-k8s.gcr.io/cluster-autoscaler-amd64:dev ghcr.io/github_username_lowercase/cluster-autoscaler-amd64
docker push ghcr.io/github_username_lowercase/cluster-autoscaler-amd64

Make sure relevant clsuter has access to this registry/image.

Follow the documentation for deploying the image and using the autoscaler.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildKamatera 

func BuildKamatera(
	opts config.AutoscalingOptions,
	do cloudprovider.NodeGroupDiscoveryOptions,
	rl *cloudprovider.ResourceLimiter,
) cloudprovider.CloudProvider

BuildKamatera builds the Kamatera cloud provider.

Types

type GcfgGlobalConfig 

type GcfgGlobalConfig struct {
	KamateraApiClientId   string   `gcfg:"kamatera-api-client-id"`
	KamateraApiSecret     string   `gcfg:"kamatera-api-secret"`
	KamateraApiUrl        string   `gcfg:"kamatera-api-url"`
	ClusterName           string   `gcfg:"cluster-name"`
	DefaultMinSize        string   `gcfg:"default-min-size"`
	DefaultMaxSize        string   `gcfg:"default-max-size"`
	DefaultNamePrefix     string   `gcfg:"default-name-prefix"`
	DefaultPassword       string   `gcfg:"default-password"`
	DefaultSshKey         string   `gcfg:"default-ssh-key"`
	DefaultDatacenter     string   `gcfg:"default-datacenter"`
	DefaultImage          string   `gcfg:"default-image"`
	DefaultCpu            string   `gcfg:"default-cpu"`
	DefaultRam            string   `gcfg:"default-ram"`
	DefaultDisks          []string `gcfg:"default-disk"`
	DefaultDailybackup    bool     `gcfg:"default-dailybackup"`
	DefaultManaged        bool     `gcfg:"default-managed"`
	DefaultNetworks       []string `gcfg:"default-network"`
	DefaultBillingCycle   string   `gcfg:"default-billingcycle"`
	DefaultMonthlyPackage string   `gcfg:"default-monthlypackage"`
	DefaultScriptBase64   string   `gcfg:"default-script-base64"`
}

GcfgGlobalConfig is the gcfg representation of the global section in the cloud config file for Kamatera.

type GcfgNodeGroupConfig 

type GcfgNodeGroupConfig struct {
	MinSize        string   `gcfg:"min-size"`
	MaxSize        string   `gcfg:"max-size"`
	NamePrefix     string   `gcfg:"name-prefix"`
	Password       string   `gcfg:"password"`
	SshKey         string   `gcfg:"ssh-key"`
	Datacenter     string   `gcfg:"datacenter"`
	Image          string   `gcfg:"image"`
	Cpu            string   `gcfg:"cpu"`
	Ram            string   `gcfg:"ram"`
	Disks          []string `gcfg:"disk"`
	Dailybackup    bool     `gcfg:"dailybackup"`
	Managed        bool     `gcfg:"managed"`
	Networks       []string `gcfg:"network"`
	BillingCycle   string   `gcfg:"billingcycle"`
	MonthlyPackage string   `gcfg:"monthlypackage"`
	ScriptBase64   string   `gcfg:"script-base64"`
}

GcfgNodeGroupConfig is the gcfg representation of the section in the cloud config file to change defaults for a node group.

type Instance 

type Instance struct {
	// Id is the Kamatera server Name.
	Id string
	// Status represents status of the node. (Optional)
	Status *cloudprovider.InstanceStatus
	// Kamatera specific fields
	PowerOn bool
	Tags    []string
}

Instance implements cloudprovider.Instance interface. Instance contains configuration info and functions to control a single Kamatera server instance.

type KamateraApiClientRest 

type KamateraApiClientRest struct {
	// contains filtered or unexported fields
}

KamateraApiClientRest is the struct to perform API calls

func NewKamateraApiClientRest 

func NewKamateraApiClientRest(clientId string, secret string, url string) (client KamateraApiClientRest)

NewKamateraApiClientRest factory to create new Rest API Client struct

func (*KamateraApiClientRest) CreateServers 

func (c *KamateraApiClientRest) CreateServers(ctx context.Context, count int, config ServerConfig) ([]Server, error)

CreateServers creates new servers according to the given configuration

func (*KamateraApiClientRest) DeleteServer 

func (c *KamateraApiClientRest) DeleteServer(ctx context.Context, name string) error

DeleteServer deletes a server according to the given name

func (*KamateraApiClientRest) ListServers 

func (c *KamateraApiClientRest) ListServers(ctx context.Context, instances map[string]*Instance) ([]Server, error)

ListServers returns a list of all servers in the relevant account and fetches their tags

type KamateraServerCreatePostRequest 

type KamateraServerCreatePostRequest struct {
	Name               string `json:"name"`
	Password           string `json:"password"`
	PasswordValidate   string `json:"passwordValidate"`
	SshKey             string `json:"ssh-key"`
	Datacenter         string `json:"datacenter"`
	Image              string `json:"image"`
	Cpu                string `json:"cpu"`
	Ram                string `json:"ram"`
	Disk               string `json:"disk"`
	Dailybackup        string `json:"dailybackup"`
	Managed            string `json:"managed"`
	Network            string `json:"network"`
	Quantity           int    `json:"quantity"`
	BillingCycle       string `json:"billingCycle"`
	MonthlyPackage     string `json:"monthlypackage"`
	Poweronaftercreate string `json:"poweronaftercreate"`
	ScriptFile         string `json:"script-file"`
	UserdataFile       string `json:"userdata-file"`
	Tag                string `json:"tag"`
}

KamateraServerCreatePostRequest struct for Kamatera server create post request

type KamateraServerPostRequest 

type KamateraServerPostRequest struct {
	ServerName string `json:"name"`
}

KamateraServerPostRequest struct for Kamatera server post request

type KamateraServerTerminatePostRequest 

type KamateraServerTerminatePostRequest struct {
	ServerName string `json:"name"`
	Force      bool   `json:"force"`
}

KamateraServerTerminatePostRequest struct for Kamatera server terminate post request

type NodeGroup 

type NodeGroup struct {
	// contains filtered or unexported fields
}

NodeGroup implements cloudprovider.NodeGroup interface. NodeGroup contains configuration info and functions to control a set of nodes that have the same capacity and set of labels.

func (*NodeGroup) AtomicIncreaseSize 

func (n *NodeGroup) AtomicIncreaseSize(delta int) error

AtomicIncreaseSize is not implemented.

func (*NodeGroup) Autoprovisioned 

func (n *NodeGroup) Autoprovisioned() bool

Autoprovisioned returns true if the node group is autoprovisioned. An autoprovisioned group was created by CA and can be deleted when scaled to 0.

func (*NodeGroup) Create 

func (n *NodeGroup) Create() (cloudprovider.NodeGroup, error)

Create creates the node group on the cloud provider side. Implementation optional.

func (*NodeGroup) Debug 

func (n *NodeGroup) Debug() string

Debug returns a string containing all information regarding this node group.

func (*NodeGroup) DecreaseTargetSize 

func (n *NodeGroup) DecreaseTargetSize(delta int) error

DecreaseTargetSize decreases the target size of the node group. This function doesn't permit to delete any existing node and can be used only to reduce the request for new nodes that have not been yet fulfilled. Delta should be negative. It is assumed that cloud provider will not delete the existing nodes when there is an option to just decrease the target. Implementation required.

func (*NodeGroup) Delete 

func (n *NodeGroup) Delete() error

Delete deletes the node group on the cloud provider side. This will be executed only for autoprovisioned node groups, once their size drops to 0. Implementation optional.

func (*NodeGroup) DeleteNodes 

func (n *NodeGroup) DeleteNodes(nodes []*apiv1.Node) error

DeleteNodes deletes nodes from this node group. Error is returned either on failure or if the given node doesn't belong to this node group. This function should wait until node group size is updated. Implementation required.

func (*NodeGroup) Exist 

func (n *NodeGroup) Exist() bool

Exist checks if the node group really exists on the cloud provider side. Allows to tell the theoretical node group from the real one. Implementation required.

func (*NodeGroup) ForceDeleteNodes 

func (n *NodeGroup) ForceDeleteNodes(nodes []*apiv1.Node) error

ForceDeleteNodes deletes nodes from the group regardless of constraints.

func (*NodeGroup) GetOptions 

func (n *NodeGroup) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)

GetOptions returns NodeGroupAutoscalingOptions that should be used for this particular NodeGroup. Returning a nil will result in using default options. Implementation optional.

func (*NodeGroup) Id 

func (n *NodeGroup) Id() string

Id returns an unique identifier of the node group.

func (*NodeGroup) IncreaseSize 

func (n *NodeGroup) IncreaseSize(delta int) error

IncreaseSize increases the size of the node group. To delete a node you need to explicitly name it and use DeleteNode. This function should wait until node group size is updated. Implementation required.

func (*NodeGroup) MaxSize 

func (n *NodeGroup) MaxSize() int

MaxSize returns maximum size of the node group.

func (*NodeGroup) MinSize 

func (n *NodeGroup) MinSize() int

MinSize returns minimum size of the node group.

func (*NodeGroup) Nodes 

func (n *NodeGroup) Nodes() ([]cloudprovider.Instance, error)

Nodes returns a list of all nodes that belong to this node group. It is required that Instance objects returned by this method have Id field set. Other fields are optional. This list should include also instances that might have not become a kubernetes node yet.

func (*NodeGroup) TargetSize 

func (n *NodeGroup) TargetSize() (int, error)

TargetSize returns the current target size of the node group. It is possible that the number of nodes in Kubernetes is different at the moment but should be equal to Size() once everything stabilizes (new nodes finish startup and registration or removed nodes are deleted completely). Implementation required.

func (*NodeGroup) TemplateNodeInfo 

func (n *NodeGroup) TemplateNodeInfo() (*framework.NodeInfo, error)

TemplateNodeInfo returns a framework.NodeInfo structure of an empty (as if just started) node. This will be used in scale-up simulations to predict what would a new node look like if a node group was expanded. The returned NodeInfo is expected to have a fully populated Node object, with all of the labels, capacity and allocatable information as well as all pods that are started on the node by default, using manifest (most likely only kube-proxy). Implementation optional.

type ProviderConfig 

type ProviderConfig struct {
	ApiUrl      string
	ApiClientID string
	ApiSecret   string
}

ProviderConfig is the configuration for the Kamatera cloud provider

type Server 

type Server struct {
	Name    string
	Tags    []string
	PowerOn bool
}

Server contains information about a Kamatera server, as fetched from the API It is used by the manager to keep track of the servers in the cluster.

type ServerConfig 

type ServerConfig struct {
	NamePrefix     string
	Password       string
	SshKey         string
	Datacenter     string
	Image          string
	Cpu            string
	Ram            string
	Disks          []string
	Dailybackup    bool
	Managed        bool
	Networks       []string
	BillingCycle   string
	MonthlyPackage string
	ScriptFile     string
	UserdataFile   string
	Tags           []string
}

ServerConfig struct for Kamatera server config

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.