hetzner

README

Cluster Autoscaler for Hetzner Cloud

The cluster autoscaler for Hetzner Cloud scales worker nodes.

Configuration

HCLOUD_TOKEN Required Hetzner Cloud token.

HCLOUD_CLOUD_INIT Base64 encoded Cloud Init yaml with commands to join the cluster, Sample examples/cloud-init.txt for (Kubernetes 1.20.1)

HCLOUD_IMAGE Defaults to ubuntu-20.04, @see https://docs.hetzner.cloud/#images. You can also use an image ID here (e.g. 15512617), or a label selector associated with a custom snapshot (e.g. customized_ubuntu=true). The most recent snapshot will be used in the latter case.

HCLOUD_CLUSTER_CONFIG This is the new format replacing

• HCLOUD_CLOUD_INIT
• HCLOUD_IMAGE

Base64 encoded JSON according to the following structure

{
    "imagesForArch": { // These should be the same format as HCLOUD_IMAGE
        "arm64": "", 
        "amd64": ""
    },
    "nodeConfigs": {
        "pool1": { // This equals the pool name. Required for each pool that you have
            "cloudInit": "", // HCLOUD_CLOUD_INIT make sure it isn't base64 encoded twice ;]
            "labels": {
                "node.kubernetes.io/role": "autoscaler-node"
            },
            "taints": 
            [
                {
                    "key": "node.kubernetes.io/role",
                    "value": "autoscaler-node",
                    "effect": "NoExecute"
                }
            ]
        }
    }
}

HCLOUD_NETWORK Default empty , The id or name of the network that is used in the cluster , @see https://docs.hetzner.cloud/#networks

HCLOUD_FIREWALL Default empty , The id or name of the firewall that is used in the cluster , @see https://docs.hetzner.cloud/#firewalls

HCLOUD_SSH_KEY Default empty , The id or name of SSH Key that will have access to the fresh created server, @see https://docs.hetzner.cloud/#ssh-keys

HCLOUD_PUBLIC_IPV4 Default true , Whether the server is created with a public IPv4 address or not, @see https://docs.hetzner.cloud/#primary-ips

HCLOUD_PUBLIC_IPV6 Default true , Whether the server is created with a public IPv6 address or not, @see https://docs.hetzner.cloud/#primary-ips

Node groups must be defined with the --nodes=<min-servers>:<max-servers>:<instance-type>:<region>:<name> flag.

Multiple flags will create multiple node pools. For example:

--nodes=1:10:CPX51:FSN1:pool1
--nodes=1:10:CPX51:NBG1:pool2
--nodes=1:10:CX41:NBG1:pool3

You can find a deployment sample under examples/cluster-autoscaler-run-on-master.yaml. Please be aware that you should change the values within this deployment to reflect your cluster.

Development

Make sure you're inside the cluster-autoscaler root folder.

1.) Build the cluster-autoscaler binary:

make build-in-docker

2.) Build the docker image:

docker build -t hetzner/cluster-autoscaler:dev .

3.) Push the docker image to Docker hub:

docker push hetzner/cluster-autoscaler:dev

Updating vendored hcloud-go

To update the vendored hcloud-go code, navigate to the directory and run the hack/update-vendor.sh script:

cd cluster-autoscaler/cloudprovider/hetzner
UPSTREAM_REF=v2.0.0 hack/update-vendor.sh
git add hcloud-go/

Debugging

To enable debug logging, set the log level of the autoscaler to at least level 5 via cli flag: --v=5
The logs will include all requests and responses made towards the Hetzner API including headers and body.

Documentation

Index

• Constants
• func BuildHetzner(_ config.AutoscalingOptions, do cloudprovider.NodeGroupDiscoveryOptions, ...) cloudprovider.CloudProvider
• type ClusterConfig
• type HetznerCloudProvider
  • func (d *HetznerCloudProvider) Cleanup() error
  • func (d *HetznerCloudProvider) GPULabel() string
  • func (d *HetznerCloudProvider) GetAvailableGPUTypes() map[string]struct{}
  • func (d *HetznerCloudProvider) GetAvailableMachineTypes() ([]string, error)
  • func (d *HetznerCloudProvider) GetNodeGpuConfig(node *apiv1.Node) *cloudprovider.GpuConfig
  • func (d *HetznerCloudProvider) GetResourceLimiter() (*cloudprovider.ResourceLimiter, error)
  • func (d *HetznerCloudProvider) HasInstance(node *apiv1.Node) (bool, error)
  • func (d *HetznerCloudProvider) Name() string
  • func (d *HetznerCloudProvider) NewNodeGroup(machineType string, labels map[string]string, systemLabels map[string]string, ...) (cloudprovider.NodeGroup, error)
  • func (d *HetznerCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)
  • func (d *HetznerCloudProvider) NodeGroups() []cloudprovider.NodeGroup
  • func (d *HetznerCloudProvider) Pricing() (cloudprovider.PricingModel, autoscalerErrors.AutoscalerError)
  • func (d *HetznerCloudProvider) Refresh() error
• type ImageList
• type LegacyConfig
• type NodeConfig

Constants

const (
	// GPULabel is the label added to nodes with GPU resource.
	GPULabel = hcloudLabelNamespace + "/gpu-node"
)

Functions

func BuildHetzner

func BuildHetzner(_ config.AutoscalingOptions, do cloudprovider.NodeGroupDiscoveryOptions, rl *cloudprovider.ResourceLimiter) cloudprovider.CloudProvider

BuildHetzner builds the Hetzner cloud provider.

Types

type ClusterConfig

type ClusterConfig struct {
	ImagesForArch    ImageList
	NodeConfigs      map[string]*NodeConfig
	IsUsingNewFormat bool
	LegacyConfig     LegacyConfig
}

ClusterConfig holds the configuration for all the nodepools

type HetznerCloudProvider

type HetznerCloudProvider struct {
	// contains filtered or unexported fields
}

HetznerCloudProvider implements CloudProvider interface.

func (*HetznerCloudProvider) Cleanup

func (d *HetznerCloudProvider) Cleanup() error

Cleanup cleans up open resources before the cloud provider is destroyed, i.e. go routines etc.

func (*HetznerCloudProvider) GPULabel

func (d *HetznerCloudProvider) GPULabel() string

GPULabel returns the label added to nodes with GPU resource.

func (*HetznerCloudProvider) GetAvailableGPUTypes

func (d *HetznerCloudProvider) GetAvailableGPUTypes() map[string]struct{}

GetAvailableGPUTypes return all available GPU types cloud provider supports.

func (*HetznerCloudProvider) GetAvailableMachineTypes

func (d *HetznerCloudProvider) GetAvailableMachineTypes() ([]string, error)

GetAvailableMachineTypes get all machine types that can be requested from the cloud provider. Implementation optional.

func (*HetznerCloudProvider) GetNodeGpuConfig

func (d *HetznerCloudProvider) GetNodeGpuConfig(node *apiv1.Node) *cloudprovider.GpuConfig

GetNodeGpuConfig returns the label, type and resource name for the GPU added to node. If node doesn't have any GPUs, it returns nil.

func (*HetznerCloudProvider) GetResourceLimiter

func (d *HetznerCloudProvider) GetResourceLimiter() (*cloudprovider.ResourceLimiter, error)

GetResourceLimiter returns struct containing limits (max, min) for resources (cores, memory etc.).

func (*HetznerCloudProvider) HasInstance

func (d *HetznerCloudProvider) HasInstance(node *apiv1.Node) (bool, error)

HasInstance returns whether a given node has a corresponding instance in this cloud provider

func (*HetznerCloudProvider) Name

func (d *HetznerCloudProvider) Name() string

Name returns name of the cloud provider.

func (*HetznerCloudProvider) NewNodeGroup

func (d *HetznerCloudProvider) NewNodeGroup(
	machineType string,
	labels map[string]string,
	systemLabels map[string]string,
	taints []apiv1.Taint,
	extraResources map[string]resource.Quantity,
) (cloudprovider.NodeGroup, error)

NewNodeGroup builds a theoretical node group based on the node definition provided. The node group is not automatically created on the cloud provider side. The node group is not returned by NodeGroups() until it is created. Implementation optional.

func (*HetznerCloudProvider) NodeGroupForNode

func (d *HetznerCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)

NodeGroupForNode returns the node group for the given node, nil if the node should not be processed by cluster autoscaler, or non-nil error if such occurred. Must be implemented.

func (*HetznerCloudProvider) NodeGroups

func (d *HetznerCloudProvider) NodeGroups() []cloudprovider.NodeGroup

NodeGroups returns all node groups configured for this cloud provider.

func (*HetznerCloudProvider) Pricing

func (d *HetznerCloudProvider) Pricing() (cloudprovider.PricingModel, autoscalerErrors.AutoscalerError)

Pricing returns pricing model for this cloud provider or error if not available. Implementation optional.

func (*HetznerCloudProvider) Refresh

func (d *HetznerCloudProvider) Refresh() error

Refresh is called before every main loop and can be used to dynamically update cloud provider state. In particular the list of node groups returned by NodeGroups() can change as a result of CloudProvider.Refresh().

type ImageList

type ImageList struct {
	Arm64 string
	Amd64 string
}

ImageList holds the image id/names for the different architectures

type LegacyConfig

type LegacyConfig struct {
	CloudInit string
	ImageName string
}

LegacyConfig holds the configuration in the legacy format

type NodeConfig

type NodeConfig struct {
	CloudInit      string
	PlacementGroup string
	Taints         []apiv1.Taint
	Labels         map[string]string
}

NodeConfig holds the configuration for a single nodepool