cloudstack

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 10, 2024 License: Apache-2.0 Imports: 14 Imported by: 0

README

CloudStack Kubernetes Provider

A Cloud Controller Manager to facilitate Kubernetes deployments on Cloudstack.

Based on the old Cloudstack provider in Kubernetes was removed.

Refer:

Deployment

The CloudStack Kubernetes Provider is automatically deployed when a Kubernetes Cluster is created on CloudStack 4.16+

In order to communicate with CloudStack, a separate service user kubeadmin is created in the same account as the cluster owner. The provider uses this user's API keys to get the details of the cluster as well as update the networking rules. It is imperative that this user is not altered or have its keys regenerated.

The provider can also be manually deployed as follows :

Kubernetes

Prebuilt containers are posted on Docker Hub.

To configure API access to your CloudStack management server, you need to create a secret containing a cloud-config that is suitable for your environment.

cloud-config should look like this:

[Global]
api-url = <CloudStack API URL>
api-key = <CloudStack API Key>
secret-key = <CloudStack API Secret>
project-id = <CloudStack Project UUID (optional)>
zone = <CloudStack Zone Name (optional)>
ssl-no-verify = <Disable SSL certificate validation: true or false (optional)>

The access token needs to be able to fetch VM information and deploy load balancers in the project or domain where the nodes reside.

To create the secret, use the following command:

kubectl -n kube-system create secret generic cloudstack-secret --from-file=cloud-config

You can then use the provided example deployment.yaml to deploy the controller:

kubectl apply -f deployment.yaml
Protocols

This CCM supports TCP, UDP and TCP-Proxy LoadBalancer deployments.

For UDP and Proxy Protocol support, CloudStack 4.6 or later is required.

Since kube-proxy does not support the Proxy Protocol or UDP, you should connect this directly to pods, for example by deploying a DaemonSet and setting hostPort: <TCP port> on the desired container port. Important: The service running in the pod must support the chosen protocol. Do not try to enable TCP-Proxy when the service only supports regular TCP.

traefik-ingress-controller.yml contains a basic deployment for the Træfik ingress controller that illustrates how to use it with the proxy protocol.

For the nginx ingress controller, please refer to the official documentation at kubernetes.github.io/ingress-nginx/deploy. After applying the deployment, patch it for proxy protocol support with the provided fragment:

kubectl apply -f nginx-ingress-controller-patch.yml
Node Labels

⚠ The node name must match the host name, so the controller can fetch and assign metadata from CloudStack.

It is recommended to launch kubelet with the following parameter:

--register-with-taints=node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule

This will treat the node as 'uninitialized' and cause the CCM to apply metadata labels from CloudStack automatically.

Supported labels for Kubernetes versions up to 1.16 are:

  • kubernetes.io/hostname (= the instance name)
  • beta.kubernetes.io/instance-type (= the compute offering)
  • failure-domain.beta.kubernetes.io/zone (= the zone)
  • failure-domain.beta.kubernetes.io/region (also = the zone)

Supported labels for Kubernetes versions 1.17 and later are:

  • kubernetes.io/hostname (= the instance name)
  • node.kubernetes.io/instance-type (= the compute offering)
  • topology.kubernetes.io/zone (= the zone)
  • topology.kubernetes.io/region (also = the zone)

It is also possible to trigger this process manually by issuing the following command:

kubectl taint nodes <my-node-without-labels> node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule

Migration Guide

There are several notable differences to the old Kubernetes CloudStack cloud provider that need to be taken into account when migrating from the old cloud provider to the standalone controller.

Load Balancer

Load balancer rule names now include the protocol in addition to the LB name and service port. This was added to distinguish tcp, udp and tcp-proxy services operating on the same port. Without this change, it would not be possible to map a service that runs on both TCP and UDP port 8000, for example.

⚠ If you have existing rules, remove them before the migration, and add them back afterwards.

If you don't do this, you will end up with duplicate rules for the same service, which won't work.

Metadata

Since the controller is now intended to be run inside a pod and not on the node, it will not be able to fetch metadata from the Virtual Router's DHCP server.

Instead, it first obtains the name of the node from Kubernetes, then fetches information from the CloudStack API.

Development

Building

At least Go 1.13 is required to build cloudstack-ccm.

To build the controller with correct versioning, some build flags need to be passed. A Makefile is provided that sets these build flags to automatically derived values.

go get github.com/apache/cloudstack-kubernetes-provider
cd ${GOPATH}/src/github.com/apache/cloudstack-kubernetes-provider
make

To build the cloudstack-cloud-controller-manager container, please use the provided Dockerfile. The Makefile will also with that and properly tag the resulting container.

make docker
Testing

You need a local instance of the CloudStack Management Server or a 'real' one to connect to. The CCM supports the same cloud-config configuration file format used by the cs tool, so you can simply point it to that.

./cloudstack-ccm --cloud-provider external-cloudstack --cloud-config ~/.cloud-config --master k8s-apiserver

Replace k8s-apiserver with the host name of your Kubernetes development clusters's API server.

If you don't have a 'real' CloudStack installation, you can also launch a local simulator instance instead. This is very useful for dry-run testing.

Copyright 2019 The Apache Software Foundation

This product includes software developed at The Apache Software Foundation (http://www.apache.org/).

Documentation

Index

Constants

View Source 
const (

	// ServiceAnnotationLoadBalancerProxyProtocol is the annotation used on the
	// service to enable the proxy protocol on a CloudStack load balancer.
	// Note that this protocol only applies to TCP service ports and
	// CloudStack >= 4.6 is required for it to work.
	ServiceAnnotationLoadBalancerProxyProtocol = "service.beta.kubernetes.io/cloudstack-load-balancer-proxy-protocol"

	ServiceAnnotationLoadBalancerLoadbalancerHostname = "service.beta.kubernetes.io/cloudstack-load-balancer-hostname"
)
View Source 
const ProviderName = "external-cloudstack"

ProviderName is the name of this cloud provider.

Variables

This section is empty.

Functions

This section is empty.

Types

type CSCloud 

type CSCloud struct {
	// contains filtered or unexported fields
}

CSCloud is an implementation of Interface for CloudStack.

func (*CSCloud) AddSSHKeyToAllInstances 

func (cs *CSCloud) AddSSHKeyToAllInstances(ctx context.Context, user string, keyData []byte) error

AddSSHKeyToAllInstances is currently not implemented.

func (*CSCloud) Clusters 

func (cs *CSCloud) Clusters() (cloudprovider.Clusters, bool)

Clusters returns an implementation of Clusters for CloudStack.

func (*CSCloud) CurrentNodeName 

func (cs *CSCloud) CurrentNodeName(ctx context.Context, hostname string) (types.NodeName, error)

CurrentNodeName returns the name of the node we are currently running on.

func (*CSCloud) EnsureLoadBalancer 

func (cs *CSCloud) EnsureLoadBalancer(ctx context.Context, clusterName string, service *corev1.Service, nodes []*corev1.Node) (status *corev1.LoadBalancerStatus, err error)

EnsureLoadBalancer creates a new load balancer, or updates the existing one. Returns the status of the balancer.

func (*CSCloud) EnsureLoadBalancerDeleted 

func (cs *CSCloud) EnsureLoadBalancerDeleted(ctx context.Context, clusterName string, service *corev1.Service) error

EnsureLoadBalancerDeleted deletes the specified load balancer if it exists, returning nil if the load balancer specified either didn't exist or was successfully deleted.

func (*CSCloud) GetLoadBalancer 

func (cs *CSCloud) GetLoadBalancer(ctx context.Context, clusterName string, service *corev1.Service) (*corev1.LoadBalancerStatus, bool, error)

GetLoadBalancer returns whether the specified load balancer exists, and if so, what its status is.

func (*CSCloud) GetLoadBalancerName 

func (cs *CSCloud) GetLoadBalancerName(ctx context.Context, clusterName string, service *corev1.Service) string

GetLoadBalancerName retrieves the name of the LoadBalancer.

func (*CSCloud) GetZone 

func (cs *CSCloud) GetZone(ctx context.Context) (cloudprovider.Zone, error)

GetZone returns the Zone containing the region that the program is running in.

func (*CSCloud) GetZoneByNodeName 

func (cs *CSCloud) GetZoneByNodeName(ctx context.Context, nodeName types.NodeName) (cloudprovider.Zone, error)

GetZoneByNodeName returns the Zone, found by using the node name.

func (*CSCloud) GetZoneByProviderID 

func (cs *CSCloud) GetZoneByProviderID(ctx context.Context, providerID string) (cloudprovider.Zone, error)

GetZoneByProviderID returns the Zone, found by using the provider ID.

func (*CSCloud) HasClusterID 

func (cs *CSCloud) HasClusterID() bool

HasClusterID returns true if the cluster has a clusterID

func (*CSCloud) Initialize 

func (cs *CSCloud) Initialize(clientBuilder cloudprovider.ControllerClientBuilder, stop <-chan struct{})

Initialize passes a Kubernetes clientBuilder interface to the cloud provider

func (*CSCloud) InstanceExists 

func (cs *CSCloud) InstanceExists(ctx context.Context, node *corev1.Node) (bool, error)

func (*CSCloud) InstanceExistsByProviderID 

func (cs *CSCloud) InstanceExistsByProviderID(ctx context.Context, providerID string) (bool, error)

InstanceExistsByProviderID returns if the instance still exists.

func (*CSCloud) InstanceID 

func (cs *CSCloud) InstanceID(ctx context.Context, name types.NodeName) (string, error)

InstanceID returns the cloud provider ID of the specified instance.

func (*CSCloud) InstanceMetadata 

func (cs *CSCloud) InstanceMetadata(ctx context.Context, node *corev1.Node) (*cloudprovider.InstanceMetadata, error)

func (*CSCloud) InstanceShutdown 

func (cs *CSCloud) InstanceShutdown(ctx context.Context, node *corev1.Node) (bool, error)

func (*CSCloud) InstanceShutdownByProviderID 

func (cs *CSCloud) InstanceShutdownByProviderID(ctx context.Context, providerID string) (bool, error)

InstanceShutdownByProviderID returns true if the instance is in safe state to detach volumes

func (*CSCloud) InstanceType 

func (cs *CSCloud) InstanceType(ctx context.Context, name types.NodeName) (string, error)

InstanceType returns the type of the specified instance.

func (*CSCloud) InstanceTypeByProviderID 

func (cs *CSCloud) InstanceTypeByProviderID(ctx context.Context, providerID string) (string, error)

InstanceTypeByProviderID returns the type of the specified instance.

func (*CSCloud) Instances 

func (cs *CSCloud) Instances() (cloudprovider.Instances, bool)

Instances returns an implementation of Instances for CloudStack.

func (*CSCloud) InstancesV2 

func (cs *CSCloud) InstancesV2() (cloudprovider.InstancesV2, bool)

func (*CSCloud) LoadBalancer 

func (cs *CSCloud) LoadBalancer() (cloudprovider.LoadBalancer, bool)

LoadBalancer returns an implementation of LoadBalancer for CloudStack.

func (*CSCloud) NodeAddresses 

func (cs *CSCloud) NodeAddresses(ctx context.Context, name types.NodeName) ([]corev1.NodeAddress, error)

NodeAddresses returns the addresses of the specified instance.

func (*CSCloud) NodeAddressesByProviderID 

func (cs *CSCloud) NodeAddressesByProviderID(ctx context.Context, providerID string) ([]corev1.NodeAddress, error)

NodeAddressesByProviderID returns the addresses of the specified instance.

func (*CSCloud) ProviderName 

func (cs *CSCloud) ProviderName() string

ProviderName returns the cloud provider ID.

func (*CSCloud) Routes 

func (cs *CSCloud) Routes() (cloudprovider.Routes, bool)

Routes returns an implementation of Routes for CloudStack.

func (*CSCloud) UpdateLoadBalancer 

func (cs *CSCloud) UpdateLoadBalancer(ctx context.Context, clusterName string, service *corev1.Service, nodes []*corev1.Node) error

UpdateLoadBalancer updates hosts under the specified load balancer.

func (*CSCloud) Zones 

func (cs *CSCloud) Zones() (cloudprovider.Zones, bool)

Zones returns an implementation of Zones for CloudStack.

type CSConfig 

type CSConfig struct {
	Global struct {
		APIURL      string `gcfg:"api-url"`
		APIKey      string `gcfg:"api-key"`
		SecretKey   string `gcfg:"secret-key"`
		SSLNoVerify bool   `gcfg:"ssl-no-verify"`
		ProjectID   string `gcfg:"project-id"`
		Zone        string `gcfg:"zone"`
	}
}

CSConfig wraps the config for the CloudStack cloud provider.

type LoadBalancerProtocol 

type LoadBalancerProtocol int

LoadBalancerProtocol represents a specific network protocol supported by the CloudStack load balancer.

It also allows easy mapping to standard protocol names. 

const (
	LoadBalancerProtocolTCP LoadBalancerProtocol = iota
	LoadBalancerProtocolUDP
	LoadBalancerProtocolTCPProxy
	LoadBalancerProtocolInvalid
)

func ProtocolFromLoadBalancer 

func ProtocolFromLoadBalancer(protocol string) LoadBalancerProtocol

ProtocolFromLoadBalancer returns the protocol corresponding to the CloudStack load balancer protocol name.

func ProtocolFromServicePort 

func ProtocolFromServicePort(port v1.ServicePort, service *v1.Service) LoadBalancerProtocol

ProtocolFromServicePort selects a suitable CloudStack protocol type based on a ServicePort object and annotations from a LoadBalancer definition.

Supported combinations include: 

v1.ProtocolTCP="tcp" -> "tcp"
v1.ProtocolTCP="udp" -> "udp" (CloudStack 4.6 and later)
v1.ProtocolTCP="tcp" + annotation "service.beta.kubernetes.io/cloudstack-load-balancer-proxy-protocol"
                     -> "tcp-proxy" (CloudStack 4.6 and later)

Other values return LoadBalancerProtocolInvalid.

func (LoadBalancerProtocol) CSProtocol 

func (p LoadBalancerProtocol) CSProtocol() string

CSProtocol returns the full CloudStack protocol name. Returns "" if the value is unknown.

func (LoadBalancerProtocol) IPProtocol 

func (p LoadBalancerProtocol) IPProtocol() string

IPProtocol returns the standard IP protocol name. Returns "" if the value is unknown.

func (LoadBalancerProtocol) String 

func (p LoadBalancerProtocol) String() string

String returns the same value as CSProtocol.

Source Files

View all Source files

Directories

Path Synopsis
cmd
cloudstack-ccm

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.