aws

README

Cluster Autoscaler on AWS

The cluster autoscaler on AWS scales worker nodes within an autoscaling group. It will run as a Deployment in your cluster. This README will go over some of the necessary steps required to get the cluster autoscaler up and running.

Kubernetes Version

Cluster autoscaler must run on v1.3.0 or greater.

Permissions

The worker running the cluster autoscaler will need access to certain resources and actions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup"
            ],
            "Resource": "*"
        }
    ]
}

Unfortunately AWS does not support ARNs for autoscaling groups yet so you must use "*" as the resource. More information here.

Deployment Specification

Your deployment configuration should look something like this:

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cluster-autoscaler
  labels:
    app: cluster-autoscaler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      containers:
        - image: {{ YOUR IMAGE HERE }}
          name: cluster-autoscaler
          resources:
            limits:
              cpu: 100m
              memory: 300Mi
            requests:
              cpu: 100m
              memory: 300Mi
          command:
            - ./cluster-autoscaler
            - --v=4
            - --cloud-provider=aws
            - --skip-nodes-with-local-storage=false
            - --nodes={{ ASG MIN e.g. 1 }}:{{ASG MAX e.g. 5}}:{{ASG NAME e.g. k8s-worker-asg}}
          env:
            - name: AWS_REGION
              value: us-east-1
          volumeMounts:
            - name: ssl-certs
              mountPath: /etc/ssl/certs/ca-certificates.crt
              readOnly: true
          imagePullPolicy: "Always"
      volumes:
        - name: ssl-certs
          hostPath:
            path: "/etc/ssl/certs/ca-certificates.crt"

Note:

• The /etc/ssl/certs/ca-certificates.crt should exist by default on your ec2 instance.
• The autoscaling group should span 1 availability zone for the cluster autoscaler to work. If you want to distribute workloads evenly across zones, set up multiple ASGs, with a cluster autoscaler for each ASG. At the time of writing this, cluster autoscaler is unaware of availability zones and although autoscaling groups can contain instances in multiple availability zones when configured so, the cluster autoscaler can't reliably add nodes to desired zones. That's because AWS AutoScaling determines which zone to add nodes which is out of the control of the cluster autoscaler. For more information, see https://github.com/kubernetes/contrib/pull/1552#discussion_r75533090.

Documentation

Index

• type Asg
  • func (asg *Asg) Belongs(node *apiv1.Node) (bool, error)
  • func (asg *Asg) Debug() string
  • func (asg *Asg) DeleteNodes(nodes []*apiv1.Node) error
  • func (asg *Asg) Id() string
  • func (asg *Asg) IncreaseSize(delta int) error
  • func (asg *Asg) MaxSize() int
  • func (asg *Asg) MinSize() int
  • func (asg *Asg) TargetSize() (int, error)
• type AwsCloudProvider
  • func BuildAwsCloudProvider(awsManager *AwsManager, specs []string) (*AwsCloudProvider, error)
  • func (aws *AwsCloudProvider) Name() string
  • func (aws *AwsCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)
  • func (aws *AwsCloudProvider) NodeGroups() []cloudprovider.NodeGroup
• type AwsManager
  • func CreateAwsManager(configReader io.Reader) (*AwsManager, error)
  • func (m *AwsManager) DeleteInstances(instances []*AwsRef) error
  • func (m *AwsManager) GetAsgForInstance(instance *AwsRef) (*Asg, error)
  • func (m *AwsManager) GetAsgSize(asgConfig *Asg) (int64, error)
  • func (m *AwsManager) RegisterAsg(asg *Asg)
  • func (m *AwsManager) SetAsgSize(asg *Asg, size int64) error
• type AwsRef
  • func AwsRefFromProviderId(id string) (*AwsRef, error)

Types

type Asg

type Asg struct {
	AwsRef
	// contains filtered or unexported fields
}

Asg implements NodeGroup interfrace.

func (*Asg) Belongs

func (asg *Asg) Belongs(node *apiv1.Node) (bool, error)

Belongs returns true if the given node belongs to the NodeGroup.

func (*Asg) Debug

func (asg *Asg) Debug() string

Debug returns a debug string for the Asg.

func (*Asg) DeleteNodes

func (asg *Asg) DeleteNodes(nodes []*apiv1.Node) error

DeleteNodes deletes the nodes from the group.

func (*Asg) Id

func (asg *Asg) Id() string

Id returns asg id.

func (*Asg) IncreaseSize

func (asg *Asg) IncreaseSize(delta int) error

IncreaseSize increases Asg size

func (*Asg) MaxSize

func (asg *Asg) MaxSize() int

MaxSize returns maximum size of the node group.

func (*Asg) MinSize

func (asg *Asg) MinSize() int

MinSize returns minimum size of the node group.

func (*Asg) TargetSize

func (asg *Asg) TargetSize() (int, error)

TargetSize returns the current TARGET size of the node group. It is possible that the number is different from the number of nodes registered in Kuberentes.

type AwsCloudProvider

type AwsCloudProvider struct {
	// contains filtered or unexported fields
}

AwsCloudProvider implements CloudProvider interface.

func BuildAwsCloudProvider

func BuildAwsCloudProvider(awsManager *AwsManager, specs []string) (*AwsCloudProvider, error)

BuildAwsCloudProvider builds CloudProvider implementation for AWS.

func (*AwsCloudProvider) Name

func (aws *AwsCloudProvider) Name() string

Name returns name of the cloud provider.

func (*AwsCloudProvider) NodeGroupForNode

func (aws *AwsCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)

NodeGroupForNode returns the node group for the given node.

func (*AwsCloudProvider) NodeGroups

func (aws *AwsCloudProvider) NodeGroups() []cloudprovider.NodeGroup

NodeGroups returns all node groups configured for this cloud provider.

type AwsManager

type AwsManager struct {
	// contains filtered or unexported fields
}

AwsManager is handles aws communication and data caching.

func CreateAwsManager

func CreateAwsManager(configReader io.Reader) (*AwsManager, error)

CreateAwsManager constructs awsManager object.

func (*AwsManager) DeleteInstances

func (m *AwsManager) DeleteInstances(instances []*AwsRef) error

DeleteInstances deletes the given instances. All instances must be controlled by the same ASG.

func (*AwsManager) GetAsgForInstance

func (m *AwsManager) GetAsgForInstance(instance *AwsRef) (*Asg, error)

GetAsgForInstance returns AsgConfig of the given Instance

func (*AwsManager) GetAsgSize

func (m *AwsManager) GetAsgSize(asgConfig *Asg) (int64, error)

GetAsgSize gets ASG size.

func (*AwsManager) RegisterAsg

func (m *AwsManager) RegisterAsg(asg *Asg)

RegisterAsg registers asg in Aws Manager.

func (*AwsManager) SetAsgSize

func (m *AwsManager) SetAsgSize(asg *Asg, size int64) error

SetAsgSize sets ASG size.

type AwsRef

type AwsRef struct {
	Name string
}

AwsRef contains a reference to some entity in AWS/GKE world.

func AwsRefFromProviderId

func AwsRefFromProviderId(id string) (*AwsRef, error)

AwsRefFromProviderId creates InstanceConfig object from provider id which must be in format: aws:///zone/name