azure

package
v0.0.0-...-6b2d771 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 11, 2017 License: Apache-2.0 Imports: 20 Imported by: 0

README

Cluster Autoscaler on Azure

The cluster autoscaler on Azure scales worker nodes within any specified autoscaling group. It will run as a Deployment in your cluster. This README will go over some of the necessary steps required to get the cluster autoscaler up and running.

Kubernetes Version

Cluster autoscaler must run on Kubernetes with Azure VMSS support (kubernetes#43287). It is planed in Kubernetes v1.10.

Permissions

Get azure credentials by running the following command

# replace <subscription-id> with yours.
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/<subscription-id>" --output json

And fill the values with the result you got into the configmap

apiVersion: v1
data:
  ClientID: <client-id>
  ClientSecret: <client-secret>
  ResourceGroup: <resource-group>
  SubscriptionID: <subscription-id>
  TenantID: <tenand-id>
  ScaleSetName: <scale-set-name>
kind: ConfigMap
metadata:
  name: cluster-autoscaler-azure
  namespace: kube-system

Create the configmap by running

kubectl create -f cluster-autoscaler-azure-configmap.yaml

Deployment

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    app: cluster-autoscaler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      containers:
      - image: gcr.io/google_containers/cluster-autoscaler:{{ ca_version }}
        name: cluster-autoscaler
        resources:
          limits:
            cpu: 100m
            memory: 300Mi
          requests:
            cpu: 100m
            memory: 300Mi
        env:
        - name: ARM_SUBSCRIPTION_ID
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: SubscriptionID
        - name: ARM_RESOURCE_GROUP
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ResourceGroup
        - name: ARM_TENANT_ID
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: TenantID
        - name: ARM_CLIENT_ID
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ClientID
        - name: ARM_CLIENT_SECRET
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ClientSecret
        - name: ARM_SCALE_SET_NAME
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ScaleSetName
        command:
          - ./cluster-autoscaler
          - --v=4
          - --cloud-provider=azure
          - --skip-nodes-with-local-storage=false
          - --nodes="1:10:$(ARM_SCALE_SET_NAME)"
        volumeMounts:
          - name: ssl-certs
            mountPath: /etc/ssl/certs/ca-certificates.crt
            readOnly: true
        imagePullPolicy: "Always"
      volumes:
      - name: ssl-certs
        hostPath:
          path: "/etc/ssl/certs/ca-certificates.crt"

Deploy in master node

To run a CA pod in master node - CA deployment should tolerate the master taint and nodeSelector should be used to schedule the pods in master node.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    app: cluster-autoscaler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      nodeSelector:
        kubernetes.io/role: master
      containers:
      - image: gcr.io/google_containers/cluster-autoscaler:{{ ca_version }}
        name: cluster-autoscaler
        resources:
          limits:
            cpu: 100m
            memory: 300Mi
          requests:
            cpu: 100m
            memory: 300Mi
        env:
        - name: ARM_SUBSCRIPTION_ID
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: SubscriptionID
        - name: ARM_RESOURCE_GROUP
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ResourceGroup
        - name: ARM_TENANT_ID
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: TenantID
        - name: ARM_CLIENT_ID
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ClientID
        - name: ARM_CLIENT_SECRET
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ClientSecret
        - name: ARM_SCALE_SET_NAME
          valueFrom:
            configMapKeyRef:
              name: cluster-autoscaler-azure
              key: ScaleSetName
        command:
          - ./cluster-autoscaler
          - --v=4
          - --cloud-provider=azure
          - --skip-nodes-with-local-storage=false
          - --nodes="1:10:$(ARM_SCALE_SET_NAME)"
        volumeMounts:
          - name: ssl-certs
            mountPath: /etc/ssl/certs/ca-certificates.crt
            readOnly: true
        imagePullPolicy: "Always"
      volumes:
      - name: ssl-certs
        hostPath:
          path: "/etc/ssl/certs/ca-certificates.crt"

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewServicePrincipalTokenFromCredentials 

func NewServicePrincipalTokenFromCredentials(tenantID string, clientID string, clientSecret string, scope string) (*adal.ServicePrincipalToken, error)

NewServicePrincipalTokenFromCredentials creates a new ServicePrincipalToken using values of the passed credentials map.

Types

type AzureCloudProvider 

type AzureCloudProvider struct {
	// contains filtered or unexported fields
}

AzureCloudProvider provides implementation of CloudProvider interface for Azure.

func BuildAzureCloudProvider 

func BuildAzureCloudProvider(azureManager *AzureManager, specs []string, resourceLimiter *cloudprovider.ResourceLimiter) (*AzureCloudProvider, error)

BuildAzureCloudProvider creates new AzureCloudProvider

func (*AzureCloudProvider) Cleanup 

func (azure *AzureCloudProvider) Cleanup() error

Cleanup stops the go routine that is handling the current view of the ASGs in the form of a cache

func (*AzureCloudProvider) GetAvailableMachineTypes 

func (azure *AzureCloudProvider) GetAvailableMachineTypes() ([]string, error)

GetAvailableMachineTypes get all machine types that can be requested from the cloud provider.

func (*AzureCloudProvider) GetResourceLimiter 

func (azure *AzureCloudProvider) GetResourceLimiter() (*cloudprovider.ResourceLimiter, error)

GetResourceLimiter returns struct containing limits (max, min) for resources (cores, memory etc.).

func (*AzureCloudProvider) Name 

func (azure *AzureCloudProvider) Name() string

Name returns name of the cloud provider.

func (*AzureCloudProvider) NewNodeGroup 

func (azure *AzureCloudProvider) NewNodeGroup(machineType string, labels map[string]string, systemLabels map[string]string, extraResources map[string]resource.Quantity) (cloudprovider.NodeGroup, error)

NewNodeGroup builds a theoretical node group based on the node definition provided. The node group is not automatically created on the cloud provider side. The node group is not returned by NodeGroups() until it is created.

func (*AzureCloudProvider) NodeGroupForNode 

func (azure *AzureCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)

NodeGroupForNode returns the node group for the given node.

func (*AzureCloudProvider) NodeGroups 

func (azure *AzureCloudProvider) NodeGroups() []cloudprovider.NodeGroup

NodeGroups returns all node groups configured for this cloud provider.

func (*AzureCloudProvider) Pricing 

func (azure *AzureCloudProvider) Pricing() (cloudprovider.PricingModel, errors.AutoscalerError)

Pricing returns pricing model for this cloud provider or error if not available.

func (*AzureCloudProvider) Refresh 

func (azure *AzureCloudProvider) Refresh() error

Refresh is called before every main loop and can be used to dynamically update cloud provider state. In particular the list of node groups returned by NodeGroups can change as a result of CloudProvider.Refresh().

type AzureManager 

type AzureManager struct {
	// contains filtered or unexported fields
}

AzureManager handles Azure communication and data caching.

func CreateAzureManager 

func CreateAzureManager(configReader io.Reader) (*AzureManager, error)

CreateAzureManager creates Azure Manager object to work with Azure.

func (*AzureManager) Cleanup 

func (m *AzureManager) Cleanup()

Cleanup closes the channel to signal the go routine to stop that is handling the cache

func (*AzureManager) DeleteInstances 

func (m *AzureManager) DeleteInstances(instances []*AzureRef) error

DeleteInstances deletes the given instances. All instances must be controlled by the same ASG.

func (*AzureManager) GetScaleSetForInstance 

func (m *AzureManager) GetScaleSetForInstance(instance *AzureRef) (*ScaleSet, error)

GetScaleSetForInstance returns ScaleSetConfig of the given Instance

func (*AzureManager) GetScaleSetSize 

func (m *AzureManager) GetScaleSetSize(asConfig *ScaleSet) (int64, error)

GetScaleSetSize gets Scale Set size.

func (*AzureManager) GetScaleSetVms 

func (m *AzureManager) GetScaleSetVms(scaleSet *ScaleSet) ([]string, error)

GetScaleSetVms returns list of nodes for the given scale set.

func (*AzureManager) RegisterScaleSet 

func (m *AzureManager) RegisterScaleSet(scaleSet *ScaleSet)

RegisterScaleSet registers scale set in Azure Manager.

func (*AzureManager) SetScaleSetSize 

func (m *AzureManager) SetScaleSetSize(asConfig *ScaleSet, size int64) error

SetScaleSetSize sets ScaleSet size.

type AzureRef 

type AzureRef struct {
	Name string
}

AzureRef contains a reference to some entity in Azure world.

func AzureRefFromProviderId 

func AzureRefFromProviderId(id string) (*AzureRef, error)

AzureRefFromProviderId creates InstanceConfig object from provider id which must be in format: azure:///resourceGroupName/name

func (*AzureRef) GetKey 

func (m *AzureRef) GetKey() string

GetKey returns key of the given azure reference.

type Config 

type Config struct {
	Cloud                      string `json:"cloud" yaml:"cloud"`
	TenantID                   string `json:"tenantId" yaml:"tenantId"`
	SubscriptionID             string `json:"subscriptionId" yaml:"subscriptionId"`
	ResourceGroup              string `json:"resourceGroup" yaml:"resourceGroup"`
	Location                   string `json:"location" yaml:"location"`
	VnetName                   string `json:"vnetName" yaml:"vnetName"`
	SubnetName                 string `json:"subnetName" yaml:"subnetName"`
	SecurityGroupName          string `json:"securityGroupName" yaml:"securityGroupName"`
	RouteTableName             string `json:"routeTableName" yaml:"routeTableName"`
	PrimaryAvailabilitySetName string `json:"primaryAvailabilitySetName" yaml:"primaryAvailabilitySetName"`

	AADClientID     string `json:"aadClientId" yaml:"aadClientId"`
	AADClientSecret string `json:"aadClientSecret" yaml:"aadClientSecret"`
	AADTenantID     string `json:"aadTenantId" yaml:"aadTenantId"`
}

Config holds the configuration parsed from the --cloud-config flag

type ScaleSet 

type ScaleSet struct {
	AzureRef
	// contains filtered or unexported fields
}

ScaleSet implements NodeGroup interface.

func (*ScaleSet) Autoprovisioned 

func (scaleSet *ScaleSet) Autoprovisioned() bool

Autoprovisioned returns true if the node group is autoprovisioned.

func (*ScaleSet) Belongs 

func (scaleSet *ScaleSet) Belongs(node *apiv1.Node) (bool, error)

Belongs returns true if the given node belongs to the NodeGroup.

func (*ScaleSet) Create 

func (scaleSet *ScaleSet) Create() error

Create creates the node group on the cloud provider side.

func (*ScaleSet) Debug 

func (scaleSet *ScaleSet) Debug() string

Debug returns a debug string for the Scale Set.

func (*ScaleSet) DecreaseTargetSize 

func (scaleSet *ScaleSet) DecreaseTargetSize(delta int) error

DecreaseTargetSize decreases the target size of the node group. This function doesn't permit to delete any existing node and can be used only to reduce the request for new nodes that have not been yet fulfilled. Delta should be negative. It is assumed that cloud provider will not delete the existing nodes if the size when there is an option to just decrease the target.

func (*ScaleSet) Delete 

func (scaleSet *ScaleSet) Delete() error

Delete deletes the node group on the cloud provider side. This will be executed only for autoprovisioned node groups, once their size drops to 0.

func (*ScaleSet) DeleteNodes 

func (scaleSet *ScaleSet) DeleteNodes(nodes []*apiv1.Node) error

DeleteNodes deletes the nodes from the group.

func (*ScaleSet) Exist 

func (scaleSet *ScaleSet) Exist() bool

Exist checks if the node group really exists on the cloud provider side. Allows to tell the theoretical node group from the real one.

func (*ScaleSet) Id 

func (scaleSet *ScaleSet) Id() string

Id returns ScaleSet id.

func (*ScaleSet) IncreaseSize 

func (scaleSet *ScaleSet) IncreaseSize(delta int) error

IncreaseSize increases Scale Set size

func (*ScaleSet) MaxSize 

func (scaleSet *ScaleSet) MaxSize() int

MaxSize returns maximum size of the node group.

func (*ScaleSet) MinSize 

func (scaleSet *ScaleSet) MinSize() int

MinSize returns minimum size of the node group.

func (*ScaleSet) Nodes 

func (scaleSet *ScaleSet) Nodes() ([]string, error)

Nodes returns a list of all nodes that belong to this node group.

func (*ScaleSet) TargetSize 

func (scaleSet *ScaleSet) TargetSize() (int, error)

TargetSize returns the current TARGET size of the node group. It is possible that the number is different from the number of nodes registered in Kubernetes.

func (*ScaleSet) TemplateNodeInfo 

func (scaleSet *ScaleSet) TemplateNodeInfo() (*schedulercache.NodeInfo, error)

TemplateNodeInfo returns a node template for this scale set.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.