README
¶
Cluster Autoscaler on Azure
The cluster autoscaler on Azure dynamically scales Kubernetes worker nodes. It runs as a deployment in your cluster.
This README will help you get cluster autoscaler running on your Azure Kubernetes cluster.
Kubernetes Version
Kubernetes v1.10.x or later is required to use cluster autoscaler on Azure. See the "Releases" section in the README for more information.
CA Version
Cluster autoscaler v1.2.x or later is required for Azure. See the "Releases" section in the README for more information.
NOTE: In the deployment manifests referenced below, be sure to replace the
{{ ca_version }}
placeholder with an actual release, such asv1.14.2
.
Permissions
Get Azure credentials by running the following Azure CLI command:
# replace <subscription-id> with yours.
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/<subscription-id>" --output json
This will create a new service principal with "Contributor" role scoped to your subscription. Save the JSON output, because it will be needed to configure the cluster autoscaler deployment in the next step.
Scaling a VMSS node group to and from 0
If you are using nodeSelector
, you need to tag the VMSS with a node-template key "k8s.io_cluster-autoscaler_node-template_label_"
for using labels and "k8s.io_cluster-autoscaler_node-template_taint_"
if you are using taints.
Note that these tags use the pipe
_
character compared to a forward slash due to Azure tag name restrictions.
Examples
Labels
To add the label of foo=bar
to a node from a VMSS pool, you would add the following tag to the VMSS k8s.io_cluster-autoscaler_node-template_label_foo: bar
.
You can also use forward slashes in the labels by setting them as an underscore in the tag name. For example to add the label of k8s.io/foo=bar
to a node from a VMSS pool, you would add the following tag to the VMSS k8s.io_cluster-autoscaler_node-template_label_k8s.io_foo: bar
. To encode a tag name containing an underscore, use "~2" (eg. "cpu~2arch" gives "cpu_arch").
Taints
To add the taint of foo=bar:NoSchedule
to a node from a VMSS pool, you would add the following tag to the VMSS k8s.io_cluster-autoscaler_node-template_taint_foo: bar:NoSchedule
.
You can also use forward slashes in taints by setting them as an underscore in the tag name. For example to add the taint of k8s.io/foo=bar:NoSchedule
to a node from a VMSS pool, you would add the following tag to the VMSS k8s.io_cluster-autoscaler_node-template_taint_k8s.io_foo: bar:NoSchedule
. To encode a taint name containing an underscore, use "~2".
Resources
When scaling from an empty VM Scale Set (0 instances), Cluster Autoscaler will evaluate the provided resources (cpu, memory, ephemeral-storage) based on that VM Scale Set's backing instance type.
This can be overridden (for instance, to account for system reserved resources) by specifying capacities with VMSS tags, formated as: k8s.io_cluster-autoscaler_node-template_resources_<resource name>: <resource value>
. For instance:
k8s.io_cluster-autoscaler_node-template_resources_cpu: 3800m
k8s.io_cluster-autoscaler_node-template_resources_memory: 11Gi
NOTE: GPU autoscaling on VMSS is informed by the presence of the
kubernetes.azure.com/accelerator
Node label. A VMSS with GPUs whose Nodes do not have the label may not be scaled correctly. Theaccelerator
label was used for this purpose in versions 1.31 and older.
Autoscaling options
Some autoscaling options can be defined per VM Scale Set, with tags. Those tags values have the format as the respective cluster-autoscaler flags they override: floats or durations encoded as strings.
Supported options tags (with example values) are:
# overrides --scale-down-utilization-threshold global value for that specific VM Scale Set
k8s.io_cluster-autoscaler_node-template_autoscaling-options_scaledownutilizationthreshold: "0.5"
# overrides --scale-down-gpu-utilization-threshold global value for that specific VM Scale Set
k8s.io_cluster-autoscaler_node-template_autoscaling-options_scaledowngpuutilizationthreshold: "0.5"
# overrides --scale-down-unneeded-time global value for that specific VM Scale Set
k8s.io_cluster-autoscaler_node-template_autoscaling-options_scaledownunneededtime: "10m0s"
# overrides --scale-down-unready-time global value for that specific VM Scale Set
k8s.io_cluster-autoscaler_node-template_autoscaling-options_scaledownunreadytime: "20m0s"
Deployment manifests
Cluster autoscaler supports four Kubernetes cluster options on Azure:
- vmss: Autoscale VMSS instances by setting the Azure cloud provider's
vmType
parameter tovmss
or to an empty string. This supports clusters deployed with aks-engine. - standard: Autoscale VMAS (Virtual Machine Availability Set) VMs by setting the Azure cloud provider's
vmType
parameter tostandard
. This supports clusters deployed with aks-engine.
NOTE: only the
vmss
option supports scaling down to zero nodes.
NOTE: The
subscriptionID
parameter is optional. When skipped, the subscription will be fetched from the instance metadata.
VMSS deployment
Prerequisites:
- Get Azure credentials from the Permissions step above.
- Get the name of the VM scale set associated with the cluster's node pool. You can find this in the Azure Portal or with the
az vmss list
command.
Make a copy of cluster-autoscaler-vmss.yaml. Fill in the placeholder values for the cluster-autoscaler-azure
secret data by base64-encoding each of your Azure credential fields.
- ClientID:
<base64-encoded-client-id>
- ClientSecret:
<base64-encoded-client-secret>
- ResourceGroup:
<base64-encoded-resource-group>
- SubscriptionID:
<base64-encoded-subscription-id>
- TenantID:
<base64-encoded-tenant-id>
NOTE: Use a command such as
echo $CLIENT_ID | base64
to encode each of the fields above.
In the cluster-autoscaler
spec, find the image:
field and replace {{ ca_version }}
with a specific cluster autoscaler release.
Auto-Discovery Setup
To run a cluster-autoscaler which auto-discovers VMSSs with nodes use the --node-group-auto-discovery
flag.
For example, --node-group-auto-discovery=label:cluster-autoscaler-enabled=true,cluster-autoscaler-name=<YOUR CLUSTER NAME>
will find the VMSSs tagged with those tags containing those values.
Note that:
- It is recommended to use a second tag like
cluster-autoscaler-name=<YOUR CLUSTER NAME>
whencluster-autoscaler-enabled=true
is used across many clusters to prevent VMSSs from different clusters recognized as the node groups - There are no
--nodes
flags passed to cluster-autoscaler because the node groups are automatically discovered by tags - No min/max values are provided when using Auto-Discovery, cluster-autoscaler will detect the "min" and "max" tags on the VMSS resource in Azure, adjusting the desired number of nodes within these limits.
kubectl apply -f examples/cluster-autoscaler-autodiscover.yaml
Explicit setup
Below that, in the command:
section, update the --nodes=
arguments to reference your node limits and VMSS name. For example, if node pool "k8s-nodepool-1-vmss" should scale from 1 to 10 nodes:
- --nodes=1:10:k8s-nodepool-1-vmss
or to autoscale multiple VM scale sets:
- --nodes=1:10:k8s-nodepool-1-vmss
- --nodes=1:10:k8s-nodepool-2-vmss
Note that it doesn't mean the number of nodes in nodepool is restricted in the range from 1 to 10. It means when ca is downscaling (upscaling) the nodepool, it will never break the limit of 1 (10). If the current node pool size is lower than the specified minimum or greater than the specified maximum when you enable autoscaling, the autoscaler waits to take effect until a new node is needed in the node pool or until a node can be safely deleted from the node pool.
To allow scaling similar node pools simultaneously, or when using separate node groups per zone and to keep nodes balanced across zones, use the --balance-similar-node-groups
flag (default false). Add it to the command
section to enable it:
- --balance-similar-node-groups=true
See the FAQ for more details.
Save the updated deployment manifest, then deploy cluster-autoscaler by running:
kubectl create -f cluster-autoscaler-vmss.yaml
To run a cluster autoscaler pod on a control plane (previously referred to as master) node, the deployment should tolerate the master
taint, and nodeSelector
should be used to schedule pods. Use cluster-autoscaler-vmss-control-plane.yaml in this case.
To run a cluster autoscaler pod with Azure managed service identity (MSI), use cluster-autoscaler-vmss-msi.yaml instead.
Azure API Throttling
Azure has hard limits on the number of read and write requests against Azure APIs per subscription, per region. Running lots of clusters in a single subscription, or running a single large, dynamic cluster in a subscription can produce side effects that exceed the number of calls permitted within a given time window for a particular category of requests. See the following documents for more detail on Azure API throttling in general:
- https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling
- https://docs.microsoft.com/en-us/azure/virtual-machines/troubleshooting/troubleshooting-throttling-errors
Given the dynamic nature of cluster autoscaler, it can be a trigger for hitting those rate limits on the subscriptions. This in turn can affect other components running in the cluster that depend on Azure APIs such as kube-controller-manager.
When using K8s versions older than v1.18, we recommend using at least v.1.17.5, v1.16.9, v1.15.12 which include various improvements on the cloud-provider side that have an impact on the number of API calls during scale down operations.
As for CA versions older than 1.18, we recommend using at least v.1.17.2, v1.16.5, v1.15.6.
In addition, cluster-autoscaler exposes a AZURE_VMSS_CACHE_TTL
environment variable which controls the rate of GetVMScaleSet
being made. By default, this is 15 seconds but setting this to a higher value such as 60 seconds can protect against API throttling. The caches used are proactively incremented and decremented with the scale up and down operations and this higher value doesn't have any noticeable impact on performance. Note that the value is in seconds
Config Name | Default | Environment Variable | Cloud Config File |
---|---|---|---|
VmssCacheTTL | 60 | AZURE_VMSS_CACHE_TTL | vmssCacheTTL |
The AZURE_VMSS_VMS_CACHE_TTL
environment variable affects the GetScaleSetVms
(VMSS VM List) calls rate. The default value is 300 seconds.
A configurable jitter (AZURE_VMSS_VMS_CACHE_JITTER
environment variable, default 0) expresses the maximum number of second that will be subtracted from that initial VMSS cache TTL after a new VMSS is discovered by the cluster-autoscaler: this can prevent a dogpile effect on clusters having many VMSS.
Config Name | Default | Environment Variable | Cloud Config File |
---|---|---|---|
vmssVmsCacheTTL | 300 | AZURE_VMSS_VMS_CACHE_TTL | vmssVmsCacheTTL |
vmssVmsCacheJitter | 0 | AZURE_VMSS_VMS_CACHE_JITTER | vmssVmsCacheJitter |
The AZURE_ENABLE_DYNAMIC_INSTANCE_LIST
environment variable enables workflow that fetched SKU information dynamically using SKU API calls. By default, it uses static list of SKUs.
Config Name | Default | Environment Variable | Cloud Config File |
---|---|---|---|
enableDynamicInstanceList | false | AZURE_ENABLE_DYNAMIC_INSTANCE_LIST | enableDynamicInstanceList |
The AZURE_ENABLE_VMSS_FLEX
environment variable enables VMSS Flex support. By default, support is disabled.
Config Name | Default | Environment Variable | Cloud Config File |
---|---|---|---|
enableVmssFlex | false | AZURE_ENABLE_VMSS_FLEX | enableVmssFlex |
When using K8s 1.18 or higher, it is also recommended to configure backoff and retries on the client as described here
Standard deployment
Prerequisites:
- Get Azure credentials from the Permissions step above.
- Get the name of the initial Azure deployment resource for the cluster. You can find this in the Azure Portal or with the
az deployment list
command. If there are multiple deployments, get the name of the first one.
Make a copy of cluster-autoscaler-standard-control-plane.yaml. Fill in the placeholder values for the cluster-autoscaler-azure
secret data by base64-encoding each of your Azure credential fields.
- ClientID:
<base64-encoded-client-id>
- ClientSecret:
<base64-encoded-client-secret>
- ResourceGroup:
<base64-encoded-resource-group>
- SubscriptionID:
<base64-encoded-subscription-id>
- TenantID:
<base64-encoded-tenant-id>
- Deployment:
<base64-encoded-azure-initial-deployment-name>
NOTE: Use a command such as
echo $CLIENT_ID | base64
to encode each of the fields above.
In the cluster-autoscaler
spec, find the image:
field and replace {{ ca_version }}
with a specific cluster autoscaler release.
Below that, in the command:
section, update the --nodes=
arguments to reference your node limits and node pool name (tips: node pool name is NOT availability set name, e.g., the corresponding node pool name of the availability set
agentpool1-availabilitySet-xxxxxxxx
would be agentpool1
). For example, if node pool "k8s-nodepool-1" should scale from 1 to 10 nodes:
- --nodes=1:10:k8s-nodepool-1
or to autoscale multiple VM scale sets:
- --nodes=1:10:k8s-nodepool-1
- --nodes=1:10:k8s-nodepool-2
Create the Azure deploy parameters secret cluster-autoscaler-azure-deploy-parameters
by running:
kubectl -n kube-system create secret generic cluster-autoscaler-azure-deploy-parameters --from-file=deploy-parameters=./_output/<your-output-path>/azuredeploy.parameters.json
Then deploy cluster-autoscaler by running:
kubectl create -f cluster-autoscaler-standard-control-plane.yaml
To run a cluster autoscaler pod with Azure managed service identity (MSI), use cluster-autoscaler-standard-msi.yaml instead.
WARNING: Cluster autoscaler depends on user-provided deployment parameters to provision new nodes. After upgrading your Kubernetes cluster, cluster autoscaler must also be redeployed with new parameters to prevent provisioning nodes with an old version.
AKS Autoscaler
Node Pool Autoscaling is a first class feature of your AKS cluster. The option to enable cluster autoscaler is available in the Azure Portal or with the Azure CLI:
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--kubernetes-version 1.25.11 \
--node-count 1 \
--enable-cluster-autoscaler \
--min-count 1 \
--max-count 3
Please see the AKS autoscaler documentation for details.
Rate limit and back-off retries
The new version of Azure client supports rate limit and back-off retries when the cluster hits the throttling issue. These can be set by either environment variables, or cloud config file. With config file, defaults values are false or 0.
Config Name | Default | Environment Variable | Cloud Config File |
---|---|---|---|
CloudProviderBackoff | false | ENABLE_BACKOFF | cloudProviderBackoff |
CloudProviderBackoffRetries | 6 | BACKOFF_RETRIES | cloudProviderBackoffRetries |
CloudProviderBackoffExponent | 1.5 | BACKOFF_EXPONENT | cloudProviderBackoffExponent |
CloudProviderBackoffDuration | 5 | BACKOFF_DURATION | cloudProviderBackoffDuration |
CloudProviderBackoffJitter | 1.0 | BACKOFF_JITTER | cloudProviderBackoffJitter |
CloudProviderRateLimit * | false | CLOUD_PROVIDER_RATE_LIMIT | cloudProviderRateLimit |
CloudProviderRateLimitQPS * | 1 | RATE_LIMIT_READ_QPS | cloudProviderRateLimitQPS |
CloudProviderRateLimitBucket * | 5 | RATE_LIMIT_READ_BUCKETS | cloudProviderRateLimitBucket |
CloudProviderRateLimitQPSWrite * | 1 | RATE_LIMIT_WRITE_QPS | cloudProviderRateLimitQPSWrite |
CloudProviderRateLimitBucketWrite * | 5 | RATE_LIMIT_WRITE_BUCKETS | cloudProviderRateLimitBucketWrite |
NOTE: * These rate limit configs can be set per-client. Customizing
QPS
andBucket
through environment variables per client is not supported.
Documentation
¶
Index ¶
- Constants
- Variables
- func BuildAzure(opts config.AutoscalingOptions, do cloudprovider.NodeGroupDiscoveryOptions, ...) cloudprovider.CloudProvider
- func BuildAzureCloudProvider(azureManager *AzureManager, resourceLimiter *cloudprovider.ResourceLimiter) (cloudprovider.CloudProvider, error)
- func GetVMNameIndex(osType compute.OperatingSystemTypes, vmName string) (int, error)
- func ParseLabelAutoDiscoverySpecs(o cloudprovider.NodeGroupDiscoveryOptions) ([]labelAutoDiscoveryConfig, error)
- type AgentPool
- func (as *AgentPool) AtomicIncreaseSize(delta int) error
- func (as *AgentPool) Autoprovisioned() bool
- func (as *AgentPool) Belongs(node *apiv1.Node) (bool, error)
- func (as *AgentPool) Create() (cloudprovider.NodeGroup, error)
- func (as *AgentPool) Debug() string
- func (as *AgentPool) DecreaseTargetSize(delta int) error
- func (as *AgentPool) Delete() error
- func (as *AgentPool) DeleteInstances(instances []*azureRef) error
- func (as *AgentPool) DeleteNodes(nodes []*apiv1.Node) error
- func (as *AgentPool) Exist() bool
- func (as *AgentPool) ForceDeleteNodes(nodes []*apiv1.Node) error
- func (m *AgentPool) GetKey() string
- func (as *AgentPool) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)
- func (as *AgentPool) GetVMIndexes() ([]int, map[int]string, error)
- func (as *AgentPool) Id() string
- func (as *AgentPool) IncreaseSize(delta int) error
- func (as *AgentPool) MaxSize() int
- func (as *AgentPool) MinSize() int
- func (as *AgentPool) Nodes() ([]cloudprovider.Instance, error)
- func (m *AgentPool) String() string
- func (as *AgentPool) TargetSize() (int, error)
- func (as *AgentPool) TemplateNodeInfo() (*framework.NodeInfo, error)
- type AgentPoolsClient
- type AzUtil
- type AzureCloudProvider
- func (azure *AzureCloudProvider) Cleanup() error
- func (azure *AzureCloudProvider) GPULabel() string
- func (azure *AzureCloudProvider) GetAvailableGPUTypes() map[string]struct{}
- func (azure *AzureCloudProvider) GetAvailableMachineTypes() ([]string, error)
- func (azure *AzureCloudProvider) GetNodeGpuConfig(node *apiv1.Node) *cloudprovider.GpuConfig
- func (azure *AzureCloudProvider) GetResourceLimiter() (*cloudprovider.ResourceLimiter, error)
- func (azure *AzureCloudProvider) HasInstance(node *apiv1.Node) (bool, error)
- func (azure *AzureCloudProvider) Name() string
- func (azure *AzureCloudProvider) NewNodeGroup(machineType string, labels map[string]string, systemLabels map[string]string, ...) (cloudprovider.NodeGroup, error)
- func (azure *AzureCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)
- func (azure *AzureCloudProvider) NodeGroups() []cloudprovider.NodeGroup
- func (azure *AzureCloudProvider) Pricing() (cloudprovider.PricingModel, errors.AutoscalerError)
- func (azure *AzureCloudProvider) Refresh() error
- type AzureManager
- func (m *AzureManager) Cleanup()
- func (m *AzureManager) GetNodeGroupForInstance(instance *azureRef) (cloudprovider.NodeGroup, error)
- func (m *AzureManager) GetScaleSetOptions(scaleSetName string, defaults config.NodeGroupAutoscalingOptions) *config.NodeGroupAutoscalingOptions
- func (m *AzureManager) Refresh() error
- func (m *AzureManager) RegisterNodeGroup(nodeGroup cloudprovider.NodeGroup) bool
- func (m *AzureManager) UnregisterNodeGroup(nodeGroup cloudprovider.NodeGroup) bool
- type Config
- type DeploymentClientMock
- func (m *DeploymentClientMock) CreateOrUpdate(ctx context.Context, resourceGroupName string, deploymentName string, ...) (err *retry.Error)
- func (m *DeploymentClientMock) Delete(ctx context.Context, resourceGroupName, deploymentName string) (err *retry.Error)
- func (m *DeploymentClientMock) ExportTemplate(ctx context.Context, resourceGroupName string, deploymentName string) (result resources.DeploymentExportResult, err *retry.Error)
- func (m *DeploymentClientMock) Get(ctx context.Context, resourceGroupName string, deploymentName string) (result resources.DeploymentExtended, err *retry.Error)
- func (m *DeploymentClientMock) List(ctx context.Context, resourceGroupName string) (result []resources.DeploymentExtended, err *retry.Error)
- type GetVMSSFailedError
- type InstanceCache
- type InstanceType
- type MockAgentPoolsClient
- func (m *MockAgentPoolsClient) BeginCreateOrUpdate(arg0 context.Context, arg1, arg2, arg3 string, ...) (*runtime.Poller[armcontainerservice.AgentPoolsClientCreateOrUpdateResponse], ...)
- func (m *MockAgentPoolsClient) BeginDeleteMachines(arg0 context.Context, arg1, arg2, arg3 string, ...) (*runtime.Poller[armcontainerservice.AgentPoolsClientDeleteMachinesResponse], ...)
- func (m *MockAgentPoolsClient) EXPECT() *MockAgentPoolsClientMockRecorder
- func (m *MockAgentPoolsClient) Get(arg0 context.Context, arg1, arg2, arg3 string, ...) (armcontainerservice.AgentPoolsClientGetResponse, error)
- type MockAgentPoolsClientMockRecorder
- func (mr *MockAgentPoolsClientMockRecorder) BeginCreateOrUpdate(arg0, arg1, arg2, arg3, arg4, arg5 any) *gomock.Call
- func (mr *MockAgentPoolsClientMockRecorder) BeginDeleteMachines(arg0, arg1, arg2, arg3, arg4, arg5 any) *gomock.Call
- func (mr *MockAgentPoolsClientMockRecorder) Get(arg0, arg1, arg2, arg3, arg4 any) *gomock.Call
- type ScaleSet
- func (scaleSet *ScaleSet) AtomicIncreaseSize(delta int) error
- func (scaleSet *ScaleSet) Autoprovisioned() bool
- func (scaleSet *ScaleSet) Belongs(node *apiv1.Node) (bool, error)
- func (scaleSet *ScaleSet) Create() (cloudprovider.NodeGroup, error)
- func (scaleSet *ScaleSet) Debug() string
- func (scaleSet *ScaleSet) DecreaseTargetSize(delta int) error
- func (scaleSet *ScaleSet) Delete() error
- func (scaleSet *ScaleSet) DeleteInstances(instances []*azureRef, hasUnregisteredNodes bool) error
- func (scaleSet *ScaleSet) DeleteNodes(nodes []*apiv1.Node) error
- func (scaleSet *ScaleSet) Exist() bool
- func (scaleSet *ScaleSet) ForceDeleteNodes(nodes []*apiv1.Node) error
- func (scaleSet *ScaleSet) GetFlexibleScaleSetVms() ([]compute.VirtualMachine, *retry.Error)
- func (m *ScaleSet) GetKey() string
- func (scaleSet *ScaleSet) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)
- func (scaleSet *ScaleSet) GetScaleSetVms() ([]compute.VirtualMachineScaleSetVM, *retry.Error)
- func (scaleSet *ScaleSet) Id() string
- func (scaleSet *ScaleSet) IncreaseSize(delta int) error
- func (scaleSet *ScaleSet) MaxSize() int
- func (scaleSet *ScaleSet) MinSize() int
- func (scaleSet *ScaleSet) Nodes() ([]cloudprovider.Instance, error)
- func (m *ScaleSet) String() string
- func (scaleSet *ScaleSet) TargetSize() (int, error)
- func (scaleSet *ScaleSet) TemplateNodeInfo() (*framework.NodeInfo, error)
- type VMsPool
- func (agentPool *VMsPool) AtomicIncreaseSize(delta int) error
- func (agentPool *VMsPool) Autoprovisioned() bool
- func (agentPool *VMsPool) Create() (cloudprovider.NodeGroup, error)
- func (agentPool *VMsPool) Debug() string
- func (agentPool *VMsPool) DecreaseTargetSize(delta int) error
- func (agentPool *VMsPool) Delete() error
- func (agentPool *VMsPool) DeleteNodes(nodes []*apiv1.Node) error
- func (agentPool *VMsPool) Exist() bool
- func (agentPool *VMsPool) ForceDeleteNodes(nodes []*apiv1.Node) error
- func (m *VMsPool) GetKey() string
- func (agentPool *VMsPool) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)
- func (agentPool *VMsPool) Id() string
- func (agentPool *VMsPool) IncreaseSize(delta int) error
- func (agentPool *VMsPool) MaxSize() int
- func (agentPool *VMsPool) MinSize() int
- func (agentPool *VMsPool) Nodes() ([]cloudprovider.Instance, error)
- func (m *VMsPool) String() string
- func (agentPool *VMsPool) TargetSize() (int, error)
- func (agentPool *VMsPool) TemplateNodeInfo() (*framework.NodeInfo, error)
Constants ¶
const ( // AKSLabelPrefixValue represents the constant prefix for AKSLabelKeyPrefixValue AKSLabelPrefixValue = "kubernetes.azure.com" // AKSLabelKeyPrefixValue represents prefix for AKS Labels AKSLabelKeyPrefixValue = AKSLabelPrefixValue + "/" )
const ( // GPULabel is the label added to nodes with GPU resource. GPULabel = AKSLabelKeyPrefixValue + "accelerator" )
Variables ¶
var GetVMSSTypeDynamically = func(template compute.VirtualMachineScaleSet, azCache *azureCache) (InstanceType, error) { ctx := context.Background() var vmssType InstanceType sku, err := azCache.GetSKU(ctx, *template.Sku.Name, *template.Location) if err != nil { promoRe := regexp.MustCompile(`(?i)_promo`) skuName := promoRe.ReplaceAllString(*template.Sku.Name, "") if skuName != *template.Sku.Name { klog.V(1).Infof("No exact match found for %q, checking standard type %q. Error %v", *template.Sku.Name, skuName, err) sku, err = azCache.GetSKU(ctx, skuName, *template.Location) } if err != nil { return vmssType, fmt.Errorf("instance type %q not supported. Error %v", *template.Sku.Name, err) } } vmssType.VCPU, err = sku.VCPU() if err != nil { klog.V(1).Infof("Failed to parse vcpu from sku %q %v", *template.Sku.Name, err) return vmssType, err } gpu, err := getGpuFromSku(sku) if err != nil { klog.V(1).Infof("Failed to parse gpu from sku %q %v", *template.Sku.Name, err) return vmssType, err } vmssType.GPU = gpu memoryGb, err := sku.Memory() if err != nil { klog.V(1).Infof("Failed to parse memoryMb from sku %q %v", *template.Sku.Name, err) return vmssType, err } vmssType.MemoryMb = int64(memoryGb) * 1024 return vmssType, nil }
GetVMSSTypeDynamically fetched vmss instance information using sku api calls. It is declared as a variable for testing purpose.
var GetVMSSTypeStatically = func(template compute.VirtualMachineScaleSet) (*InstanceType, error) { var vmssType *InstanceType for k := range InstanceTypes { if strings.EqualFold(k, *template.Sku.Name) { vmssType = InstanceTypes[k] break } } promoRe := regexp.MustCompile(`(?i)_promo`) if promoRe.MatchString(*template.Sku.Name) { if vmssType == nil { klog.V(4).Infof("No exact match found for %s, checking standard types", *template.Sku.Name) skuName := promoRe.ReplaceAllString(*template.Sku.Name, "") for k := range InstanceTypes { if strings.EqualFold(k, skuName) { vmssType = InstanceTypes[k] break } } } } if vmssType == nil { return vmssType, fmt.Errorf("instance type %q not supported", *template.Sku.Name) } return vmssType, nil }
GetVMSSTypeStatically uses static list of vmss generated at azure_instance_types.go to fetch vmss instance information. It is declared as a variable for testing purpose.
var InstanceTypes = map[string]*InstanceType{}/* 1048 elements not displayed */
InstanceTypes is a map of azure resources
var ( // NvidiaEnabledSKUs represents a list of NVIDIA gpus. // If a new GPU sku becomes available, add a key to this map, but only if you have a confirmation // that we have an agreement with NVIDIA for this specific gpu. NvidiaEnabledSKUs = map[string]bool{ "standard_nc6": true, "standard_nc12": true, "standard_nc24": true, "standard_nc24r": true, "standard_nv6": true, "standard_nv12": true, "standard_nv12s_v3": true, "standard_nv24": true, "standard_nv24s_v3": true, "standard_nv24r": true, "standard_nv48s_v3": true, "standard_nd6s": true, "standard_nd12s": true, "standard_nd24s": true, "standard_nd24rs": true, "standard_nc6s_v2": true, "standard_nc12s_v2": true, "standard_nc24s_v2": true, "standard_nc24rs_v2": true, "standard_nc6s_v3": true, "standard_nc12s_v3": true, "standard_nc24s_v3": true, "standard_nc24rs_v3": true, "standard_nd40s_v3": true, "standard_nd40rs_v2": true, "standard_nc4as_t4_v3": true, "standard_nc8as_t4_v3": true, "standard_nc16as_t4_v3": true, "standard_nc64as_t4_v3": true, "standard_nd96asr_v4": true, "standard_nd112asr_a100_v4": true, "standard_nd120asr_a100_v4": true, "standard_nd96amsr_a100_v4": true, "standard_nd112amsr_a100_v4": true, "standard_nd120amsr_a100_v4": true, "standard_nc24ads_a100_v4": true, "standard_nc48ads_a100_v4": true, "standard_nc96ads_a100_v4": true, } )
Functions ¶
func BuildAzure ¶
func BuildAzure(opts config.AutoscalingOptions, do cloudprovider.NodeGroupDiscoveryOptions, rl *cloudprovider.ResourceLimiter) cloudprovider.CloudProvider
BuildAzure builds Azure cloud provider, manager etc.
func BuildAzureCloudProvider ¶
func BuildAzureCloudProvider(azureManager *AzureManager, resourceLimiter *cloudprovider.ResourceLimiter) (cloudprovider.CloudProvider, error)
BuildAzureCloudProvider creates new AzureCloudProvider
func GetVMNameIndex ¶
func GetVMNameIndex(osType compute.OperatingSystemTypes, vmName string) (int, error)
GetVMNameIndex return the index of VM in the node pools.
func ParseLabelAutoDiscoverySpecs ¶
func ParseLabelAutoDiscoverySpecs(o cloudprovider.NodeGroupDiscoveryOptions) ([]labelAutoDiscoveryConfig, error)
ParseLabelAutoDiscoverySpecs returns any provided NodeGroupAutoDiscoverySpecs parsed into configuration appropriate for node group autodiscovery.
Types ¶
type AgentPool ¶
type AgentPool struct {
// contains filtered or unexported fields
}
AgentPool implements NodeGroup interface for agent pools deployed by aks-engine.
func NewAgentPool ¶
func NewAgentPool(spec *dynamic.NodeGroupSpec, az *AzureManager) (*AgentPool, error)
NewAgentPool creates a new AgentPool.
func (*AgentPool) AtomicIncreaseSize ¶
AtomicIncreaseSize is not implemented.
func (*AgentPool) Autoprovisioned ¶
Autoprovisioned returns true if the node group is autoprovisioned.
func (*AgentPool) Create ¶
func (as *AgentPool) Create() (cloudprovider.NodeGroup, error)
Create creates the node group on the cloud provider side.
func (*AgentPool) DecreaseTargetSize ¶
DecreaseTargetSize decreases the target size of the node group. This function doesn't permit to delete any existing node and can be used only to reduce the request for new nodes that have not been yet fulfilled. Delta should be negative. It is assumed that cloud provider will not delete the existing nodes if the size when there is an option to just decrease the target.
func (*AgentPool) Delete ¶
Delete deletes the node group on the cloud provider side. This will be executed only for autoprovisioned node groups, once their size drops to 0.
func (*AgentPool) DeleteInstances ¶
DeleteInstances deletes the given instances. All instances must be controlled by the same ASG.
func (*AgentPool) DeleteNodes ¶
DeleteNodes deletes the nodes from the group.
func (*AgentPool) Exist ¶
Exist checks if the node group really exists on the cloud provider side. Allows to tell the theoretical node group from the real one.
func (*AgentPool) ForceDeleteNodes ¶
ForceDeleteNodes deletes nodes from the group regardless of constraints.
func (*AgentPool) GetKey ¶
func (m *AgentPool) GetKey() string
GetKey returns key of the given azure reference.
func (*AgentPool) GetOptions ¶
func (as *AgentPool) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)
GetOptions returns NodeGroupAutoscalingOptions that should be used for this particular NodeGroup. Returning a nil will result in using default options.
func (*AgentPool) GetVMIndexes ¶
GetVMIndexes gets indexes of all virtual machines belonging to the agent pool.
func (*AgentPool) IncreaseSize ¶
IncreaseSize increases agent pool size
func (*AgentPool) Nodes ¶
func (as *AgentPool) Nodes() ([]cloudprovider.Instance, error)
Nodes returns a list of all nodes that belong to this node group.
func (*AgentPool) String ¶
func (m *AgentPool) String() string
String is represented by calling GetKey()
func (*AgentPool) TargetSize ¶
TargetSize returns the current TARGET size of the node group. It is possible that the number is different from the number of nodes registered in Kubernetes.
type AgentPoolsClient ¶
type AgentPoolsClient interface { Get(ctx context.Context, resourceGroupName, resourceName, agentPoolName string, options *armcontainerservice.AgentPoolsClientGetOptions) ( armcontainerservice.AgentPoolsClientGetResponse, error) BeginCreateOrUpdate( ctx context.Context, resourceGroupName, resourceName, agentPoolName string, parameters armcontainerservice.AgentPool, options *armcontainerservice.AgentPoolsClientBeginCreateOrUpdateOptions) ( *runtime.Poller[armcontainerservice.AgentPoolsClientCreateOrUpdateResponse], error) BeginDeleteMachines( ctx context.Context, resourceGroupName, resourceName, agentPoolName string, machines armcontainerservice.AgentPoolDeleteMachinesParameter, options *armcontainerservice.AgentPoolsClientBeginDeleteMachinesOptions) ( *runtime.Poller[armcontainerservice.AgentPoolsClientDeleteMachinesResponse], error) }
AgentPoolsClient interface defines the methods needed for scaling vms pool. it is implemented by track2 sdk armcontainerservice.AgentPoolsClient
type AzUtil ¶
type AzUtil struct {
// contains filtered or unexported fields
}
AzUtil consists of utility functions which utilizes clients to different services. Since they span across various clients they cannot be fitted into individual client structs so adding them here.
func (*AzUtil) DeleteBlob ¶
DeleteBlob deletes the blob using the storage client.
type AzureCloudProvider ¶
type AzureCloudProvider struct {
// contains filtered or unexported fields
}
AzureCloudProvider provides implementation of CloudProvider interface for Azure.
func (*AzureCloudProvider) Cleanup ¶
func (azure *AzureCloudProvider) Cleanup() error
Cleanup stops the go routine that is handling the current view of the ASGs in the form of a cache
func (*AzureCloudProvider) GPULabel ¶
func (azure *AzureCloudProvider) GPULabel() string
GPULabel returns the label added to nodes with GPU resource.
func (*AzureCloudProvider) GetAvailableGPUTypes ¶
func (azure *AzureCloudProvider) GetAvailableGPUTypes() map[string]struct{}
GetAvailableGPUTypes return all available GPU types cloud provider supports
func (*AzureCloudProvider) GetAvailableMachineTypes ¶
func (azure *AzureCloudProvider) GetAvailableMachineTypes() ([]string, error)
GetAvailableMachineTypes get all machine types that can be requested from the cloud provider.
func (*AzureCloudProvider) GetNodeGpuConfig ¶
func (azure *AzureCloudProvider) GetNodeGpuConfig(node *apiv1.Node) *cloudprovider.GpuConfig
GetNodeGpuConfig returns the label, type and resource name for the GPU added to node. If node doesn't have any GPUs, it returns nil.
func (*AzureCloudProvider) GetResourceLimiter ¶
func (azure *AzureCloudProvider) GetResourceLimiter() (*cloudprovider.ResourceLimiter, error)
GetResourceLimiter returns struct containing limits (max, min) for resources (cores, memory etc.).
func (*AzureCloudProvider) HasInstance ¶
func (azure *AzureCloudProvider) HasInstance(node *apiv1.Node) (bool, error)
HasInstance returns whether a given node has a corresponding instance in this cloud provider.
Used to prevent undercount of existing VMs (taint-based overcount of deleted VMs), and so should not return false, nil (no instance) if uncertain; return error instead. (Think "has instance for sure, else error".) Returning an error causes fallback to taint-based determination; use ErrNotImplemented for silent fallback, any other error will be logged.
Expected behavior (should work for VMSS Uniform/Flex, and VMs): - exists : return true, nil - !exists : return *, ErrNotImplemented (could use custom error for autoscaled nodes) - unimplemented case : return *, ErrNotImplemented - any other error : return *, error
func (*AzureCloudProvider) Name ¶
func (azure *AzureCloudProvider) Name() string
Name returns name of the cloud provider.
func (*AzureCloudProvider) NewNodeGroup ¶
func (azure *AzureCloudProvider) NewNodeGroup(machineType string, labels map[string]string, systemLabels map[string]string, taints []apiv1.Taint, extraResources map[string]resource.Quantity) (cloudprovider.NodeGroup, error)
NewNodeGroup builds a theoretical node group based on the node definition provided. The node group is not automatically created on the cloud provider side. The node group is not returned by NodeGroups() until it is created.
func (*AzureCloudProvider) NodeGroupForNode ¶
func (azure *AzureCloudProvider) NodeGroupForNode(node *apiv1.Node) (cloudprovider.NodeGroup, error)
NodeGroupForNode returns the node group for the given node.
func (*AzureCloudProvider) NodeGroups ¶
func (azure *AzureCloudProvider) NodeGroups() []cloudprovider.NodeGroup
NodeGroups returns all node groups configured for this cloud provider.
func (*AzureCloudProvider) Pricing ¶
func (azure *AzureCloudProvider) Pricing() (cloudprovider.PricingModel, errors.AutoscalerError)
Pricing returns pricing model for this cloud provider or error if not available.
func (*AzureCloudProvider) Refresh ¶
func (azure *AzureCloudProvider) Refresh() error
Refresh is called before every main loop and can be used to dynamically update cloud provider state. In particular the list of node groups returned by NodeGroups can change as a result of CloudProvider.Refresh().
type AzureManager ¶
type AzureManager struct {
// contains filtered or unexported fields
}
AzureManager handles Azure communication and data caching.
func CreateAzureManager ¶
func CreateAzureManager(configReader io.Reader, discoveryOpts cloudprovider.NodeGroupDiscoveryOptions) (*AzureManager, error)
CreateAzureManager creates Azure Manager object to work with Azure.
func (*AzureManager) GetNodeGroupForInstance ¶
func (m *AzureManager) GetNodeGroupForInstance(instance *azureRef) (cloudprovider.NodeGroup, error)
GetNodeGroupForInstance returns the NodeGroup of the given Instance
func (*AzureManager) GetScaleSetOptions ¶
func (m *AzureManager) GetScaleSetOptions(scaleSetName string, defaults config.NodeGroupAutoscalingOptions) *config.NodeGroupAutoscalingOptions
GetScaleSetOptions parse options extracted from VMSS tags and merges them with provided defaults
func (*AzureManager) Refresh ¶
func (m *AzureManager) Refresh() error
Refresh is called before every main loop and can be used to dynamically update cloud provider state. In particular the list of node groups returned by NodeGroups can change as a result of CloudProvider.Refresh().
func (*AzureManager) RegisterNodeGroup ¶
func (m *AzureManager) RegisterNodeGroup(nodeGroup cloudprovider.NodeGroup) bool
RegisterNodeGroup registers an a NodeGroup.
func (*AzureManager) UnregisterNodeGroup ¶
func (m *AzureManager) UnregisterNodeGroup(nodeGroup cloudprovider.NodeGroup) bool
UnregisterNodeGroup unregisters a NodeGroup.
type Config ¶
type Config struct { // Azure cloud provider configuration, which is generally shared with other Azure components. providerazure.Config `json:",inline" yaml:",inline"` ClusterName string `json:"clusterName" yaml:"clusterName"` // ClusterResourceGroup is the resource group where the cluster is located. ClusterResourceGroup string `json:"clusterResourceGroup" yaml:"clusterResourceGroup"` // ARMBaseURLForAPClient is the URL to use for operations for the VMs pool. // It can override the default public ARM endpoint for VMs pool scale operations. ARMBaseURLForAPClient string `json:"armBaseURLForAPClient" yaml:"armBaseURLForAPClient"` // AuthMethod determines how to authorize requests for the Azure // cloud. Valid options are "principal" (= the traditional // service principle approach) and "cli" (= load az command line // config file). The default is "principal". // 08/16/2024: This field is awkward, given the existence of UseManagedIdentityExtension and UseFederatedWorkloadIdentityExtension. // Ideally, either it should be deprecated, or reworked to be on the same "dimension" as the two above, if not reworking those two. AuthMethod string `json:"authMethod" yaml:"authMethod"` // Configs only for standard vmType (agent pools). Deployment string `json:"deployment" yaml:"deployment"` DeploymentParameters map[string]interface{} `json:"deploymentParameters" yaml:"deploymentParameters"` // Jitter in seconds subtracted from the VMSS cache TTL before the first refresh VmssVmsCacheJitter int `json:"vmssVmsCacheJitter" yaml:"vmssVmsCacheJitter"` // number of latest deployments that will not be deleted MaxDeploymentsCount int64 `json:"maxDeploymentsCount" yaml:"maxDeploymentsCount"` // EnableForceDelete defines whether to enable force deletion on the APIs EnableForceDelete bool `json:"enableForceDelete,omitempty" yaml:"enableForceDelete,omitempty"` // (DEPRECATED, DO NOT USE) EnableDynamicInstanceList defines whether to enable dynamic instance workflow for instance information check EnableDynamicInstanceList bool `json:"enableDynamicInstanceList,omitempty" yaml:"enableDynamicInstanceList,omitempty"` // (DEPRECATED, DO NOT USE) EnableDetailedCSEMessage defines whether to emit error messages in the CSE error body info EnableDetailedCSEMessage bool `json:"enableDetailedCSEMessage,omitempty" yaml:"enableDetailedCSEMessage,omitempty"` // (DEPRECATED, DO NOT USE) GetVmssSizeRefreshPeriod (seconds) defines how frequently to call GET VMSS API to fetch VMSS info per nodegroup instance GetVmssSizeRefreshPeriod int `json:"getVmssSizeRefreshPeriod,omitempty" yaml:"getVmssSizeRefreshPeriod,omitempty"` // StrictCacheUpdates updates cache values only after positive validation from Azure APIs StrictCacheUpdates bool `json:"strictCacheUpdates,omitempty" yaml:"strictCacheUpdates,omitempty"` // EnableFastDeleteOnFailedProvisioning defines whether to delete the experimental faster VMSS instance deletion on failed provisioning EnableFastDeleteOnFailedProvisioning bool `json:"enableFastDeleteOnFailedProvisioning,omitempty" yaml:"enableFastDeleteOnFailedProvisioning,omitempty"` // contains filtered or unexported fields }
Config holds the configuration parsed from the --cloud-config flag or the environment variables. Contains both general Azure cloud provider configuration (i.e., in azure.json) and CAS configurations/options specifically for Azure provider.
type DeploymentClientMock ¶
type DeploymentClientMock struct { mock.Mock FakeStore map[string]resources.DeploymentExtended // contains filtered or unexported fields }
DeploymentClientMock mocks for DeploymentsClient.
func (*DeploymentClientMock) CreateOrUpdate ¶
func (m *DeploymentClientMock) CreateOrUpdate(ctx context.Context, resourceGroupName string, deploymentName string, parameters resources.Deployment, etag string) (err *retry.Error)
CreateOrUpdate creates or updates the Deployment.
func (*DeploymentClientMock) Delete ¶
func (m *DeploymentClientMock) Delete(ctx context.Context, resourceGroupName, deploymentName string) (err *retry.Error)
Delete deletes the given deployment
func (*DeploymentClientMock) ExportTemplate ¶
func (m *DeploymentClientMock) ExportTemplate(ctx context.Context, resourceGroupName string, deploymentName string) (result resources.DeploymentExportResult, err *retry.Error)
ExportTemplate exports the deployment's template.
func (*DeploymentClientMock) Get ¶
func (m *DeploymentClientMock) Get(ctx context.Context, resourceGroupName string, deploymentName string) (result resources.DeploymentExtended, err *retry.Error)
Get gets the DeploymentExtended by deploymentName.
func (*DeploymentClientMock) List ¶
func (m *DeploymentClientMock) List(ctx context.Context, resourceGroupName string) (result []resources.DeploymentExtended, err *retry.Error)
List gets all the deployments for a resource group.
type GetVMSSFailedError ¶
type GetVMSSFailedError struct {
// contains filtered or unexported fields
}
GetVMSSFailedError is used to differentiate between NotFound and other errors
func (*GetVMSSFailedError) Error ¶
func (v *GetVMSSFailedError) Error() string
type InstanceCache ¶
type InstanceCache struct {
// contains filtered or unexported fields
}
InstanceCache tracks the VMs in the ScaleSet, in the form of corresponding cloudprovider.Instances. This struct also contains related locks and cache interval variables.
type InstanceType ¶
type InstanceType struct { InstanceType string SkuFamily string VCPU int64 MemoryMb int64 GPU int64 }
InstanceType is the sepc of Azure instance
type MockAgentPoolsClient ¶
type MockAgentPoolsClient struct {
// contains filtered or unexported fields
}
MockAgentPoolsClient is a mock of AgentPoolsClient interface.
func NewMockAgentPoolsClient ¶
func NewMockAgentPoolsClient(ctrl *gomock.Controller) *MockAgentPoolsClient
NewMockAgentPoolsClient creates a new mock instance.
func (*MockAgentPoolsClient) BeginCreateOrUpdate ¶
func (m *MockAgentPoolsClient) BeginCreateOrUpdate(arg0 context.Context, arg1, arg2, arg3 string, arg4 armcontainerservice.AgentPool, arg5 *armcontainerservice.AgentPoolsClientBeginCreateOrUpdateOptions) (*runtime.Poller[armcontainerservice.AgentPoolsClientCreateOrUpdateResponse], error)
BeginCreateOrUpdate mocks base method.
func (*MockAgentPoolsClient) BeginDeleteMachines ¶
func (m *MockAgentPoolsClient) BeginDeleteMachines(arg0 context.Context, arg1, arg2, arg3 string, arg4 armcontainerservice.AgentPoolDeleteMachinesParameter, arg5 *armcontainerservice.AgentPoolsClientBeginDeleteMachinesOptions) (*runtime.Poller[armcontainerservice.AgentPoolsClientDeleteMachinesResponse], error)
BeginDeleteMachines mocks base method.
func (*MockAgentPoolsClient) EXPECT ¶
func (m *MockAgentPoolsClient) EXPECT() *MockAgentPoolsClientMockRecorder
EXPECT returns an object that allows the caller to indicate expected use.
func (*MockAgentPoolsClient) Get ¶
func (m *MockAgentPoolsClient) Get(arg0 context.Context, arg1, arg2, arg3 string, arg4 *armcontainerservice.AgentPoolsClientGetOptions) (armcontainerservice.AgentPoolsClientGetResponse, error)
Get mocks base method.
type MockAgentPoolsClientMockRecorder ¶
type MockAgentPoolsClientMockRecorder struct {
// contains filtered or unexported fields
}
MockAgentPoolsClientMockRecorder is the mock recorder for MockAgentPoolsClient.
func (*MockAgentPoolsClientMockRecorder) BeginCreateOrUpdate ¶
func (mr *MockAgentPoolsClientMockRecorder) BeginCreateOrUpdate(arg0, arg1, arg2, arg3, arg4, arg5 any) *gomock.Call
BeginCreateOrUpdate indicates an expected call of BeginCreateOrUpdate.
func (*MockAgentPoolsClientMockRecorder) BeginDeleteMachines ¶
func (mr *MockAgentPoolsClientMockRecorder) BeginDeleteMachines(arg0, arg1, arg2, arg3, arg4, arg5 any) *gomock.Call
BeginDeleteMachines indicates an expected call of BeginDeleteMachines.
type ScaleSet ¶
type ScaleSet struct { InstanceCache // contains filtered or unexported fields }
ScaleSet implements NodeGroup interface.
func NewScaleSet ¶
func NewScaleSet(spec *dynamic.NodeGroupSpec, az *AzureManager, curSize int64, dedicatedHost bool) (*ScaleSet, error)
NewScaleSet creates a new NewScaleSet.
func (*ScaleSet) AtomicIncreaseSize ¶
AtomicIncreaseSize is not implemented.
func (*ScaleSet) Autoprovisioned ¶
Autoprovisioned returns true if the node group is autoprovisioned.
func (*ScaleSet) Create ¶
func (scaleSet *ScaleSet) Create() (cloudprovider.NodeGroup, error)
Create creates the node group on the cloud provider side.
func (*ScaleSet) DecreaseTargetSize ¶
DecreaseTargetSize decreases the target size of the node group. This function doesn't permit to delete any existing node and can be used only to reduce the request for new nodes that have not been yet fulfilled. Delta should be negative. It is assumed that cloud provider will not delete the existing nodes if the size when there is an option to just decrease the target.
func (*ScaleSet) Delete ¶
Delete deletes the node group on the cloud provider side. This will be executed only for autoprovisioned node groups, once their size drops to 0.
func (*ScaleSet) DeleteInstances ¶
DeleteInstances deletes the given instances. All instances must be controlled by the same nodegroup.
func (*ScaleSet) DeleteNodes ¶
DeleteNodes deletes the nodes from the group.
func (*ScaleSet) Exist ¶
Exist checks if the node group really exists on the cloud provider side. Allows to tell the theoretical node group from the real one.
func (*ScaleSet) ForceDeleteNodes ¶
ForceDeleteNodes deletes nodes from the group regardless of constraints.
func (*ScaleSet) GetFlexibleScaleSetVms ¶
func (scaleSet *ScaleSet) GetFlexibleScaleSetVms() ([]compute.VirtualMachine, *retry.Error)
GetFlexibleScaleSetVms returns list of nodes for flexible scale set.
func (*ScaleSet) GetKey ¶
func (m *ScaleSet) GetKey() string
GetKey returns key of the given azure reference.
func (*ScaleSet) GetOptions ¶
func (scaleSet *ScaleSet) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)
GetOptions returns NodeGroupAutoscalingOptions that should be used for this particular NodeGroup. Returning a nil will result in using default options.
func (*ScaleSet) GetScaleSetVms ¶
func (scaleSet *ScaleSet) GetScaleSetVms() ([]compute.VirtualMachineScaleSetVM, *retry.Error)
GetScaleSetVms returns list of nodes for the given scale set.
func (*ScaleSet) IncreaseSize ¶
IncreaseSize increases Scale Set size
func (*ScaleSet) Nodes ¶
func (scaleSet *ScaleSet) Nodes() ([]cloudprovider.Instance, error)
Nodes returns a list of all nodes that belong to this node group.
func (*ScaleSet) String ¶
func (m *ScaleSet) String() string
String is represented by calling GetKey()
func (*ScaleSet) TargetSize ¶
TargetSize returns the current TARGET size of the node group. It is possible that the number is different from the number of nodes registered in Kubernetes.
type VMsPool ¶
type VMsPool struct {
// contains filtered or unexported fields
}
VMsPool is single instance VM pool this is a placeholder for now, no real implementation
func NewVMsPool ¶
func NewVMsPool(spec *dynamic.NodeGroupSpec, am *AzureManager) *VMsPool
NewVMsPool creates a new VMsPool
func (*VMsPool) AtomicIncreaseSize ¶
AtomicIncreaseSize is not implemented.
func (*VMsPool) Autoprovisioned ¶
Autoprovisioned is always false since we are initialized with an existing agentpool
func (*VMsPool) Create ¶
func (agentPool *VMsPool) Create() (cloudprovider.NodeGroup, error)
Create creates the node group on the cloud provider side.
func (*VMsPool) DecreaseTargetSize ¶
DecreaseTargetSize decreases the target size of the node group.
func (*VMsPool) DeleteNodes ¶
DeleteNodes extracts the providerIDs from the node spec and delete or deallocate the nodes from the agent pool based on the scale down policy.
func (*VMsPool) ForceDeleteNodes ¶
ForceDeleteNodes deletes nodes from the group regardless of constraints.
func (*VMsPool) GetKey ¶
func (m *VMsPool) GetKey() string
GetKey returns key of the given azure reference.
func (*VMsPool) GetOptions ¶
func (agentPool *VMsPool) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)
GetOptions returns NodeGroupAutoscalingOptions that should be used for this particular NodeGroup. Returning a nil will result in using default options.
func (*VMsPool) IncreaseSize ¶
IncreaseSize increase the size through a PUT AP call. It calculates the expected size based on a delta provided as parameter
func (*VMsPool) MaxSize ¶
MaxSize returns the maximum size scale limit provided by --node parameter to the autoscaler main
func (*VMsPool) MinSize ¶
MinSize returns the minimum size the cluster is allowed to scaled down to as provided by the node spec in --node parameter.
func (*VMsPool) Nodes ¶
func (agentPool *VMsPool) Nodes() ([]cloudprovider.Instance, error)
Nodes returns the list of nodes in the vms agentPool.
func (*VMsPool) String ¶
func (m *VMsPool) String() string
String is represented by calling GetKey()
func (*VMsPool) TargetSize ¶
TargetSize returns the current TARGET size of the node group. It is possible that the number is different from the number of nodes registered in Kubernetes.
Source Files
¶
- azure_agent_pool.go
- azure_autodiscovery.go
- azure_cache.go
- azure_client.go
- azure_cloud_provider.go
- azure_config.go
- azure_fakes.go
- azure_force_delete_scale_set.go
- azure_instance.go
- azure_instance_gpu_sku.go
- azure_instance_types.go
- azure_manager.go
- azure_mock_agentpool_client.go
- azure_scale_set.go
- azure_scale_set_instance_cache.go
- azure_template.go
- azure_util.go
- azure_vms_pool.go