volcengine

package
v0.0.0-...-de5d64f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 16, 2024 License: Apache-2.0 Imports: 26 Imported by: 0

README

Cluster Autoscaler on Volcengine

The Cluster Autoscaler on Volcengine dynamically scales Kubernetes worker nodes. It runs as a deployment within your cluster. This README provides a step-by-step guide for setting up cluster autoscaler on your Kubernetes cluster.

Permissions

Using Volcengine Credentials

To use Volcengine credentials, create a Secret with your access key and access key secret:

apiVersion: v1
kind: Secret
metadata:
  name: cloud-config
  namespace: kube-system
type: Opaque
data:
  access-key: [YOUR_BASE64_AK_ID]
  secret-key: [YOUR_BASE64_AK_SECRET]
  region-id: [YOUR_BASE64_REGION_ID]

See the Volcengine Access Key User Manual and Volcengine Autoscaling Region for more information.

Manual Configuration

Auto Scaling Group Setup

  1. Create an Auto Scaling Group in the Volcengine Console with valid configurations, and set the desired instance number to zero.

  2. Create a Scaling Configuration for the Scaling Group with valid configurations. In User Data, specify the script to initialize the environment and join this node to the Kubernetes cluster.

Cluster Autoscaler Deployment
  1. Create a service account.
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
  name: cluster-autoscaler-account
  namespace: kube-system

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cluster-autoscaler
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
rules:
  - apiGroups: [""]
    resources: ["events", "endpoints"]
    verbs: ["create", "patch"]
  - apiGroups: [""]
    resources: ["pods/eviction"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["pods/status"]
    verbs: ["update"]
  - apiGroups: [""]
    resources: ["endpoints"]
    resourceNames: ["cluster-autoscaler"]
    verbs: ["get", "update"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["watch", "list", "get", "update", "delete"]
  - apiGroups: [""]
    resources:
      - "namespaces"
      - "pods"
      - "services"
      - "replicationcontrollers"
      - "persistentvolumeclaims"
      - "persistentvolumes"
    verbs: ["watch", "list", "get"]
  - apiGroups: ["batch", "extensions"]
    resources: ["jobs"]
    verbs: ["watch", "list", "get", "patch"]
  - apiGroups: [ "policy" ]
    resources: [ "poddisruptionbudgets" ]
    verbs: [ "watch", "list" ]
  - apiGroups: ["apps"]
    resources: ["daemonsets", "replicasets", "statefulsets"]
    verbs: ["watch", "list", "get"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses", "csinodes", "csidrivers", "csistoragecapacities"]
    verbs: ["watch", "list", "get"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["create","list","watch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["cluster-autoscaler-status", "cluster-autoscaler-priority-expander"]
    verbs: ["delete", "get", "update"]
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["watch", "list", "get", "create", "update", "patch", "delete", "deletecollection"]
  - apiGroups: ["extensions"]
    resources: ["replicasets", "daemonsets"]
    verbs: ["watch", "list", "get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["create","list","watch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["cluster-autoscaler-status", "cluster-autoscaler-priority-expander"]
    verbs: ["delete","get","update","watch"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-autoscaler
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-autoscaler
subjects:
  - kind: ServiceAccount
    name: cluster-autoscaler-account
    namespace: kube-system

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cluster-autoscaler
subjects:
  - kind: ServiceAccount
    name: cluster-autoscaler
    namespace: kube-system
  1. Create a deployment.
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    app: cluster-autoscaler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      namespace: kube-system
      labels:
        app: cluster-autoscaler
    spec:
      serviceAccountName: cluster-autoscaler-account
      containers:
        - name: cluster-autoscaler
          image: registry.k8s.io/autoscaling/cluster-autoscaler:latest
          imagePullPolicy: Always
          command:
            - ./cluster-autoscaler
            - --alsologtostderr
            - --cloud-config=/config/cloud-config
            - --cloud-provider=volcengine
            - --nodes=[min]:[max]:[ASG_ID]
            - --scale-down-delay-after-add=1m0s
            - --scale-down-unneeded-time=1m0s
          env:
            - name: ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: cloud-config
                  key: access-key
            - name: SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: cloud-config
                  key: secret-key
            - name: REGION_ID
              valueFrom:
                secretKeyRef:
                  name: cloud-config
                  key: region-id

Auto-Discovery Setup

Auto Discovery is not currently supported in Volcengine.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildVolcengine 

func BuildVolcengine(opts config.AutoscalingOptions, do cloudprovider.NodeGroupDiscoveryOptions, rl *cloudprovider.ResourceLimiter) cloudprovider.CloudProvider

BuildVolcengine builds CloudProvider implementation for Volcengine

func StringSliceInGroupsOf 

func StringSliceInGroupsOf(arr []string, num int64) [][]string

StringSliceInGroupsOf split arr into num parts

Types

type AutoScalingGroup 

type AutoScalingGroup struct {
	// contains filtered or unexported fields
}

AutoScalingGroup represents a Volcengine 'Auto Scaling Group' which also can be treated as a node group.

func (*AutoScalingGroup) AtomicIncreaseSize 

func (asg *AutoScalingGroup) AtomicIncreaseSize(delta int) error

AtomicIncreaseSize is not implemented.

func (*AutoScalingGroup) Autoprovisioned 

func (asg *AutoScalingGroup) Autoprovisioned() bool

Autoprovisioned returns true if the node group is autoprovisioned. An autoprovisioned group was created by CA and can be deleted when scaled to 0.

func (*AutoScalingGroup) Create 

func (asg *AutoScalingGroup) Create() (cloudprovider.NodeGroup, error)

Create creates the node group on the cloud provider side. Implementation optional.

func (*AutoScalingGroup) Debug 

func (asg *AutoScalingGroup) Debug() string

Debug returns a string containing all information regarding this node group.

func (*AutoScalingGroup) DecreaseTargetSize 

func (asg *AutoScalingGroup) DecreaseTargetSize(delta int) error

DecreaseTargetSize decreases the target size of the node group. This function doesn't permit to delete any existing node and can be used only to reduce the request for new nodes that have not been yet fulfilled. Delta should be negative. It is assumed that cloud provider will not delete the existing nodes when there is an option to just decrease the target. Implementation required.

func (*AutoScalingGroup) Delete 

func (asg *AutoScalingGroup) Delete() error

Delete deletes the node group on the cloud provider side. This will be executed only for autoprovisioned node groups, once their size drops to 0. Implementation optional.

func (*AutoScalingGroup) DeleteNodes 

func (asg *AutoScalingGroup) DeleteNodes(nodes []*apiv1.Node) error

DeleteNodes deletes nodes from this node group. Error is returned either on failure or if the given node doesn't belong to this node group. This function should wait until node group size is updated. Implementation required.

func (*AutoScalingGroup) Exist 

func (asg *AutoScalingGroup) Exist() bool

Exist checks if the node group really exists on the cloud provider side. Allows to tell the theoretical node group from the real one. Implementation required.

func (*AutoScalingGroup) ForceDeleteNodes 

func (asg *AutoScalingGroup) ForceDeleteNodes(nodes []*apiv1.Node) error

ForceDeleteNodes deletes nodes from the group regardless of constraints.

func (*AutoScalingGroup) GetOptions 

func (asg *AutoScalingGroup) GetOptions(defaults config.NodeGroupAutoscalingOptions) (*config.NodeGroupAutoscalingOptions, error)

GetOptions returns NodeGroupAutoscalingOptions that should be used for this particular NodeGroup. Returning a nil will result in using default options. Implementation optional.

func (*AutoScalingGroup) Id 

func (asg *AutoScalingGroup) Id() string

Id returns an unique identifier of the node group.

func (*AutoScalingGroup) IncreaseSize 

func (asg *AutoScalingGroup) IncreaseSize(delta int) error

IncreaseSize increases the size of the node group. To delete a node you need to explicitly name it and use DeleteNode. This function should wait until node group size is updated. Implementation required.

func (*AutoScalingGroup) MaxSize 

func (asg *AutoScalingGroup) MaxSize() int

MaxSize returns maximum size of the node group.

func (*AutoScalingGroup) MinSize 

func (asg *AutoScalingGroup) MinSize() int

MinSize returns minimum size of the node group.

func (*AutoScalingGroup) Nodes 

func (asg *AutoScalingGroup) Nodes() ([]cloudprovider.Instance, error)

Nodes returns a list of all nodes that belong to this node group. It is required that Instance objects returned by this method have Id field set. Other fields are optional. This list should include also instances that might have not become a kubernetes node yet.

func (*AutoScalingGroup) TargetSize 

func (asg *AutoScalingGroup) TargetSize() (int, error)

TargetSize returns the current target size of the node group. It is possible that the number of nodes in Kubernetes is different at the moment but should be equal to Size() once everything stabilizes (new nodes finish startup and registration or removed nodes are deleted completely). Implementation required.

func (*AutoScalingGroup) TemplateNodeInfo 

func (asg *AutoScalingGroup) TemplateNodeInfo() (*framework.NodeInfo, error)

TemplateNodeInfo returns a framework.NodeInfo structure of an empty (as if just started) node. This will be used in scale-up simulations to predict what would a new node look like if a node group was expanded. The returned NodeInfo is expected to have a fully populated Node object, with all of the labels, capacity and allocatable information as well as all pods that are started on the node by default, using manifest (most likely only kube-proxy). Implementation optional.

type AutoScalingService 

type AutoScalingService interface {
	GetScalingGroupById(groupId string) (*autoscaling.ScalingGroupForDescribeScalingGroupsOutput, error)
	ListScalingInstancesByGroupId(groupId string) ([]*autoscaling.ScalingInstanceForDescribeScalingInstancesOutput, error)
	GetScalingConfigurationById(configurationId string) (*autoscaling.ScalingConfigurationForDescribeScalingConfigurationsOutput, error)
	RemoveInstances(groupId string, instanceIds []string) error
	SetAsgTargetSize(groupId string, targetSize int) error
	SetAsgDesireCapacity(groupId string, desireCapacity int) error
}

AutoScalingService is the interface for volcengine auto-scaling service

type EcsService 

type EcsService interface {
	GetInstanceTypeById(instanceTypeId string) (*ecs.InstanceTypeForDescribeInstanceTypesOutput, error)
}

EcsService represents the ECS interfaces

type VolcengineManager 

type VolcengineManager interface {
	// RegisterAsg registers the given ASG with the manager.
	RegisterAsg(asg *AutoScalingGroup)

	// GetAsgForInstance returns the ASG of the given instance.
	GetAsgForInstance(instanceId string) (*AutoScalingGroup, error)

	// GetAsgById returns the ASG of the given id.
	GetAsgById(id string) (*AutoScalingGroup, error)

	// GetAsgDesireCapacity returns the desired capacity of the given ASG.
	GetAsgDesireCapacity(asgId string) (int, error)

	// SetAsgTargetSize sets the target size of the given ASG.
	SetAsgTargetSize(asgId string, targetSize int) error

	// DeleteScalingInstances deletes the given instances from the given ASG.
	DeleteScalingInstances(asgId string, instanceIds []string) error

	// GetAsgNodes returns the scaling instance ids of the given ASG.
	GetAsgNodes(asgId string) ([]cloudprovider.Instance, error)

	// SetAsgDesireCapacity sets the desired capacity of the given ASG.
	SetAsgDesireCapacity(groupId string, desireCapacity int) error
	// contains filtered or unexported methods
}

VolcengineManager define the interface that implements Cloud Provider and Node Group

func CreateVolcengineManager 

func CreateVolcengineManager(cloudConfig *cloudConfig) (VolcengineManager, error)

CreateVolcengineManager returns the VolcengineManager interface implementation

Source Files

View all Source files

Directories

Path Synopsis
volc-sdk-golang
base
service/sts
volcengine-go-sdk
internal/ini
internal/sdkio
internal/sdkmath
internal/sdkrand
internal/shareddefaults
private/protocol
private/protocol/json/jsonutil
Package jsonutil provides JSON serialization of VOLCSTACK requests and responses.
 Package jsonutil provides JSON serialization of VOLCSTACK requests and responses.
private/protocol/query
Package query provides serialization of VOLCSTACK volcenginequery requests, and responses.
 Package query provides serialization of VOLCSTACK volcenginequery requests, and responses.
private/protocol/query/queryutil
private/protocol/rest
Package rest provides RESTFUL serialization of VOLCSTACK requests and responses.
 Package rest provides RESTFUL serialization of VOLCSTACK requests and responses.
private/protocol/volcenginequery
private/protocol/xml/xmlutil
Package xmlutil Package xml util provides XML serialization of VOLCSTACK requests and responses.
 Package xmlutil Package xml util provides XML serialization of VOLCSTACK requests and responses.
private/util
service/autoscaling
Package autoscalingiface provides an interface to enable mocking the AUTO_SCALING service client for testing your code.
 Package autoscalingiface provides an interface to enable mocking the AUTO_SCALING service client for testing your code.
service/ecs
Package ecsiface provides an interface to enable mocking the ECS service client for testing your code.
 Package ecsiface provides an interface to enable mocking the ECS service client for testing your code.
volcengine
Package volcengine provides core functionality for making requests to volcengine services.
 Package volcengine provides core functionality for making requests to volcengine services.
volcengine/client
volcengine/client/metadata
volcengine/corehandlers
volcengine/credentials
Package credentials provides credential retrieval and management
 Package credentials provides credential retrieval and management
volcengine/credentials/endpointcreds
volcengine/credentials/processcreds
volcengine/custom
volcengine/defaults
Package defaults is a collection of helpers to retrieve the SDK's default configuration and handlers.
 Package defaults is a collection of helpers to retrieve the SDK's default configuration and handlers.
volcengine/endpoints
volcengine/request
volcengine/response
volcengine/session
volcengine/signer/volc
volcengine/special
volcengine/universal
volcengine/volcenginebody
volcengine/volcengineerr
Package volcengineerr represents API error interface accessors for the SDK.
 Package volcengineerr represents API error interface accessors for the SDK.
volcengine/volcenginequery
volcengine/volcengineutil

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.