coscheduling

package
v0.0.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2022 License: Apache-2.0 Imports: 17 Imported by: 0

README

Overview

This folder holds the coscheduling plugin implementations based on Coscheduling based on PodGroup CRD. The old version coscheduling is based on Lightweight coscheduling based on back-to-back queue sorting.

Maturity Level

  • 💡 Sample (for demonstrating and inspiring purpose)
  • 👶 Alpha (used in companies for pilot projects)
  • 👦 Beta (used in companies and developed actively)
  • 👨 Stable (used in companies for production workloads)

Tutorial

PodGroup

We use a special label named pod-group.scheduling.sigs.k8s.io to define a PodGroup. Pods that set this label and use the same value belong to the same PodGroup.

# PodGroup CRD spec
apiVersion: scheduling.sigs.k8s.io/v1alpha1
kind: PodGroup
metadata:
  name: nginx
spec:
  scheduleTimeoutSeconds: 10
  minMember: 3
---
# Add a label `pod-group.scheduling.sigs.k8s.io` to mark the pod belongs to a group
labels:
  pod-group.scheduling.sigs.k8s.io: nginx

We will calculate the sum of the Running pods and the Waiting pods (assumed but not bind) in scheduler, if the sum is greater than or equal to the minMember, the Waiting pods will be created.

Pods in the same PodGroup with different priorities might lead to unintended behavior, so need to ensure Pods in the same PodGroup with the same priority.

Expectation
  1. If 2 PodGroups with different priorities come in, the PodGroup with high priority has higher precedence.
  2. If 2 PodGroups with same priority come in when there are limited resources, the PodGroup created first one has higher precedence.
Config
  1. queueSort, permit and unreserve must be enabled in coscheduling.
  2. preFilter is enhanced feature to reduce the overall scheduling time for the whole group. It will check the total number of pods belonging to the same PodGroup. If the total number is less than minMember, the pod will reject in preFilter, then the scheduling cycle will interrupt. And the preFilter is user selectable according to the actual situation of users. If the minMember of PodGroup is relatively small, for example less than 5, you can disable this plugin. But if the minMember of PodGroup is relatively large, please enable this plugin to reduce the overall scheduling time.
apiVersion: kubescheduler.config.k8s.io/v1beta2
kind: KubeSchedulerConfiguration
leaderElection:
  leaderElect: false
clientConnection:
  kubeconfig: "REPLACE_ME_WITH_KUBE_CONFIG_PATH"
profiles:
- schedulerName: default-scheduler
  plugins:
    queueSort:
      enabled:
      - name: Coscheduling
      disabled:
      - name: "*"
    preFilter:
      enabled:
      - name: Coscheduling
    postFilter:
      enabled:
      - name: Coscheduling
    permit:
      enabled:
      - name: Coscheduling
    reserve:
      enabled:
      - name: Coscheduling
    postBind:
      enabled:
      - name: Coscheduling
Demo

Suppose we have a cluster which can only afford 3 nginx pods. We create a ReplicaSet with replicas=6, and set the value of minMember to 3.

apiVersion: scheduling.sigs.k8s.io/v1alpha1
kind: PodGroup
metadata:
  name: nginx
spec:
  scheduleTimeoutSeconds: 10
  minMember: 3
---
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 6
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
        pod-group.scheduling.sigs.k8s.io: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        resources:
          limits:
            cpu: 3000m
            memory: 500Mi
          requests:
            cpu: 3000m
            memory: 500Mi

$ kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
nginx-4jw2m   0/1     Pending   0          55s
nginx-4mn52   1/1     Running   0          55s
nginx-c9gv8   1/1     Running   0          55s
nginx-frm24   0/1     Pending   0          55s
nginx-hsflk   0/1     Pending   0          55s
nginx-qtj5f   1/1     Running   0          55s

If minMember is set to 4 at this time, all nginx pods are in pending state because the resource does not meet the requirements of minMember

$ kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
nginx-4vqrk   0/1     Pending   0          3s
nginx-bw9nn   0/1     Pending   0          3s
nginx-gnjsv   0/1     Pending   0          3s
nginx-hqhhz   0/1     Pending   0          3s
nginx-n47r7   0/1     Pending   0          3s
nginx-n7vtq   0/1     Pending   0          3s

Documentation

Index

Constants

View Source 
const (
	// Name is the name of the plugin used in Registry and configurations.
	Name = "Coscheduling"
)

Variables

This section is empty.

Functions

func New 

func New(obj runtime.Object, handle framework.Handle) (framework.Plugin, error)

New initializes and returns a new Coscheduling plugin.

Types

type Coscheduling 

type Coscheduling struct {
	// contains filtered or unexported fields
}

Coscheduling is a plugin that schedules pods in a group.

func (*Coscheduling) EventsToRegister 

func (cs *Coscheduling) EventsToRegister() []framework.ClusterEvent

func (*Coscheduling) Less 

func (cs *Coscheduling) Less(podInfo1, podInfo2 *framework.QueuedPodInfo) bool

Less is used to sort pods in the scheduling queue in the following order. 1. Compare the priorities of Pods. 2. Compare the initialization timestamps of PodGroups or Pods. 3. Compare the keys of PodGroups/Pods: <namespace>/<podname>.

func (*Coscheduling) Name 

func (cs *Coscheduling) Name() string

Name returns name of the plugin. It is used in logs, etc.

func (*Coscheduling) Permit 

func (cs *Coscheduling) Permit(ctx context.Context, state *framework.CycleState, pod *v1.Pod, nodeName string) (*framework.Status, time.Duration)

Permit is the functions invoked by the framework at "Permit" extension point.

func (*Coscheduling) PostBind 

func (cs *Coscheduling) PostBind(ctx context.Context, _ *framework.CycleState, pod *v1.Pod, nodeName string)

PostBind is called after a pod is successfully bound. These plugins are used update PodGroup when pod is bound.

func (*Coscheduling) PostFilter 

func (cs *Coscheduling) PostFilter(ctx context.Context, state *framework.CycleState, pod *v1.Pod,
	filteredNodeStatusMap framework.NodeToStatusMap) (*framework.PostFilterResult, *framework.Status)

PostFilter is used to reject a group of pods if a pod does not pass PreFilter or Filter.

func (*Coscheduling) PreFilter 

func (cs *Coscheduling) PreFilter(ctx context.Context, state *framework.CycleState, pod *v1.Pod) (*framework.PreFilterResult, *framework.Status)

PreFilter performs the following validations. 1. Whether the PodGroup that the Pod belongs to is on the deny list. 2. Whether the total number of pods in a PodGroup is less than its `minMember`.

func (*Coscheduling) PreFilterExtensions 

func (cs *Coscheduling) PreFilterExtensions() framework.PreFilterExtensions

PreFilterExtensions returns a PreFilterExtensions interface if the plugin implements one.

func (*Coscheduling) Reserve 

func (cs *Coscheduling) Reserve(ctx context.Context, state *framework.CycleState, pod *v1.Pod, nodeName string) *framework.Status

Reserve is the functions invoked by the framework at "reserve" extension point.

func (*Coscheduling) Unreserve 

func (cs *Coscheduling) Unreserve(ctx context.Context, state *framework.CycleState, pod *v1.Pod, nodeName string)

Unreserve rejects all other Pods in the PodGroup when one of the pods in the group times out.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.