scheduler-stress-test

README

中文

scheduler-stress-test

This is a stress testing tool for the scheduler in a large-scale scenario.

Preparatory Work

To simulate a large-scale scheduling scenario, you can create a large number of nodes as needed using kwok. The nodes you create may be in a NotReady state. In order to use the nodes to schedule pods, the "pod.yaml" file must add a toleration to tolerate all NoSchedule taints.

To prepare for this, you should do the following:

1. Install kwok on your k8s cluster. Refer to https://kwok.sigs.k8s.io/docs/user/kwok-in-cluster/.
2. Create fake nodes on your k8s cluster.

fake node example:

cat << EOF > node.yaml 
apiVersion: v1
kind: Node
metadata:
  annotations:
    node.alpha.kubernetes.io/ttl: "0"
    kwok.x-k8s.io/node: fake
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/os: linux
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: {NODE_NAME}
    kubernetes.io/os: linux
    kubernetes.io/role: agent
    node-role.kubernetes.io/agent: ""
    type: kwok
  name: {NODE_NAME}
spec:
  taints: # Avoid scheduling actual running pods to fake Node
    - effect: NoSchedule
      key: kwok.x-k8s.io/node
      value: fake
status:
  allocatable:
    cpu: "64"
    ephemeral-storage: 1Ti
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 250Gi
    pods: "110"
  capacity:
    cpu: "64"
    ephemeral-storage: 1Ti
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 250Gi
    pods: "128"
  nodeInfo:
    architecture: amd64
    bootID: ""
    containerRuntimeVersion: ""
    kernelVersion: ""
    kubeProxyVersion: fake
    kubeletVersion: fake
    machineID: ""
    operatingSystem: linux
    osImage: ""
    systemUUID: ""
  phase: Running
EOF

# create nodes as you needed
for i in {0..99}; do sed "s/{NODE_NAME}/kwok-node-$i/g" node.yaml | kubectl apply -f -; done

How to use

Download the code and build it:

git clone https://github.com/k-cloud-labs/scheduler-stress-test.git
make build

Two commands are supported: create and wait.

The create command creates a specified number of pods from a template file in the k8s cluster with a specified level of concurrency.

The wait command waits for all the pods created above to be scheduled and continuously prints the results.

Example:

# create 1000 pod by 1000 concurrency (namespace: scheduler-stress-test)
sst create --kubeconfig=/root/.kube/config --count 1000 --concurrency 1000 --pod-template=pod.yaml
# wait result
sst wait --kubeconfig=/root/.kube/config --namespace=scheduler-stress-test

The example above uses "pod.yaml" as a template and creates 1000 pods in the "scheduler-stress-test" namespace of the k8s cluster. It then waits and continuously prints the results. You can modify the "pod.yaml" file as needed.

Documentation

Overview

Copyright © 2023 likakuli <1154584512@qq.com>

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.