large-scale-perf

README

Usage

In order to use this to in your local ENV,

You can do:

1. Install binary to you ENV

go install github.com/open-cluster-management/large-scale-perf

1. Run the binary as usual

For example: apply 5 applications to your cluster(the cluster is define at the kubeconfig file).

large-scale-perf apply -r 5

1. Other functionality

❯ large-scale-perf
Generate and apply application to hub cluster

Usage:
  perf [command]

Available Commands:
  apply       Generate application(s) instance and apply them
  check       Check if application(s) is subscribed successfully
  clean       Delete created applications
  help        Help about any command

Flags:
      --cfg string   kubeconfig file path (default "/Users/ianzhang/.kube/config")
      --d string     numbers of application replica (default "sets")
  -h, --help         help for perf
      --r int        numbers of application replica (default 1)
  -t, --toggle       Help message for toggle

Use "perf [command] --help" for more information about a command.

Application set up

https://github.com/open-cluster-management/demo-subscription-gitops/tree/master/book-import

The application contains: 1 service, 1 route and 1 deployment

Data collection

Data is collect from the hub cluster dashboard.

1. single application to multiple clusters

index	applications	clusters	start	complete
0	1	1
1	1	2
2	1	4
3	1	8
4	1	16
5	1	32
6	1	50

data collected:

• Start timestamps: 2020-11-13 10:03:26.55439 -0500 EST m=+1.005345364

• CPU

• Memory

2. 128 applications to multiple clusters

index	applications	clusters	start	end
0	128	50

data collected:

• Start timestamps: start time: 2020-11-13 16:10:13.077395 -0500 EST m=+1.182538019

• CPU

• Memory

NOTE: Upon the deletion, we observed that on the managed clusters,

a. there are lots of resources left over

b. the subscription pod restarted, due to OOMKill

c. the subscription resources is deleted successfully

d. the managed pod requested 2G memory

e. 128 applications, each of it deployed to all the managed clusters. Then on each cluster, we will have around 128 applications, each applications will have 1 subscription and 3 sub-resources. Assume each resources consume 4M, then we are having 400M in the resources map. Then we might have some resource hold on k8s watch, in addition, some on the watch queue.

3. delta add applications to multiple clusters

index	applications	clusters	delta	start	end
0	1	54	1
1	2	54	1
2	4	54	2
3	7	54	3
4	11	54	4
5	16	54	5
6	22	54	6
6	29	54	7

data collected:

• Start timestamps: 2020-11-17 15:06:02.642625 -0500 EST m=+1.144518753

• CPU

• Memory

Next

• helm type(download all the deployable)
• more details of the spoke subscription operator(memory)

cacahed client, uncacahed client

• type application
• application with limited resource
• [ ]