README ¶
Karmada operator
Overview
The Karmada operator is a method for installing, upgrading, and deleting Karmada instances. It builds upon the basic Karmada resource and controller concepts, provides convenience to centrally manage entire lifecycle of Karmada instances in a global cluster. With the operator, you can extend Karmada with custom resources (CRs) to manage your instances not only in local clusters but also in remote clusters.
This document is an overview of how the operator works from a user perspective.
Developer quick start
This section describes how to install karmada-operator
and create a Karmada instance with CR.
Prerequisites
- Kubernetes 1.16+
- Helm v3+
Deploy karmada-operator
Helm install
Go to the root directory of the karmada-io/karmada
repo. To install the Helm Chart with
the release name karmada-operator
in the namespace karmada-system
, simply run the helm command:
helm install karmada-operator -n karmada-system --create-namespace --dependency-update ./charts/karmada-operator --debug
Using YAML resource
The karmada-operator
workload requires a kubeconfig of the local cluster to establish a connection with the cluster and watch CR resources.
In preparation for this, create a secret containing the kubeconfig for the karmada-operator.
kubectl create namespace karmada-system
kubectl create secret generic my-kubeconfig --from-file=$HOME/.kube/config -n karmada-system
Deploy the karmada-operator
workload.
kubectl apply -f operator/config/deploy/karmada-operator.yaml
The pod of karmada-operator
in the karmada-system
namespace will be running.
kubectl get po -n karmada-system
NAME READY STATUS RESTARTS AGE
karmada-operator-5b7f485c5-g5lj5 1/1 Running 0 26s
Install Karmada operator crds
kubectl apply -f operator/config/crds/
Create a Karmada instance
The Karmada operator provides a Karmada CR that can define most configurations for Karmada components.
It includes image
messages, replica
, the args
of binary file, and custom label
, annotation
, and featuregate
.
For details, see API.
A Karmada CR represents a Karmada instance, which is a namespace-scoped resource.
The example below is to create a simple Karmada CR in the test
namespace:
kubectl create namespace test
kubectl apply -f - <<EOF
apiVersion: operator.karmada.io/v1alpha1
kind: Karmada
metadata:
name: karmada-demo
namespace: test
EOF
You can also create a Karmada CR directly using the sample provided by the Karmada operator.
kubectl create namespace test
kubectl apply -f operator/config/samples/karmada.yaml
Wait for around 40 seconds, and the pods of the Karmada components will be running in the same namespace as the Karmada CR.
kubectl get po -n test
karmada-demo-aggregated-apiserver-587bc5c697-v27vb 1/1 Running 0 12s
karmada-demo-apiserver-55968d9f8c-mp8hf 1/1 Running 0 35s
karmada-demo-controller-manager-64455f7fd4-stls6 1/1 Running 0 5s
karmada-demo-etcd-0 1/1 Running 0 37s
karmada-demo-kube-controller-manager-584f978bbd-fftwq 1/1 Running 0 5s
karmada-demo-metrics-adapter-57cb5f56b6-4vwk2 1/1 Running 0 5s
karmada-demo-metrics-adapter-57cb5f56b6-zbhjk 1/1 Running 0 5s
karmada-demo-scheduler-6d77b7547-hgz8n 1/1 Running 0 5s
karmada-demo-webhook-6f5944f5d8-bpkqz 1/1 Running 0 5s
Generate kubeconfig for karmada
kubectl get secret -n test karmada-demo-admin-config -o jsonpath={.data.kubeconfig} | base64 -d > ~/.kube/karmada-apiserver.config
export KUBECONFIG=~/.kube/karmada-apiserver.config
Tip:
If no
spec.hostCluster.secretRef
is specified in CR, the Karmada instance will be installed in the cluster wherekarmada-operator
is located.
Upgrade a Karmada instance
Once a Karmada instance is created, the CR resource is automatically filled with default values. To upgrade the Karmada instance, for example, you can upgrade the Karmada version to v1.5.0 or higher:
kubectl patch karmada karmada-demo -n test --type merge -p '
{
"spec": {
"components": {
"karmadaAggregatedAPIServer": {
"imageTag": "v1.5.0"
},
"karmadaControllerManager": {
"imageTag": "v1.5.0"
},
"karmadaScheduler": {
"imageTag": "v1.5.0"
},
"karmadaWebhook": {
"imageTag": "v1.5.0"
}
}
}
}'
Delete a Karmada instance
Deleting a Karmada CR is a delicate operation that requires careful attention. Once the Karmada CR is deleted, the associated Karmada instance will also be deleted. It is important to proceed with caution when deleting a Karmada CR due to the potential risks involved.
kubectl delete karmada karmada-demo -n test
If you want to delete a Karmada CR without cascading deletion of the associated Karmada instance, you can run the following command before performing the deletion operation.
kubectl label karmada karmada-demo -n test operator.karmada.io/disable-cascading-deletion=true
Custom Karmada CR
This feature allows you to configure the Karmada CR to install Karmada instances flexibly. For details, see karmada.yaml.
Set Karmada component replicas
The replicas
of all Karmada components can be modified.
For example, you can scale the etcd pod replicas
to 3:
apiVersion: operator.karmada.io/v1alpha1
kind: Karmada
metadata:
name: karmada-demo
namespace: test
spec:
components:
etcd:
local:
replicas: 3
Custom label and annotation
All Karmada components allow for custom labels and annotations to be set. These are merged into both pod and workload resources.
apiVersion: operator.karmada.io/v1alpha1
kind: Karmada
metadata:
name: karmada-demo
namespace: test
spec:
components:
karmadaAPIServer:
labels:
<custom-label-key>: <custom-label-value>
annotations:
<custom-annotation-key>: <custom-annotation-value>
Change karmada-apiserver service type
The service type of karmada-apiserver is ClusterIP
by default.
You can change it to NodePort
:
...
karmadaAPIServer:
imageRepository: registry.k8s.io/kube-apiserver
imageTag: v1.26.12
replicas: 1
serviceType: NodePort
serviceSubnet: 10.96.0.0/12
...
Add karmada-apiserver SANs
You can add more SANs to karmada-apiserver certificate:
...
karmadaAPIServer:
imageRepository: registry.k8s.io/kube-apiserver
imageTag: v1.26.12
replicas: 1
serviceSubnet: 10.96.0.0/12
certSANs:
- "kubernetes.default.svc"
- "127.0.0.1"
...
Install karmada addon
By default, the Karmada operator does not install the descheduler
and search
addons.
If you want to use them, you should add definitions to the Karmada CR.
Here is an example of the descheduler
addon:
apiVersion: operator.karmada.io/v1alpha1
kind: Karmada
metadata:
name: karmada-demo
namespace: test
spec:
components:
karmadaDescheduler: {}
If you want to install with the defaults, simply define an empty struct for descheduler
.
Contributing
The karmada/operator
repo is part of Karmada from 1.5 onwards. If you're interested in
the Karmada operator and want to contribute your code and ideas, welcome to open PRs and issues.
See CONTRIBUTING for details on submitting patches and the contribution workflow.
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
generated/clientset/versioned
This package has the automatically generated clientset.
|
This package has the automatically generated clientset. |
generated/clientset/versioned/fake
This package has the automatically generated fake clientset.
|
This package has the automatically generated fake clientset. |
generated/clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
|
This package contains the scheme of the automatically generated clientset. |
generated/clientset/versioned/typed/operator/v1alpha1
This package has the automatically generated typed clients.
|
This package has the automatically generated typed clients. |
generated/clientset/versioned/typed/operator/v1alpha1/fake
Package fake has the automatically generated clients.
|
Package fake has the automatically generated clients. |