README
¶
Overview / User Guides
Kubermatic Kubernetes Platform is in an open source project to centrally manage the global automation of thousands of Kubernetes clusters across multicloud, on-prem and edge with unparalleled density and resilience.
All user documentation is available at the Kubermatic Kubernetes Platform docs website.
Editions
There are two editions of Kubermatic Kubernetes Platform:
Kubermatic Kubernetes Platform Community Edition (CE) is available freely under the Apache License, Version 2.0. Kubermatic Kubernetes Platform Enterprise Edition (EE) includes premium features that are most useful for organizations with large-scale Kubernetes installations with more than 50 clusters. To access the Enterprise Edition and get official support please become a subscriber.
Licensing
See the LICENSE file for licensing information as it pertains to files in this repository.
Installation
We strongly recommend that you use an official release of Kubermatic Kubernetes Platform. Follow the instructions under the Installation section of our documentation to get started.
The code and sample YAML files in the main branch of the kubermatic repository are under active development and are not guaranteed to be stable. Use them at your own risk!
More information
The documentation provides a getting started guide, plus information about building from source, architecture, extending kubermatic, and more.
Please use the version selector at the top of the site to ensure you are using the appropriate documentation for your version of kubermatic.
Troubleshooting
If you encounter issues file an issue or talk to us on the #kubermatic channel on the Kubermatic Community Slack (click here to join).
Contributing
Thanks for taking the time to join our community and start contributing!
Before you start
- Please familiarize yourself with the Code of Conduct before contributing.
- See CONTRIBUTING.md for instructions on the developer certificate of origin that we require.
Repository layout
├── addons # Default Kubernetes addons
├── charts # The Helm charts we use to deploy
├── cmd # Various Kubermatic binaries for the controller-managers, operator etc.
├── codegen # Helper programs to generate Go code and Helm charts
├── docs # Some basic developer-oriented documentation
├── hack # scripts for development and CI
└── pkg # most of the actual codebase
Development environment
git clone git@github.com:kubermatic/kubermatic.git
cd kubermatic
There are a couple of scripts in the hacks directory to aid in running the components locally
for testing purposes.
Running components locally
user-cluster-controller-manager
In order to instrument the seed-controller to allow for a local user-cluster-controller-manager, you need to add a worker-name label with your local machine's name as its value. Additionally, you need to scale down the already running deployment.
# Using a kubeconfig, which points to the seed-cluster
export cluster_id="<id-of-your-user-cluster>"
kubectl label cluster ${cluster_id} worker-name=$(uname -n)
kubectl scale deployment -n cluster-${cluster_id} usercluster-controller --replicas=0
Afterwards, you can start your local user-cluster-controller-manager.
# Using a kubeconfig, which points to the seed-cluster
./hack/run-user-cluster-controller-manager.sh
seed-controller-manager
./hack/run-seed-controller-manager.sh
master-controller-manager
./hack/run-master-controller-manager.sh
Run linters
Before every push, make sure you run:
make lint
Run tests
make test
Update code generation
The Kubernetes code-generator tool does not work outside of GOPATH
(upstream issue), so the script
below will automatically run the code generation in a Docker container.
hack/update-codegen.sh
Pull requests
- We welcome pull requests. Feel free to dig through the issues and jump in.
Changelog
See the list of releases to find out about feature changes.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
codegen
|
|
|
pkg
|
|
|
apis/apps.kubermatic/v1
+groupName=apps.kubermatic.k8c.io +versionName=v1 +kubebuilder:object:generate=true
|
+groupName=apps.kubermatic.k8c.io +versionName=v1 +kubebuilder:object:generate=true |
|
apis/kubermatic/v1
+groupName=kubermatic.k8c.io +versionName=v1 +kubebuilder:object:generate=true
|
+groupName=kubermatic.k8c.io +versionName=v1 +kubebuilder:object:generate=true |
|
applications
Package applications contains logic to download the source of an application (e.g.
|
Package applications contains logic to download the source of an application (e.g. |
|
applications/helmclient
Package helmclient contains tools to interact with Helm API for downloading, installing, upgrading, and uninstalling Helm Chart.
|
Package helmclient contains tools to interact with Helm API for downloading, installing, upgrading, and uninstalling Helm Chart. |
|
applications/providers
Package providers provide the different implementations to download the application's source and install / uninstall the application into the user-cluster.
|
Package providers provide the different implementations to download the application's source and install / uninstall the application into the user-cluster. |
|
applications/test
Package test contains helper functions for tests related to helm.
|
Package test contains helper functions for tests related to helm. |
|
cni
Package cni contains supported CNI version definitions and helpers for managing CNIs in KKP.
|
Package cni contains supported CNI version definitions and helpers for managing CNIs in KKP. |
|
cni/cilium
Package cilium contains Cilium CNI related helpers for managing CNI using Applications infra.
|
Package cilium contains Cilium CNI related helpers for managing CNI using Applications infra. |
|
controller
Package controller contains all our controllers.
|
Package controller contains all our controllers. |
|
controller/kubeletdnat-controller
Package kubeletdnatcontroller contains the kubeletdnat controller which:
|
Package kubeletdnatcontroller contains the kubeletdnat controller which: |
|
controller/master-controller-manager
Package mastercontrollermanager contains all controllers that run within the master-controller-manager binary.
|
Package mastercontrollermanager contains all controllers that run within the master-controller-manager binary. |
|
controller/master-controller-manager/application-definition-synchronizer
Package applicationdefinitionsynchronizer contains a controller that is responsible for ensuring that ApplicationDefinition are synced from master to the seed clusters.
|
Package applicationdefinitionsynchronizer contains a controller that is responsible for ensuring that ApplicationDefinition are synced from master to the seed clusters. |
|
controller/master-controller-manager/application-secret-synchronizer
Package applicationsecretsynchronizer contains a controller that is responsible for ensuring that ApplicationSecrets are synced from master to the seed clusters.
|
Package applicationsecretsynchronizer contains a controller that is responsible for ensuring that ApplicationSecrets are synced from master to the seed clusters. |
|
controller/master-controller-manager/cluster-template-synchronizer
Package clustertemplatesynchronizer contains a controller that is responsible for ensuring that the kubermatic ClusterTemplate objects are synced from master to the seed clusters.
|
Package clustertemplatesynchronizer contains a controller that is responsible for ensuring that the kubermatic ClusterTemplate objects are synced from master to the seed clusters. |
|
controller/master-controller-manager/kc-status-controller
Package kcstatuscontroller contains a controller that is responsible for managing the basic parts of the KubermaticConfigurationStatus:
|
Package kcstatuscontroller contains a controller that is responsible for managing the basic parts of the KubermaticConfigurationStatus: |
|
controller/master-controller-manager/master-constraint-controller
Package masterconstraintsyncercontroller contains a controller that is responsible for ensuring that the kubermatic constraint are synced from master to the seed clusters.
|
Package masterconstraintsyncercontroller contains a controller that is responsible for ensuring that the kubermatic constraint are synced from master to the seed clusters. |
|
controller/master-controller-manager/master-constraint-template-controller
Package masterconstrainttemplatecontroller contains a controller that is responsible for ensuring that the kubermatic constraint templates are synced from master to the seed clusters.
|
Package masterconstrainttemplatecontroller contains a controller that is responsible for ensuring that the kubermatic constraint templates are synced from master to the seed clusters. |
|
controller/master-controller-manager/preset-synchronizer
Package presetsynchronizer contains a controller that is responsible for ensuring that the kubermatic Preset objects are synced from master to the seed clusters.
|
Package presetsynchronizer contains a controller that is responsible for ensuring that the kubermatic Preset objects are synced from master to the seed clusters. |
|
controller/master-controller-manager/project-label-synchronizer
Package projectlabelsynchronizer contains a controller that synchronizes labels from a project onto all the clusters that belong to the project, without allowing them to be overridden.
|
Package projectlabelsynchronizer contains a controller that synchronizes labels from a project onto all the clusters that belong to the project, without allowing them to be overridden. |
|
controller/master-controller-manager/project-synchronizer
Package projectsynchronizer contains a controller that is responsible for ensuring that the Kubermatic Project objects are synced from master to the seed clusters.
|
Package projectsynchronizer contains a controller that is responsible for ensuring that the Kubermatic Project objects are synced from master to the seed clusters. |
|
controller/master-controller-manager/rbac
Package rbac contains a controller that is responsible for setting up RBAC to allow users to access the clusters they should be able to access.
|
Package rbac contains a controller that is responsible for setting up RBAC to allow users to access the clusters they should be able to access. |
|
controller/master-controller-manager/seed-proxy
Package seedproxy contains a controller that is responsible for creating various resources in the master cluster to be able to access the other seeds clusters prometheus, leveraging `kubectl proxy` under the hood.
|
Package seedproxy contains a controller that is responsible for creating various resources in the master cluster to be able to access the other seeds clusters prometheus, leveraging `kubectl proxy` under the hood. |
|
controller/master-controller-manager/seed-status-controller
Package seedstatuscontroller contains a controller that is responsible for managing the basic parts of the SeedStatus:
|
Package seedstatuscontroller contains a controller that is responsible for managing the basic parts of the SeedStatus: |
|
controller/master-controller-manager/seed-sync
Package seedsync contains a controller that is responsible for synchronizing the `Seed` custom resources onto the corresponding seed clusters, so that the seed-controller-manager can use them.
|
Package seedsync contains a controller that is responsible for synchronizing the `Seed` custom resources onto the corresponding seed clusters, so that the seed-controller-manager can use them. |
|
controller/master-controller-manager/serviceaccount-projectbinding-controller
Package serviceaccountprojectbindingcontroller contains a controller that watches Users, but only those that are KKP service accounts.
|
Package serviceaccountprojectbindingcontroller contains a controller that watches Users, but only those that are KKP service accounts. |
|
controller/master-controller-manager/user-synchronizer
Package usersynchronizer contains a controller that is responsible for ensuring that the kubermatic User objects (excluding service account users) are synced from master to the seed clusters.
|
Package usersynchronizer contains a controller that is responsible for ensuring that the kubermatic User objects (excluding service account users) are synced from master to the seed clusters. |
|
controller/master-controller-manager/usersshkey-project-ownership
Package usersshkeyprojectownershipcontroller contains a controller that ensures that each UserSSHKey has an owner reference pointing to the project specified in spec.project.
|
Package usersshkeyprojectownershipcontroller contains a controller that ensures that each UserSSHKey has an owner reference pointing to the project specified in spec.project. |
|
controller/master-controller-manager/usersshkey-synchronizer
The usersshkeysynchronizer controller is responsible for synchronizing usersshkeys into a secret in the cluster namespace.
|
The usersshkeysynchronizer controller is responsible for synchronizing usersshkeys into a secret in the cluster namespace. |
|
controller/operator
Package operator contains all controllers that run within the kubermatic operator binary.
|
Package operator contains all controllers that run within the kubermatic operator binary. |
|
controller/operator/seed-init
Package seedinit contains a controller that is responsible for setting up new Seed clusters.
|
Package seedinit contains a controller that is responsible for setting up new Seed clusters. |
|
controller/operator/seed/resources/nodeportproxy
Package nodeportproxy is responsible for reconciling a seed-cluster-wide proxy based on Envoy and a custom envoy-manager/lb-updater tools.
|
Package nodeportproxy is responsible for reconciling a seed-cluster-wide proxy based on Envoy and a custom envoy-manager/lb-updater tools. |
|
controller/seed-controller-manager
Package seedcontrollermanager contains a package for each controller that runs within the seed controller manager binary.
|
Package seedcontrollermanager contains a package for each controller that runs within the seed controller manager binary. |
|
controller/seed-controller-manager/addon
Package addon contains a controller that applies addons based on a Addon CRD.
|
Package addon contains a controller that applies addons based on a Addon CRD. |
|
controller/seed-controller-manager/addoninstaller
Package addoninstaller contains a controller that is responsible for making sure a set of addons that are configured via a flag on the controller-manager and are required for basic cluster functionality exist for all clusters.
|
Package addoninstaller contains a controller that is responsible for making sure a set of addons that are configured via a flag on the controller-manager and are required for basic cluster functionality exist for all clusters. |
|
controller/seed-controller-manager/application-secret-cluster-controller
Package applicationsecretclustercontroller contains a controller that ensure ApplicationSecrets are synced from kubermatic namespace to the user cluster namespaces.
|
Package applicationsecretclustercontroller contains a controller that ensure ApplicationSecrets are synced from kubermatic namespace to the user cluster namespaces. |
|
controller/seed-controller-manager/auto-update-controller
Package autoupdatecontroller contains a controller that auto applies updates to the cluster version and potentially to the MachineDeployments inside the usercluster.
|
Package autoupdatecontroller contains a controller that auto applies updates to the cluster version and potentially to the MachineDeployments inside the usercluster. |
|
controller/seed-controller-manager/cloud
Package cloud contains a controller that is responsible for creating cluster-level resources at the cloud provider, like networks, subnets or security groups.
|
Package cloud contains a controller that is responsible for creating cluster-level resources at the cloud provider, like networks, subnets or security groups. |
|
controller/seed-controller-manager/cluster-credentials-controller
Package clustercredentialscontroller contains a controller that watches Cluster objects and is responsible for moving inline credentials (from the CloudSpec) into dedicated Kubernetes Secrets in the KKP namespace.
|
Package clustercredentialscontroller contains a controller that watches Cluster objects and is responsible for moving inline credentials (from the CloudSpec) into dedicated Kubernetes Secrets in the KKP namespace. |
|
controller/seed-controller-manager/cluster-phase-controller
Package clusterphasecontroller contains a controller that updates the Phase on a Cluster object, based on the ClusterStatus.
|
Package clusterphasecontroller contains a controller that updates the Phase on a Cluster object, based on the ClusterStatus. |
|
controller/seed-controller-manager/cluster-stuck-controller
Package clusterstuckcontroller contains a controller that is only useful during development.
|
Package clusterstuckcontroller contains a controller that is only useful during development. |
|
controller/seed-controller-manager/cluster-template-controller
Package clustertemplatecontroller contains a controller that is responsible for managing cluster template instances.
|
Package clustertemplatecontroller contains a controller that is responsible for managing cluster template instances. |
|
controller/seed-controller-manager/cni-application-installation-controller
Package cniapplicationinstallationcontroller contains a controller that watches Cluster resources, and if the CNI for the Cluster is managed by the Applications infra, reconciles ApplicationInstallation Resources in the user cluster with necessary CNI configuration in ApplicationInstallation's Values.
|
Package cniapplicationinstallationcontroller contains a controller that watches Cluster resources, and if the CNI for the Cluster is managed by the Applications infra, reconciles ApplicationInstallation Resources in the user cluster with necessary CNI configuration in ApplicationInstallation's Values. |
|
controller/seed-controller-manager/constraint-controller
Package seedconstraintsynchronizer contains a controller that is responsible for ensuring that the kubermatic constraints are synced to the user cluster namespace.
|
Package seedconstraintsynchronizer contains a controller that is responsible for ensuring that the kubermatic constraints are synced to the user cluster namespace. |
|
controller/seed-controller-manager/constraint-template-controller
Package constrainttemplatecontroller contains a controller that is responsible for ensuring that the kubermatic constraint templates are synced to the user cluster.
|
Package constrainttemplatecontroller contains a controller that is responsible for ensuring that the kubermatic constraint templates are synced to the user cluster. |
|
controller/seed-controller-manager/initial-application-installation-controller
Package initialapplicationinstallationcontroller contains a controller that watches Cluster resources that have the InitialApplicationInstallations annotation.
|
Package initialapplicationinstallationcontroller contains a controller that watches Cluster resources that have the InitialApplicationInstallations annotation. |
|
controller/seed-controller-manager/initial-machinedeployment-controller
Package initialmachinedeployment contains a controller that watches Cluster resources and will transform the InitialMachineDeployment annotation into an actual MachineDeployment once the cluster has become ready.
|
Package initialmachinedeployment contains a controller that watches Cluster resources and will transform the InitialMachineDeployment annotation into an actual MachineDeployment once the cluster has become ready. |
|
controller/seed-controller-manager/ipam
Package ipam contains a controller that is responsible for managing IPAM (Multi-Cluster IP Address Management) pools.
|
Package ipam contains a controller that is responsible for managing IPAM (Multi-Cluster IP Address Management) pools. |
|
controller/seed-controller-manager/kubernetes
Package kubernetes contains a controller that reconciles all controlplane components in the seed needed for Kubernetes clusters.
|
Package kubernetes contains a controller that reconciles all controlplane components in the seed needed for Kubernetes clusters. |
|
controller/seed-controller-manager/mla
Package mla contains controllers that are responsible for configuring MLA (Monitoring, Logging, and Alerting) for user clusters.
|
Package mla contains controllers that are responsible for configuring MLA (Monitoring, Logging, and Alerting) for user clusters. |
|
controller/seed-controller-manager/monitoring
Package monitoring contains a controller that reconciles all monitoring-related components for all cluster types in the seed.
|
Package monitoring contains a controller that reconciles all monitoring-related components for all cluster types in the seed. |
|
controller/seed-controller-manager/operating-system-profile-synchronizer
Package operatingsystemprofilesynchronizer contains a controller that is responsible for ensuring that OperatingSystemProfiles are synced from the seed namespace to the user cluster namespace.
|
Package operatingsystemprofilesynchronizer contains a controller that is responsible for ensuring that OperatingSystemProfiles are synced from the seed namespace to the user cluster namespace. |
|
controller/seed-controller-manager/preset-controller
Package presetcontroller contains a controller that is responsible for managing presets.
|
Package presetcontroller contains a controller that is responsible for managing presets. |
|
controller/seed-controller-manager/project
Package project contains a controller responsible for removing all clusters in a project whenever a project is deleted, and (importantly) waiting until all clusters are gone before releasing the project.
|
Package project contains a controller responsible for removing all clusters in a project whenever a project is deleted, and (importantly) waiting until all clusters are gone before releasing the project. |
|
controller/seed-controller-manager/pvwatcher
Package pvwatcher contains a controller that is responsible for monitoring and recovering users' clusters etcd plane PVs.
|
Package pvwatcher contains a controller that is responsible for monitoring and recovering users' clusters etcd plane PVs. |
|
controller/seed-controller-manager/seedresourcesuptodatecondition
Package seedresourcesuptodatecondition contains a controller that is responsible for:
|
Package seedresourcesuptodatecondition contains a controller that is responsible for: |
|
controller/seed-controller-manager/update-controller
Package updatecontroller contains a controller that coordinates the upgrade of clusters.
|
Package updatecontroller contains a controller that coordinates the upgrade of clusters. |
|
controller/shared
Package shared contains all controllers that are imported by more than one controller-manager
|
Package shared contains all controllers that are imported by more than one controller-manager |
|
controller/user-cluster-controller-manager
Package userclustercontrollermanager contains all controllers running in the usercluster controller manager binary.
|
Package userclustercontrollermanager contains all controllers running in the usercluster controller manager binary. |
|
controller/user-cluster-controller-manager/application-installation-controller
Package applicationinstallationcontroller contains a controller that is responsible for reconciling ApplicationInstallation (ie install, update or uninstall applications into the user-cluster)
|
Package applicationinstallationcontroller contains a controller that is responsible for reconciling ApplicationInstallation (ie install, update or uninstall applications into the user-cluster) |
|
controller/user-cluster-controller-manager/ccm-csi-migrator
Package ccmcsimigrator contains a controller that is responsible to ensure the cluster status related to CCM/CSI machine migration.
|
Package ccmcsimigrator contains a controller that is responsible to ensure the cluster status related to CCM/CSI machine migration. |
|
controller/user-cluster-controller-manager/cluster-role-labeler
Package clusterrolelabeler contains a controller that is responsible for ensuring that the viewer, editor and admin clusterroles have a `component: userClusterRole` label associated.
|
Package clusterrolelabeler contains a controller that is responsible for ensuring that the viewer, editor and admin clusterroles have a `component: userClusterRole` label associated. |
|
controller/user-cluster-controller-manager/constraint-syncer
Package constraint syncer contains the controller which is responsible for syncing the kubermatic Constraints to the user cluster as gatekeeper constraints.
|
Package constraint syncer contains the controller which is responsible for syncing the kubermatic Constraints to the user cluster as gatekeeper constraints. |
|
controller/user-cluster-controller-manager/flatcar
Package flatcar linux contains the flatcar linux controller that is responsible for deploying the [Flatcar Linux Update Operator](https://github.com/flatcar/flatcar-linux-update-operator) operator and DaemonSet
|
Package flatcar linux contains the flatcar linux controller that is responsible for deploying the [Flatcar Linux Update Operator](https://github.com/flatcar/flatcar-linux-update-operator) operator and DaemonSet |
|
controller/user-cluster-controller-manager/ipam
Package ipam contains a controller responsible for assigning IP addresses from a configured pool to machines that have an annotation keyed `machine-controller.kubermatic.io/initializers` which contains the value ipam.
|
Package ipam contains a controller responsible for assigning IP addresses from a configured pool to machines that have an annotation keyed `machine-controller.kubermatic.io/initializers` which contains the value ipam. |
|
controller/user-cluster-controller-manager/kubevirt-vmi-eviction
Package kubevirtvmieviction contains a controller that is responsible for handling bare-metal nodes draining: - watches in the KubeVirt infra cluster the VMI status.evacuationNodeName.
|
Package kubevirtvmieviction contains a controller that is responsible for handling bare-metal nodes draining: - watches in the KubeVirt infra cluster the VMI status.evacuationNodeName. |
|
controller/user-cluster-controller-manager/node-labeler
Package nodelabeler contains a controller that ensures Nodes have various labels present at all times:
|
Package nodelabeler contains a controller that ensures Nodes have various labels present at all times: |
|
controller/user-cluster-controller-manager/node-version-controller
Package nodeversioncontroller contains a controller that watches Nodes and updates the Cluster's status with the oldestNodeVersion.
|
Package nodeversioncontroller contains a controller that watches Nodes and updates the Cluster's status with the oldestNodeVersion. |
|
controller/user-cluster-controller-manager/owner-binding-creator
The ownerbindingcreator controller is responsible for making sure that the binding exists and if it doesn't, will create it and use the cluster owner as subject.
|
The ownerbindingcreator controller is responsible for making sure that the binding exists and if it doesn't, will create it and use the cluster owner as subject. |
|
controller/user-cluster-controller-manager/resources
Package resources contains the resources controller responsible for making sure the various Kubernetes resources we need in the usercluster like RBAC, Deployments, ServiceAccounts and much more are present.
|
Package resources contains the resources controller responsible for making sure the various Kubernetes resources we need in the usercluster like RBAC, Deployments, ServiceAccounts and much more are present. |
|
controller/user-cluster-controller-manager/role-cloner-controller
Package roleclonercontroller contains a controller that duplicates all roles with the `component=userClusterRole` label that are in the kube-system namespace into all other namespaces.
|
Package roleclonercontroller contains a controller that duplicates all roles with the `component=userClusterRole` label that are in the kube-system namespace into all other namespaces. |
|
controller/usersshkeysagent
Package usersshkeysagent contains the usersshkeysagent controller, which is deployed as a DaemonSet on all usercluster nodes and responsible for synchronizing the `$HOME/.ssh/authorized_keys` file for all users we know about (root, core, ubuntu, centos) and that exist with the content of a secret.
|
Package usersshkeysagent contains the usersshkeysagent controller, which is deployed as a DaemonSet on all usercluster nodes and responsible for synchronizing the `$HOME/.ssh/authorized_keys` file for all users we know about (root, core, ubuntu, centos) and that exist with the content of a secret. |
|
controller/util
Package util contains helpers for building controllers.
|
Package util contains helpers for building controllers. |
|
provider
Package provider contains abstract cloud provider types and interfaces.
|
Package provider contains abstract cloud provider types and interfaces. |
|
provider/cloud/aws
Package aws contains the Amazon Web Services (AWS) cloud provider implementation.
|
Package aws contains the Amazon Web Services (AWS) cloud provider implementation. |
|
provider/cloud/aws/data
Package data is separate from the regular AWS provider package because it initializes a large chunk of data that might not be relevant to everyone using the AWS provider code.
|
Package data is separate from the regular AWS provider package because it initializes a large chunk of data that might not be relevant to everyone using the AWS provider code. |
|
resources/certificates/triple
Package triple generates key-certificate pairs for the triple (CA, Server, Client).
|
Package triple generates key-certificate pairs for the triple (CA, Server, Client). |
|
resources/registry
Package registry groups all container registry related types and helpers in one place.
|
Package registry groups all container registry related types and helpers in one place. |
Click to show internal directories.
Click to hide internal directories.