csm-replication

README

Dell Container Storage Modules (CSM) for Replication

Dell CSM for Replication is part of the CSM (Container Storage Modules) open-source suite of Kubernetes storage enablers for Dell products.

This project aims at extending native Kubernetes functionality to support Disaster Recovery workflows by utilizing storage array based replication.

CSM Replication project includes the following components:

• CSM Replication CRDs - Custom Resource definitions
• CSM Replication sidecar container for CSI drivers
• CSM Replication Controller - Multi-Cluster Controller
• repctl - helper utility for setup and queries

CSM Replication sidecar is installed as part of the CSI driver controller pod and utilizes custom Dell CSI extensions to communicate with Dell CSI drivers. It includes a set of Kubernetes controllers which keep watch on requests related to Persistent Volumes (PVs) and Persistent Volume Claims(PVCs). It uses dell-csi-extensions APIs to perform replication related actions. You can read more about the extensions here.

CSM Replication Controller is a set of Kubernetes controllers which are multi-cluster aware and can communicate with various Kubernetes API servers to reconcile state. These controllers are responsible for the syncing of objects and the associated metadata.

This project has been created using kubebuilder, and the various kubernetes controllers have been implemented using controller-runtime. Additionally, this project uses kustomize for generating various yaml manifests for Kubernetes objects.

For documentation, please visit Container Storage Modules documentation.

Table of Contents

• Code of Conduct
• Maintainer Guide
• Committer Guide
• Contributing Guide
• List of Adopters
• Support
• Security
• About

Build

This project contains multiple go modules, and you can build multiple binaries & images with the provided Makefile.

Dependencies

This project relies on the following tools which have to be installed in order to generate certain manifests.

Tool	Version
controller-gen	v0.15.0
kustomize	v5.4.3

The above tools can also be installed by running the command make tools.

Custom Resource Definitions

Run the command make manifests to build the CSM Replication Custom Resource Definitions (CRDs). This command invokes controller-gen to generate the CRDs. The API code is annotated with kubebuilder tags which are used by the controller-gen generators. The generated definitions are present in the form of a kustomize recipe in the config/crd folder.

Binaries

To build all the binaries associated with CSM Replication, run make build.

To compile individual binaries run:

• make build-controller-manager to build the CSM Replication Controller, dell-replication-controller.
• make build-sidecar-manager to build the CSM Replication sidecar, dell-csi-replicator.
• make build-sidecar-migrator to build the CSM Replication sidecar, dell-csi-migrator.
• make build-sidecar-node-rescanner to build the CSM Replication sidecar, dell-csi-node-rescanner.

Repctl

To build the repctl binary, run cd repctl && make build.

Static manifests

In order to simplify installation for users who don't want to use kustomize, statically generated manifests are stored in the repository.

• deploy/controller.yaml - manifest for installing the CSM Replication Controller.
• deploy/role.yaml - manifest for creating ClusterRole for CSM Replication Controller.
• deploy/replicationcrds.all.yaml - manifest for install CSM Replication CRDs.

In case there are modifications done which update the CRD, run make static-crd and commit changes to the CRD file in the repository.

In case there are modifications done which update any manifests for the installation of the CSM Replication Controller, run make static-controller and commit changes to the file deploy/controller.yaml in the repository.

Container Images

There are various Makefile targets available for building and pushing container images

Target	Description
image-sidecar	Build dell-csi-replicator sidecar image
image-sidecar-push	Build dell-csi-replicator sidecar image & push to an image repo
image-controller	Build dell-replication-controller image
image-controller-push	Build dell-replication-controller image & push to an image repo
images	Build both dell-csi-replicator & dell-replication-controller images
images-push	Push both dell-csi-replicator & dell-replication-controller images

Note: make images will also tag the built images with the release tags and push those tags to the image repo.

RBAC manifests

For building the RBAC rules for the sidecar, run make sidecar-rbac. For building the RBAC rules for the controller, run make controller-rbac.

Installation

Please consult the Installation Guide for instructions on installing the replication controller and sidecar.

To build and install your own images using the provided Makefile targets, below information may be useful:

Custom Resource Definitions

You can run the command make install to install the Custom Resource Definitions in your Kubernetes cluster.

CSM Replication sidecar

The CSM Replication sidecar container can only be installed as a sidecar within the CSI driver controller pod. The kubernetes controllers which run as part of the sidecar container, communicate with the CSI drivers using gRPC over a Unix Domain Socket (UDS). The address for this socket can be passed as an argument to the sidecar container.

The RBAC permissions required by the kubernetes controllers are present in the file config/sidecar-rbac/role.yaml and should be included in the CSI driver's RBAC permissions.

You can also run the dell-csi-replicator process directly in your Kubernetes cluster by running the command make run-sidecar. Make sure that the kubernetes user has the desired RBAC permissions as described above.

CSM Replication Controller

You can install the CSM Replication Controller by running the command make deploy-controller. The namespace dell-replication-controller must exist in the cluster before this installation.

You can also run the dell-replication-controller process directly in your Kubernetes cluster by running the command make run-controller. Make sure that the kubernetes user has the desired RBAC permissions.

Platform Notes

If you wish to install CSM Replication Controller with repctl on your openSUSE server, when referring to this section, you need to install glibc-devel-static devel package before running make build.

Testing

Click here for details on how to test.