csm-replication

README

Dell Container Storage Modules (CSM) for Replication

Dell CSM for Replication is part of the CSM (Container Storage Modules) open-source suite of Kubernetes storage enablers for Dell products.

This project aims at extending native Kubernetes functionality to support Disaster Recovery workflows by utilizing storage array based replication.

CSM Replication project includes the following components:

• CSM Replication CRDs - Custom Resource definitions
• CSM Replication sidecar container for CSI drivers
• CSM Replication Controller - Multi-Cluster Controller
• repctl - helper utility for setups, queries

CSM Replication sidecar is installed as part of the CSI driver controller plugin and utilizes custom Dell CSI extensions to communicate with Dell CSI drivers. It includes a set of Kubernetes controllers which keep watch on requests related to Persistent Volumes (PVs) and Persistent Volume Claims(PVCs). It uses dell-csi-extensions APIs to perform replication related actions. You can read more about the extensions here.

CSM Replication Controller is a set of Kubernetes controllers which are multi-cluster aware and can communicate with various Kubernetes API servers to reconcile state. These controllers are responsible for the syncing of objects and the associated metadata.

This project has been created using kubebuilder, and the various kubernetes controllers have been implemented using controller-runtime. Additionally, this project uses kustomize for generating various yaml manifests for Kubernetes objects.

For documentation, please visit Container Storage Modules documentation.

Table of Contents

• Code of Conduct
• Maintainer Guide
• Committer Guide
• Contributing Guide
• Branching Strategy
• List of Adopters
• Maintainers
• Support
• Security
• About

Build

This project contains multiple go modules, and you can build multiple binaries & images with the provided Makefile.

Dependencies

This project relies on the following tools which have to be installed in order to generate certain manifests.

Tool	Version
controller-gen	v0.4.1
kustomize	v3.10.0

The above tools can also be installed by running the command make tools.

Custom Resource Definitions

Run the command make manifests to build the CSM Replication CRDs. This command invokes controller-gen to generate the CRDs. The API code is annotated with kubebuilder tags which are used by thecontroller-gen generators. The generated definitions are present in the form of a kustomize recipe in the config/crd folder.

Binaries

For building the CSM Replication sidecar, dell-csi-replicator, run make sidecar-manager.

For building the CSM Replication Controller, dell-replication-controller, run make controller-manager.

For building the repctl binary, run cd repctl && make build.

Static manifests

In order to simplify installation for users who don't want to use kustomize, statically generated manifests are stored in the repository.

• deploy/controller.yaml - manifest for installing the CSM Replication Controller.
• deploy/role.yaml - manifest for creating ClusterRole for CSM Replication Controller.
• deploy/replicationcrds.all.yaml - manifest for install CSM Replication CRDs.

In case there are modifications done which update the CRD, run make static-crd and commit changes to the CRD file in the repository.

In case there are modifications done which update any manifests for the installation of the CSM Replication Controller, run make static-controller and commit changes to the file deploy/controller.yaml in the repository.

Container Images

There are various Makefile targets available for building and pushing container images

Target	Description
image-sidecar	Build dell-csi-replicator sidecar image
image-sidecar-push	Build dell-csi-replicator sidecar image & push to an image repo
image-controller	Build dell-replication-controller image
image-controller-push	Build dell-replication-controller image & push to an image repo
images	Build both dell-csi-replicator & dell-replication-controller images
images-push	Push both dell-csi-replicator & dell-replication-controller images

Note: make images will also tag the built images with the release tags and push those tags to the image repo.

RBAC manifests

For building the RBAC rules for the sidecar, run make sidecar-rbac. For building the RBAC rules for the controller, run make controller-rbac.

Installation

Custom Resource Definitions

Run the command make install to install the Custom Resource Definitions in your Kubernetes cluster. Alternatively, you can directly install the CRDs using the static manifest file deploy/replicationcrds.all.yaml by running the command -

kubectl create -f deploy/replicationcrds.all.yaml

Note: CRDs must be installed before installing either CSM Replication sidecar or CSM Replication Controller.

CSM Replication sidecar

The CSM Replication sidecar container can only be installed as a sidecar within the CSI driver controller pod. The kubernetes controllers which run as part of the sidecar container, communicate with the CSI drivers using gRPC over a Unix Domain Socket (UDS). The address for this socket can be passed as an argument to the sidecar container.

The RBAC permissions required by the kubernetes controllers are present in the file config/sidecar-rbac/role.yaml and should be included in the CSI driver's RBAC permissions.

You can also run the dell-csi-replicator process directly in your Kubernetes cluster by running the command make run-sidecar. Make sure that the kubernetes user has the desired RBAC permissions as described above.

CSM Replication Controller

Pre-requisites

CSM Replication Controller is always installed in the namespace dell-replication-controller. You should also edit the file deploy/config.yaml with the required details before proceeding with the installation.

Using Install script

Use the script scripts/install.sh to install the CSM Replication Controller. This script will do the following:

• Create namespace dell-replication-controller
• Create/Update configmap using the contents of deploy/config.yaml
• Install/Update the CRDs
• Install/Update the dell-replication-controller Deployment

Note: In case you want to install a specific version, make sure to update the deploy/controller.yaml to point to the desired image tag.

Using Makefile target

If you are building your own images using the provided Makefile targets, you can install the CSM Replication Controller by running the command make deploy-controller. The namespace dell-replication-controller must exist in the cluster before this installation.

You can also run the dell-replication-controller process directly in your Kubernetes cluster by running the command make run-controller. Make sure that the kubernetes user has the desired RBAC permissions.

Testing

Click here for more details on how to test.