sleepcycles

README

Define sleep & wake up cycles for your Kubernetes resources. Automatically schedule to shutdown Deployments, CronJobs, StatefulSets and HorizontalPodAutoscalers that occupy resources in your cluster and wake them up only when you need them; in that way you can:

• schedule resource-hungry workloads (migrations, synchronizations, replications) in hours that do not impact your daily business
• depressurize your cluster
• decrease your costs
• reduce your power consumption
• lower you carbon footprint

Getting Started

You’ll need a Kubernetes cluster to run against. You can use KIND or K3D to get a local cluster for testing, or run against a remote cluster.

Samples

Under config/samples you will find a set manifests that you can use to test this sleepcycles on your cluster:

SleepCycles

• core_v1alpha1_sleepcycle_app_x.yaml, manifests to deploy 2 SleepCycle resources in namespaces app-1 and app-2

apiVersion: core.rekuberate.io/v1alpha1
kind: SleepCycle
metadata:
  name: sleepcycle-app-1
  namespace: app-1
spec:
  shutdown: "1/2 * * * *"
  shutdownTimeZone: "Europe/Athens"
  wakeup: "*/2 * * * *"
  wakeupTimeZone: "Europe/Dublin"
  enabled: true

[!NOTE] The cron expressions of the samples are tailored so you perform a quick demo. The shutdown expression schedules the deployment to scale down on odd minutes and the wakeup schedule to scale up on even minutes.

Every SleepCycle has the following mandatory properties:

• shutdown: cron expression for your shutdown schedule
• enabled: whether this sleepcycle policy is enabled

and the following non-mandatory properties:

• shutdownTimeZone: the timezone for your shutdown schedule, defaults to UTC
• wakeup: cron expression for your wake-up schedule
• wakeupTimeZone: the timezone for your wake-up schedule, defaults to UTC
• successfulJobsHistoryLimit: how many completed CronJob Runner Pods to retain for debugging reasons, defaults to 1
• failedJobsHistoryLimit: how many failed CronJob Runner Pods to retain for debugging reasons, defaults to 1
• runnerImage: the image to use when spawn CronJob Runner pods, defaults to akyriako78/rekuberate-io-sleepcycles-runners

[!IMPORTANT] DO NOT ADD seconds or timezone information to you cron expressions.

Demo workloads

• whoami-app-1_x-deployment.yaml, manifests to deploy 2 Deployment that provisions traefik/whoami in namespace app-1
• whoami-app-2_x-deployment.yaml, manifests to deploy a Deploymentthat provisions traefik/whoami in namespace app-2
• apache-hpa.yaml, manifest to deploy an HorizontalPodAutoscaler for a PHP application in namespace app-2
• nginx-statefulset.yaml, manifest to deploy a Statefulsetin namespace app-2
• busybox-cronjob.yaml, manifest to deploy a Statefulsetin namespace app-1

SleepCycle is a namespace-scoped custom resource; the controller will monitor all the resources in that namespace that are marked with a Label that has as key rekuberate.io/sleepcycle: and as value the name of the manifest you created:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-2
  namespace: app-2
  labels:
    app: app-2
    rekuberate.io/sleepcycle: sleepcycle-app-2
spec:
  replicas: 9
  selector:
    matchLabels:
      app: app-2
  template:
    metadata:
      name: app-2
      labels:
        app: app-2
    spec:
      containers:
        - name: app-2
          image: traefik/whoami
          imagePullPolicy: IfNotPresent

[!IMPORTANT] Any workload in namespace kube-system marked with rekuberate.io/sleepcycle will be ignored by the controller by design.

How it works

The diagram below describes how rekuberate.io/sleepcycles are dealing with scheduling a Deployment:

1. The sleepcycle-controller watches periodically, every 1min, all the SleepCycle custom resources for changes (in all namespaces).
2. The controller, for every SleepCycle resource within the namespace app-1, collects all the resources that have been marked with the label rekuberate.io/sleepcycle: sleepcycle-app1.
3. It provisions, for every workload - in this case deployment deployment-app1 a CronJob for the shutdown schedule and optionally a second CronJob if a wake-up schedule is provided.
4. It provisions a ServiceAccount, a Role and a RoleBinding per namespace, in order to make possible for runner-pods to update resources' specs.
5. The Runner pods will be created automatically by the cron jobs and are responsible for scaling the resources up or down.

[!NOTE] In the diagram it was depicted how rekuberate.io/sleepcycles scales Deployment. The same steps count for a StatefulSet and a HorizontalPodAutoscaler. There are two exception though:

• a HorizontalPodAutoscaler will scale down to 1 replica and not to 0 as for a Deployment or a Statefulset.
• a CronJob has no replicas to scale up or down, it is going to be enabled or suspended respectively.

Using it together with ArgoCD

You can combine rekuberate.io/sleepcycles with applications provisioned with ArgoCD, as long as you disable self-healing when an automatic sync policy is enabled, otherwise ArgoCD's sync mechanism will operate antagonistically towards the shutdown and wakeup cronjobs. In this case ArgoCD will always automatically revert to the state described in the git manifests and practically will cancel the effect of the sleepcycle schedule. An ArgoCD application with Manual or Automatic sync policy without self-healing will work as expected.

[!TIP] There is a git repository in place, with all the necessary artifacts to deploy via ArgoCD an nginx application and a preconfigured sleepcycle. You can find the git repo here.

Deploy

From sources

1. Build and push your image to the location specified by IMG in Makefile:

# Image URL to use all building/pushing image targets
DOCKER_HUB_NAME ?= $(shell docker info | sed '/Username:/!d;s/.* //')
# sleepcycles
IMG_TAG ?= $(shell git rev-parse --short HEAD)
IMG_NAME ?= rekuberate-io-sleepcycles
IMG ?= $(DOCKER_HUB_NAME)/$(IMG_NAME):$(IMG_TAG)
# runners
RUNNERS_IMG_NAME ?= rekuberate-io-sleepcycles-runners
RUNNERS_IMG ?= $(DOCKER_HUB_NAME)/$(RUNNERS_IMG_NAME)
# targets
PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
# CONTAINER_TOOL defines the container tool to be used for building images.
# Be aware that the target commands are only tested with Docker which is
# scaffolded by default. However, you might want to replace it to use other
# tools. (i.e. podman)
CONTAINER_TOOL ?= docker

make docker-buildx

1. Deploy the controller to the cluster using the image defined in IMG:

make install && make deploy

and then deploy the samples:

kubectl create namespace app-1
kubectl create namespace app-2
kubectl apply -f config/samples

Uninstall

make undeploy

Using Helm (from sources)

If you are on a development environment, you can quickly test & deploy the controller to the cluster using a Helm chart directly from config/helm:

helm install rekuberate-io-sleepcycles config/helm/ -n <namespace> --create-namespace

and then deploy the samples:

kubectl create namespace app-1
kubectl create namespace app-2
kubectl apply -f config/samples

Uninstall

helm uninstall rekuberate-io-sleepcycles -n <namespace>

Using Helm (from repo)

On the other hand if you are deploying on a production environment, it is highly recommended to deploy the controller to the cluster using a Helm chart from its repo:

helm repo add sleepcycles https://rekuberate-io.github.io/sleepcycles/
helm repo update

helm upgrade --install sleepcycles sleepcycles/sleepcycles -n rekuberate-system --create-namespace

and then deploy the samples:

kubectl create namespace app-1
kubectl create namespace app-2
kubectl apply -f config/samples

Uninstall

helm uninstall rekuberate-io-sleepcycles -n <namespace>

Develop

This project aims to follow the Kubernetes Operator pattern. It uses Controllers which provides a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster.

Controller

Modifying the API definitions

If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

make generate
make manifests

then install the CRDs in the cluster with:

make install

[!TIP] You can debug the controller in the IDE of your choice by hooking to the main.go or you can start the controller without debugging with:

make run

[!TIP] Run make --help for more information on all potential make targets More information can be found via the Kubebuilder Documentation

Build

You always need to build a new docker container and push it to your repository:

make docker-buildx 

[!IMPORTANT] In this case you will need to adjust your Helm chart values to use your repository and container image.

Runner

Build

Adjust the image and container registry name in Makefile:

# runners
RUNNERS_IMG_NAME ?= rekuberate-io-sleepcycles-runners
RUNNERS_IMG ?= $(DOCKER_HUB_NAME)/$(RUNNERS_IMG_NAME)

and then build and push the new image to the registry:

make docker-buildx-runner

[!IMPORTANT] In this case you will need to adjust the runnerImage of your SleepCycle manifest to use your own Runner image, otherwise it defaults always to akyriako78/rekuberate-io-sleepcycles-runners

Uninstall CRDs

To delete the CRDs from the cluster:

make uninstall