Orkestra

README

Orkestra

Orkestra is a cloud-native release orchestration platform that allows you to manage the lifecycle and orchestrate the release of groups of Kubernetes Helm applications through Kubernetes Custom Resource Objects. Orkestra works by generating dependency driven DAG workflows to orchestrate the deployment and upgrade of multiple applications within a Kubernetes cluster. Additionally, Orkestra can also orchestrate the deployment of multiple microservice (helm dependencies - sub-charts) within a parent Helm chart.

Overview

What is it?

Orkestra renders a DAG based workflow for deploying applications to a Kubernetes cluster by leveraging popular and mature open-source frameworks like Argo (Workflows), Flux Helm Operator and Chart-museum

What problems does it solve?

Complex applications oftentimes require intelligent release orchestration and lifecycle management which is not provided by Helm itself.

Take, for example, Continuous Deployment of mission-critical applications - like 5G core Network Functions or NFs

• Network Functions are applications that rely on a rich ecosystem of infrastructure and PaaS (platform-as-a-service) components to be present on the cluster before they can be deployed. This establishes a hard dependency between the applications and the infra/paas applications.

Getting Started

For getting started you will need,

• A Kubernetes cluster
• kubectl - Kubernetes client
• helm - Helm client
• kubebuilder - https://book.kubebuilder.io/quick-start.html#installation
• controller-gen - GO111MODULE=on go get -v -u sigs.k8s.io/controller-tools/cmd/controller-gen@v0.5.0 (this should be run from outside the Orkestra repo to be installed to your $GOBIN)
• (optional) argo - Argo workflow client (follow the instructions to install the binary at https://github.com/argoproj/argo/releases)

Install the ApplicationGroup and custom resource definitions (CRDs)

make install

/home/nitishm/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/applicationgroups.orkestra.azure.microsoft.com configured

Alternatively, you can use the kustomize directly to install the CRDs using kubectl by issuing the following command - kubectl -k config/bases

Using helm

Install the orkestra controller and supporting services like, Argo Workflow, Flux Helm-operator and Chartmuseum using the provided helm chart

helm install orkestra chart/orkestra/  --namespace orkestra --create-namespace  

You should see resources spin up in the orkestra namespace as shown below,

NAME                                                          NAMESPACE  AGE
configmap/orkestra-helm-operator-kube-config                  orkestra   4s     
configmap/orkestra-workflow-controller-configmap              orkestra   4s     
endpoints/orkestra-argo-server                                orkestra   4s     
endpoints/orkestra-chartmuseum                                orkestra   4s     
endpoints/orkestra-helm-operator                              orkestra   4s     
pod/orkestra-544949cdf8-htct9                                 orkestra   4s     
pod/orkestra-argo-server-597455875f-qdbhb                     orkestra   4s     
pod/orkestra-chartmuseum-7f78f54f5-vf8lh                      orkestra   4s     
pod/orkestra-helm-operator-cc9b5776b-xmn6d                    orkestra   4s     
pod/orkestra-workflow-controller-688bc7677-5t5tn              orkestra   4s     
secret/argo-token-mt9gw                                       orkestra   4s     
secret/default-token-2jpsq                                    orkestra   9h     
secret/flux-helm-repositories                                 orkestra   4s     
secret/orkestra-chartmuseum                                   orkestra   4s     
secret/orkestra-helm-operator-git-deploy                      orkestra   4s     
secret/orkestra-token-x5gqx                                   orkestra   4s     
secret/sh.helm.release.v1.orkestra.v1                         orkestra   4s     
serviceaccount/argo                                           orkestra   4s     
serviceaccount/default                                        orkestra   9h     
serviceaccount/orkestra                                       orkestra   4s     
service/orkestra-argo-server                                  orkestra   4s     
service/orkestra-chartmuseum                                  orkestra   4s     
service/orkestra-helm-operator                                orkestra   4s     
deployment.apps/orkestra                                      orkestra   4s     
deployment.apps/orkestra-argo-server                          orkestra   4s     
deployment.apps/orkestra-chartmuseum                          orkestra   4s     
deployment.apps/orkestra-helm-operator                        orkestra   4s     
deployment.apps/orkestra-workflow-controller                  orkestra   4s     
replicaset.apps/orkestra-544949cdf8                           orkestra   4s     
replicaset.apps/orkestra-argo-server-597455875f               orkestra   4s     
replicaset.apps/orkestra-chartmuseum-7f78f54f5                orkestra   4s     
replicaset.apps/orkestra-helm-operator-cc9b5776b              orkestra   4s     
replicaset.apps/orkestra-workflow-controller-688bc7677        orkestra   4s     
endpointslice.discovery.k8s.io/orkestra-argo-server-qfk6p     orkestra   4s     
endpointslice.discovery.k8s.io/orkestra-chartmuseum-x8dqq     orkestra   4s     
endpointslice.discovery.k8s.io/orkestra-helm-operator-w88tw   orkestra   4s   

(optional) Argo Workflow Dashboard

The following command should open a browser window to the Argo Workflow Dashboard (the command port-forwards the Argo server to http://localhost:2476).

argo server --browser

INFO[2021-02-03T01:02:15.839Z]                                               authModes="[server]" baseHRef=/ managedNamespace= namespace=orkestra secure=false
WARN[2021-02-03T01:02:15.840Z] You are running in insecure mode. Learn how to enable transport layer security: https://argoproj.github.io/argo/tls/
WARN[2021-02-03T01:02:15.840Z] You are running without client authentication. Learn how to enable client authentication: https://argoproj.github.io/argo/argo-server-auth-mode/
INFO[2021-02-03T01:02:15.840Z] config map                                    name=workflow-controller-configmap
INFO[2021-02-03T01:02:15.840Z] SSO disabled
INFO[2021-02-03T01:02:15.851Z] Starting Argo Server                          instanceID= version=v2.12.6
INFO[2021-02-03T01:02:15.851Z] Creating event controller                     operationQueueSize=16 workerCount=4
INFO[2021-02-03T01:02:15.852Z] Argo Server started successfully on http://localhost:2746
INFO[2021-02-03T01:02:15.852Z] Argo UI is available at http://localhost:2746

How it works

To solve the complex application orchestration problem Orkestra builds a Directed Acyclic Graph using the application, and it's dependencies and submits it to Argo Workflow. The Workflow nodes use workflow-executor nodes to deploy a HelmRelease object into the cluster. This HelmRelease object signals Flux's HelmOperator to perform a "Helm Action" on the referenced chart.

1. Submit ApplicationGroup CRs
2. For each application in ApplicationGroup download Helm chart from “primary” Helm Registry
3. (*optional) For each dependency in the Application chart, if subcharts found in charts/ directory, push to ”staging” Helm Registry (Chart-museum).
4. Generate and submit Argo Workflow DAG
5. (Executor nodes only) Submit and probe deployment state of HelmRelease CR (.Status.Phase)
6. Fetch and deploy Helm charts referred to by each HelmRelease CR to the Kubernetes cluster.

Features

• Built for Kubernetes - custom controller built using the kubebuilder project
• Easy to use - familiar declarative spec using Kubernetes Custom Resource Definitions
• Dependency management - DAG-based workflows for groups of application charts and their sub-charts using Argo Workflow
• Works with any Continous Deployment system - bring your own CD to deploy Orkestra Custom Resources. Works with any Kubernetes compatible Continuous Deployment framework like FluxCD and ArgoCD.
• Built for GitOps - describe your desired set of applications (and dependencies) declaratively and manage them from a version-controlled git repository.

Examples

Try out the examples

Roadmap

Features

• Rollback ApplicationGroup to previous version on failure by re-deploying last-applied workflow. (MVP)
• Switch from helm-operator to helm-controller (MVP #65)
• Leverage helm-controller's remediation and rollback features for remediation on failures/errors. (MVP #144)
• Support multiple remediation strategies on failure
• Bring your own executor template for Argo Workflow leaf node HelmRelease deployment (allows a user to do custom checks before deeming the leaf node successful)

Development

Follow the development docs

Contributing

For instructions about setting up your environment to develop and extend the operator, please see contributing.md

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Reporting security issues and security bugs

For instructions on reporting security issues and bugs, please see security.md