New Relic Kubernetes Operator
Overview
The newrelic-kubernetes-operator is a Kubernetes Operator that facilitates management of New Relic resources from within your Kubernetes configuration. Managing New Relic resources via custom Kubernetes objects can be done the same way you manage built-in Kubernetes objects.
Currently the operator supports managing the following resources:
- Alert Policies
- NRQL Alert Conditions.
- Alert Conditions for APM, Browser and mobile
- Alert Channels
Quick Start
Note: These quick start instructions do not require you to clone the repo.
Running Kubernetes in a Docker container locally with kind
-
Install docker, kubectl, kustomize, and kind
brew cask install docker
brew install kubernetes-cli kustomize kind
-
Create a test cluster with kind
kind create cluster --name newrelic
kubectl cluster-info
-
Install cert-manager
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.15.0/cert-manager.yaml
Note: This takes a minute or two to finish so wait a minute before going on to the next step.
You can also confirm it's running with the command kubectl rollout status deployment -n cert-manager cert-manager-webhook
-
Install the operator in the test cluster.
kustomize build https://github.com/newrelic/newrelic-kubernetes-operator/config/default | kubectl apply -f -
Note: This will install operator on whatever kubernetes cluster kubectl is configured to use.
Using a custom container
If you want to deploy the operator in a custom container you can override the image name with a kustomize.yaml
file.
-
Create a new kustomize.yaml
file
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: newrelic-kubernetes-operator-system
resources:
- github.com/newrelic/newrelic-kubernetes-operator/config/default
images:
- name: newrelic/kubernetes-operator:snapshot
newName: <CUSTOM_IMAGE>
newTag: <CUSTOM_TAG>
-
The apply the file with:
kustomize build . | kubectl apply -f -
Provision New Relic resources with the operator
Once you've completed the Quick Start section, you can start provisioning New Relic resources with our New Relic Kubernetes objects.
Create a New Relic alert policy with NRQL alert conditions
-
We'll be using the following example policy configuration file. You will need to update the api_key
field with your New Relic personal API key.
Once you've added your API key, we can apply it your local cluster.
kubectl apply -f examples/example_policy.yaml
Note: You can also use a Kubernetes secret for providing your API key. We've provided an example secret configuration file in case you want to use this method. You'll need to replace api_key
with api_key_secret
.
-
See your configured policies with the following command.
kubectl describe alertspolicies.nr.k8s.newrelic.com
Note: You should also see the newly created policy within your New Relic account.
The operator will create and update alert policies and NRQL alert conditions as needed by applying your configuration files with kubectl apply -f <filename>
Create a NRQL alert condition and add it to an existing alert policy
- We'll be using the following example NRQL alert condition configuration file. You will need to update the
api_key
field with your New Relic personal API key.
Create an Alerts Channel
-
We'll be using the following example alerts channel configuration file. You will need to update the api_key
field with your New Relic personal API key.
Note: The New Relic Alerts API does not allow updating Alerts Channels. In order to change a channel, you will need to either rename the k8s AlertsChannel object to create a new one and delete the old one or manually delete the k8s AlertsChannel object and create a new one.
Monitoring the New Relic Operator
The New Relic Operator uses the New Relic Go Agent to report monitoring statistics.
This can be activated by adding a Kubernetes Secret and an option ConfigMap to configure the agent.
We'll be using Example_New_Relic_Agent_Config
apiVersion: v1
kind: Secret
metadata:
name: nr-agent
namespace: newrelic-kubernetes-operator-system
type: Opaque
stringData:
license-key: <your New Relic license key, base64 encoded>
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nr-agent
namespace: newrelic-kubernetes-operator-system
data:
# defaults to New Relic Kubernetes Operator
app-name: <your desired New Relic App name for the Operator>
# Defaults to collector.newrelic.com
host: <New Relic endpoint>
# Defaults to false
log-level: debug
Note: If the agent isn't reporting, make sure to check your base64 encoding didn't include a /n
character.
Uninstall the operator
The Operator can be removed with the reverse of installation, namely building the kubernetes resource files with kustomize
and running kubectl delete
kustomize build github.com/newrelic/newrelic-kubernetes-operator/config/default | kubectl delete -f -
Development
This section should get you set up properly for doing development on the operator.
Requirements
Code
-
Clone the repo
git clone git@github.com:newrelic/newrelic-kubernetes-operator.git
-
Install kubebuilder following the instructions for your operating system. This installation will also get etcd
and kube-apiserver
which are needed for the tests.
Note: Do not install kubebuilder
with brew
. Homebrew's kubebuilder
package will not provide all the necessary dependencies for running the tests.
-
Run the test suite, which uses the Ginkgo testing framework. Using the make
targets is the quickest way to get started with testing.
-
Perform the steps from the Quick Start section, except use the following instead of step 4 to install with the :snapshot image:
kustomize build config/development | kubectl apply -f -
-
Confirm your configuration was deployed to your local Kubernetes cluster (the one that we created with kind
).
-
Personalize your development alert policy
Edit examples/development/personal*
with your name and API key.
Next, create your customized alert policy:
kustomize build examples | kubectl apply -f -
To delete your customized policy:
kustomize build examples | kubectl delete -f -
Helpful commands
# Describe the currently configured policies.
kubectl describe alertspolicies.nr.k8s.newrelic.com
# Describe the currently configured alert conditions.
kubectl describe alertsnrqlconditions.nr.k8s.newrelic.com
# Get the node being used for the newrelic operator.
kubectl get nodes -n newrelic-kubernetes-operator-system
# Describe the node being used for the newrelic operator.
kubectl describe node <your-node-name>
# Tail logs of the operator's manager container (useful during development).
# Use the `describe node` command above to locate your manager controller.
kubectl logs -f -n newrelic-kubernetes-operator-system -c manager newrelic-kubernetes-operator-controller-manager-<hash from>