kubernetes

README

Kubernetes Mutating Webhook

To consume Berglas secrets from Kubernetes, you can deploy a MutatingWebhookConfiguration. This mutation will inspect all pods admitted into the cluster and adjust their manifest to pull secrets from Berglas.

The webhook must be deployed and accessible by the Kubernetes API server. You can deploy the webhook via knative, Cloud Run, Cloud Functions, Heroku, Cloud Foundry, etc, so long as it offers a TLS endpoint accessible by the Kubernetes API server.

Deployment

One of the easiest ways to deploy the mutation webhook is using Cloud Run. To deploy on Cloud Run:

1. Set environment variables (replace with your values):

   export PROJECT_ID=my-project
   

2. Enable the Cloud Run API (this only needs to be done once per project):

   gcloud services enable --project ${PROJECT_ID} \
     artifactregistry.googleapis.com \
     cloudbuild.googleapis.com \
     run.googleapis.com
   

3. Ensure you are in the examples/kubernetes/ directory!

   cd examples/kubernetes
   

4. Deploy the mutating webhook:

   gcloud run deploy berglas-webhook \
     --project ${PROJECT_ID} \
     --region us-central1 \
     --allow-unauthenticated \
     --source .
   

   Note: This service does not require authentication. It does not need permission to access secrets. The function purely mutates YAML configurations before sending them to the scheduler.

   You may be prompted to create an Artifact Registry repository. Say "Yes".

5. Extract the Cloud Run URL:

   ENDPOINT=$(gcloud run services describe berglas-webhook --project ${PROJECT_ID} --region us-central1 --format 'value(status.url)')
   

6. Register the webhook with this URL:

   sed "s|REPLACE_WITH_YOUR_URL|$ENDPOINT|" deploy/webhook.yaml | kubectl apply -f -
   

7. (Optional) Verify the webhook is running:

   kubectl get mutatingwebhookconfiguration
   
   NAME
   berglas-webhook
   

Setup and Usage

Either the pods or the Kubernetes cluster needs to be able to authenticate to Google Cloud using IAM with Default Application Credentials. See the authentication section of the main README for more details.

On Google Cloud, it is strongly recommended that you use Workload Identity or have a dedicated service account for the GKE cluster. For example:

1. Create a service account:

   gcloud iam service-accounts create berglas-accessor \
     --project ${PROJECT_ID} \
     --display-name "Berglas secret accessor account"
   

   export SA_EMAIL=berglas-accessor@${PROJECT_ID}.iam.gserviceaccount.com
   

2. Grant the Google Cloud service account permissions to access the secrets you require. For example, to grant access to the secret named "my-secret":

   Using Secret Manager storage:

   berglas grant sm://${PROJECT_ID}/my-secret --member serviceAccount:$SA_EMAIL
   

   Using Cloud Storage storage:

   berglas grant ${BUCKET_ID}/my-secret --member serviceAccount:$SA_EMAIL
   

3. Create the GKE cluster:

   gcloud container clusters create berglas-k8s-test \
     --project ${PROJECT_ID} \
     --region us-east1 \
     --num-nodes 1 \
     --machine-type n1-standard-2 \
     --no-issue-client-certificate \
     --no-enable-basic-auth \
     --enable-autoupgrade \
     --scopes cloud-platform \
     --workload-pool ${PROJECT_ID}.svc.id.goog \
     --metadata disable-legacy-endpoints=true
   

Note: If using existing cluster, remember that in order to use Workload Identity, node pools have to be updated or created new ones (after enabling Workload Identity on whole cluster). Otherwise, berglas Service Account will not be bound to pods.

1. Create a Kubernetes service account:

   kubectl create serviceaccount "envserver"
   

2. Grant the Kubernetes service account permissions to act as the Google Cloud service account:

   gcloud iam service-accounts add-iam-policy-binding \
     --project ${PROJECT_ID} \
     --role "roles/iam.workloadIdentityUser" \
     --member "serviceAccount:${PROJECT_ID}.svc.id.goog[default/envserver]" \
     berglas-accessor@${PROJECT_ID}.iam.gserviceaccount.com
   

3. Annotate the Kubernetes service account with the name of the Google Cloud service account:

   kubectl annotate serviceaccount "envserver" \
     iam.gke.io/gcp-service-account=berglas-accessor@${PROJECT_ID}.iam.gserviceaccount.com
   

4. Update deploy/sample.yaml to refer to your secret. If you're using Secret Manager storage, use the sm:// prefix. If you're using the Cloud Storage storage, use the berglas:// prefix. See more in the Berglas [reference syntax][reference-syntax].

5. Deploy it:

   kubectl apply -f deploy/sample.yaml
   

Limitations

• The mutator requires that containers specify a command in their manifest. If a container requests Berglas secrets and does not specify a command, the mutator will log an error and not mutate the spec.

• Berglas requires CA certificates to authenticate outbound TLS connections. The target container must have CA certificates in a well-known path.