k8s-audit-metrics
K8s-audit-metrics is a service that processes Kubernetes apiserver's audit logs and exposes metrics from it.
Tips & tricks
Kubernetes client user-agent
In order to have nicer labels and easier way to distinct different clients, it's good to configure appropriate user-agent header to your k8s client.
Client-go rest.Config has a field UserAgent
that is useful to set to <component>/<version>
.
Example (from azure-operator):
restConfig.UserAgent = fmt.Sprintf("%s/%s", project.Name(), project.Version())
Prometheus queries
authorization failures
Each entry in the audit log has information about authorization status and we expose that information in the metrics - authorization_decision
tells you whether or not a request was authorized and authorization_decision_reason
tells you why. The following query gives you the count of all requests that got forbidden:
count({authorization_decision="forbid"})
request duration
k8s_api_audit_request_duration_nanoseconds
gives you information about request duration and potential latencies.
req/min per component
Grouping metrics by user-agent and computing rate of requests gives a metric for req/min e.g. as follows:
sum by (user_agent) (rate(k8s_api_audit_requests_total[5m])*60)
Prerequisites
Getting Project
Download the latest release: https://github.com/giantswarm/k8s-audit-metrics/releases/latest
Clone the git repository: https://github.com/giantswarm/k8s-audit-metrics.git
Download the latest docker image from here: https://hub.docker.com/r/giantswarm/k8s-audit-metrics/
How to build
Building the standard way
go build
Contributing & Reporting Bugs
See CONTRIBUTING.md for details on submitting patches, the contribution workflow as well as reporting bugs.
For security issues, please see the security policy.
License
PROJECT is under the Apache 2.0 license. See the LICENSE file for details.