metrics

README

Stats Reporter

This code provides the ability to register with OpenCensus the Views required to have the various metrics associated with eventing-kafka exported to whatever backend is configured (Prometheus by default). The METRICS_DOMAIN and METRICS_PORT environment variables will be used by Knative to produce unique metrics names and start the metrics service when InitializeObservability is called.

Note that this will serve only to expose the metrics on the specified port. The creation of the K8S Service and any external monitoring is left up to the individual component to provide.

Metrics Endpoint

Assuming the use of the default Prometheus backend and port, you may manually test the exposed /metrics endpoint by running a Kubernetes port forward to the service.

kubectl port-forward svc/<service> -n <namespace> 8081:8081

...and point your browser at http://localhost:8081/metrics, or you can use telepresence and curl...

telepresence
curl http://<service>.<namespace>.svc.cluster.local:8081/metrics

Prometheus Alerts

The following are sample Prometheus alerts based on the distributed KafkaChannel metrics and are provided only as a starting point for creating your own alerts.

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  labels:
    prometheus: kube-prometheus
    role: alert-rules
  name: eventing-kafka-channel.rules
spec:
  groups:
  - name: eventing-kafka-channel.rules
    rules:
    - alert: EventingKafkaReceiverBrokerLatencyWarning
      annotations:
        summary: "Receiver Kafka broker request latency is excessively high."
        description: "{{`Kafka broker request latency ({{ $value }}ms) has exceeded 200ms for more than 5 minutes.`}}"
      expr: eventing_kafka_request_latency_in_ms{job="eventing-kafka-channels", percentile="95%"} > 200
      for: 5m
      labels:
        severity: warning
    - alert: EventingKafkaDispatcherBrokerLatencyWarning
      annotations:
        summary: "Dispatcher Kafka broker request latency is excessively high."
        description: "{{`Kafka broker request latency ({{ $value }}ms) has exceeded 400ms for more than 5 minutes.`}}"
      expr: eventing_kafka_request_latency_in_ms{job="eventing-kafka-dispatchers", percentile="95%"} > 400
      for: 5m
      labels:
        severity: warning

Documentation

Index

• type Reporter
  • func (r *Reporter) Read() []*metricdata.Metric
  • func (r *Reporter) Report(list ReportingList)
  • func (r *Reporter) Shutdown()
• type ReportingItem
• type ReportingList
• type StatsReporter
  • func NewStatsReporter(log *zap.Logger) StatsReporter

Types

type Reporter

type Reporter struct {
	// contains filtered or unexported fields
}

Define StatsReporter Structure, which implements the OpenCensus Producer interface

func (*Reporter) Read

func (r *Reporter) Read() []*metricdata.Metric

Read implements the OpenCensus Producer interface

func (*Reporter) Report

func (r *Reporter) Report(list ReportingList)

Report The Sarama Metrics (go-metrics) Via Knative / OpenCensus Metrics

func (*Reporter) Shutdown

func (r *Reporter) Shutdown()

Remove this producer from the global manager's list so that it will no longer call Read()

type ReportingItem

type ReportingItem = map[string]interface{}

Some type aliases for the otherwise unwieldy metric collection map-of-maps-to-interfaces

type ReportingList

type ReportingList = map[string]ReportingItem

type StatsReporter

type StatsReporter interface {
	Report(ReportingList)
	Shutdown()
}

StatsReporter defines the interface for sending ingress metrics.

func NewStatsReporter

func NewStatsReporter(log *zap.Logger) StatsReporter

StatsReporter Constructor