ocagent

README

Exporting Metrics with OpenCensus and Prometheus

This tutorial provides a minimum example to verify that metrics can be exported to OpenCensus from Go tools.

Setting up the OpenCensus Agent

1. Follow the instructions for setting up the OpenCensus Service. You will need to be able to run the agent.
2. Create a file named config.yaml with the following contents:

exporters:
  prometheus:
    namespace: "promdemo"
    address: "localhost:8888"
    const_labels: {
        "vendor": "gotools"
    }
receivers:
  opencensus:
    address: "localhost:55678"

1. Run the OpenCensus Agent with the configuration file. The following command assumes that you are running from binary:

bin/ocagent_linux --config config.yaml

1. If you see output similar to the following, the OpenCensus Agent is now running:

{"level":"info","ts":1574381470.1922305,"caller":"config/config.go:497","msg":"Metrics Exporter enabled","exporter":"prometheus"}
2019/11/21 18:11:11 Running OpenCensus Trace and Metrics receivers as a gRPC service at "localhost:55678"
2019/11/21 18:11:11 Running zPages on port 55679

1. You can check the status of the agent using zPages at http://localhost:55679/debug/tracez.

Setting up Prometheus

1. Follow the instructions for setting up Prometheus.
2. Create a file named prom.yaml with the following contents:

scrape_configs:
  - job_name: 'agent1'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:8888']

1. Run Prometheus with the new configuration file. The following command assumes that you are running from pre-compiled binaries:

./prometheus --config.file=prom.yaml

1. If you see output similar to the following, Prometheus is now running:

level=info ts=2019-11-22T00:27:13.772Z caller=main.go:626 msg="Server is ready to receive web requests."

1. You can now access the Prometheus UI at http://localhost:9090.
2. Check to make sure Prometheus is able to scrape metrics from OpenCensus at http://localhost:9090/targets. If the state for the endpoint http://localhost:8888/metrics is not UP, make sure the OpenCensus agent is running. If you are running Prometheus using Docker, you may have to add --net="host" to your run command so that localhost resolves correctly.

Exporting Metrics

1. Clone the tools subrepository.
2. Inside internal, create a file named main.go with the following contents:

package main

import (
	"context"
	"fmt"
	"math/rand"
	"net/http"
	"time"

	"github.com/anz-bank/sysl/pkg/lspimpl/lspframework/telemetry/export"
	"github.com/anz-bank/sysl/pkg/lspimpl/lspframework/telemetry/export/ocagent"
	"github.com/anz-bank/sysl/pkg/lspimpl/lspframework/telemetry/metric"
	"github.com/anz-bank/sysl/internal/telemetry/stats"
)

func main() {

	exporter := ocagent.Connect(&ocagent.Config{
		Start:   time.Now(),
		Address: "http://127.0.0.1:55678",
		Service: "go-tools-test",
		Rate:    5 * time.Second,
		Client:  &http.Client{},
	})
	export.SetExporter(exporter)

	ctx := context.TODO()
	mLatency := stats.Float64("latency", "the latency in milliseconds", "ms")
	distribution := metric.HistogramFloat64Data{
		Info: &metric.HistogramFloat64{
			Name:        "latencyDistribution",
			Description: "the various latencies",
			Buckets:     []float64{0, 10, 50, 100, 200, 400, 800, 1000, 1400, 2000, 5000, 10000, 15000},
		},
	}

	distribution.Info.Record(mLatency)

	for {
		sleep := randomSleep()
		time.Sleep(time.Duration(sleep) * time.Millisecond)
		mLatency.Record(ctx, float64(sleep))

		fmt.Println("Latency: ", float64(sleep))
	}
}

func randomSleep() int64 {
	var max int64
	switch modulus := time.Now().Unix() % 5; modulus {
	case 0:
		max = 17001
	case 1:
		max = 8007
	case 2:
		max = 917
	case 3:
		max = 87
	case 4:
		max = 1173
	}
	return rand.Int63n(max)
}

1. Run the new file from within the tools repository:

go run internal/main.go

1. After about 5 seconds, OpenCensus should start receiving your new metrics, which you can see at http://localhost:8888/metrics. This page will look similar to the following:

# HELP promdemo_latencyDistribution the various latencies
# TYPE promdemo_latencyDistribution histogram
promdemo_latencyDistribution_bucket{vendor="otc",le="0"} 0
promdemo_latencyDistribution_bucket{vendor="otc",le="10"} 2
promdemo_latencyDistribution_bucket{vendor="otc",le="50"} 9
promdemo_latencyDistribution_bucket{vendor="otc",le="100"} 22
promdemo_latencyDistribution_bucket{vendor="otc",le="200"} 35
promdemo_latencyDistribution_bucket{vendor="otc",le="400"} 49
promdemo_latencyDistribution_bucket{vendor="otc",le="800"} 63
promdemo_latencyDistribution_bucket{vendor="otc",le="1000"} 78
promdemo_latencyDistribution_bucket{vendor="otc",le="1400"} 93
promdemo_latencyDistribution_bucket{vendor="otc",le="2000"} 108
promdemo_latencyDistribution_bucket{vendor="otc",le="5000"} 123
promdemo_latencyDistribution_bucket{vendor="otc",le="10000"} 138
promdemo_latencyDistribution_bucket{vendor="otc",le="15000"} 153
promdemo_latencyDistribution_bucket{vendor="otc",le="+Inf"} 15
promdemo_latencyDistribution_sum{vendor="otc"} 1641
promdemo_latencyDistribution_count{vendor="otc"} 15

1. After a few more seconds, Prometheus should start displaying your new metrics. You can view the distribution at http://localhost:9090/graph?g0.range_input=5m&g0.stacked=1&g0.expr=rate(promdemo_latencyDistribution_bucket%5B5m%5D)&g0.tab=0.

Documentation

Overview

Package ocagent adds the ability to export all telemetry to an ocagent. This keeps the compile time dependencies to zero and allows the agent to have the exporters needed for telemetry aggregation and viewing systems.

Index

• type Config
  • func Discover() *Config
• type Exporter
  • func Connect(config *Config) *Exporter
  • func (e *Exporter) Flush()
  • func (e *Exporter) Metric(ctx context.Context, data telemetry.MetricData)
  • func (e *Exporter) ProcessEvent(ctx context.Context, event telemetry.Event) context.Context

Types

type Config

type Config struct {
	Start   time.Time
	Host    string
	Process uint32
	Client  *http.Client
	Service string
	Address string
	Rate    time.Duration
}

func Discover

func Discover() *Config

Discover finds the local agent to export to, it will return nil if there is not one running. TODO: Actually implement a discovery protocol rather than a hard coded address

type Exporter

type Exporter struct {
	// contains filtered or unexported fields
}

func Connect

func Connect(config *Config) *Exporter

Connect creates a process specific exporter with the specified serviceName and the address of the ocagent to which it will upload its telemetry.

func (*Exporter) Flush

func (e *Exporter) Flush()

func (*Exporter) Metric

func (e *Exporter) Metric(ctx context.Context, data telemetry.MetricData)

func (*Exporter) ProcessEvent

func (e *Exporter) ProcessEvent(ctx context.Context, event telemetry.Event) context.Context