sinks

README

Sinks

Veneur is all about sending observability primitives on to other places.

• Blackhole
• Datadog
• Kafka
• LightStep
• SignalFx
• SSFMetrics

Looking For Something Else?

We love new sinks! You learn more about contributing and send us a PR for a new sink!

Documentation

Index

• Constants
• func IsAcceptableMetric(metric samplers.InterMetric, sink MetricSink) bool
• type MetricSink
• type SpanSink

Constants

const EventReportedCount = "sink.events_reported_total"

EventReportedCount number of events processed by a sink. Tagged with `sink:sink.Name()`.

const MetricKeyMetricFlushDuration = "sink.metric_flush_total_duration_ns"

MetricKeyMetricFlushDuration should be emitted as a timer by a MetricSink if possible. Tagged with `sink:sink.Name()`. The `Flush` function is a great place to do this.

const MetricKeySpanFlushDuration = "sink.span_flush_total_duration_ns"

MetricKeySpanFlushDuration should be emitted as a timer by a SpanSink if possible. Tagged with `sink:sink.Name()`. The `Flush` function is a great place to do this. If your sync does async sends, this might not be necessary.

const MetricKeySpanIngestDuration = "sink.span_ingest_total_duration_ns"

const MetricKeyTotalMetricsFlushed = "sink.metrics_flushed_total"

MetricKeyTotalMetricsFlushed should be emitted as a counter by a MetricSink if possible. Tagged with `sink:sink.Name()`. The `Flush` function is a great place to do this.

const MetricKeyTotalMetricsSkipped = "sink.metrics_skipped_total"

MetricKeyTotalMetricsSkipped should be emitted as a counter by a MetricSink if possible. Tagged with `sink:sink.Name()`. Track the number of metrics skipped, not applicable to this MetricSink.

const MetricKeyTotalSpansDropped = "sink.spans_dropped_total"

MetricKeyTotalSpansDropped tracks the number of spans that the sink is aware it has dropped. It should be emitted as a counter by a SpanSink if possible. Tagged with `sink:sink.Name()`. The `Flush` function is a great place to do this.

const MetricKeyTotalSpansFlushed = "sink.spans_flushed_total"

MetricKeyTotalSpansFlushed should be emitted as a counter by a SpanSink if possible. Tagged with `sink:sink.Name()`. The `Flush` function is a great place to do this.

Functions

func IsAcceptableMetric

func IsAcceptableMetric(metric samplers.InterMetric, sink MetricSink) bool

IsAcceptableMetric returns true if a metric is meant to be ingested by a given sink.

Types

type MetricSink

type MetricSink interface {
	Name() string
	// Start finishes setting up the sink and starts any
	// background processing tasks that the sink might have to run
	// in the background. It's invoked when the server starts.
	Start(traceClient *trace.Client) error
	// Flush receives `InterMetric`s from Veneur and is
	// responsible for "sinking" these metrics to whatever it's
	// backend wants. Note that the sink must **not** mutate the
	// incoming metrics as they are shared with other sinks. Sinks
	// must also check each metric with IsAcceptableMetric to
	// verify they are eligible to consume the metric.
	Flush(context.Context, []samplers.InterMetric) error
	// Handle non-metric, non-span samples.
	FlushOtherSamples(ctx context.Context, samples []ssf.SSFSample)
}

MetricSink is a receiver of `InterMetric`s when Veneur periodically flushes it's aggregated metrics.

type SpanSink

type SpanSink interface {
	// Start finishes setting up the sink and starts any
	// background processing tasks that the sink might have to run
	// in the background. It's invoked when the server starts.
	Start(*trace.Client) error

	// Name returns the span sink's name for debugging purposes
	Name() string

	// Flush receives `SSFSpan`s from Veneur **as they arrive**. If the sink wants
	// to buffer spans it may do so and defer sending until `Flush` is called.
	Ingest(*ssf.SSFSpan) error

	// Invoked at the same interval as metric flushes, this can be used as a
	// signal for the sink to write out if it was buffering or something.
	Flush()
}

SpanSink is a receiver of spans that handles sending those spans to some downstream sink. Calls to `Ingest(span)` are meant to give the sink control of the span, with periodic calls to flush as a signal for sinks that don't handle their own flushing in a separate goroutine, etc. Note that SpanSinks differ from MetricSinks because Veneur does *not* aggregate Spans.