spancheck

README

go-spancheck

Checks usage of OpenTelemetry spans from go.opentelemetry.io/otel/trace.

Installation & Usage

go install github.com/jjti/go-spancheck/cmd/spancheck@latest
spancheck ./...

Configuration

Only the span.End() check is enabled by default. The others can be enabled with -enable-all, -enable-record-error-check, or -enable-set-status-check.

$ spancheck -h
...
Flags:
  -disable-end-check
        disable the check for calling span.End() after span creation
  -enable-all
        enable all checks, overriding individual check flags
  -enable-record-error-check
        enable check for a span.RecordError(err) call when returning an error
  -enable-set-status-check
        enable check for a span.SetStatus(codes.Error, msg) call when returning an error
  -ignore-record-error-check-signatures string
        comma-separated list of function signature regex that disable the span.RecordError(err) check on errors
  -ignore-set-status-check-signatures string
        comma-separated list of function signature regex that disable the span.SetStatus(codes.Error, msg) check on errors

Ignore check signatures

The span.SetStatus() and span.RecordError() checks warn when there is a path to return statement, with an error, without a call (to SetStatus, or RecordError, respectively). But it's convenient to set spans' status and record errors from utility methods [1]. To support that, the ignore-*-check-signatures settings can be used to ignore paths to return statements if that signature is present.

For example, by default, the code below would have the warning shown:

func task(ctx context.Context) error {
    ctx, span := otel.Tracer("foo").Start(ctx, "bar") // span.SetStatus is not called on all paths
    defer span.End()

    if err := subTask(ctx); err != nil {
        return recordErr(span, err) // return can be reached without calling span.SetStatus
    }

    return nil

func recordErr(span trace.Span, err error) error {
	span.SetStatus(codes.Error, err.Error())
	span.RecordError(err)
	return err
}

Using the -ignore-set-status-check-signatures flag, the error above can be suppressed:

spancheck -enable-set-status-check -ignore-set-status-check-signatures 'recordErr' ./...

Background

Tracing is a celebrated [1,2] and well marketed [3,4] pillar of observability. But self-instrumented traces requires a lot of easy-to-forget boilerplate:

import (
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/codes"
)

func task(ctx context.Context) error {
    ctx, span := otel.Tracer("foo").Start(ctx, "bar")
    defer span.End() // call `.End()`

    if err := subTask(ctx); err != nil {
        span.SetStatus(codes.Error, err.Error()) // call SetStatus(codes.Error, msg) to set status:error
        span.RecordError(err) // call RecordError(err) to record an error event
        return err
    }

    return nil
}

For spans to be really useful, developers need to:

1. call span.End()
2. call span.SetStatus(codes.Error, msg) on error
3. call span.RecordError(err) on error
4. call span.SetAttributes() liberally

OpenTelemetry docs: Creating spans

Uptrace tutorial: OpenTelemetry Go Tracing API

Checks

This linter supports three checks, each documented below. Only the check for span.End() is enabled by default. See Configuration for instructions on enabling the others.

span.End() Check

Not calling End can cause memory leaks and prevents spans from being closed.

Any Span that is created MUST also be ended. This is the responsibility of the user. Implementations of this API may leak memory or other resources if Spans are not ended.

source: trace.go

func task(ctx context.Context) error {
    otel.Tracer("app").Start(ctx, "foo") // span is unassigned, probable memory leak
    _, span := otel.Tracer().Start(ctx, "foo") // span.End is not called on all paths, possible memory leak
    return nil // return can be reached without calling span.End
}

span.SetStatus(codes.Error, "msg") Check

Developers should call SetStatus on spans. The status attribute is an important, first-class attribute:

1. observability platforms and APMs differentiate "success" vs "failure" using span's status codes.
2. telemetry collector agents, like the Open Telemetry Collector's Tail Sampling Processor, are configurable to sample Error spans at a higher rate than OK spans.
3. observability platforms, like DataDog, have trace retention filters that use spans' status. In other words, status:error spans often receive special treatment with the assumption they are more useful for debugging. And forgetting to set the status can lead to spans, with useful debugging information, being dropped.

func _() error {
    _, span := otel.Tracer("foo").Start(context.Background(), "bar") // span.SetStatus is not called on all paths
    defer span.End()

    if err := subTask(); err != nil {
        span.RecordError(err)
        return errors.New(err) // return can be reached without calling span.SetStatus
    }

    return nil
}

OpenTelemetry docs: Set span status.

span.RecordError(err) Check

Calling RecordError creates a new exception-type event (structured log message) on the span. This is recommended to capture the error's stack trace.

func _() error {
    _, span := otel.Tracer("foo").Start(context.Background(), "bar") // span.RecordError is not called on all paths
    defer span.End()

    if err := subTask(); err != nil {
        span.SetStatus(codes.Error, err.Error())
        return errors.New(err) // return can be reached without calling span.RecordError
    }

    return nil
}

OpenTelemetry docs: Record errors.

Attribution

This linter is the result of liberal copying of:

• github.com/golang/tools/go/analysis/passes/lostcancel (half the linter)
• github.com/tomarrell/wrapcheck (error type checking and config)
• github.com/Antonboom/testifylint (README)
• github.com/ghostiam/protogetter (test setup)

Documentation

Overview

Package spancheck defines a linter that checks for mistakes with OTEL trace spans.

Analyzer spancheck

spancheck: check for mistakes with OpenTelemetry trace spans.

Common mistakes with OTEL trace spans include forgetting to call End:

func(ctx context.Context) {
	ctx, span := otel.Tracer("app").Start(ctx, "span")
	// defer span.End() should be here

	// do stuff
}

Forgetting to set an Error status:

ctx, span := otel.Tracer("app").Start(ctx, "span")
defer span.End()

if err := task(); err != nil {
	// span.SetStatus(codes.Error, err.Error()) should be here
	span.RecordError(err)
	return fmt.Errorf("failed to run task: %w", err)
}

Forgetting to record the Error:

ctx, span := otel.Tracer("app").Start(ctx, "span")
defer span.End()

if err := task(); err != nil {
	span.SetStatus(codes.Error, err.Error())
	// span.RecordError(err) should be here
	return fmt.Errorf("failed to run task: %w", err)
}

Index

• func NewAnalyzer() *analysis.Analyzer
• func NewAnalyzerWithConfig(config *Config) *analysis.Analyzer
• type Config
  • func NewConfig() *Config

Functions

func NewAnalyzer

func NewAnalyzer() *analysis.Analyzer

NewAnalyzer returns a new analyzer that checks for mistakes with OTEL trace spans. Its config is sourced from flags.

func NewAnalyzerWithConfig

func NewAnalyzerWithConfig(config *Config) *analysis.Analyzer

NewAnalyzerWithConfig returns a new analyzer configured with the Config passed in. Its config can be set for testing.

Types

type Config

type Config struct {

	// EnableAll enables all checks and takes precedence over other fields like
	// DisableEndCheck. Ignore*CheckSignatures still apply.
	EnableAll bool

	// DisableEndCheck enables the check for calling span.End().
	DisableEndCheck bool

	// EnableSetStatusCheck enables the check for calling span.SetStatus.
	EnableSetStatusCheck bool

	// IgnoreSetStatusCheckSignatures is a regex that, if matched, disables the
	// SetStatus check for a particular error.
	IgnoreSetStatusCheckSignatures *regexp.Regexp

	// EnableRecordErrorCheck enables the check for calling span.RecordError.
	// By default, this check is disabled.
	EnableRecordErrorCheck bool

	// IgnoreRecordErrorCheckSignatures is a regex that, if matched, disables the
	// RecordError check for a particular error.
	IgnoreRecordErrorCheckSignatures *regexp.Regexp
	// contains filtered or unexported fields
}

Config is a configuration for the spancheck analyzer.

func NewConfig

func NewConfig() *Config

NewConfig returns a new Config with default values and flags for cli usage.