spancheck

README

go-spancheck

Checks usage of OpenTelemetry spans from go.opentelemetry.io/otel/trace.

Installation & Usage

go install github.com/jjti/go-spancheck/cmd/spancheck@latest
spancheck ./...

Example

func _() error {
    // span.End is not called on all paths, possible memory leak
    // span.SetStatus is not called on all paths
    // span.RecordError is not called on all paths
    _, span := otel.Tracer("foo").Start(context.Background(), "bar")

    if true {
        // return can be reached without calling span.End
        // return can be reached without calling span.SetStatus
        // return can be reached without calling span.RecordError
        return errors.New("err")
    }

    return nil // return can be reached without calling span.End
}

Configuration

Only the span.End() check is enabled by default. The others can be enabled with -enable-all, -enable-record-error-check, or -enable-set-status-check.

$ spancheck -h
...
Flags:
  -disable-end-check
        disable the check for calling span.End() after span creation
  -enable-all
        enable all checks, overriding individual check flags
  -enable-record-error-check
        enable check for a span.RecordError(err) call when returning an error
  -enable-set-status-check
        enable check for a span.SetStatus(codes.Error, msg) call when returning an error
  -ignore-record-error-check-signatures string
        comma-separated list of function signature regex that disable the span.RecordError(err) check on errors
  -ignore-set-status-check-signatures string
        comma-separated list of function signature regex that disable the span.SetStatus(codes.Error, msg) check on errors

Ignore check signatures

The span.SetStatus() and span.RecordError() checks warn when there is:

1. a path to return statement
2. that returns an error
3. without a call (to SetStatus or RecordError, respectively)

But it's convenient to call SetStatus and RecordError from utility methods [1]. To support that, the ignore-*-check-signatures settings will suppress warnings if the configured function is present in the path.

For example, by default, the code below would have warnings as shown:

func task(ctx context.Context) error {
    ctx, span := otel.Tracer("foo").Start(ctx, "bar") // span.SetStatus is not called on all paths
    defer span.End()

    if err := subTask(ctx); err != nil {
        return recordErr(span, err) // return can be reached without calling span.SetStatus
    }

    return nil

func recordErr(span trace.Span, err error) error {
	span.SetStatus(codes.Error, err.Error())
	span.RecordError(err)
	return err
}

The warnings are can be ignored by setting -ignore-set-status-check-signatures flag to recordErr:

spancheck -enable-set-status-check -ignore-set-status-check-signatures 'recordErr' ./...

Problem Statement

Tracing is a celebrated [1,2] and well marketed [3,4] pillar of observability. But self-instrumented traces requires a lot of easy-to-forget boilerplate:

import (
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/codes"
)

func task(ctx context.Context) error {
    ctx, span := otel.Tracer("foo").Start(ctx, "bar")
    defer span.End() // call `.End()`

    if err := subTask(ctx); err != nil {
        span.SetStatus(codes.Error, err.Error()) // call SetStatus(codes.Error, msg) to set status:error
        span.RecordError(err) // call RecordError(err) to record an error event
        return err
    }

    return nil
}

For spans to be really useful, developers need to:

1. call span.End()
2. call span.SetStatus(codes.Error, msg) on error
3. call span.RecordError(err) on error
4. call span.SetAttributes() liberally

OpenTelemetry docs: Creating spans

Uptrace tutorial: OpenTelemetry Go Tracing API

Checks

This linter supports three checks, each documented below. Only the check for span.End() is enabled by default. See Configuration for instructions on enabling the others.

span.End() Check

Enabled by default. Disable with -disable-end-check.

Not calling End can cause memory leaks and prevents spans from being closed.

Any Span that is created MUST also be ended. This is the responsibility of the user. Implementations of this API may leak memory or other resources if Spans are not ended.

source: trace.go

func task(ctx context.Context) error {
    otel.Tracer("app").Start(ctx, "foo") // span is unassigned, probable memory leak
    _, span := otel.Tracer().Start(ctx, "foo") // span.End is not called on all paths, possible memory leak
    return nil // return can be reached without calling span.End
}

span.SetStatus(codes.Error, "msg") Check

Disabled by default. Enable with -enable-set-status-check.

Developers should call SetStatus on spans. The status attribute is an important, first-class attribute:

1. observability platforms and APMs differentiate "success" vs "failure" using span's status codes.
2. telemetry collector agents, like the Open Telemetry Collector's Tail Sampling Processor, are configurable to sample Error spans at a higher rate than OK spans.
3. observability platforms, like DataDog, have trace retention filters that use spans' status. In other words, status:error spans often receive special treatment with the assumption they are more useful for debugging. And forgetting to set the status can lead to spans, with useful debugging information, being dropped.

func _() error {
    _, span := otel.Tracer("foo").Start(context.Background(), "bar") // span.SetStatus is not called on all paths
    defer span.End()

    if err := subTask(); err != nil {
        span.RecordError(err)
        return errors.New(err) // return can be reached without calling span.SetStatus
    }

    return nil
}

OpenTelemetry docs: Set span status.

span.RecordError(err) Check

Disabled by default. Enable with -enable-record-error-check.

Calling RecordError creates a new exception-type event (structured log message) on the span. This is recommended to capture the error's stack trace.

func _() error {
    _, span := otel.Tracer("foo").Start(context.Background(), "bar") // span.RecordError is not called on all paths
    defer span.End()

    if err := subTask(); err != nil {
        span.SetStatus(codes.Error, err.Error())
        return errors.New(err) // return can be reached without calling span.RecordError
    }

    return nil
}

OpenTelemetry docs: Record errors.

Attribution

This linter is the product of liberal copying of:

• github.com/golang/tools/go/analysis/passes/lostcancel (half the linter)
• github.com/tomarrell/wrapcheck (error type checking and config)
• github.com/Antonboom/testifylint (README)
• github.com/ghostiam/protogetter (test setup)

Documentation

Overview

Package spancheck defines a linter that checks for mistakes with OTEL trace spans.

Analyzer spancheck

spancheck: check for mistakes with OpenTelemetry trace spans.

Common mistakes with OTEL trace spans include forgetting to call End:

func(ctx context.Context) {
	ctx, span := otel.Tracer("app").Start(ctx, "span")
	// defer span.End() should be here

	// do stuff
}

Forgetting to set an Error status:

ctx, span := otel.Tracer("app").Start(ctx, "span")
defer span.End()

if err := task(); err != nil {
	// span.SetStatus(codes.Error, err.Error()) should be here
	span.RecordError(err)
	return fmt.Errorf("failed to run task: %w", err)
}

Forgetting to record the Error:

ctx, span := otel.Tracer("app").Start(ctx, "span")
defer span.End()

if err := task(); err != nil {
	span.SetStatus(codes.Error, err.Error())
	// span.RecordError(err) should be here
	return fmt.Errorf("failed to run task: %w", err)
}

Index

• func NewAnalyzer() *analysis.Analyzer
• func NewAnalyzerWithConfig(config *Config) *analysis.Analyzer
• type Config
  • func NewConfigFromFlags() *Config

Functions

func NewAnalyzer

func NewAnalyzer() *analysis.Analyzer

NewAnalyzer returns a new analyzer that checks for mistakes with OTEL trace spans. Its config is sourced from flags.

func NewAnalyzerWithConfig

func NewAnalyzerWithConfig(config *Config) *analysis.Analyzer

NewAnalyzerWithConfig returns a new analyzer configured with the Config passed in. Its config can be set for testing.

Types

type Config

type Config struct {

	// EnableAll enables all checks and takes precedence over other fields like
	// DisableEndCheck. Ignore*CheckSignatures still apply.
	EnableAll bool

	// DisableEndCheck enables the check for calling span.End().
	DisableEndCheck bool

	// EnableSetStatusCheck enables the check for calling span.SetStatus.
	EnableSetStatusCheck bool

	// IgnoreSetStatusCheckSignaturesSlice is a slice of strings that are turned into
	// the IgnoreSetStatusCheckSignatures regex.
	IgnoreSetStatusCheckSignaturesSlice []string

	// EnableRecordErrorCheck enables the check for calling span.RecordError.
	// By default, this check is disabled.
	EnableRecordErrorCheck bool

	// IgnoreRecordErrorCheckSignaturesSlice is a slice of strings that are turned into
	// the IgnoreRecordErrorCheckSignatures regex.
	IgnoreRecordErrorCheckSignaturesSlice []string
	// contains filtered or unexported fields
}

Config is a configuration for the spancheck analyzer.

func NewConfigFromFlags

func NewConfigFromFlags() *Config

NewConfigFromFlags returns a new Config with default values and flags for CLI usage.