trace

README

Trace package

Trace is a golang package that implements a distributed tracing capability inspired by Google's Dapper. Traces may be optionally recorded to a database backend or local file. HTTP middleware is provided to facilitate easy tracing of requests and cascading of trace contexts during request fanout scenarios.

To install the package:

$ go get github.com/SpirentOrion/trace

Tracing Data Model

Each trace is logically comprised of one or more spans in a tree-like structure. You are free to determine the granularity of traces and spans. Typical usages include:

• Each request received at a web API endpoint starts a new trace.
• Each scheduled background task starts a new trace.
• Each span represents some type of start-to-finish activity. By creating new spans you can differentiate between different types or stages of activity within a single trace.

Traces are identified by a probabilistically unique 64-bit integer. All spans within in a trace share the same trace id. Identifiers are randomly generated within this number space and thus do not require use of a centralized allocator.

Spans are also identified by their own unique 64-bit integer values. Each span records its trace id, the id of its parent span, its location, its start time, its finish time, and other data that you may provide.

With this structure it is possible to build a causal record of activity within trace. For any trace, activity began with the first span -- the span with a parent id of 0. Any spans within the same trace that have a parent id matching the first span's id were caused by the first span. And so on.

If processing of a span requires fanout to other services or processes the trace context may be propagated using HTTP request headers or other appropriate mechanisms. Each span's location indicates where the activity actually occurred, in terms of both process and hostname. When activity spans multiple hosts, start and finish times are based on the host's clock and aren't necessarily synchronized.

Recording Backends

Currently, a single recording backend (JSON) is provided. Recording to other formats or backends (e.g. databases) may be performed by types implementing the Recorder interface:

type Recorder interface {
    Record(s *Span) error
}

Example

A simple example is provided to demonstrate trace recording via the JSON recorder:

$ cd $GOPATH/src/gopkg.in/SpirentOrion/trace.v2/example
$ go run main.go

Separately:

$ curl -i http://127.0.0.1:8000/foo/bar
$ cat example.json

Documentation

Index

• func Annotate(ctx context.Context) map[string]interface{}
• func CurrentSpanID(ctx context.Context) (spanID int64)
• func CurrentTraceID(ctx context.Context) (traceID int64)
• func Do(ctx context.Context, kind string, name string, act func(context.Context))
• func GenerateID(ctx context.Context) (int64, error)
• func Join(ctx context.Context, traceCtx context.Context) (context.Context, error)
• func Record(ctx context.Context, rec Recorder) (context.Context, error)
• func WithBuffer(ctx context.Context, buffer int) context.Context
• func WithContext(ctx context.Context, traceCtx context.Context) context.Context
• func WithLogger(ctx context.Context, logger Logger) context.Context
• func WithParentID(ctx context.Context, parentID int64) context.Context
• func WithProcess(ctx context.Context, process string) context.Context
• func WithTraceID(ctx context.Context, traceID int64) context.Context
• type Handler
  • func NewHandler(ctx context.Context) *Handler
  • func (h *Handler) ServeHTTP(rw http.ResponseWriter, req *http.Request, next http.HandlerFunc)
• type Logger
• type Recorder
  • func NewJSONRecorder(writer io.Writer) Recorder
• type Span

Functions

func Annotate

func Annotate(ctx context.Context) map[string]interface{}

Annotate returns a map that can be used to store trace span-specific data if a trace is active in ctx. Otherwise it returns nil.

func CurrentSpanID

func CurrentSpanID(ctx context.Context) (spanID int64)

CurrentSpanID returns the current span id if a trace is active in the context. Otherwise it returns 0.

func CurrentTraceID

func CurrentTraceID(ctx context.Context) (traceID int64)

CurrentTraceID returns the current trace id if a trace is active in the context. Otherwise it returns 0.

func Do

func Do(ctx context.Context, kind string, name string, act func(context.Context))

Do starts a new trace span if recording is active in the context. If a trace is already active in ctx then the new trace span continues under the existing trace id, otherwise a new trace id is generated.

The activity function act is always invoked, either with a new context representing a new trace span, or with the caller-provided ctx if recording is not active or an error occurs.

func GenerateID

func GenerateID(ctx context.Context) (int64, error)

GenerateId returns a probablistically unique 64-bit id if a trace is active in ctx. All id values returned by this function will be positive integers. This may be useful for callers that want to generate their own id values.

func Join

func Join(ctx context.Context, traceCtx context.Context) (context.Context, error)

Join creates a new context that joins trace recording already in progress. If the original trace recording context is canceled then recording stops in all joined contexts. Cancellation of joined contexts has no effect on recording.

func Record

func Record(ctx context.Context, rec Recorder) (context.Context, error)

Record starts trace recording in a context. If the context is canceled then trace recording stops.

func WithBuffer

func WithBuffer(ctx context.Context, buffer int) context.Context

WithBuffer returns a new context.Context instance with a buffer depth included as a value. This buffer value is used when allocating internal channels for trace span recording.

func WithContext

func WithContext(ctx context.Context, traceCtx context.Context) context.Context

WithContext returns a new context.Context instance merging trace and parent id values from a context.Context where tracing is active.

func WithLogger

func WithLogger(ctx context.Context, logger Logger) context.Context

WithLogger returns a new context.Context instance with a Logger as a value. This logger is used to log errors that occur during tracing.

func WithParentID

func WithParentID(ctx context.Context, parentID int64) context.Context

WithParentID returns a new context.Context instance with a parent id included as a value. This may be used to override parent id determination when new trace spans are created.

func WithProcess

func WithProcess(ctx context.Context, process string) context.Context

WithProcess returns a new context.Context instance with a process identifier included as a value. This process identifier is used in all trace spans created from the new context.

func WithTraceID

func WithTraceID(ctx context.Context, traceID int64) context.Context

WithTraceID returns a new context.Context instance with a trace id included as a value. This may be used to override automatic trace id generation when new trace spans are created.

Types

type Handler

type Handler struct {
	// Kind is the kind value used when starting new traces.
	Kind string

	// HeaderKey is the key used when ServeHTTP inserts an id header in requests or responses.
	HeaderKey string

	// HonorReqHeader determines whether or not ServeHTTP honors id headers in requests.
	HonorReqHeader bool

	// AddRespHeader determines whether or not ServeHTTP adds id headers to responses.
	AddRespHeader bool
	// contains filtered or unexported fields
}

func NewHandler

func NewHandler(ctx context.Context) *Handler

NewHandler creates a middleware handler that facilitates HTTP request tracing.

If the request contains an id header and HonorReqHeader is true, then the id values are used (allowing trace contexts to span services). Otherwise a new trace id is generated. An id header is optionally added to the response.

func (*Handler) ServeHTTP

func (h *Handler) ServeHTTP(rw http.ResponseWriter, req *http.Request, next http.HandlerFunc)

type Logger

type Logger interface {
	Println(v ...interface{})
}

Logger is an interface compatible with log.Logger.

type Recorder

type Recorder interface {
	Record(s *Span) error
}

Recorder persists a Span to an external file or datastore.

func NewJSONRecorder

func NewJSONRecorder(writer io.Writer) Recorder

type Span

type Span struct {
	SpanID    int64                  `json:"span_id,string" yaml:"span_id"`
	TraceID   int64                  `json:"trace_id,string" yaml:"trace_id"`
	ParentID  int64                  `json:"parent_id,string" yaml:"parent_id"`
	Process   string                 `json:"process,omitempty" yaml:",omitempty"`
	Kind      string                 `json:"kind,omitempty" yaml:",omitempty"`
	Name      string                 `json:"name,omitempty" yaml:",omitempty"`
	Start     time.Time              `json:"-" yaml:"-"`
	StartStr  string                 `json:"start,omitempty" yaml:"start,omitempty"`
	Finish    time.Time              `json:"-" yaml:"-"`
	FinishStr string                 `json:"finish,omitempty" yaml:"finish,omitempty"`
	Data      map[string]interface{} `json:"data,omitempty" yaml:",omitempty,inline"`
}

Span tracks an activity within a trace. Note: YAML struct tags are included for backwards-compatibility with V1 code.