ctrace

README

ctrace-go

Canonical OpenTracing for Go

Why

OpenTracing is a young specification and for most (if not all) SDK implementations, output format and wire protocol are specific to the backend platform implementation. ctrace attempts to decouple the format and wire protocol from the backend tracer implementation.

What

ctrace specifies a canonical format for trace logs. By default the logs are output to stdout but you can configure them to go to any WritableStream.

Required Reading

To fully understand this platform API, it's helpful to be familiar with the OpenTracing project project, terminology, and ctrace specification more specifically.

Install

Install via glide as follows:

$ glide get github.com/Nordstrom/ctrace-go

Usage

Add instrumentation to the operations you want to track. Most of this is done by middleware.

Initialization

First import ctrace.

import (
	ctrace "github.com/Nordstrom/ctrace-go"
)

Instrumentation

The primary way to use ctrace-go is to hook in one or more of the auto-instrumentation decorators. These are the decorators we currently support.

• TracedHttpHandler - this wraps the default serve mux to trace Incoming HTTP Requests.
• TracedHttpClientTransport - this wraps the http.Transport for tracing Outgoing HTTP Requests.
• TracedAPIGwLambdaProxyHandler - this wraps AWS Gateway Lambda Proxy handler using AWS Lambda Go Shim to trace requests coming into an AWS Lambda from an API Gateway proxy event.

TracedHttpHandler

To automatically instrument incoming HTTP Requests use the TracedHttpHandler.

import (
	"net/http"

	ctrace "github.com/Nordstrom/ctrace-go"
)

func ok(w http.ResponseWriter, r *http.Request) {
	region := r.URL.Query().Get("region")
	msg := hello(r.Context(), region)
	w.Write([]byte(msg))
}

func main() {
	http.HandleFunc("/ok", ok)
	http.ListenAndServe(":8004", ctrace.TracedHTTPHandler(http.DefaultServeMux))
}

TracedHTTPClientTransport

To automatically instrument outgoing HTTP Requests use the TracedHttpClientTransport.

import (
	"context"
	"net/http"

	ctrace "github.com/Nordstrom/ctrace-go"
)

var httpClient = &http.Client{
	Transport: ctrace.TracedHTTPClientTransport(&http.Transport{}),
}

func send(ctx context.Context, method string, url string, body io.Reader) (*http.Response, error) {
	req, err := http.NewRequest(method, url, body)
	if err != nil {
		return nil, err
	}
	return httpClient.Do(req.WithContext(ctx))
}

TracedAPIGwLambdaProxyHandler

To automatically instrument incoming API Gateway Lambda proxy requests use TracedAPIGwLambdaProxyHandler.

import (
	"github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
	"github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/apigatewayproxyevt"
	ctrace "github.com/Nordstrom/ctrace-go"
)

handler := func(
	ctx context.Context,
	evt *apigatewayproxyevt.Event,
	lambdaCtx *runtime.Context,
) (interface{}, error) {
  // ...
}

var TracedHandler = ctrace.TracedAPIGwLambdaProxyHandler(handler)

Log Events

To log events within the context of the current Span, use span.LogFields.

import (
	ctrace "github.com/Nordstrom/ctrace-go"
	log "github.com/Nordstrom/ctrace-go/log"
)

func hello(ctx context.Context, region string) string {
	msg := fmt.Sprintf("Hello %v!", region)
	ctrace.LogInfo(ctx, "generate-msg", log.Message(message))
	return msg
}

Advanced Usage

If middleware does not fully meet your needs, you can manually instrument spans operations of interest and adding log statements to capture useful data relevant to those operations.

func main() {
	ctrace.Init(TracerOptions{
		MultiEvent: true,
		Writer:     fileWriter,
	})
}

Creating a Span given an existing Go context.Context

If you use context.Context in your application, OpenTracing's Go library will happily rely on it for Span propagation. To start a new (blocking child) Span, you can use StartSpanFromContext.

func xyz(ctx context.Context, ...) {
    // ...
    span, ctx := opentracing.StartSpanFromContext(ctx, "operation_name")
    defer span.Finish()
    span.LogFields(
        log.String("event", "soft error"),
        log.String("type", "cache timeout"),
        log.Int("waited.millis", 1500))
    // ...
}

Starting an empty trace by creating a "root span"

It's always possible to create a "root" Span with no parent or other causal reference.

func xyz() {
   // ...
   sp := opentracing.StartSpan("operation_name")
   defer sp.Finish()
   // ...
}

Best Practices

The following are recommended practices for using opentracing and ctrace-go within a GoLang project.

Context Propagation

Require a context.Context argument as the first parameter of every(a) API call

func (h handler) HandleRequest(ctx context.Context, r MyRequest) error {
  // ...
}

Rule of thumb: because contexts are always request-scopeed, never hold a reference to them on a struct. Always pass as a function parameter.

a. Obviously not every function in your codebase. You'll get a feel for the balance when you start writing context-aware code.

Custom Context Types

At some point, you will be tempted to invent your own "custom" context type. For example to provide convenience methods, since Value() takes an interface{} key and returns an interface{}.

You will regret it. Use extractor functions instead.

Contributing

Please see the contributing guide for details on contributing to ctrace-go.

License

Apache License 2.0

Documentation

Index

• Variables
• func ContextWithSpan(ctx context.Context, span opentracing.Span) context.Context
• func Global() core.Tracer
• func Init(opts TracerOptions) core.Tracer
• func LogErrorMessage(ctx context.Context, message string, fields ...log.Field)
• func LogErrorObject(ctx context.Context, e error, fields ...log.Field)
• func LogInfo(ctx context.Context, event string, fields ...log.Field)
• func SpanFromContext(ctx context.Context) opentracing.Span
• func StartSpanFromContext(ctx context.Context, operationName string, opts ...opentracing.StartSpanOption) (opentracing.Span, context.Context)
• func TracedHTTPClientTransport(t http.RoundTripper, interceptor ...TracedHTTPInterceptor) http.RoundTripper
• func TracedHTTPHandler(h http.Handler, interceptor ...TracedHTTPInterceptor) http.Handler
• func TracedHTTPHandlerFunc(fn func(http.ResponseWriter, *http.Request), ...) http.HandlerFunc
• type LambdaFunction
  • func TracedAPIGwLambdaProxyHandler(fn TracedLambdaFunction, interceptor ...LambdaFunctionInterceptor) LambdaFunction
• type LambdaFunctionInterceptor
• type SpanConfig
  • func ConfigSpan(operationName string, ignored *regexp.Regexp, ...) SpanConfig
• type TracedHTTPInterceptor
• type TracedLambdaFunction
• type TracerOptions

Variables

var ChildOf = opentracing.ChildOf

ChildOf returns a StartSpanOption pointing to a dependent parent span. If sc == nil, the option has no effect.

See ChildOfRef, SpanReference

Functions

func ContextWithSpan

func ContextWithSpan(ctx context.Context, span opentracing.Span) context.Context

ContextWithSpan returns a new `context.Context` that holds a reference to `span`'s SpanContext.

func Global

func Global() core.Tracer

Global returns the global Tracer

func Init

func Init(opts TracerOptions) core.Tracer

Init initializes the global Tracer returned by Global().

func LogErrorMessage

func LogErrorMessage(ctx context.Context, message string, fields ...log.Field)

LogErrorMessage allows the logging of an Error with a Message based on the current context.Context. If a running span does not exist on the current context, nothing is logged.

func LogErrorObject

func LogErrorObject(ctx context.Context, e error, fields ...log.Field)

LogErrorObject allows the logging of an Error Object based on the current context.Context. If a running span does not exist on the current context, nothing is logged.

func LogInfo

func LogInfo(ctx context.Context, event string, fields ...log.Field)

LogInfo allows the logging of an Info Event based on the current context.Context. If a running span does not exist on the current context, nothing is logged.

func SpanFromContext

func SpanFromContext(ctx context.Context) opentracing.Span

SpanFromContext returns the `Span` previously associated with `ctx`, or `nil` if no such `Span` could be found.

NOTE: context.Context != SpanContext: the former is Go's intra-process context propagation mechanism, and the latter houses OpenTracing's per-Span identity and baggage information.

func StartSpanFromContext

func StartSpanFromContext(ctx context.Context, operationName string, opts ...opentracing.StartSpanOption) (opentracing.Span, context.Context)

StartSpanFromContext starts and returns a Span with `operationName`, using any Span found within `ctx` as a ChildOfRef. If no such parent could be found, StartSpanFromContext creates a root (parentless) Span.

The second return value is a context.Context object built around the returned Span.

Example usage:

SomeFunction(ctx context.Context, ...) {
    sp, ctx := opentracing.StartSpanFromContext(ctx, "SomeFunction")
    defer sp.Finish()
    ...
}

func TracedHTTPClientTransport

func TracedHTTPClientTransport(
	t http.RoundTripper,
	interceptor ...TracedHTTPInterceptor,
) http.RoundTripper

TracedHTTPClientTransport creates a new Transporter (http.RoundTripper) that intercepts and traces egress requests.

func TracedHTTPHandler

func TracedHTTPHandler(
	h http.Handler,
	interceptor ...TracedHTTPInterceptor,
) http.Handler

TracedHTTPHandler returns a http.Handler that is traced as an opentracing.Span

func TracedHTTPHandlerFunc

func TracedHTTPHandlerFunc(
	fn func(http.ResponseWriter, *http.Request),
	interceptor ...TracedHTTPInterceptor,
) http.HandlerFunc

TracedHTTPHandlerFunc returns a http.HandlerFunc that is traced as an opentracing.Span

Types

type LambdaFunction

type LambdaFunction func(
	evt *apigatewayproxyevt.Event,
	lambdaCtx *runtime.Context,
) (interface{}, error)

LambdaFunction is the defined function for Lambda handlers

func TracedAPIGwLambdaProxyHandler

func TracedAPIGwLambdaProxyHandler(
	fn TracedLambdaFunction,
	interceptor ...LambdaFunctionInterceptor,
) LambdaFunction

TracedAPIGwLambdaProxyHandler is a decorator (wrapper) that wraps the Lambda handler function for tracing. It handles starting a span when the handler is called and finishing it upon completion. To customize the OperationName or Tags pass in a LambdaFunctionInterceptor

type LambdaFunctionInterceptor

type LambdaFunctionInterceptor func(evt *apigatewayproxyevt.Event, ctx *runtime.Context) SpanConfig

LambdaFunctionInterceptor is the defined function for intercepting the TracedApiGwLambdaProxyHandler calls for providing custom OperationName and/or custom Tags

type SpanConfig

type SpanConfig struct {
	// OperationName is the custom operation name decided by interceptor
	OperationName string

	// IgnoredPaths are URL paths ignored by tracer (ex.. /health to be ignored as it is used by a load balancer)
	Ignored *regexp.Regexp

	// Tags are the custom start span options decided by interceptor.
	Tags []opentracing.StartSpanOption
}

SpanConfig is used by middleware interceptors to return custom OperationName and Tags

func ConfigSpan

func ConfigSpan(
	operationName string,
	ignored *regexp.Regexp,
	tags ...opentracing.StartSpanOption,
) SpanConfig

ConfigSpan function is used by middleware interceptors to construct a SpanConfig for customizing the starting of a span. For example:

ctrace.TracedHttpHandler(
  http.DefaultServeMux,
  func (r *http.Request) SpanConfig {
    return ConfigSpan(
      "MyOperation:" + r.URL.String(),
      ext.String("MyTag", "MyValue")
    )
  },
)

type TracedHTTPInterceptor

type TracedHTTPInterceptor func(r *http.Request) SpanConfig

TracedHTTPInterceptor is a defined function called during HTTP tracing to return custom OperationName and/or Tags for the given request

type TracedLambdaFunction

type TracedLambdaFunction func(
	ctx context.Context,
	evt *apigatewayproxyevt.Event,
	lambdaCtx *runtime.Context,
) (interface{}, error)

TracedLambdaFunction is the defined function for traced Lambda handlers it adds context.Context as the first parameter.

type TracerOptions

type TracerOptions core.TracerOptions

TracerOptions allows creating a customized Tracer via NewWithOptions. The object must not be updated when there is an active tracer using it. This is an alias of core.TracerOptions.