otelslog

README

otelslog

otelslog is a Go package that seamlessly integrates structured logging (slog) with OpenTelemetry tracing. It enriches your observability stack by automatically correlating logs with distributed traces, providing deep insights into your application's behavior. The package maintains the simplicity of slog while adding powerful tracing capabilities that help you understand the flow of operations across your distributed system.

中文版

Key Features

otelslog enhances your application's observability through several powerful features:

• Automatic Trace Context Integration

  • Seamless injection of trace and span IDs into log records
  • Built-in context propagation across service boundaries
  • Support for nested spans with proper parent-child relationships

• Flexible Configuration

  • Customizable field names for trace and span IDs
  • Configurable minimum log level for trace creation
  • Optional span event recording
  • Support for mandatory spans that bypass log level filtering

• Rich Structured Data Support

  • Complete support for nested attribute groups in both logs and traces
  • Type-aware attribute conversion between slog and OpenTelemetry formats
  • Preservation of attribute hierarchies in distributed traces

• Operational Excellence

  • Thread-safe design for concurrent use
  • Memory-efficient attribute handling

Installation

To add otelslog to your project, use Go modules:

go get github.com/yakumioto/otelslog

Quick Start

Here's a minimal example to get you started with otelslog:

package main

import (
    "context"
    "log/slog"
    "os"
    
    "github.com/yakumioto/otelslog"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"
)

func main() {
    // Initialize a basic tracer for demonstration
    tp := trace.NewTracerProvider()
    otel.SetTracerProvider(tp)
    defer tp.Shutdown(context.Background())
    
    // Set up the default logger with otelslog handler
    slog.SetDefault(slog.New(
        otelslog.NewHandler(slog.NewJSONHandler(os.Stdout, nil)),
    ))

    // Create a span and include it in logging
    span := otelslog.NewSpanContext("service", "process-request")
    slog.Info("handling request", 
        "operation", span,
        slog.Group("user",
            slog.String("id", "123"),
            slog.String("role", "admin"),
        ),
    )
    defer span.End()
}

Advanced Usage

Configuring the Handler

Customize the handler's behavior using functional options:

slog.SetDefault(slog.New(
    otelslog.NewHandler(
        slog.NewJSONHandler(os.Stdout, nil),
        otelslog.WithTraceIDKey("trace_id"),     // Custom trace ID field
        otelslog.WithSpanIDKey("span_id"),       // Custom span ID field
        otelslog.WithTraceLevel(slog.LevelDebug), // Set minimum trace level
        otelslog.WithNoSpanEvents(),             // Disable span events
    ),
))

Context Propagation and Nested Spans

Track operations across your application with proper context propagation:

// Create a root span
span1 := otelslog.NewSpanContext("service", "parent-operation")
slog.Info("starting parent operation", 
    "operation", span1,
    "request_id", "req-123",
)

// Create a child span with context
span2Ctx := otelslog.NewSpanContextWithContext(span1, "service", "child-operation")
slog.InfoContext(span2Ctx, "processing sub-operation",
    slog.Group("metrics",
        slog.Int("items_processed", 42),
        slog.Duration("processing_time", time.Second),
    ),
)

defer span2Ctx.Done()
defer span1.End()

Mandatory Spans and Critical Operations

Ensure critical operations are always traced regardless of log level:

span := otelslog.NewMustSpanContext("service", "critical-operation")
slog.Info("processing critical request",
    "operation", span,
    slog.Group("transaction",
        slog.String("id", "tx-789"),
        slog.Float64("amount", 1299.99),
    ),
)
defer span.End()

Working with Structured Data

Organize your logging data using slog's powerful grouping features:

span := otelslog.NewSpanContext("service", "user-management")
slog.Default().WithGroup("request").Info("updating user profile",
    "operation", span,
    slog.Group("user",
        slog.String("id", "user-456"),
        slog.Group("changes",
            slog.String("email", "new@example.com"),
            slog.String("role", "admin"),
        ),
    ),
)
defer span.End()

Integration with OpenTelemetry

Set up a complete tracing pipeline with OTLP export:

func initTracer(ctx context.Context) (func(), error) {
    exporter, err := otlptrace.New(
        ctx,
        otlptracegrpc.NewClient(
            otlptracegrpc.WithEndpoint("localhost:4317"),
            otlptracegrpc.WithInsecure(),
        ),
    )
    if err != nil {
        return nil, err
    }

    tp := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(exporter),
        sdktrace.WithResource(resource.NewWithAttributes(
            semconv.ServiceName("your-service"),
            semconv.ServiceVersion("1.0.0"),
        )),
        sdktrace.WithSampler(sdktrace.AlwaysSample()),
    )
    otel.SetTracerProvider(tp)

    return func() { tp.Shutdown(ctx) }, nil
}

Best Practices

To maximize the benefits of otelslog in your application:

• Design spans to reflect your application's logical operations. Choose span names that clearly describe what operation is being performed, making traces easier to understand and analyze.

• Create a consistent attribute hierarchy using groups. Organize related attributes together to maintain clarity in both logs and traces, making it easier to correlate information across your observability tools.

• Use context propagation effectively. Always pass context through your application's call chain to maintain proper parent-child relationships between spans and ensure accurate distributed tracing.

• Consider performance implications. Configure appropriate trace levels and use WithNoSpanEvents when span events aren't needed to optimize performance in high-throughput scenarios.

• Handle span lifecycle properly. Always use defer for span.End() calls immediately after span creation to ensure proper cleanup and accurate duration measurements.

• Leverage mandatory spans judiciously. Use NewMustSpanContext for operations that must be traced regardless of log level, but be mindful of the additional overhead.

Acknowledgements

This project was inspired by slog-otel. We extend our gratitude to its creators and contributors for their pioneering work in combining structured logging with OpenTelemetry.

Related Projects

Explore these related projects to enhance your observability stack:

• slog-otel - Another approach to bringing OpenTelemetry to slog
• slog - Go's official structured logging package
• OpenTelemetry Go - The official OpenTelemetry SDK for Go

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Documentation

Overview

Package otelslog provides a powerful integration between Go's structured logging (slog) and OpenTelemetry tracing. This package enables developers to correlate logs and traces seamlessly, creating a comprehensive observability solution that enhances debugging and monitoring capabilities in distributed systems.

Core Concepts

The package centers around a custom slog.Handler implementation that automatically enriches log entries with distributed tracing context while maintaining the flexibility and simplicity of slog's interface. It preserves the hierarchical structure of logged attributes in both logs and trace spans, ensuring consistency across your observability data.

Key Features

Trace Context Integration:

• Automatic injection of trace and span IDs into log records
• Preservation of parent-child relationships between spans
• Support for context propagation across service boundaries

Flexible Configuration:

• Customizable trace and span ID field names
• Configurable minimum log level for trace creation
• Optional span event recording
• Support for mandatory spans that bypass log level filtering

Structured Data Support:

• Full support for slog's group functionality
• Hierarchical attribute preservation in both logs and spans
• Type-aware attribute conversion between slog and OpenTelemetry formats

Basic Usage

1. Setting up the handler with default configuration:

slog.SetDefault(slog.New(otelslog.NewHandler(slog.NewJSONHandler(os.Stdout, nil))))
slog.Info("hello, world")

2. Configuring the handler with custom options:

slog.SetDefault(slog.New(
    otelslog.NewHandler(
        slog.NewJSONHandler(os.Stdout, nil),
        otelslog.WithTraceIDKey("trace_id"),
        otelslog.WithSpanIDKey("span_id"),
        otelslog.WithTraceLevel(slog.LevelDebug),
    ),
))

Advanced Usage

1. Creating spans with context propagation:

// Create a root span
span1 := otelslog.NewSpanContext("trace", "span1")
slog.Info("processing request",
    "operation", span1,
    "key1", "value1",
)
defer span1.End()

// Create a child span
span2Ctx := otelslog.NewSpanContextWithContext(span1, "trace", "span2")
slog.InfoContext(span2Ctx, "nested operation")
defer span2Ctx.Done()

2. Working with attribute groups:

span := otelslog.NewSpanContext("trace", "span")
slog.Default().WithGroup("request").Info("processing",
    "operation", span,
    slog.Group("user",
        slog.String("id", "123"),
        slog.String("role", "admin"),
    ),
)
defer span.End()

3. Creating mandatory spans:

span := otelslog.NewMustSpanContext("trace", "critical-operation")
slog.Info("critical processing", "operation", span)
defer span.End()

Configuration Options

The handler supports several functional options for customization:

WithTraceIDKey(key string):

Customizes the field name for trace IDs in log records

WithSpanIDKey(key string):

Customizes the field name for span IDs in log records

WithSpanEventKey(key string):

Customizes the field name used when recording log entries as span events

WithTraceLevel(level slog.Level):

Sets the minimum log level at which spans are created

WithNoSpanEvents():

Disables the recording of log entries as span events

Best Practices

1. Span Management:

• Use defer for span.End() calls to ensure proper cleanup
• Create spans with meaningful names that describe the operation
• Use NewMustSpanContext for critical operations that should always be traced

2. Context Handling:

• Propagate context through your application using NewSpanContextWithContext
• Use InfoContext/ErrorContext when you have an existing context
• Maintain proper parent-child relationships between spans

3. Attribute Organization:

• Use groups to logically organize related attributes
• Maintain consistent attribute naming across your application
• Consider the hierarchical structure when designing attribute groups

4. Performance Considerations:

• Configure appropriate trace levels to control span creation
• Use WithNoSpanEvents when span events are not needed
• Consider the overhead of attribute conversion in high-throughput scenarios

Thread Safety

The handler implementation is fully thread-safe and can be safely used concurrently from multiple goroutines. All operations on spans and contexts are designed to be thread-safe as well.

Integration with OpenTelemetry

The package seamlessly integrates with OpenTelemetry's trace API and SDK. It supports:

• Standard OpenTelemetry trace exporters
• Custom trace samplers
• Resource attributes for service identification
• Context propagation across service boundaries

Example Setup with OTLP Exporter:

func initTracer(ctx context.Context) (func(), error) {
    exporter, err := otlptrace.New(
        ctx,
        otlptracegrpc.NewClient(
            otlptracegrpc.WithEndpoint("localhost:4317"),
            otlptracegrpc.WithInsecure(),
        ),
    )
    if err != nil {
        return nil, err
    }

    tp := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(exporter),
        sdktrace.WithResource(resource.NewWithAttributes(
            semconv.ServiceName("your-service"),
        )),
    )
    otel.SetTracerProvider(tp)

    return func() { tp.Shutdown(ctx) }, nil
}

The package is designed to be a comprehensive solution for correlating logs and traces in Go applications, providing the flexibility and features needed for effective observability in modern distributed systems.

Index

• type Handler
  • func NewHandler(handler slog.Handler, opts ...Options) *Handler
  • func (h *Handler) Enabled(_ context.Context, _ slog.Level) bool
  • func (h *Handler) Handle(ctx context.Context, record slog.Record) error
  • func (h *Handler) WithAttrs(attrs []slog.Attr) slog.Handler
  • func (h *Handler) WithGroup(name string) slog.Handler
• type Options
  • func WithNoSpanEvents() Options
  • func WithSpanEventKey(key string) Options
  • func WithSpanIDKey(key string) Options
  • func WithTraceIDKey(key string) Options
  • func WithTraceLevel(level slog.Level) Options
• type SpanContext
  • func NewMustSpanContext(spanName string, traceNameOpt ...string) *SpanContext
  • func NewMustSpanContextWithContext(ctx context.Context, spanName string, traceNameOpt ...string) *SpanContext
  • func NewSpanContext(spanName string, traceNameOpt ...string) *SpanContext
  • func NewSpanContextWithContext(ctx context.Context, spanName string, traceNameOpt ...string) *SpanContext
  • func (s *SpanContext) Done() <-chan struct{}
  • func (s *SpanContext) End()

Examples

• NewHandler

Types

type Handler

type Handler struct {

	// Next slog.Handler in the chain
	Next slog.Handler
	// contains filtered or unexported fields
}

Handler is responsible for managing OpenTelemetry trace context and handling slog attributes. It contains keys for trace and span IDs, controls for recording span events, and options for including baggage attributes in slog records.

func NewHandler

func NewHandler(handler slog.Handler, opts ...Options) *Handler

NewHandler creates a new slog.Handler with the given options.

Example

ExampleNewHandler shows how to use the default logger.

// /*
//  * Copyright (c) 2024 yakumioto <yaku.mioto@gmail.com>
//  * All rights reserved.
//  */

package main

import (
	"context"
	"log/slog"
	"os"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
	"go.opentelemetry.io/otel/sdk/resource"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	semconv "go.opentelemetry.io/otel/semconv/v1.24.0"

	"github.com/yakumioto/otelslog"
)

// initTracer initializes an OTLP exporter, and configures the corresponding trace provider.
func initTracer(ctx context.Context) (func(), error) {
	// Create OTLP exporter
	exporter, err := otlptrace.New(
		ctx,
		otlptracegrpc.NewClient(
			otlptracegrpc.WithEndpoint("127.0.0.1:4317"), // Your collector endpoint
			otlptracegrpc.WithInsecure(),                 // For testing only
		),
	)
	if err != nil {
		return nil, err
	}

	// Create resource with service information
	res, err := resource.New(ctx,
		resource.WithAttributes(
			semconv.ServiceName("your-service-name"),
			semconv.ServiceVersion("1.0.0"),
		),
	)
	if err != nil {
		return nil, err
	}

	// Create TracerProvider
	tp := sdktrace.NewTracerProvider(
		sdktrace.WithBatcher(exporter),
		sdktrace.WithResource(res),
		sdktrace.WithSampler(sdktrace.AlwaysSample()),
	)

	// Set global TracerProvider
	otel.SetTracerProvider(tp)

	// Return a cleanup function
	return func() {
		if err := tp.Shutdown(ctx); err != nil {
			slog.Error("Error shutting down tracer provider", "error", err)
		}
	}, nil
}

// ExampleNewHandler shows how to use the default logger.
func main() {
	// Set the default logger to use the OTEL SLOG handler with JSON output to standard output.
	slog.SetDefault(slog.New(otelslog.NewHandler(slog.NewJSONHandler(os.Stdout, nil))))
	slog.Info("hello, world")

	// Set the default logger to use the OTEL SLOG handler with JSON output to standard output.
	// And set the trace ID key to "trace_id", span ID key to "span_id", and the trace level to debug.
	slog.SetDefault(slog.New(
		otelslog.NewHandler(
			slog.NewJSONHandler(os.Stdout, nil),
			otelslog.WithTraceIDKey("trace_id"),
			otelslog.WithSpanIDKey("span_id"),
			otelslog.WithTraceLevel(slog.LevelDebug),
		),
	))

	// Initialize the tracer.
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	cleanup, err := initTracer(ctx)
	if err != nil {
		panic(err)
	}
	defer cleanup()

	// no trace log
	slog.Info("hello, world")

	// trace with slog attributes
	span1 := otelslog.NewSpanContextWithContext(ctx, "", "span1")
	slog.Info("processing request1",
		"trace1", span1,
		"key", "1",
		slog.Group("group1",
			slog.String("key1", "value1"),
			slog.String("key2", "value2"),
		),
	)
	defer span1.End()

	// trace with slog.XXXContext
	span2Ctx := otelslog.NewSpanContextWithContext(span1, "trace2", "span2")
	slog.InfoContext(span2Ctx, "processing request2",
		"key", "1",
		slog.Group("group2",
			slog.String("key1", "value1"),
			slog.String("key2", "value2"),
		),
	)
	defer span2Ctx.Done()

	// trace with slog.With
	span3Ctx := otelslog.NewSpanContextWithContext(span2Ctx, "trace3", "span3")
	slog.Default().WithGroup("group3").With("trace3", span3Ctx).Error("processing request3",
		"key", "1",
		slog.Group("group4",
			slog.String("key1", "value1"),
			slog.String("key2", "value2"),
		),
	)
	defer span3Ctx.End()
}

func (*Handler) Enabled

func (h *Handler) Enabled(_ context.Context, _ slog.Level) bool

Enabled checks if the handler is enabled for the given slog.Level.

func (*Handler) Handle

func (h *Handler) Handle(ctx context.Context, record slog.Record) error

Handle processes the slog.Record and adds OpenTelemetry attributes and events.

func (*Handler) WithAttrs

func (h *Handler) WithAttrs(attrs []slog.Attr) slog.Handler

WithAttrs returns a new slog.Handler that includes the given slog.Attrs.

func (*Handler) WithGroup

func (h *Handler) WithGroup(name string) slog.Handler

WithGroup returns a new slog.Handler that includes the given slog.Handler.

type Options

type Options func(*Handler)

Options is a functional option for the Handler.

func WithNoSpanEvents

func WithNoSpanEvents() Options

WithNoSpanEvents disables recording slog attributes as span events.

func WithSpanEventKey

func WithSpanEventKey(key string) Options

WithSpanEventKey sets the key used to record slog attributes as span events.

func WithSpanIDKey

func WithSpanIDKey(key string) Options

WithSpanIDKey sets the key used to record the span ID in slog records.

func WithTraceIDKey

func WithTraceIDKey(key string) Options

WithTraceIDKey sets the key used to record the trace ID in slog records.

func WithTraceLevel

func WithTraceLevel(level slog.Level) Options

type SpanContext

type SpanContext struct {
	trace.Span
	context.Context
	// contains filtered or unexported fields
}

SpanContext is a wrapper around trace.Span that provides a context.Context. It contains the span, context, trace name, span name, and a flag to ensure the span is created.

func NewMustSpanContext

func NewMustSpanContext(spanName string, traceNameOpt ...string) *SpanContext

NewMustSpanContext creates a new SpanContext with the given span name and ensures it is always created.

func NewMustSpanContextWithContext

func NewMustSpanContextWithContext(ctx context.Context, spanName string, traceNameOpt ...string) *SpanContext

NewMustSpanContextWithContext creates a new SpanContext with the given context and ensures it is always created.

func NewSpanContext

func NewSpanContext(spanName string, traceNameOpt ...string) *SpanContext

NewSpanContext creates a new SpanContext with the given span name.

func NewSpanContextWithContext

func NewSpanContextWithContext(ctx context.Context, spanName string, traceNameOpt ...string) *SpanContext

NewSpanContextWithContext creates a new SpanContext with the given context.

func (*SpanContext) Done

func (s *SpanContext) Done() <-chan struct{}

Done ends the span and returns the context's done channel.

func (*SpanContext) End

func (s *SpanContext) End()

End ends the span.