log

README

package log

package log provides a minimal interface for structured logging in services. It may be wrapped to encode conventions, enforce type-safety, provide leveled logging, and so on. It can be used for both typical application log events, and log-structured data streams.

Structured logging

Structured logging is, basically, conceding to the reality that logs are data, and warrant some level of schematic rigor. Using a stricter, key/value-oriented message format for our logs, containing contextual and semantic information, makes it much easier to get insight into the operational activity of the systems we build. Consequently, package log is of the strong belief that "the benefits of structured logging outweigh the minimal effort involved".

Migrating from unstructured to structured logging is probably a lot easier than you'd expect.

// Unstructured
log.Printf("HTTP server listening on %s", addr)

// Structured
logger.Log("transport", "HTTP", "addr", addr, "msg", "listening")

Usage

Typical application logging

logger := log.NewLogfmtLogger(os.Stderr)
logger.Log("question", "what is the meaning of life?", "answer", 42)

// Output:
// question="what is the meaning of life?" answer=42

Log contexts

func main() {
	var logger log.Logger
	logger = log.NewLogfmtLogger(os.Stderr)
	logger = log.NewContext(logger).With("instance_id", 123)

	logger.Log("msg", "starting")
	NewWorker(log.NewContext(logger).With("component", "worker")).Run()
	NewSlacker(log.NewContext(logger).With("component", "slacker")).Run()
}

// Output:
// instance_id=123 msg=starting
// instance_id=123 component=worker msg=running
// instance_id=123 component=slacker msg=running

Interact with stdlib logger

Redirect stdlib logger to Go kit logger.

import (
	"os"
	stdlog "log"
	kitlog "github.com/go-kit/kit/log"
)

func main() {
	logger := kitlog.NewJSONLogger(os.Stdout)
	stdlog.SetOutput(kitlog.NewStdlibAdapter(logger))
	stdlog.Print("I sure like pie")
}

// Output:
// {"msg":"I sure like pie","ts":"2016/01/01 12:34:56"}

Or, if, for legacy reasons, you need to pipe all of your logging through the stdlib log package, you can redirect Go kit logger to the stdlib logger.

logger := kitlog.NewLogfmtLogger(kitlog.StdlibWriter{})
logger.Log("legacy", true, "msg", "at least it's something")

// Output:
// 2016/01/01 12:34:56 legacy=true msg="at least it's something"

Timestamps and callers

var logger log.Logger
logger = log.NewLogfmtLogger(os.Stderr)
logger = log.NewContext(logger).With("ts", log.DefaultTimestampUTC, "caller", log.DefaultCaller)

logger.Log("msg", "hello")

// Output:
// ts=2016-01-01T12:34:56Z caller=main.go:15 msg=hello

Supported output formats

• Logfmt
• JSON

Enhancements

package log is centered on the one-method Logger interface.

type Logger interface {
	Log(keyvals ...interface{}) error
}

This interface, and its supporting code like log.Context, is the product of much iteration and evaluation. For more details on the evolution of the Logger interface, see The Hunt for a Logger Interface, a talk by Chris Hines. Also, please see #63, #76, #131, #157, #164, and #252 to review historical conversations about package log and the Logger interface.

Value-add packages and suggestions, like improvements to the leveled logger, are of course welcome. Good proposals should

• Be composable with log.Context,
• Not break the behavior of log.Caller in any wrapped context, and
• Be friendly to packages that accept only an unadorned log.Logger.

Benchmarks & comparisons

There are a few Go logging benchmarks and comparisons that include Go kit's package log.

• imkira/go-loggers-bench includes kit/log
• uber-common/zap, a zero-alloc logging library, includes a comparison with kit/log

Documentation

Overview

Package log provides basic interfaces for structured logging.

The fundamental interface is Logger. Loggers create log events from key/value data.

Index

• Variables
• func NewStdlibAdapter(logger Logger, options ...StdlibAdapterOption) io.Writer
• type Context
  • func NewContext(logger Logger) *Context
  • func (l *Context) Log(keyvals ...interface{}) error
  • func (l *Context) With(keyvals ...interface{}) *Context
  • func (l *Context) WithPrefix(keyvals ...interface{}) *Context
• type Logger
  • func NewJSONLogger(w io.Writer) Logger
  • func NewLogfmtLogger(w io.Writer) Logger
  • func NewNopLogger() Logger
• type LoggerFunc
  • func (f LoggerFunc) Log(keyvals ...interface{}) error
• type StdlibAdapter
  • func (a StdlibAdapter) Write(p []byte) (int, error)
• type StdlibAdapterOption
  • func FileKey(key string) StdlibAdapterOption
  • func MessageKey(key string) StdlibAdapterOption
  • func TimestampKey(key string) StdlibAdapterOption
• type StdlibWriter
  • func (w StdlibWriter) Write(p []byte) (int, error)
• type SwapLogger
  • func (l *SwapLogger) Log(keyvals ...interface{}) error
  • func (l *SwapLogger) Swap(logger Logger)
• type Valuer
  • func Caller(depth int) Valuer
  • func Timestamp(t func() time.Time) Valuer

Examples

• Context

Variables

var (
	// DefaultCaller is a Valuer that returns the file and line where the Log
	// method was invoked. It can only be used with log.With.
	DefaultCaller = Caller(3)
)

var ErrMissingValue = errors.New("(MISSING)")

ErrMissingValue is appended to keyvals slices with odd length to substitute the missing value.

Functions

func NewStdlibAdapter

func NewStdlibAdapter(logger Logger, options ...StdlibAdapterOption) io.Writer

NewStdlibAdapter returns a new StdlibAdapter wrapper around the passed logger. It's designed to be passed to log.SetOutput.

Types

type Context

type Context struct {
	// contains filtered or unexported fields
}

A Context wraps a Logger and holds keyvals that it includes in all log events. When logging, a Context replaces all value elements (odd indexes) containing a Valuer with their generated value for each call to its Log method.

Example

package main

import (
	"os"

	"github.com/go-kit/kit/log"
)

func main() {
	logger := log.NewLogfmtLogger(os.Stdout)
	logger.Log("foo", 123)
	ctx := log.NewContext(logger).With("level", "info")
	ctx.Log()
	ctx = ctx.With("msg", "hello")
	ctx.Log()
	ctx.With("a", 1).Log("b", 2)

}

Output:

foo=123
level=info
level=info msg=hello
level=info msg=hello a=1 b=2

func NewContext

func NewContext(logger Logger) *Context

NewContext returns a new Context that logs to logger.

func (*Context) Log

func (l *Context) Log(keyvals ...interface{}) error

Log replaces all value elements (odd indexes) containing a Valuer in the stored context with their generated value, appends keyvals, and passes the result to the wrapped Logger.

func (*Context) With

func (l *Context) With(keyvals ...interface{}) *Context

With returns a new Context with keyvals appended to those of the receiver.

func (*Context) WithPrefix

func (l *Context) WithPrefix(keyvals ...interface{}) *Context

WithPrefix returns a new Context with keyvals prepended to those of the receiver.

type Logger

type Logger interface {
	Log(keyvals ...interface{}) error
}

Logger is the fundamental interface for all log operations. Log creates a log event from keyvals, a variadic sequence of alternating keys and values. Implementations must be safe for concurrent use by multiple goroutines. In particular, any implementation of Logger that appends to keyvals or modifies any of its elements must make a copy first.

func NewJSONLogger

func NewJSONLogger(w io.Writer) Logger

NewJSONLogger returns a Logger that encodes keyvals to the Writer as a single JSON object.

func NewLogfmtLogger

func NewLogfmtLogger(w io.Writer) Logger

NewLogfmtLogger returns a logger that encodes keyvals to the Writer in logfmt format. The passed Writer must be safe for concurrent use by multiple goroutines if the returned Logger will be used concurrently.

func NewNopLogger

func NewNopLogger() Logger

NewNopLogger returns a logger that doesn't do anything.

type LoggerFunc

type LoggerFunc func(...interface{}) error

LoggerFunc is an adapter to allow use of ordinary functions as Loggers. If f is a function with the appropriate signature, LoggerFunc(f) is a Logger object that calls f.

func (LoggerFunc) Log

func (f LoggerFunc) Log(keyvals ...interface{}) error

Log implements Logger by calling f(keyvals...).

type StdlibAdapter

type StdlibAdapter struct {
	Logger
	// contains filtered or unexported fields
}

StdlibAdapter wraps a Logger and allows it to be passed to the stdlib logger's SetOutput. It will extract date/timestamps, filenames, and messages, and place them under relevant keys.

func (StdlibAdapter) Write

func (a StdlibAdapter) Write(p []byte) (int, error)

type StdlibAdapterOption

type StdlibAdapterOption func(*StdlibAdapter)

StdlibAdapterOption sets a parameter for the StdlibAdapter.

func FileKey

func FileKey(key string) StdlibAdapterOption

FileKey sets the key for the file and line field. By default, it's "file".

func MessageKey

func MessageKey(key string) StdlibAdapterOption

MessageKey sets the key for the actual log message. By default, it's "msg".

func TimestampKey

func TimestampKey(key string) StdlibAdapterOption

TimestampKey sets the key for the timestamp field. By default, it's "ts".

type StdlibWriter

type StdlibWriter struct{}

StdlibWriter implements io.Writer by invoking the stdlib log.Print. It's designed to be passed to a Go kit logger as the writer, for cases where it's necessary to redirect all Go kit log output to the stdlib logger.

If you have any choice in the matter, you shouldn't use this. Prefer to redirect the stdlib log to the Go kit logger via NewStdlibAdapter.

func (StdlibWriter) Write

func (w StdlibWriter) Write(p []byte) (int, error)

Write implements io.Writer.

type SwapLogger

type SwapLogger struct {
	// contains filtered or unexported fields
}

SwapLogger wraps another logger that may be safely replaced while other goroutines use the SwapLogger concurrently. The zero value for a SwapLogger will discard all log events without error.

SwapLogger serves well as a package global logger that can be changed by importers.

func (*SwapLogger) Log

func (l *SwapLogger) Log(keyvals ...interface{}) error

Log implements the Logger interface by forwarding keyvals to the currently wrapped logger. It does not log anything if the wrapped logger is nil.

func (*SwapLogger) Swap

func (l *SwapLogger) Swap(logger Logger)

Swap replaces the currently wrapped logger with logger. Swap may be called concurrently with calls to Log from other goroutines.

type Valuer

type Valuer func() interface{}

A Valuer generates a log value. When passed to Context.With in a value element (odd indexes), it represents a dynamic value which is re-evaluated with each log event.

var (
	// DefaultTimestamp is a Valuer that returns the current wallclock time,
	// respecting time zones, when bound.
	DefaultTimestamp Valuer = func() interface{} { return time.Now().Format(time.RFC3339) }

	// DefaultTimestampUTC is a Valuer that returns the current time in UTC
	// when bound.
	DefaultTimestampUTC Valuer = func() interface{} { return time.Now().UTC().Format(time.RFC3339) }
)

func Caller

func Caller(depth int) Valuer

Caller returns a Valuer that returns a file and line from a specified depth in the callstack. Users will probably want to use DefaultCaller.

func Timestamp

func Timestamp(t func() time.Time) Valuer

Timestamp returns a Valuer that invokes the underlying function when bound, returning a time.Time. Users will probably want to use DefaultTimestamp or DefaultTimestampUTC.