kvlog

README

kvlog

kvlog is a logging library based on key-value logging implemented using go (golang).

Description

kvlog provides types and functions to produce a log stream of key-value based log events. Key-value based log messages differ from conventional string-based log messages. They do not contain a bare string message but any number of key-value-tuples which are encoded using a simple to parse syntax. This allows log processor systems such as the ELK-stack to analyze and index the log messages based on key-value-tuples.

Components

kvlog is built from a set of components that interact to implement logging functionality.

A Message is produced by the client. Every Message consists of Pairs each representing a single key-value-pair. kvlog provides a convenient and idiomatic API to create Pairs and Messages.

The Message is then given to a Logger. A Logger may augment the message with additional Pairs. It's common for a Logger to add at least a level and a ts (timestamp) Pair, but Loggers may add other Pairs.

Every Logger uses a set of Handlers. A Handler is responsible for

• formatting the Message using a Formatter
• delivering the Message using an Output

Log Format

The format used by kvlog by default follows the defaults of the logstash KV filter. The following lines show examples of the log output

ts=2019-08-16T12:58:22 level=info event=<started>
ts=2019-08-16T12:58:34 level=info event=<request> method=<GET> url=</> status=200 duration=0.001s
ts=2019-08-16T12:58:34 level=info event=<request> method=<GET> url=</favicon.ico> status=404 duration=0.000s
ts=2019-08-16T12:58:35 level=info event=<request> method=<POST> url=</pdf> status=200 duration=0.009s

Installation

$ go get -u bitbucket.org/halimath/kvlog

Usage

kvlog can be used in different ways.

Module functions

The most simple usage uses module functions.

package main

import (
    "bitbucket.org/halimath/kvlog"
)

func main () {
    // ...

    kvlog.Info(kvlog.KV("event", "App started"))
}

The module provides functions for all log level (Debug, Info, Warn, Error) as well as a configuration function for initializing the package level logger (i.e. configuring output and threshold as well as other filters). The default is to log everything of level Info or above to stdout using the default log format.

Logger instance

A more advanced usage involves a dedicated Logger instance which can be used in dependency injection scenarios.

package main

import (
    "bitbucket.org/halimath/kvlog"
)

func main () {
    l := kvlog.NewLogger(kvlog.NewHandler(kvlog.KVFormatter, kvlog.Stdout(), kvlog.Threshold(kvlog.LevelWarn)))

    // ...

    l.Info(kvlog.KV("event", "App started"))
}

HTTP Middleware

kvlog contains a HTTP middleware that generates an access log. It wraps another http.Hander allowing you to log only requests on those handlers you are interested in.

package main

import (
	"net/http"

	"bitbucket.org/halimath/kvlog"
)

func main() {
    mux := http.NewServeMux()
    // ...
	kvlog.Info(kvlog.KV("event", "started"))
	http.ListenAndServe(":8000", kvlog.Middleware(kvlog.L, mux))
}

Changelog

0.4.0

• Export package level logger instance L

0.3.0

Caution, breaking changes: This version provides a new API which is not compatible to the API exposed before.

• Introduction of new component structure (see description above)

0.2.0

• Improve log message rendering

0.1.0

• Initial release

License

Copyright 2019, 2020 Alexander Metzner.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Documentation

Overview

Package kvlog provides a key-value based logging system.

Example

package main

import (
	"bitbucket.org/halimath/kvlog"
)

func main() {
	kvlog.Debug(kvlog.KV("event", "test"), kvlog.KV("foo", "bar"))
	kvlog.Info(kvlog.KV("event", "test"), kvlog.KV("foo", "bar"))
}

Index

• Constants
• Variables
• func Debug(pairs ...KVPair)
• func Error(pairs ...KVPair)
• func Info(pairs ...KVPair)
• func Init(handler ...*Handler)
• func Middleware(l *Logger, h http.Handler) http.Handler
• func Warn(pairs ...KVPair)
• type Filter
  • func Threshold(threshold Level) Filter
• type FilterFunc
  • func (f FilterFunc) Filter(m Message) Message
• type Formatter
• type FormatterFunc
  • func (ff FormatterFunc) Format(m Message, w io.Writer) error
• type Handler
  • func NewHandler(formatter Formatter, output Output, filter ...Filter) *Handler
  • func (h *Handler) Close()
  • func (h *Handler) Deliver(m Message)
• type KVPair
  • func KV(key string, value interface{}) KVPair
• type Level
  • func (l Level) String() string
• type Logger
  • func NewLogger(handler ...*Handler) *Logger
  • func (l *Logger) Close()
  • func (l *Logger) Debug(pairs ...KVPair)
  • func (l *Logger) Error(pairs ...KVPair)
  • func (l *Logger) Info(pairs ...KVPair)
  • func (l *Logger) Log(m Message)
  • func (l *Logger) Warn(pairs ...KVPair)
• type Message
  • func NewMessage(l Level, pairs ...KVPair) Message
  • func (m Message) Level() Level
• type Output
  • func Stderr() Output
  • func Stdout() Output

Examples

• Package

Constants

const (
	// KeyLevel defines the message key containing the message's level.
	KeyLevel = "level"
	// KeyTimestamp defines the message key containing the message's timestamp.
	KeyTimestamp = "ts"
)

Variables

var KVFormatter = FormatterFunc(formatMessage)

KVFormatter implements a Formatter that writes the default KV format.

Functions

func Debug

func Debug(pairs ...KVPair)

Debug emits a log message of level debug.

func Error

func Error(pairs ...KVPair)

Error emits a log message of level error.

func Info

func Info(pairs ...KVPair)

Info emits a log message of level info.

func Init

func Init(handler ...*Handler)

Init initializes the package global logger to a new logger using the given handler. The previous logger is closed if it had been set before.

func Middleware

func Middleware(l *Logger, h http.Handler) http.Handler

Middleware returns a http.Handler that acts as an access log middleware.

func Warn

func Warn(pairs ...KVPair)

Warn emits a log message of level warn.

Types

type Filter

type Filter interface {
	// Filter filters the given message m and returns
	// either a message (which may be m) to be handled
	// or nil if the given message should be dropped.
	Filter(m Message) Message
}

Filter defines the interface for types that filter messages.

func Threshold

func Threshold(threshold Level) Filter

Threshold is a factory for a Filter that drops messages if their level is less then the given threshold.

type FilterFunc

type FilterFunc func(m Message) Message

FilterFunc is a wrapper type implementing Filter that wraps a plain function.

func (FilterFunc) Filter

func (f FilterFunc) Filter(m Message) Message

Filter just calls f to perform filtering.

type Formatter

type Formatter interface {
	// Formats the given message into a slice of bytes.
	Format(m Message, w io.Writer) error
}

Formatter defines the interface implemented by all message formatters.

type FormatterFunc

type FormatterFunc func(m Message, w io.Writer) error

FormatterFunc is a converter type that allows using a plain function as a Formatter.

func (FormatterFunc) Format

func (ff FormatterFunc) Format(m Message, w io.Writer) error

Format simply calls ff.

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

Handler implements a threshold

func NewHandler

func NewHandler(formatter Formatter, output Output, filter ...Filter) *Handler

NewHandler creates a new Handler using the provided values.

func (*Handler) Close

func (h *Handler) Close()

Close closes the handler terminating its service. The underlying output is also closed.

func (*Handler) Deliver

func (h *Handler) Deliver(m Message)

Deliver performs the delivery of the given message.

type KVPair

type KVPair struct {
	// Key stores the key of the pair
	Key string

	// Value stores the value
	Value interface{}
}

KVPair implements a key-value pair

func KV

func KV(key string, value interface{}) KVPair

KV is a factory method for KVPair objects

type Level

type Level int

Level defines the valid log levels.

const (
	// LevelDebug log level
	LevelDebug Level = iota
	// LevelInfo log level
	LevelInfo
	// LevelWarn log level
	LevelWarn
	// LevelError log level
	LevelError
)

func (Level) String

func (l Level) String() string

String provides a string representation of the log level.

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger implements a logger component.

var L *Logger

L is the Logger instance used by package level functions. Use this logger as a convenience.

func NewLogger

func NewLogger(handler ...*Handler) *Logger

NewLogger constructs a new Logger and returns a pointer to it.

func (*Logger) Close

func (l *Logger) Close()

Close closes the handlers registered to this logger and waits for the goroutines to finish.

func (*Logger) Debug

func (l *Logger) Debug(pairs ...KVPair)

Debug logs a message with level Debug.

func (*Logger) Error

func (l *Logger) Error(pairs ...KVPair)

Error logs a message with level Error.

func (*Logger) Info

func (l *Logger) Info(pairs ...KVPair)

Info logs a message with level Info.

func (*Logger) Log

func (l *Logger) Log(m Message)

Log logs the given message.

func (*Logger) Warn

func (l *Logger) Warn(pairs ...KVPair)

Warn logs a message with level Warn.

type Message

type Message []KVPair

Message represents a single log message expressed as an ordered list of key value pairs

func NewMessage

func NewMessage(l Level, pairs ...KVPair) Message

NewMessage creates a new message from the given log level and key-value pairs

func (Message) Level

func (m Message) Level() Level

Level returns the message's level

type Output

type Output interface {
	io.Writer
}

Output defines the interface that must be implemented by types that handle output.

func Stderr

func Stderr() Output

Stderr returns an Output that writes the STDOUT but ignores any request to close the stream.

func Stdout

func Stdout() Output

Stdout returns an Output that writes the STDOUT but ignores any request to close the stream.