log

README

Logging

Logging will be to STDOUT, then captured by the fluentd and finally delivered to ElasticSearch or Graylog.

Reasoning:

• Local log storage allows for easy maintenance on ELK/GL, no problem on outage for production system

Format

The format is implemented in the log package and generally refered to as KVP stile logging. It is easy to parse and generate in code when logging to STDOUT and not as heavy as JSON on the eyes. See example below:

2018-09-06 15:31:23 |INFO| Starting testserver ... addr=:3000

The KVP logs are only generated if the stdout device is a terminal. If the device is not a terminal JSON output will be generated. This way there is very little parse overhead for imports into storage systems like elasticsearch.

JSON log format attributes

Name	Type	Example	Notes
level	string	"info"
req_id	string	"be922tboo3smmkmppjfg"	The generated id is a URL safe base64 encoded mongo object-id-like unique id. Mongo unique id generation algorithm has been selected as a trade-off between size and ease of use: UUID is less space efficient and snowflake re`quires machine configuration.
method	string	"GET"
url	string	"/foo/bar"
status	int	501
host	string	"example.com"	taken from the request
size	int	121
duration	float	0.217762
ip	string	"192.0.2.1"	respects X-Forwarded-For and X-Real-Ip
referer	string	"https://google.de"
user_agent	string	"Mozilla/5.0 (Macintosh;"
time	string	"2018-09-07 06:57:57"	iso8601 UTC
message	string	"Request Completed"
-	-	Microservice specific	-
handler	string	"GetPumpHandler"	Name of the handler func in case of a panic
error	string	"Can't open file"	text representation of the error

Constraints

• Encoding of all entries is in UTF-8
• Timestamps are UTC format is iso8601
• Use keys consistently

Environment based configuration

• LOG_LEVEL default: debug
  • Allows for logging at the following levels (from highest to lowest):
    • panic
    • fatal
    • error
    • warn
    • info
    • debug
    • disabled don't log at all
• LOG_FORMAT default: auto
  • If set to auto will detect if stdout is attached to a TTY and set the format to console otherwise the format will be json. Formats can be set directly.
• LOG_COMPLETED_REQUEST default: true
  • If set to true allows log handler to log request related information once at the end of the request

Resources

• https://logz.io/blog/logging-best-practices/
• https://docs.logentries.com/docs/best-practices-logs

Documentation

Index

• Constants
• func ContextWithSink(ctx context.Context, sink *Sink) context.Context
• func Ctx(ctx context.Context) *zerolog.Logger
• func Debug(v ...interface{})
• func Debugf(format string, v ...interface{})
• func Error(v ...interface{})
• func Errorf(format string, v ...interface{})
• func Fatal(v ...interface{})
• func Fatalf(format string, v ...interface{})
• func Fatalln(v ...interface{})
• func Handler(silentPrefixes ...string) func(http.Handler) http.Handler
• func Info(v ...interface{})
• func Infof(format string, v ...interface{})
• func Logger() *zerolog.Logger
• func Output(w io.Writer) *zerolog.Logger
• func Print(v ...interface{})
• func Printf(format string, v ...interface{})
• func Println(v ...interface{})
• func ProxyAwareRemote(r *http.Request) string
• func Req(r *http.Request) *zerolog.Logger
• func RequestID(r *http.Request) string
• func RequestIDFromContext(ctx context.Context) string
• func RequestIDHandler(fieldKey, headerName string) func(next http.Handler) http.Handler
• func SinkContextTransfer(sourceCtx, targetCtx context.Context) context.Context
• func Stack(ctx context.Context)
• func TraceIDFromContext(ctx context.Context) string
• func Warn(v ...interface{})
• func Warnf(format string, v ...interface{})
• func WithContext(ctx context.Context) context.Context
• type Sink
  • func NewSink(opts ...SinkOption) *Sink
  • func SinkFromContext(ctx context.Context) (*Sink, bool)
  • func (s *Sink) Pretty() string
  • func (s *Sink) ToJSON() []byte
  • func (s *Sink) Write(b []byte) (int, error)
• type SinkOption
  • func CustomSize(size int) SinkOption
  • func Silent() SinkOption

Constants

const RequestIDHeader = "Request-Id"

RequestIDHeader name of the header that can contain a request ID

Functions

func ContextWithSink

func ContextWithSink(ctx context.Context, sink *Sink) context.Context

ContextWithSink wraps the given context in a new context with the given Sink stored as value.

func Ctx

func Ctx(ctx context.Context) *zerolog.Logger

Ctx returns the logger for the passed context

func Debug

func Debug(v ...interface{})

Debug implements logrus Debug interface

func Debugf

func Debugf(format string, v ...interface{})

Debugf implements logrus Debugf interface

func Error

func Error(v ...interface{})

Error implements logrus Error interface

func Errorf

func Errorf(format string, v ...interface{})

Errorf implements logrus Errorf interface

func Fatal

func Fatal(v ...interface{})

Fatal implements log Fatal interface

func Fatalf

func Fatalf(format string, v ...interface{})

Fatalf implements log Fatalf interface

func Fatalln

func Fatalln(v ...interface{})

Fatalln implements log Fatalln interface

func Handler

func Handler(silentPrefixes ...string) func(http.Handler) http.Handler

Handler returns a middleware that handles all of the logging aspects of any incoming http request. Optionally several path prefixes like "/health" can be provided to decrease log spamming. All url paths with these prefixes will not be logged to the standard output but still be available in the request specific Sink.

func Info

func Info(v ...interface{})

Info implements logrus Info interface

func Infof

func Infof(format string, v ...interface{})

Infof implements logrus Infof interface

func Logger

func Logger() *zerolog.Logger

Logger returns the current logger instance

func Output

func Output(w io.Writer) *zerolog.Logger

Output duplicates the current logger and sets w as its output.

func Print

func Print(v ...interface{})

Print implements log Print interface

func Printf

func Printf(format string, v ...interface{})

Printf implements log Printf interface

func Println

func Println(v ...interface{})

Println implements log Println interface

func ProxyAwareRemote

func ProxyAwareRemote(r *http.Request) string

ProxyAwareRemote return the most likely remote address

func Req

func Req(r *http.Request) *zerolog.Logger

Req returns the logger for the passed request

func RequestID

func RequestID(r *http.Request) string

RequestID returns a unique request id or an empty string if there is none

func RequestIDFromContext

func RequestIDFromContext(ctx context.Context) string

RequestIDFromContext returns a unique request id or an empty string if there is none

func RequestIDHandler

func RequestIDHandler(fieldKey, headerName string) func(next http.Handler) http.Handler

func SinkContextTransfer

func SinkContextTransfer(sourceCtx, targetCtx context.Context) context.Context

SinkContextTransfer gets the sink from the sourceCtx and returns a new context based on targetCtx with the extracted sink. If no sink is present this is a noop

func Stack

func Stack(ctx context.Context)

Stack prints the stack of the calling goroutine

func TraceIDFromContext

func TraceIDFromContext(ctx context.Context) string

TraceIDFromContext returns a unique request id or an empty string if there is none

func Warn

func Warn(v ...interface{})

Warn implements logrus Warn interface

func Warnf

func Warnf(format string, v ...interface{})

Warnf implements logrus Warnf interface

func WithContext

func WithContext(ctx context.Context) context.Context

WithContext returns context with enabled logger. This overwrites a logger that is set on the context already use this if you are not inside a request context.

Types

type Sink

type Sink struct {
	Silent bool
	// contains filtered or unexported fields
}

Sink respresents a log sink which is used to store logs, created with log.Ctx(ctx), inside the context and use them at a later point in time

func NewSink

func NewSink(opts ...SinkOption) *Sink

NewSink initializes a new sink. This will deprecate the public properties of the sink struct sometime in the future

func SinkFromContext

func SinkFromContext(ctx context.Context) (*Sink, bool)

SinkFromContext returns the Sink of the given context if it exists

func (*Sink) Pretty

func (s *Sink) Pretty() string

Pretty returns the logs as string while using the zerolog.ConsoleWriter to format them in a human readable way

func (*Sink) ToJSON

func (s *Sink) ToJSON() []byte

ToJSON returns a copy of the currently available logs in the Sink as json formatted []byte.

func (*Sink) Write

func (s *Sink) Write(b []byte) (int, error)

Write implements the io.Writer interface. This makes it possible to use the Sink as output in the zerolog.Output() func. Write stores all incoming logs in its internal store and calls Write() on the default output writer.

type SinkOption

type SinkOption func(*Sink)

func CustomSize

func CustomSize(size int) SinkOption

func Silent

func Silent() SinkOption