Documentation ¶
Overview ¶
Package flume is a logging package, build on top of zap. It's structured and leveled logs, like zap/logrus/etc. It adds global, runtime re-configuration of all loggers, via an internal logger registry.
There are two interaction points with flume: code that generates logs, and code that configures logging output. Code which generates logs needs to create named logger instances, and call log functions on it, like Info() and Debug(). But by default, all these logs will be silently discarded. Flume does not output log entries unless explicitly told to do so. This ensures libraries can freely use flume internally, without polluting the stdout of the programs importing the library.
The Logger type is a small interface. Libraries should allow replacement of their Logger instances so importers can entirely replace flume if they wish. Alternately, importers can use flume to configure the library's log output, and/or redirect it into the overall program's log stream.
Logging ¶
This package does not offer package level log functions, so you need to create a logger instance first: A common pattern is to create a single, package-wide logger, named after the package:
var log = flume.New("mypkg")
Then, write some logs:
log.Debug("created user", "username", "frank", "role", "admin")
Logs have a message, then matched pairs of key/value properties. Child loggers can be created and pre-seeded with a set of properties:
reqLogger := log.With("remoteAddr", req.RemoteAddr)
Expensive log events can be avoid by explicitly checking level:
if log.IsDebug() { log.Debug("created resource", "resource", resource.ExpensiveToString()) }
Loggers can be bound to context.Context, which is convenient for carrying per-transaction loggers (pre-seeded with transaction specific context) through layers of request processing code:
ctx = flume.WithLogger(ctx, log.With("transactionID", tid)) // ...later... flume.FromContext(ctx).Info("Request handled.")
The standard Logger interface only supports 3 levels of log, DBG, INF, and ERR. This is inspired by this article: https://dave.cheney.net/2015/11/05/lets-talk-about-logging. However, you can create instances of DeprecatedLogger instead, which support more levels.
Configuration ¶
There are several package level functions which reconfigure logging output. They control which levels are discarded, which fields are included in each log entry, and how those fields are rendered, and how the overall log entry is rendered (JSON, LTSV, colorized, etc).
To configure logging settings from environment variables, call the configuration function from main():
flume.ConfigFromEnv()
This reads the log configuration from the environment variable "FLUME" (the default, which can be overridden). The value is JSON, e.g.:
{"level":"INF","levels":"http=DBG","development"="true"}
The properties of the config string:
"level": ERR, INF, or DBG. The default level for all loggers.
"levels": A string configuring log levels for specific loggers, overriding the default level. See note below for syntax.
"development": true or false. In development mode, the defaults for the other settings change to be more suitable for developers at a terminal (colorized, multiline, human readable, etc). See note below for exact defaults.
"addCaller": true or false. Adds call site information to log entries (file and line).
"encoding": json, ltsv, term, or term-color. Configures how log entries are encoded in the output. "term" and "term-color" are multi-line, human-friendly formats, intended for terminal output.
"encoderConfig": a JSON object which configures advanced encoding settings, like how timestamps are formatted. See docs for go.uber.org/zap/zapcore/EncoderConfig
"messageKey": the label of the message property of the log entry. If empty, message is omitted.
"levelKey": the label of the level property of the log entry. If empty, level is omitted.
"timeKey": the label of the timestamp of the log entry. If empty, timestamp is omitted.
"nameKey": the label of the logger name in the log entry. If empty, logger name is omitted.
"callerKey": the label of the logger name in the log entry. If empty, logger name is omitted.
"lineEnding": the end of each log output line.
"levelEncoder": capital, capitalColor, color, lower, or abbr. Controls how the log entry level is rendered. "abbr" renders 3-letter abbreviations, like ERR and INF.
"timeEncoder": iso8601, millis, nanos, unix, or justtime. Controls how timestamps are rendered. "millis", "nanos", and "unix" are since UNIX epoch. "unix" is in floating point seconds. "justtime" omits the date, and just prints the time in the format "15:04:05.000".
"durationEncoder": string, nanos, or seconds. Controls how time.Duration values are rendered.
"callerEncoder": full or short. Controls how the call site is rendered. "full" includes the entire package path, "short" only includes the last folder of the package.
Defaults:
{ "level":"INF", "levels":"", "development":false, "addCaller":false, "encoding":"term-color", "encoderConfig":{ "messageKey":"msg", "levelKey":"level", "timeKey":"time", "nameKey":"name", "callerKey":"caller", "lineEnding":"\n", "levelEncoder":"abbr", "timeEncoder":"iso8601", "durationEncoder":"seconds", "callerEncoder":"short", } }
These defaults are only applied if one of the configuration functions is called, like ConfigFromEnv(), ConfigString(), Configure(), or LevelsString(). Initially, all loggers are configured to discard everything, following flume's opinion that log packages should be silent unless spoken too. Ancillary to this: library packages should *not* call these functions, or configure logging levels or output in anyway. Only program entry points, like main() or test code, should configure logging. Libraries should just create loggers and log to them.
Development mode: if "development"=true, the defaults for the rest of the settings change, equivalent to:
{ "addCaller":true, "encoding":"term-color", "encodingConfig": { "timeEncoder":"justtime", "durationEncoder":"string", } }
The "levels" value is a list of key=value pairs, configuring the level of individual named loggers. If the key is "*", it sets the default level. If "level" and "levels" both configure the default level, "levels" wins. Examples:
- // set the default level to ALL, equivalent to {"level"="ALL"} *=INF // same, but set default level to INF *,sql=WRN // set default to ALL, set "sql" logger to WRN *=INF,http=ALL // set default to INF, set "http" to ALL *=INF,http // same as above. If name has no level, level is set to ALL *=INF,-http // set default to INF, set "http" to OFF http=INF // leave default setting unchanged.
Factories ¶
Most usages of flume will use its package functions. The package functions delegate to an internal instance of Factory, which a the logger registry. You can create and manage your own instance of Factory, which will be an isolated set of Loggers.
tl;dr
The implementation is a wrapper around zap. zap does levels, structured logs, and is very fast. zap doesn't do centralized, global configuration, so this package adds that by maintaining an internal registry of all loggers, and using the sync.atomic stuff to swap out levels and writers in a thread safe way.
Example ¶
package main import ( "fmt" "github.com/gemalto/flume" ) func main() { err := flume.Configure(flume.Config{ Development: true, DefaultLevel: flume.DebugLevel, Encoding: "ltsv", }) if err != nil { fmt.Println("logging config failed:", err.Error()) return } log := flume.New("root") log.Info("Hello World!") log.Info("This entry has properties", "color", "red") log.Debug("This is a debug message") log.Error("This is an error message") log.Info("This message has a multiline value", "essay", `Four score and seven years ago our fathers brought forth on this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.`) }
Output:
Index ¶
- Constants
- Variables
- func AbbrLevelEncoder(l zapcore.Level, enc zapcore.PrimitiveArrayEncoder)
- func ClearHooks()
- func ConfigFromEnv(envvars ...string) error
- func ConfigString(s string) error
- func Configure(cfg Config) error
- func Hooks(hooks ...HookFunc)
- func JustTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
- func LogFuncWriter(l func(args ...interface{}), trimSpace bool) io.Writer
- func LoggerFuncWriter(l func(msg string, kvpairs ...interface{})) io.Writer
- func SetAddCaller(b bool)
- func SetDefaultLevel(l Level)
- func SetDevelopmentDefaults() error
- func SetEncoder(e Encoder)
- func SetLevel(name string, l Level)
- func SetOut(w io.Writer) func()
- func WithLogger(ctx context.Context, l Logger) context.Context
- type CheckedEntry
- type Colorizer
- type Colors
- type Config
- type Core
- func (l *Core) Debug(msg string, args ...interface{})
- func (l *Core) Error(msg string, args ...interface{})
- func (l *Core) Info(msg string, args ...interface{})
- func (l *Core) IsDebug() bool
- func (l *Core) IsEnabled(lvl Level) bool
- func (l *Core) IsInfo() bool
- func (l *Core) Log(lvl Level, template string, fmtArgs, context []interface{}) bool
- func (l *Core) With(args ...interface{}) Logger
- func (l *Core) WithArgs(args ...interface{}) *Core
- type CoreOption
- type Encoder
- type EncoderConfig
- type Entry
- type Factory
- func (r *Factory) ClearHooks()
- func (r *Factory) Configure(cfg Config) error
- func (r *Factory) Hooks(hooks ...HookFunc)
- func (r *Factory) LevelsString(s string) error
- func (r *Factory) NewCore(name string, options ...CoreOption) *Core
- func (r *Factory) NewLogger(name string) Logger
- func (r *Factory) SetAddCaller(b bool)
- func (r *Factory) SetDefaultLevel(l Level)
- func (r *Factory) SetEncoder(e Encoder)
- func (r *Factory) SetLevel(name string, l Level)
- func (r *Factory) SetOut(w io.Writer) func()
- type Field
- type HookFunc
- type Level
- type Logger
Examples ¶
Constants ¶
const ( // OffLevel disables all logs OffLevel = Level(127) // DebugLevel should be used for low-level, non-production logs. Typically intended only for developers. DebugLevel = Level(zapcore.DebugLevel) // InfoLevel should be used for production level logs. Typically intended for end-users and developers. InfoLevel = Level(zapcore.InfoLevel) // ErrorLevel should be used for errors. Generally, this should be reserved for events which truly // need to be looked at by an admin, and might be reported to an error-tracking system. ErrorLevel = Level(zapcore.ErrorLevel) )
Variables ¶
var Bright = ansi.ColorCode("default+b")
Bright is the color used for the message
var DefaultColors = Colors{ Debug: ansi.ColorCode("cyan"), Info: ansi.ColorCode("green+h"), Warn: ansi.ColorCode("yellow+bh"), Error: ansi.ColorCode("red+bh"), }
DefaultColors is the default instance of Colors, used as the default colors if a nil Colorizer is passed to NewColorizedConsoleEncoder.
var DefaultConfigEnvVars = []string{"FLUME"}
DefaultConfigEnvVars is a list of the environment variables that ConfigFromEnv will search by default.
var DefaultLogger = New("")
DefaultLogger is returned by FromContext if no other logger has been injected into the context.
var Dim = ansi.ColorCode("240")
Dim is the color used for context keys, time, and caller information
Functions ¶
func AbbrLevelEncoder ¶
func AbbrLevelEncoder(l zapcore.Level, enc zapcore.PrimitiveArrayEncoder)
AbbrLevelEncoder encodes logging levels to the strings in the log entries. Encodes levels as 3-char abbreviations in upper case.
encConfig := flume.EncoderConfig{} encConfig.EncodeTime = flume.AbbrLevelEncoder
func ClearHooks ¶ added in v0.13.0
func ClearHooks()
ClearHooks clears all hooks from the package-level Factory.
func ConfigFromEnv ¶
ConfigFromEnv configures flume from environment variables. It should be called from main():
func main() { flume.ConfigFromEnv() ... }
It searches envvars for the first environment variable that is set, and attempts to parse the value.
If no environment variable is set, it silently does nothing.
If an environment variable with a value is found, but parsing fails, an error is printed to stdout, and the error is returned.
If envvars is empty, it defaults to DefaultConfigEnvVars.
func ConfigString ¶
ConfigString configures the package level Factory. The string can either be a JSON-serialized Config object, or just a LevelsString (see Factory.LevelsString for format).
Note: this will reconfigure the logging levels for all loggers.
func Configure ¶
Configure configures the package level Factory from the settings in the Config object. See Config for details.
Note: this will reconfigure the logging levels for all loggers.
func Hooks ¶ added in v0.13.0
func Hooks(hooks ...HookFunc)
Hooks adds hooks to the package-level Factory.
func JustTimeEncoder ¶
func JustTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
JustTimeEncoder is a timestamp encoder function which encodes time as a simple time of day, without a date. Intended for development and testing. Not good in a production system, where you probably need to know the date.
encConfig := flume.EncoderConfig{} encConfig.EncodeTime = flume.JustTimeEncoder
func LogFuncWriter ¶
LogFuncWriter is a writer which writes to a logging function signature like that of testing.T.Log() and fmt/log.Println(). It can be used to redirect flumes *output* to some other logger.
SetOut(LogFuncWriter(fmt.Println, true)) SetOut(LogFuncWriter(t.Log, true))
func LoggerFuncWriter ¶
LoggerFuncWriter is a writer which writes lines to a logging function with a signature like that of flume.Logger's functions, like Info(), Debug(), and Error().
http.Server{ ErrorLog: log.New(LoggerFuncWriter(flume.New("http").Error), "", 0), }
func SetAddCaller ¶
func SetAddCaller(b bool)
SetAddCaller enables/disables call site logging on the package-level Factory
func SetDefaultLevel ¶
func SetDefaultLevel(l Level)
SetDefaultLevel sets the default log level on the package-level Factory.
func SetDevelopmentDefaults ¶
func SetDevelopmentDefaults() error
SetDevelopmentDefaults sets useful default settings on the package-level Factory which are appropriate for a development setting. Default log level is set to INF, all loggers are reset to the default level, call site information is logged, and the encoder is a colorized, multi-line friendly console encoder with a simplified time stamp format.
func SetEncoder ¶
func SetEncoder(e Encoder)
SetEncoder sets the encoder for the package-level Factory
Types ¶
type CheckedEntry ¶ added in v0.13.0
type CheckedEntry = zapcore.CheckedEntry
type Colorizer ¶
Colorizer returns ansi escape sequences for the colors for each log level. See Colors for a default implementation.
type Colors ¶
type Colors struct {
Debug, Info, Warn, Error string
}
Colors is an implementation of the Colorizer interface, which assigns colors to the default log levels.
type Config ¶
type Config struct { // DefaultLevel is the default log level for all loggers not // otherwise configured by Levels. Defaults to Info. DefaultLevel Level `json:"level" yaml:"level"` // Levels configures log levels for particular named loggers. See // LevelsString for format. Levels string `json:"levels" yaml:"levels"` // AddCaller annotates logs with the calling function's file // name and line number. Defaults to true when the Development // flag is set, false otherwise. AddCaller *bool `json:"addCaller" yaml:"addCaller"` // Encoding sets the logger's encoding. Valid values are "json", // "console", "ltsv", "term", and "term-color". // Defaults to "term-color" if development is true, else // "ltsv" Encoding string `json:"encoding" yaml:"encoding"` // Development toggles the defaults used for the other // settings. Defaults to false. Development bool `json:"development" yaml:"development"` // EncoderConfig sets options for the chosen encoder. See // EncoderConfig for details. Defaults to NewEncoderConfig() if // Development is false, otherwise defaults to NewDevelopmentEncoderConfig(). EncoderConfig *EncoderConfig `json:"encoderConfig" yaml:"encoderConfig"` }
Config offers a declarative way to configure a Factory.
The same things can be done by calling Factory methods, but Configs can be unmarshaled from JSON, making it a convenient way to configure most logging options from env vars or files, i.e.:
err := flume.ConfigString(os.Getenv("flume"))
Configs can be created and applied programmatically:
err := flume.Configure(flume.Config{})
Defaults are appropriate for a JSON encoded production logger:
- LTSV encoder - full timestamps - default log level set to INFO - call sites are not logged
An alternate set of defaults, more appropriate for development environments, can be configured with `Config{Development:true}`:
err := flume.Configure(flume.Config{Development:true})
- colorized terminal encoder - short timestamps - call sites are logged
err := flume.Configure(flume.Config{Development:true})
Any of the other configuration options can be specified to override the defaults.
Note: If configuring the EncoderConfig setting, if any of the *Key properties are omitted, that entire field will be omitted.
func (*Config) SetAddCaller ¶
SetAddCaller sets the Config's AddCaller flag.
func (*Config) UnsetAddCaller ¶
func (c *Config) UnsetAddCaller()
UnsetAddCaller unsets the Config's AddCaller flag (reverting to defaults).
type Core ¶ added in v0.10.0
type Core struct {
// contains filtered or unexported fields
}
Core is the concrete implementation of Logger. It has some additional lower-level methods which can be used by other logging packages which wrap flume, to build alternate logging interfaces.
func NewCore ¶ added in v0.10.0
func NewCore(name string, options ...CoreOption) *Core
NewCore returns a new Core
func (*Core) Debug ¶ added in v0.10.0
Debug logs at DBG level. args should be alternative keys and values. keys should be strings.
func (*Core) Error ¶ added in v0.10.0
Error logs at ERR level. args should be alternative keys and values. keys should be strings.
func (*Core) Info ¶ added in v0.10.0
Info logs at INF level. args should be alternative keys and values. keys should be strings.
func (*Core) Log ¶ added in v0.10.0
Log is the core logging method, used by the convenience methods Debug(), Info(), and Error().
Returns true if the log was actually logged.
AddCaller option will report the caller of this method. If wrapping this, be sure to use the AddCallerSkip option.
type CoreOption ¶ added in v0.11.0
type CoreOption interface {
// contains filtered or unexported methods
}
An CoreOption configures a Core.
func AddCallerSkip ¶ added in v0.11.0
func AddCallerSkip(skip int) CoreOption
AddCallerSkip increases the number of callers skipped by caller annotation (as enabled by the AddCaller option). When building wrappers around a Core, supplying this CoreOption prevents Core from always reporting the wrapper code as the caller.
func AddHooks ¶ added in v0.13.0
func AddHooks(hooks ...HookFunc) CoreOption
AddHooks adds hooks to this logger core. These will only execute on this logger, after the global hooks.
type Encoder ¶
Encoder serializes log entries. Re-exported from zap for now to avoid exporting zap.
func NewColorizedConsoleEncoder ¶
func NewColorizedConsoleEncoder(cfg *EncoderConfig, colorizer Colorizer) Encoder
NewColorizedConsoleEncoder creates a console encoder, like NewConsoleEncoder, but colors the text with ansi escape codes. `colorize` configures which colors to use for each level.
If `colorizer` is nil, it will default to DefaultColors.
`github.com/mgutz/ansi` is a convenient package for getting color codes, e.g.:
ansi.ColorCode("red")
func NewConsoleEncoder ¶
func NewConsoleEncoder(cfg *EncoderConfig) Encoder
NewConsoleEncoder creates an encoder whose output is designed for human - rather than machine - consumption. It serializes the core log entry data (message, level, timestamp, etc.) in a plain-text format. The context is encoded in LTSV.
Note that although the console encoder doesn't use the keys specified in the encoder configuration, it will omit any element whose key is set to the empty string.
func NewJSONEncoder ¶
func NewJSONEncoder(cfg *EncoderConfig) Encoder
NewJSONEncoder just hides the zap json encoder, to avoid exporting zap
func NewLTSVEncoder ¶
func NewLTSVEncoder(cfg *EncoderConfig) Encoder
NewLTSVEncoder creates a fast, low-allocation LTSV encoder.
type EncoderConfig ¶
type EncoderConfig zapcore.EncoderConfig
EncoderConfig captures the options for encoders. Type alias to avoid exporting zap.
func NewDevelopmentEncoderConfig ¶
func NewDevelopmentEncoderConfig() *EncoderConfig
NewDevelopmentEncoderConfig returns an EncoderConfig which is intended for local development.
func NewEncoderConfig ¶
func NewEncoderConfig() *EncoderConfig
NewEncoderConfig returns an EncoderConfig with default settings.
func (*EncoderConfig) UnmarshalJSON ¶
func (enc *EncoderConfig) UnmarshalJSON(b []byte) error
UnmarshalJSON implements json.Marshaler
type Factory ¶
Factory is a log management core. It spawns loggers. The Factory has methods for dynamically reconfiguring all the loggers spawned from Factory.
The flume package has mirrors of most of the functions which delegate to a default, package-level factory.
func NewFactory ¶
func NewFactory() *Factory
NewFactory returns a factory. The default level is set to OFF (all logs disabled)
func (*Factory) ClearHooks ¶ added in v0.13.0
func (r *Factory) ClearHooks()
ClearHooks removes all hooks.
func (*Factory) Configure ¶
Configure uses a serializable struct to configure most of the options. This is useful when fully configuring the logging from an env var or file.
The zero value for Config will set defaults for a standard, production logger:
See the Config docs for details on settings.
func (*Factory) Hooks ¶ added in v0.13.0
Hooks adds functions which are called before a log entry is encoded. The hook function is given the entry and the total set of fields to be logged. The set of fields which are returned are then logged. Hook functions can return a modified set of fields, or just return the unaltered fields.
The Entry is not modified. It is purely informational.
If a hook returns an error, that error is logged, but the in-flight log entry will proceed with the original set of fields.
These global hooks will be injected into all loggers owned by this factory. They will execute before any hooks installed in individual loggers.
func (*Factory) LevelsString ¶
LevelsString reconfigures the log level for all loggers. Calling it with an empty string will reset the default level to info, and reset all loggers to use the default level.
The string can contain a list of directives, separated by commas. Directives can set the default log level, and can explicitly set the log level for individual loggers.
Directives ¶
- Default level: Use the `*` directive to set the default log level. Examples:
// set the default log level to debug -* // set the default log level to off
If the `*` directive is omitted, the default log level will be set to info.
Logger level: Use the name of the logger to set the log level for a specific logger. Examples:
http // set the http logger to debug -http // set the http logger to off http=INF // set the http logger to info
Multiple directives can be included, separated by commas. Examples:
http // set http logger to debug http,sql // set http and sql logger to debug *,-http,sql=INF // set the default level to debug, disable the http logger, // and set the sql logger to info
func (*Factory) NewCore ¶ added in v0.10.0
func (r *Factory) NewCore(name string, options ...CoreOption) *Core
NewCore returns a new Core.
func (*Factory) SetAddCaller ¶
SetAddCaller enables adding the logging callsite (file and line number) to the log entries.
func (*Factory) SetDefaultLevel ¶
SetDefaultLevel sets the default log level for all loggers which don't have a specific level assigned to them
func (*Factory) SetEncoder ¶
SetEncoder sets the encoder for all loggers created by (in the past or future) this factory.
type HookFunc ¶ added in v0.13.0
type HookFunc func(*CheckedEntry, []Field) []Field
HookFunc adapts a single function to the Hook interface.
type Level ¶
Level is a log level
func (Level) MarshalText ¶
MarshalText implements encoding.TextMarshaler
func (*Level) UnmarshalText ¶
UnmarshalText implements encoding.TextUnmarshaler
type Logger ¶
type Logger interface { Debug(msg string, args ...interface{}) Info(msg string, args ...interface{}) Error(msg string, args ...interface{}) IsDebug() bool IsInfo() bool // With creates a new Logger with some context already attached. All // entries logged with the child logger will include this context. With(args ...interface{}) Logger }
Logger is the basic logging interface. Construct instances of Logger with a Factory, or with the package functions (which use a package level Factory).
func FromContext ¶
FromContext returns a logger from the context. If the context doesn't contain a logger, the DefaultLogger will be returned.