logger

package
v0.0.0-...-4017c26 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 3, 2021 License: Apache-2.0 Imports: 9 Imported by: 0

README

logger

Logging package for ProxyFS

About

This package is implemented on top of the logrus package to provide structured logging functionality for ProxyFS. It uses logrus's log fields functionality to generate logs that are easily parsed by log tools.

The purpose of this package is to provide a front-end to the Logrus package to provide standard field-based logging for ProxyFS.

Features

  • The package and function name of the caller are automatically added to the log's fields. See the Log fields section for more detailed information about how log fields are used internal to this package.
  • Logging of commonly-used fields is standardized to make log searches easier.
  • Provides per-calling-package control (in one place) of whether Trace and Debug logs are emitted.
  • Logger is API-compatible with many logrus APIs.

Logging Levels

Logger has seven logging levels, one more (Trace level) than Logrus.

Each Logger level maps to a particular Logrus logging level. Trace logs are sent to Logrus at the Info level.

The intent is to have the ability to have more logging levels than Logrus, with the verbosity controlled by the Logger package. While Logrus has log level controls, we do not use them. Instead we have our own logging controls.

Level Description
Debug Logs intended to debug internal package behaviour. These are not normally enabled by default.
Trace Logs intended to trace the success path through a package. These may or may not be enabled in production code.
Info Logs intended record something that might be useful.
Warning Logs intended to call attention to something.
Error Logs intended to capture an error. This level should only be used for things that the top-level caller would consider an error.
Fatal Log a fatal condition. This will cause the calling process to exit, without a clean shutdown. Should be used very sparingly.
Panic Log a panic condition. This will cause the calling process to panic. Note that golang panics do not generate a core, so put as much information as possible into the log text.

Logging verbosity control

Logs at Info, Warning, Error, Fatal and Panic levels are always on.

Logs at Trace and Debug levels have per-package controls.

The emission of trace logs can be set to true or false on a per-calling-package basis. If a trace setting for the logged package cannot be found, trace logging for that package is considered to be turned off.

NOTE:

TODO: Add support for setting trace and debug logging configuration via .conf files

Trace logs

The packageTraceSettings map controls whether tracing is enabled for particular packages. If a package is in this map and is set to "true", then tracing for that package is considered to be enabled and trace logs for that package will be emitted. If the package is in this list and is set to "false", OR if the package is not in this list, trace logs for that package will NOT be emitted.

Debug logs

The packageDebugSettings map controls which debug logs are enabled for particular packages. Unlike trace settings, debug settings are stored as a list of enabled debug tags.

The intent of using a map of tags like this is to be able to turn on/off debug logs for subsets of functionality within a package so that one doesn't have to turn on traces that aren't needed.

Bear in mind that these tags are evaluated on a package + tag basis, so the same tag can be used on different packages without conflict.

Example

The simplest way to use Logger is as a replacement for the stdlib logger:

package main

import (
	"github.com/swiftstack/ProxyFS/logger"
)

func main() {
  receivedSignal := <-signalChan
  logger.Infof("Received signal %v", receivedSignal)
}

Caveats

  • Logger APIs should not be called if one has not started up the logger package by calling logger.Up().

APIs

API Description
Error,Info Same as the Logrus version of these APIs, except function and package fields are added
Errorf, Fatalf, Infof, Warnf Same as the Logrus version of these APIs, except function and package fields are added
Tracef Behaves like *f APIs, at Trace level. Function and package fields are added.
ErrorWithError, FatalWithError, InfoWithError, WarnWithError Adds support to error field. Otherwise functionally equivalent to the non-WithError versions of these APIs
TraceWithError Behaves like *WithError APIs, at Trace level.
ErrorfWithError, FatalfWithError, InfofWithError, PanicfWithError, WarnfWithError A combination of the *f and *WithError APIs
TracefWithError *fWithError API at Trace level.
ErrorfWithErrorTxnID, InfofWithErrorTxnID Adds support for a transaction ID field to *fWithError APIs.
DebugID Adds support for a debug ID field to the Debug API. Debug IDs are how we control whether debug logs are emitted.
DebugfID Adds support for a debug ID field to the Debugf API. Debug IDs are how we control whether debug logs are emitted.
DebugfIDWithTxnID Adds support for a transaction ID field to DebugfID API.
DebugfIDWithError Adds support for the error field to DebugfID API.
TraceEnter, TraceEnterDeferred APIs for tracing function entry.
TraceExit, TraceExitErr APIs for tracing function exit.

** The following debug-related APIs are intentionally not supported **

The following plain Debug* APIs are not supported, since all Debug logging must be done using a debug ID.

Not supported API Suggested Alternative API
Debug DebugID
Debugf DebugfID
DebugWithError DebugIDWithError
DebugfWithError DebugfIDWithError

Internal design notes

Log Fields

As described in logrus fields documentation, the use of log fields allows careful, structured logging instead of long, hard-to-parse error messages.

For example, instead of:

log.Infof("%s: (txn = %v) error = %v", utils.GetFnName(), scc.hc.transID, err)

(with the corresponding log)

time="2016-08-16T22:41:49Z" level=info msg="inode.Close: (txn = 45362) error = hc.setupPutChunkedBegin error"

we now do

logger.InfofWithErrorTxnID(err, scc.hc.transIDStr(), "error")

(with the corresponding log)

time="2016-08-16T22:41:49Z" level=info msg="error" error="hc.setupPutChunkedBegin error" function=Close package=inode ss_transid=45362

The use of log fields make it much easier to parse log files, either manually or with log parsing tools.

The log fields that are currently supported by this package are:

  • package: the package of the calling function
  • function: the calling function
  • error
  • ss_transid: swift transaction ID for inode/sock_swift logic

Function context

FuncCtx is a context structure that is used by some of the Logger APIs. This struct is an optimization so that package and function are only extracted once per function.

The package, function and other fields are stored in the FuncCtx in a logrus.Entry*.

This construct is mainly used inside the logger package, but is used by TraceExit/TraceExitErr as well.

Documentation

Overview

Package logger provides logging wrappers

These wrappers allow us to standardize logging while still using a third-party logging package.

This package is currently implemented on top of the sirupsen/logrus package: 

https://github.com/sirupsen/logrus

The APIs here add package and calling function to all logs.

Where possible, fields passed to logs are parameterized, to standardize log formatting to make log searches based on these fields easier.

Logging of trace and debug logs are enabled/disabled on a per package basis.

Index

Constants

View Source 
const DbgInodeInternal string = "debug_inode_internal"

packageDebugSettings controls which debug logs are enabled for particular packages.

If a package is in this map, then debug logging for that package is considered to be enabled for debug tags in the package's map entry. If a debug log's tag is NOT in this list, OR if the package is not in the map, debug logs for that package will NOT be emitted.

Unlike trace settings, debug settings are stored as a list of enabled debug tags. Bear in mind that these tags are evaluated on a package + tag basis, so the same tag can be used on different packages without conflict.

View Source 
const DbgInternal string = "debug_internal"
View Source 
const DbgTesting string = "debug_test"

Variables

This section is empty.

Functions

func AddLogTarget 

func AddLogTarget(writer io.Writer)

Add another target for log messages to be written to. writer is an object with an io.Writer interface that's called once for each log message.

Logger.Up() must be called before this function is used.

func DebugID 

func DebugID(id string, args ...interface{})

func DebugfID 

func DebugfID(id string, format string, args ...interface{})

func DebugfIDWithError 

func DebugfIDWithError(id string, err error, format string, args ...interface{})

func DebugfIDWithTxnID 

func DebugfIDWithTxnID(id string, transIDStr string, format string, args ...interface{})

func Down 

func Down(confMap conf.ConfMap) (err error)

func Error 

func Error(args ...interface{})

func ErrorWithError 

func ErrorWithError(err error, args ...interface{})

func Errorf 

func Errorf(format string, args ...interface{})

func ErrorfWithError 

func ErrorfWithError(err error, format string, args ...interface{})

func ErrorfWithErrorTxnID 

func ErrorfWithErrorTxnID(err error, transIDStr string, format string, args ...interface{})

func FatalWithError 

func FatalWithError(err error, args ...interface{})

func Fatalf 

func Fatalf(format string, args ...interface{})

func FatalfWithError 

func FatalfWithError(err error, format string, args ...interface{})

func Info 

func Info(args ...interface{})

func InfoWithError 

func InfoWithError(err error, args ...interface{})

func Infof 

func Infof(format string, args ...interface{})

func InfofWithError 

func InfofWithError(err error, format string, args ...interface{})

func InfofWithErrorTxnID 

func InfofWithErrorTxnID(err error, transIDStr string, format string, args ...interface{})

func PanicfWithError 

func PanicfWithError(err error, format string, args ...interface{})

func ParseLogEntry 

func ParseLogEntry(entry string) (map[string]string, error)

Given a log entry captured by LogTarget (above) parse out the various fields and return them as a map.

The fields parsed are "time", "level", "msg", "error", "function", "goroutine", and "package". Not all fields are present in every message, in which case the field name is not a key in the map.

func ParseLogForFunc 

func ParseLogForFunc(logcopy LogTarget, funcName string, logEntryRE *regexp.Regexp, maxEntries int) (fields map[string]string, entryIdx int, err error)

Parse the log entries, starting with the most recent, looking for a message generated by a function named funcName and a "msg=" that matches the regular expression logEntryRE, within the most recent maxEntries lines of the log.

If found, return the parsed fields from the log message, which are a combination of the fields returned by ParseLogEntry() and the fields in the passed regular expression (which must use names for the matching parts). funcName must match the contents of the field "function" returned by ParseLogEntry(). entryIdx is the index of the entry in the log with 0 being the most recent.

If not found, return an error.

Example: logEntryRE can be very simple, but here is a complicated one from trackedlock/api_test.go that matches the following message (where "\n" in the message represents embededded newlines).

msg="trackedlock watcher: *trackedlock.Mutex at 0xc420110000 locked for 2.0 sec; stack at Lock() call:\ngoroutine 19 [running]:..."

logEntryRe is `^trackedlock watcher: (?P<type>[*a-zA-Z0-9_.]+) at (?P<ptr>0x[0-9a-f]+) locked for (?P<time>[0-9.]+) sec rank 0; stack at call to (?P<locker>[a-zA-Z0-9_()]+):\\n(?P<lockStack>.*)$`

func SignaledFinish 

func SignaledFinish(confMap conf.ConfMap) (err error)

func SignaledStart 

func SignaledStart(confMap conf.ConfMap) (err error)

func TraceWithError 

func TraceWithError(err error, args ...interface{})

func Tracef 

func Tracef(format string, args ...interface{})

func TracefWithError 

func TracefWithError(err error, format string, args ...interface{})

func Up 

func Up(confMap conf.ConfMap) (err error)

func WarnWithError 

func WarnWithError(err error, args ...interface{})

func Warnf 

func Warnf(format string, args ...interface{})

func WarnfWithError 

func WarnfWithError(err error, format string, args ...interface{})

Types

type FuncCtx 

type FuncCtx struct {
	// contains filtered or unexported fields
}

This struct is an optimization so that package and function are only extracted once per function.

func TraceEnter 

func TraceEnter(argsPrefix string, args ...interface{}) (ctx FuncCtx)

func (*FuncCtx) TraceExit 

func (ctx *FuncCtx) TraceExit(argsPrefix string, args ...interface{})

TraceExit generates a function exit trace with the provided parameters, and using the package and function set in FuncCtx (if set).

The implementation of this function assumes that it will be called deferred. This assumption only matters in the case where this API is called without having called TraceEnter* first, since that is the only case where this function will attempt to determine the calling function.

func (*FuncCtx) TraceExitErr 

func (ctx *FuncCtx) TraceExitErr(argsPrefix string, err error, args ...interface{})

type Level 

type Level int
const (
	// PanicLevel corresponds to logrus.PanicLevel; Logrus will log and then call panic with the log message
	PanicLevel Level = iota
	// FatalLevel corresponds to logrus.FatalLevel; Logrus will log and then calls `os.Exit(1)`.
	// It will exit even if the logging level is set to Panic.
	FatalLevel
	// ErrorLevel corresponds to logrus.ErrorLevel
	ErrorLevel
	// WarnLevel corresponds to logrus.WarnLevel
	WarnLevel
	// InfoLevel corresponds to logrus.InfoLevel; These are general operational entries about what's going on inside the application.
	InfoLevel

	// TraceLevel is used for operational logs that trace success path through the application.
	// Whether these are logged is controlled on a per-package basis by settings in this file.
	// When enabled, these are logged at logrus.InfoLevel.
	TraceLevel

	// DebugLevel is used for very verbose logging, intended to debug internal operations of a
	// particular area. Whether these are logged is controlled on a per-package basis by settings
	// in this file.
	// When enabled, these are logged at logrus.DebugLevel.
	DebugLevel
)

Our logging levels - These are the different logging levels supported by this package.

We have more detailed logging levels than the logrus log package. As a result, when we do our logging we need to map from our levels to the logrus ones before calling logrus APIs.

type LogBuffer 

type LogBuffer struct {
	sync.Mutex
	LogEntries   []string // most recent log entry is [0]
	TotalEntries int      // count of all entries seen
}

An example of a log target that captures the most recent n lines of log into an array. Useful for writing test cases.

There should really be a lock or clever RCU mechanism to coordinate access/updates to the array, but its not really necessary for test case code (and adds overhead).

type LogTarget 

type LogTarget struct {
	LogBuf *LogBuffer
}

func (*LogTarget) Init 

func (log *LogTarget) Init(nEntry int)

Initialize a LogTarget to hold the last nEntry log entries.

func (LogTarget) Write 

func (log LogTarget) Write(p []byte) (n int, err error)

Called by logger for each log entry

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.