clog

package
v0.0.0-...-f5c3557 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2018 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package clog implements a python-like, module-based logger with a variety of backends and formats.

This logger is based on Python's fantastic logging package, though it intends to be far simpler. Every part of the system should have its own individual logger, specific by a module name, so that its messages can be easily distinguished from other parts. In golang's log package, people have typically resorted to prefixing messages with things like "[INFO module]", but that is tedious, error prone, rather un-enforceable for external pacakges, and impossible to serialize for all logging services.

Clog takes the approach that data should be represented as data, not strings (there are methods such as Infod, Warnd, etc), and this data is properly serialized for all output formats. Each message includes a timestamp, the module that generated the message, the level, and the source file and line.

Like python's logging, clog modules are heirachical, in that rules for something like "server.http" will be applied to "server.http.requests" and "server.http.responses" as the messages propogate. Each module may have multiple outputs, so you may log to both a file and logstash for the same module, if you wish.

Configuration

Configuring logging packages has always been a rather daunting affair, but clog takes great pains to be as simple as possible.

First, there are 2 concepts in logging: Outputs and Modules. Outputs are where log messages are written to, ie. a file, an external service, syslog, etc. Modules are where messages come from, ie. "http" for the http interface, "library.name" for some library named "name". You may choose your own values for these, just know that they're arranged in a tree, such that "http.request" is a child of "http" and messages from it propagate to "http".

Since examples are worth more than descriptions, let's take a look at a pretty complex configuration, with comments explaining how it all works together (this is much simpler to write in JSON, but you get the point). 

Config{
    // If set, this creates a new root module (the module named "" (the empty
    // string)), and it records any message level >= Info to the named file in
    // JSON format.
    File: "/var/log/app.log",

    Outputs: map[string]*OutputConfig{
        // Only errors with level >= Error will be logged here
        "errors": {
            Prod: "file",
            ProdArgs: eio.Args{
                "path": "/var/log/app.error.json.log",
            },
            Fmt:   "json",
            Level: Error,
            Filters: []FilterConfig{
                FilterConfig{
                    Which: "exampleFilter",
                    Args: eio.Args{
                        "exampleConfig": "someValue",
                    },
                },
            },
        },

        // All messages will be accepted here
        "debug": {
            Prod: "file",
            ProdArgs: eio.Args{
                "path": "/var/log/app.log.json",
            },
            Fmt:   "human", // Or "logfmt", or any other valid formatter
            Level: Debug,
        },

        // Only errors level >= Warn will be accepted here
        "heroku": {
            Prod: "file",
            ProdArgs: eio.Args{
                "path": "/var/log/app.logfmt",
            },
            Fmt: "logfmt",
            FmtArgs: eio.Args{
                "ShortTime": true,
            },
            Level: Warn,
        },
    },

    Modules: map[string]*ModuleConfig{
        // All messages eventually reach here, unless DontPropagate==true in a
        // module
        "": {
            Outputs: []string{"errors"},
        },

        // This logs all messages level >= Info, where the filter allows the
        // message through, to the debug log. These messages do not propagate to
        // the root.
        "http": {
            Outputs: []string{"debug"},
            Level:   Info,
            Filters: []FilterConfig{
                FilterConfig{
                    Which: "exampleFilter",
                    Args: eio.Args{
                        "exampleConfig": "someValue",
                    },
                },
            },
            DontPropagate: true,
        },

        // This logs all messages level >= Warn, to both the heroku and errors
        // outputs. These messages do not propagate to the root.
        "templates": {
            Outputs:       []string{"heroku", "errors"},
            Level:         Warn,
            DontPropagate: true,
        },

        // This logs all messages from the external library to the debug log.
        // These messages also propagate to the root, which will log any error
        // messages. So, effectively, errors from this module will be logged
        // twice.
        "external.library": {
            Outputs: []string{"debug"},
            Level:   Debug,
        },
    },
}

Producers

A full list of Producers can be found at: https://godoc.org/github.com/iheartradio/cog/cio/eio

Filters

The following filters are available (their names are case-insensitive): 

"Level"  LevelFilter

Formatters

The following formatters exist; their arguments are specified in the listed classes. Like everything else, these names are case-insensitive. 

"Human"   HumanFormat
"JSON"    JSONFormat
"LogFmt"  LogFmtFormat

Testing

If you want log output to go to a test's log, check out: https://godoc.org/github.com/iheartradio/cog/check/chlog

Example (File) 
package main

import (
	"fmt"
	"io/ioutil"
	"os"
	"strings"

	"github.com/iheartradio/cog/clog"
)

func main() {
	path := "example_file.log"
	cfg := clog.Config{
		File: path,
	}

	l, err := clog.New(cfg)
	if err != nil {
		panic(err)
	}

	defer os.Remove(path)

	lg := l.Get("example")

	lg.Debug("Bug bug bug bug")
	lg.Info("This is a very informative message")
	lg.WithKV("variable", 1234).Warn("I must warn you, numbers are scary")

	contents, err := ioutil.ReadFile(path)
	if err != nil {
		panic(err)
	}

	log := string(contents)
	fmt.Println("contains debug", strings.Contains(log, "Bug bug bug bug"))
	fmt.Println("contains info", strings.Contains(log, "informative message"))
	fmt.Println("contains warn", strings.Contains(log, "I must warn you"))

}

Output:

contains debug false
contains info true
contains warn true
Example (Stdout) 
package main

import (
	"strings"

	"github.com/iheartradio/cog/cio/eio"
	"github.com/iheartradio/cog/clog"
)

// Rejects any messages that might be insulting
type insultFilter struct{}

func (insultFilter) Accept(e clog.Entry) bool {
	return !strings.Contains(strings.ToLower(e.Msg), "i hate you")
}

func (insultFilter) Exit() {
	// This filter has nothing to cleanup, so nothing to do here
}

func init() {
	clog.RegisterFilter("insult",
		func(args eio.Args) (clog.Filter, error) {
			// If args were used here, args.ApplyTo might come in handy
			return insultFilter{}, nil
		})
}

func main() {
	cfg := clog.Config{
		Outputs: map[string]*clog.OutputConfig{
			"stdout": {
				Prod: "stdout",
				Fmt:  "human",
				FmtArgs: eio.Args{
					"ShortTime": true,
				},
				Level: clog.Info,
			},
		},

		Modules: map[string]*clog.ModuleConfig{
			"": {
				Outputs: []string{"stdout"},
			},
			"rude.module": {
				Outputs: []string{"stdout"},
				Filters: []clog.FilterConfig{
					clog.FilterConfig{
						Which: "insult",
					},
				},
				DontPropagate: true,
			},
		},
	}

	l, err := clog.New(cfg)
	if err != nil {
		panic(err)
	}

	polite := l.Get("polite.module")
	polite.Info("You're very pretty")
	polite.Info("I like you")

	rude := l.Get("rude.module")
	rude.Info("I hate you")
	rude.Info("You're ugly and I hate you")
	rude.Error("I'm better than you")

}

Output:

[000000] I-polite.module : example_stdout_test.go:64 : You're very pretty
[000000] I-polite.module : example_stdout_test.go:65 : I like you
[000000] E-rude.module : example_stdout_test.go:70 : I'm better than you

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func DumpKnownFilters 

func DumpKnownFilters(w io.Writer)

DumpKnownFilters writes all known filters and their names to the given Writer.

func Hostname 

func Hostname() string

Hostname provides a cached version of os.Hostname() (call that a ton of times is probably a bad idea).

func RegisterFilter 

func RegisterFilter(name string, nf NewFilter)

RegisterFilter adds a Filter to the list of filters

func RegisterFormatter 

func RegisterFormatter(name string, nf NewFormatter)

RegisterFormatter adds a Formatter to the list of formats

Types

type Config 

type Config struct {
	// File is the simplest way of configuring this logger. It sets up a
	// JSONFile writing to the given path, with a root logger that only accepts
	// Info and greater.
	File string

	// Identifies all of the places where log entries are written. They keys in
	// this map name the output.
	Outputs map[string]*OutputConfig

	// Identifies all modules that you want to configure. They keys in this map
	// identify the module to work on.
	//
	// If no modules are given, everything at level Info and above goes to the
	// terminal by default.
	Modules map[string]*ModuleConfig
}

Config specifies basic config for logging.

The Config struct is meant to be embedded directly into some other struct that you're Unmarshaling your application's config into (typically, this is a struct that is being filled by json.Unmarshal, yaml.Unmarshal, etc on application start).

type Ctx 

type Ctx struct {
	// contains filtered or unexported fields
}

Ctx provides access to the logging facilities

func New 

func New(cfg Config) (ctx *Ctx, err error)

New creates a new Log

func NewFromFile 

func NewFromFile(path string) (ctx *Ctx, err error)

NewFromFile creates a new Log, configured from the given file. The file type is determined by the extension.

func NewFromJSONReader 

func NewFromJSONReader(r io.Reader) (ctx *Ctx, err error)

NewFromJSONReader creates a new Log configured from JSON in the given reader.

func (*Ctx) Flush 

func (ctx *Ctx) Flush() error

Flush can be used by servers that exit gracefully. It causes all Outputs to write anything pending. It blocks until done.

If it returns an error, nothing was flushed.

func (*Ctx) Get 

func (ctx *Ctx) Get(name string) *Log

Get a Log for the given module

func (*Ctx) Reconfigure 

func (ctx *Ctx) Reconfigure(cfg Config) error

Reconfigure reconfigures the entire logging system from the ground up. All active loggers are affected immediately, and all changes are applied atomically. If reconfiguration fails, the previous configuration remains.

func (*Ctx) ReconfigureFromFile 

func (ctx *Ctx) ReconfigureFromFile(path string) error

ReconfigureFromFile reconfigures the Log from the given file. The file type is determined by the extension.

func (*Ctx) ReconfigureFromJSONReader 

func (ctx *Ctx) ReconfigureFromJSONReader(r io.Reader) error

ReconfigureFromJSONReader reconfigures the Log from the JSON in the given reader.

func (*Ctx) Rotate 

func (ctx *Ctx) Rotate() error

Rotate causes all outputters to rotate their files, if they have any. When using an external log rotator (eg. logrotated), this is what you're looking for to use in postrotate.

func (*Ctx) Stats 

func (ctx *Ctx) Stats() (ss []Stats)

Stats gets the log stats since the last time this was called.

Don't call this directly; it's meant for use with statc.

type Data 

type Data map[string]interface{}

Data is a collection of fields to add to a log entry

type Entry 

type Entry struct {
	// When this Entry was originally logged
	Time time.Time

	// Which module this Entry belongs to
	Module string

	// Level of this Entry
	Level Level

	// Source file and line from where this was logged
	Src string

	// How much of the call stack to ignore when looking for a file:lineno
	Depth int `json:"-"`

	// Formatted text
	Msg string

	// Name of the host this originated on
	Host string

	// Data to include with the Entry
	Data Data
	// contains filtered or unexported fields
}

Entry is one complete log entry

type Filter 

type Filter interface {
	// Checks whether or not this Entry should be accepted
	Accept(Entry) bool

	// Just in case you need to cleanup when no longer needed
	Exit()
}

A Filter determines which log entries are allowed through

type FilterConfig 

type FilterConfig struct {
	// Which the filter to use. This value is case-insensitive.
	Which string

	// Filter arguments
	Args eio.Args
}

FilterConfig is for setting up a Filter

type Formatter 

type Formatter interface {
	// Format a message.
	FormatEntry(Entry) ([]byte, error)

	// Get the MimeType of data produced by this Formatter
	MimeType() string
}

Formatter formats messages

type HumanFormat 

type HumanFormat struct {
	Args struct {
		ShortTime bool // Format timestamp as "seconds since start"
	}
}

HumanFormat formats log entries so that a human can quickly decipher them

func (HumanFormat) FormatEntry 

func (f HumanFormat) FormatEntry(e Entry) ([]byte, error)

FormatEntry implements Formatter.FormatEntry

func (HumanFormat) MimeType 

func (HumanFormat) MimeType() string

MimeType implements Formatter.MimeType

type JSONFormat 

type JSONFormat struct{}

JSONFormat formats messages as JSON

func (JSONFormat) FormatEntry 

func (JSONFormat) FormatEntry(e Entry) ([]byte, error)

FormatEntry implements Formatter.FormatEntry

func (JSONFormat) MimeType 

func (JSONFormat) MimeType() string

MimeType implements Formatter.MimeType

type Level 

type Level int8

Level is a typical log level 

const (
	// Debug is the finest level: it contains things that are only useful for
	// debugging
	Debug Level = iota

	// Info is for general information that might be useful during runtime
	Info

	// Warn tells about something that doesn't seem right
	Warn

	// Error is a recoverable runtime error
	Error

	// Panic causes the current goroutine to panic after logging the message
	Panic
)

func (Level) MarshalJSON 

func (l Level) MarshalJSON() ([]byte, error)

MarshalJSON implements JSON.Marshaler

func (*Level) Parse 

func (l *Level) Parse(s string) error

Parse parses a log level from a string.

func (Level) Rune 

func (l Level) Rune() rune

Rune gets a single-letter representation of this level

func (Level) String 

func (l Level) String() string

String gets a human-readble version of this level

func (*Level) UnmarshalJSON 

func (l *Level) UnmarshalJSON(data []byte) error

UnmarshalJSON implements JSON.Unmarshaler

type LevelFilter 

type LevelFilter struct {
	Args struct {
		// Don't log anything below this level
		Level Level
	}
}

LevelFilter is the filter used by the required "Level" argument for both Modules and Outputs and is typically not used directly.

Example: 

Filters: []FilterConfig{
    FilterConfig{
        Which: "Level",
        Args: eio.Args{
            "level": clog.Info,
        },
    },
}

func (LevelFilter) Accept 

func (lf LevelFilter) Accept(e Entry) bool

Accept implements Filter.Accept()

func (LevelFilter) Exit 

func (lf LevelFilter) Exit()

Exit implements Filter.Exit()

type Log 

type Log struct {
	// contains filtered or unexported fields
}

Log provides a shared log.

func (Log) Debug 

func (l Log) Debug(args ...interface{})

func (Log) Debugf 

func (l Log) Debugf(format string, args ...interface{})

func (Log) Error 

func (l Log) Error(args ...interface{})

func (Log) Errorf 

func (l Log) Errorf(format string, args ...interface{})

func (Log) Info 

func (l Log) Info(args ...interface{})

func (Log) Infof 

func (l Log) Infof(format string, args ...interface{})

func (Log) Panic 

func (l Log) Panic(args ...interface{})

func (Log) Panicf 

func (l Log) Panicf(format string, args ...interface{})

func (Log) Warn 

func (l Log) Warn(args ...interface{})

func (Log) Warnf 

func (l Log) Warnf(format string, args ...interface{})

func (Log) With 

func (l Log) With(d Data) Logger

func (Log) WithKV 

func (l Log) WithKV(k string, v interface{}) Logger

type LogFmtFormat 

type LogFmtFormat struct{}

LogFmtFormat formats messages in heroku's LogFmt

func (LogFmtFormat) FormatEntry 

func (LogFmtFormat) FormatEntry(e Entry) ([]byte, error)

FormatEntry implements Formatter.FormatEntry

func (LogFmtFormat) MimeType 

func (LogFmtFormat) MimeType() string

MimeType implements Formatter.MimeType

type Logger 

type Logger interface {
	// With merges the given data into any current data and returns a new
	// Logger with the resulting data. New values replace older ones with the
	// same key.
	With(d Data) Logger

	// Add the KV pair and get a new Logger.
	WithKV(k string, v interface{}) Logger

	// Debug entries: things that can be ignored in production
	Debug(args ...interface{})
	Debugf(format string, args ...interface{})

	// Info: stuff that is useful, even in production
	Info(args ...interface{})
	Infof(format string, args ...interface{})

	// Something might start going wrong soon
	Warn(args ...interface{})
	Warnf(format string, args ...interface{})

	// Something went wrong
	Error(args ...interface{})
	Errorf(format string, args ...interface{})

	// A horrible error happened.
	Panic(args ...interface{})
	Panicf(format string, args ...interface{})
}

A Logger provides a simplified Log

type LogstashFormat 

type LogstashFormat struct{}

LogstashFormat formats messages as JSON, with special fields for logstash

func (LogstashFormat) FormatEntry 

func (LogstashFormat) FormatEntry(e Entry) ([]byte, error)

FormatEntry implements Formatter.FormatEntry

func (LogstashFormat) MimeType 

func (LogstashFormat) MimeType() string

MimeType implements Formatter.MimeType

type ModuleConfig 

type ModuleConfig struct {
	// The list of outputs to write to. These values come from the keys in the
	// Outputs map in Config.
	Outputs []string

	// Log level to use for this output. Use Debug to accept all. This is
	// actually an implicit (and required) Filter.
	Level Level

	// Which filters to apply to this module
	Filters []FilterConfig

	// By default, messages are propagated until the root logger. If you want
	// messages to stop here, set this to True.
	DontPropagate bool
}

ModuleConfig specifies how a module to to be treated

type NewFilter 

type NewFilter func(args eio.Args) (Filter, error)

NewFilter creates new, configured Filters.

type NewFormatter 

type NewFormatter func(args eio.Args) (Formatter, error)

NewFormatter creates a new, configured Formatter.

type OutputConfig 

type OutputConfig struct {
	// Which Producer to use. This value is case-insensitive.
	//
	// The full list of Producers can be found at:
	// https://godoc.org/github.com/iheartradio/cog/cio/eio
	Prod     string
	ProdArgs eio.Args // Args to pass to the Producer

	// Which Formatter to use for this output.
	Fmt     string
	FmtArgs eio.Args // Args to pass to the Fmt

	// Log level to use for this output. Use Debug to accept all. This is
	// actually an implicit (and required) Filter.
	Level Level

	// Which filters to apply to this output.
	Filters []FilterConfig
}

OutputConfig specifies how an output is to be built

type Stats 

type Stats struct {
	Module string
	Counts [lvlLen]int64
}

Stats about which module has logged how many times at which levels

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.