logger

README

Logger Package

The logger package provides a structured, context-aware logging solution for Go applications. It is built on top of the logrus library and is designed to facilitate easy integration with your projects, offering features like:

• JSON-formatted logs suitable for production environments.
• Support for multiple log levels (DEBUG, INFO, WARN, ERROR, FATAL).
• Context propagation to include tracing information (e.g., trace_id, span_id).
• Customizable log formatters and output destinations.
• Integration with web frameworks like Gin.

Features

• Structured Logging: Outputs logs in JSON format, making them easy to parse and analyze.
• Context-Aware: Supports logging with context.Context, allowing you to include tracing information automatically.
• Customizable Formatter: Use the default StructuredJSONFormatter or provide your own formatter to customize the log output.
• Environment and Service Name: Optionally include environment and service name in your logs for better traceability.
• No-Op Logger: Provides a no-operation logger for testing purposes, which discards all log messages.

Usage

Creating a Logger

You can create a logger using the NewLogger function, providing a Config struct to customize its behavior:

import (
    "github.com/kittipat1413/go-common/framework/logger"
    "time"
)

logConfig := logger.Config{
    Level: logger.INFO,
    Formatter: &logger.StructuredJSONFormatter{
        TimestampFormat: time.RFC3339,
        PrettyPrint:     false,
    },
    Environment: "production",
    ServiceName: "my-service",
}

log, err := logger.NewLogger(logConfig)
if err != nil {
    panic(err)
}

Alternatively, you can use the default logger:

log := logger.NewDefaultLogger()

• The NewDefaultLogger returns a logger instance with the default or user-defined configuration.
• If SetDefaultLoggerConfig has been called, it uses the user-defined configuration; otherwise, it uses the package's default configuration.

Updating the Default Logger Configuration

You can update the default logger configuration using SetDefaultLoggerConfig:

err := logger.SetDefaultLoggerConfig(logConfig)
if err != nil {
    // Handle error
    panic(err)
}

Configuration

The Config struct allows you to customize the logger:

type Config struct {
	// Level determines the minimum log level that will be processed by the logger.
	// Logs with a level lower than this will be ignored.
	Level LogLevel
	// Formatter is an optional field for specifying a custom logrus formatter.
	// If not provided, the logger will use the StructuredJSONFormatter by default.
	Formatter logrus.Formatter
	// Environment is an optional field for specifying the running environment (e.g., "production", "staging").
	// This field is used for adding environment-specific fields to logs.
	Environment string
	// ServiceName is an optional field for specifying the name of the service.
	// This field is used for adding service-specific fields to logs.
	ServiceName string
	// Output is an optional field for specifying the output destination for logs (e.g., os.Stdout, file).
	// If not provided, logs will be written to stdout by default.
	Output io.Writer
}

Logging Messages

The logger provides methods for different log levels:

type Logger interface {
    WithFields(fields Fields) Logger
	Debug(ctx context.Context, msg string, fields Fields)
	Info(ctx context.Context, msg string, fields Fields)
	Warn(ctx context.Context, msg string, fields Fields)
	Error(ctx context.Context, msg string, err error, fields Fields)
	Fatal(ctx context.Context, msg string, err error, fields Fields)
}

Example:

ctx := context.Background()
fields := logger.Fields{"user_id": 12345}

log.Info(ctx, "User logged in", fields)

Including Errors

For error and fatal logs, you can include an error object:

err := errors.New("something went wrong")
log.Error(ctx, "Failed to process request", err, fields)

Adding Persistent Fields

You can add persistent fields to the logger using WithFields, which returns a new logger instance:

logWithFields := log.WithFields(logger.Fields{
    "component": "authentication",
})
logWithFields.Info(ctx, "Authentication successful", nil)

You can find a complete working example in the repository under framework/logger/example.

---

StructuredJSONFormatter

The StructuredJSONFormatter is a custom logrus.Formatter designed to include contextual information in logs. It outputs logs in JSON format with a standardized structure, making it suitable for log aggregation and analysis tools.

Features

• Timestamp: Includes a timestamp formatted according to TimestampFormat.
• Severity: The log level (debug, info, warning, error, fatal).
• Message: The log message.
• Error Handling: Automatically includes error messages if an error is provided.
• Tracing Information: Extracts trace_id and span_id from the context if available (e.g., when using OpenTelemetry).
• Caller Information: Adds information about the function, file, and line number where the log was generated.
• Stack Trace: Includes a stack trace for logs at the error level or higher.
• Custom Fields: Supports additional fields provided via logger.Fields.
• Field Key Customization: Allows custom formatting of field keys via FieldKeyFormatter.

Configuration

You can customize the StructuredJSONFormatter when initializing the logger:

import (
    "github.com/kittipat1413/go-common/framework/logger"
    "time"
)

formatter := &logger.StructuredJSONFormatter{
    TimestampFormat: time.RFC3339, // Customize timestamp format
    PrettyPrint:     true,         // Indent JSON output
    FieldKeyFormatter: func(key string) string {
        // Customize field keys
        switch key {
        case logger.DefaultEnvironmentKey:
            return "env"
        case logger.DefaultServiceNameKey:
            return "service"
        case logger.DefaultSJsonFmtSeverityKey:
            return "level"
        case logger.DefaultSJsonFmtMessageKey:
            return "msg"
        default:
            return key
        }
    },
}

logConfig := logger.Config{
    Level:     logger.INFO,
    Formatter: formatter,
}

Example Log Entry (default FieldKeyFormatter)

{
  "caller": {
    "file": "/go-common/framework/logger/example/gin_with_logger/main.go:99",
    "function": "main.handlerWithLogger"
  },
  "environment": "development",
  "message": "Handled HTTP request",
  "request": {
    "method": "GET",
    "url": "/log"
  },
  "request_id": "afa241ba-cb59-4053-a1f5-d82e6193790c",
  "response_time": 0.000026542,
  "service_name": "logger-example",
  "severity": "info",
  "span_id": "94e92f0e1b8532e6",
  "status": 200,
  "timestamp": "2024-10-20T02:01:57+07:00",
  "trace_id": "f891fd44c417fc7efa297e6a18ddf0cf"
}

{
  "caller": {
    "file": "/go-common/framework/logger/example/gin_with_logger/main.go:78",
    "function": "main.logMessages"
  },
  "environment": "development",
  "error": "example error",
  "example_field": "error_value",
  "message": "This is an error message",
  "service_name": "logger-example",
  "severity": "error",
  "stack_trace": "goroutine 1 [running]:\ngithub.com/kittipat1413/go-common/framework/logger.getStackTrace()\n\t/Users/kittipat/go-github-repo/go-common/framework/logger/structured_json_formatter.go:181 .....\n",
  "timestamp": "2024-10-20T02:01:55+07:00"
}

Tracing Integration

The StructuredJSONFormatter can extract tracing information (trace_id and span_id) from the context.Context if you are using a tracing system like OpenTelemetry. Ensure that spans are started and the context is propagated correctly.

Caller and Stack Trace

• Caller Information: The formatter includes the function name, file, and line number where the log was generated, aiding in debugging.
• Stack Trace: For logs at the error level or higher, a stack trace is included. This can be useful for diagnosing issues in production.

---

Custom Formatter

If you need a different format or additional customization, you can implement your own formatter by satisfying the logrus.Formatter interface and providing it to the logger configuration.

type MyCustomFormatter struct {
    // Custom fields...
}

func (f *MyCustomFormatter) Format(entry *logrus.Entry) ([]byte, error) {
    // Custom formatting logic...
}

Usage:

logConfig := logger.Config{
    Formatter: &MyCustomFormatter{},
}

No-Op Logger

For testing purposes, you can use the no-operation logger, which implements the Logger interface but discards all log messages:

log := logger.NewNoopLogger()

This can be useful to avoid cluttering test output with logs or when you need a logger that does nothing.

Example

You can find a complete working example in the repository under framework/logger/example.

Documentation

Index

• Constants
• Variables
• func NewContext(ctx context.Context, logger Logger) context.Context
• func NewRequest(r *http.Request, logger Logger) *http.Request
• func NoopFieldKeyFormatter(defaultKey string) string
• func SetDefaultLoggerConfig(config Config) error
• type Config
• type FieldKeyFormatter
• type Fields
• type LogLevel
  • func (l LogLevel) IsValid() bool
  • func (l LogLevel) ToLogrusLevel() logrus.Level
• type Logger
  • func FromContext(ctx context.Context) Logger
  • func FromRequest(r *http.Request) Logger
  • func NewDefaultLogger() Logger
  • func NewLogger(config Config) (Logger, error)
  • func NewNoopLogger() Logger
• type StructuredJSONFormatter
  • func (f *StructuredJSONFormatter) Format(entry *logrus.Entry) ([]byte, error)

Constants

const (
	// DefaultEnvironmentKey is the default key used for the environment field in logs.
	DefaultEnvironmentKey = "environment"
	// DefaultServiceNameKey is the default key used for the service name field in logs.
	DefaultServiceNameKey = "service_name"
	// DefaultErrorKey is the default key used for the error field in logs.
	DefaultErrorKey = "error"
)

const (
	DefaultSJsonFmtTimestampKey  = "timestamp"
	DefaultSJsonFmtSeverityKey   = "severity"
	DefaultSJsonFmtMessageKey    = "message"
	DefaultSJsonFmtErrorKey      = "error"
	DefaultSJsonFmtTraceIDKey    = "trace_id"
	DefaultSJsonFmtSpanIDKey     = "span_id"
	DefaultSJsonFmtCallerKey     = "caller"
	DefaultSJsonFmtCallerFuncKey = "function"
	DefaultSJsonFmtCallerFileKey = "file"
	DefaultSJsonFmtStackTraceKey = "stack_trace"
)

Variables

var (
	ErrInvalidLogLevel = errors.New("invalid log level")
)

Functions

func NewContext

func NewContext(ctx context.Context, logger Logger) context.Context

NewContext returns a new Context that carries the provided Logger.

func NewRequest

func NewRequest(r *http.Request, logger Logger) *http.Request

NewRequest returns a new *http.Request that carries the provided Logger.

func NoopFieldKeyFormatter

func NoopFieldKeyFormatter(defaultKey string) string

NoopFieldKeyFormatter is the default implementation of FieldKeyFormatter. It returns the key unchanged, effectively performing no operation on the key.

func SetDefaultLoggerConfig

func SetDefaultLoggerConfig(config Config) error

SetDefaultLoggerConfig tries to set a custom configuration for the default logger. If creating a logger with the provided config fails, the default configuration remains unchanged.

Types

type Config

type Config struct {
	// Level determines the minimum log level that will be processed by the logger.
	// Logs with a level lower than this will be ignored.
	Level LogLevel
	// Formatter is an optional field for specifying a custom logrus formatter.
	// If not provided, the logger will use the StructuredJSONFormatter by default.
	Formatter logrus.Formatter
	// Environment is an optional field for specifying the running environment (e.g., "production", "staging").
	// This field is used for adding environment-specific fields to logs.
	Environment string
	// ServiceName is an optional field for specifying the name of the service.
	// This field is used for adding service-specific fields to logs.
	ServiceName string
	// Output is an optional field for specifying the output destination for logs (e.g., os.Stdout, file).
	// If not provided, logs will be written to stdout by default.
	Output io.Writer
}

Config holds the logger configuration.

type FieldKeyFormatter

type FieldKeyFormatter func(key string) string

FieldKeyFormatter is a function type that allows users to customize the keys of log fields.

Example usage:

customFieldKeyFormatter := func(key string) string {
	return strings.ToUpper(key)
}

type Fields

type Fields map[string]interface{}

Fields represents a key-value pair for structured logging.

type LogLevel

type LogLevel string

const (
	DEBUG LogLevel = "debug"
	INFO  LogLevel = "info"
	WARN  LogLevel = "warn"
	ERROR LogLevel = "error"
	FATAL LogLevel = "fatal"
)

func (LogLevel) IsValid

func (l LogLevel) IsValid() bool

func (LogLevel) ToLogrusLevel

func (l LogLevel) ToLogrusLevel() logrus.Level

type Logger

type Logger interface {
	WithFields(fields Fields) Logger
	Debug(ctx context.Context, msg string, fields Fields)
	Info(ctx context.Context, msg string, fields Fields)
	Warn(ctx context.Context, msg string, fields Fields)
	Error(ctx context.Context, msg string, err error, fields Fields)
	Fatal(ctx context.Context, msg string, err error, fields Fields)
}

func FromContext

func FromContext(ctx context.Context) Logger

FromContext retrieves the Logger from the context. It returns a default logger if the context doesn't have one.

func FromRequest

func FromRequest(r *http.Request) Logger

FromRequest retrieves the Logger from the HTTP request's context.

func NewDefaultLogger

func NewDefaultLogger() Logger

NewDefaultLogger returns a logger instance with the default or user-defined configuration. If SetDefaultLoggerConfig has been called, it uses the user-defined configuration; otherwise, it uses the package's default configuration. The default logger configuration uses the StructuredJSONFormatter, which outputs logs in JSON format with the following fields:

• timestamp: formatted in RFC3339 format.
• severity: the severity level of the log (e.g., info, debug, error).
• message: the log message.
• error: the error message for logs with error-level severity or higher.
• trace_id: the trace identifier for correlating logs with distributed traces (if available).
• span_id: the span identifier for correlating logs within specific spans of a trace (if available).
• caller: the function, file, and line number where the log was generated.
• stack_trace: included for logs with error-level severity or higher, providing additional debugging context.

func NewLogger

func NewLogger(config Config) (Logger, error)

NewLogger creates a new logger instance with the provided configuration.

func NewNoopLogger

func NewNoopLogger() Logger

NewNoopLogger returns a no-op logger that discards all log messages.

type StructuredJSONFormatter

type StructuredJSONFormatter struct {
	// TimestampFormat sets the format used for marshaling timestamps.
	TimestampFormat string
	// PrettyPrint will indent all JSON logs.
	PrettyPrint bool
	// SkipPackages is a list of packages to skip when searching for the caller.
	SkipPackages []string
	// FieldKeyFormatter is a function type that allows users to customize log field keys.
	FieldKeyFormatter FieldKeyFormatter
}

StructuredJSONFormatter is a custom logrus formatter for structured JSON logs. It includes the following fields:

• timestamp: The log timestamp in the specified format.
• severity: The log severity level (e.g., info, debug, error).
• message: The log message.
• error: The error message if present.
• trace_id: The trace ID if available.
• span_id: The span ID if available.
• caller: The caller's function name, file, and line number.
• stack_trace: The stack trace for error levels.

func (*StructuredJSONFormatter) Format

func (f *StructuredJSONFormatter) Format(entry *logrus.Entry) ([]byte, error)

Format implements the logrus.Formatter interface.