logging

README

Logging

This package supports extended settings for request scoped logging. Specifically, the log level (severity below which to suppress) and a custom log field can be set with the context logger, based on http Header values with the grpc-gateway or via grpc metadata. The context logger should then be used inside grpc method implementations instead of a global logger.

The custom field log-trace-key value is intended to be used for simplifying the process of isolating logs for a single request or a set of requests. This is a similar goal to request-ids, but request-ids are randomly generated for uniqueness, making them less versatile for debugging purposes.

Enabling request-scoped logger settings

To enable these features, the LogLevelInterceptor and the grpc_logrus.UnaryServerInterceptor have to be included in the server's middleware chain.

The LogLevelInterceptor needs to be placed after the grpc_logrus.UnaryServerInterceptor in the chain, and accepts its own default logging level, so that the grpc_logrus interceptor (and the interceptors between it and this one) can be allowed to log at a different level than the proceeding ones even without setting it in the request (for example, to always/never log the Info message in the ctxlogrus interceptor, despite having a higher/lower log level). Note that the LogLevelInterceptor cannot effect whether or not the Info level message in the grpc_logrus.UnaryServerInterceptor is printed or not.

The middleware chain code should look something like this:

import (
	"github.com/kirinse/atlas-app-toolkit/logging"
	"github.com/sirupsen/logrus"
	"google.golang.org/grpc"
)

func main() {
	server := grpc.NewServer(
		grpc.UnaryInterceptor(
			grpc_middleware.ChainUnaryServer( // middleware chain
				grpc_logrus.UnaryServerInterceptor(logrus.NewEntry(logger)),
				logging.LogLevelInterceptor(logger.Level), // Request-scoped logging middleware
				...
			),
		),
	)
	...
}

For grpc-gateway support, the MetadataAnnotator should also be added to the gateway. Using the toolkit's server package, that setup looks something like this:

import (
	"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
	"github.com/kirinse/atlas-app-toolkit/gateway"
	"github.com/kirinse/atlas-app-toolkit/logging"
	"github.com/kirinse/atlas-app-toolkit/server"
)

func main() {
	gatewayOptions := []runtime.ServeMuxOption{
		runtime.WithMetadata(logging.MetadataAnnotator),
		...
	}

	server.NewServer(
		server.WithGrpcServer(grpcServer),
		server.WithGateway(
			gateway.WithGatewayOptions(gatewayOptions...),
			...
		),
	)
}

Using the request-scoped logger settings

When using the metadata annotator and the grpc-gateway, http requests using headers -H "log-trace-key: <value>" and -H "log-level: <level>" will be stored in the grpc metadata for consumption by the interceptor.

Without the grpc-gateway, the metadata has to be added to the request context directly.

ctx := metadata.AppendToOutgoingContext(ctx, "log-level", "debug", "log-trace-key", "foobar")

// make unary RPC
response, err := client.SomeRPC(ctx, someRequest)

Gateway logging

Certain client interceptors may reject incoming queries (e.g. due to non-conformant json fields). This will cause a logging gap compared to queries that fail in the server. To alleviate this, an enhanced gateway logging interceptor is provided. The GatewayLoggingInterceptor should be in the middleware chain before any that could error out. The GatewayLoggingSentinelInterceptor should be the very last middleware in the chain.

For example:

...
	grpc_middleware.ChainUnaryClient(
		[]grpc.UnaryClientInterceptor{
			GatewayLoggingInterceptor(logger, EnableDynamicLogLevel, EnableAccountID),
			...
			GatewayLoggingSentinelInterceptor(),
		},
	)
...

Other functions

The helper function CopyLoggerWithLevel can be used to make a deep copy of a logger at a new level, or using CopyLoggerWithLevel(entry.Logger, level).WithFields(entry.Data) can copy a logrus.Entry.

Generate mocks

Mocks generated with this tool. Generate mocks for logging tests via:

    make mocks

Documentation

Index

• Constants
• func Annotator(ctx context.Context, req *http.Request) metadata.MD
• func CopyLoggerWithLevel(logger *logrus.Logger, lvl logrus.Level) *logrus.Logger
• func CustomFieldsStreamServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.StreamServerInterceptor
• func CustomFieldsUnaryServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.UnaryServerInterceptor
• func DisableRequestID(o *gwLogCfg)
• func EnableAccountID(o *gwLogCfg)
• func EnableDynamicLogLevel(o *gwLogCfg)
• func GatewayLoggingInterceptor(logger *logrus.Logger, opts ...GWLogOption) grpc.UnaryClientInterceptor
• func GatewayLoggingSentinelInterceptor() grpc.UnaryClientInterceptor
• func LogLevelInterceptor(defaultLevel logrus.Level) grpc.UnaryServerInterceptor
• func New(level string) *logrus.Logger
• func SentinelValueFromCtx(ctx context.Context) (value, ok bool)
• func StreamClientInterceptor(entry *logrus.Entry, opts ...Option) grpc.StreamClientInterceptor
• func StreamServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.StreamServerInterceptor
• func UnaryClientInterceptor(entry *logrus.Entry, opts ...Option) grpc.UnaryClientInterceptor
• func UnaryServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.UnaryServerInterceptor
• type CodeToLevel
• type GWLogOption
  • func WithAccountID(keyfunc jwt.Keyfunc) GWLogOption
  • func WithCodeFunc(codeFunc grpc_logrus.CodeToLevel) GWLogOption
  • func WithDynamicLogLevel(enable bool) GWLogOption
• type Option
  • func WithCustomFields(fields []string) Option
  • func WithCustomHeaders(headers []string) Option
  • func WithLevels(f CodeToLevel) Option

Constants

const (
	DefaultAccountIDKey     = "account_id"
	DefaultRequestIDKey     = "request_id"
	DefaultSubjectKey       = "subject" // Might be used for different purposes
	DefaultDurationKey      = "grpc.time_ms"
	DefaultGRPCCodeKey      = "grpc.code"
	DefaultGRPCMethodKey    = "grpc.method"
	DefaultGRPCServiceKey   = "grpc.service"
	DefaultGRPCStartTimeKey = "grpc.start_time"
	DefaultClientKindValue  = "client"
	DefaultServerKindValue  = "server"
)

Functions

func Annotator

func Annotator(ctx context.Context, req *http.Request) metadata.MD

Annotator is a function that reads the http headers of incoming requests searching for special logging arguments

func CopyLoggerWithLevel

func CopyLoggerWithLevel(logger *logrus.Logger, lvl logrus.Level) *logrus.Logger

CopyLoggerWithLevel makes a copy of the given (logrus) logger at the logger level. If copying an entry, use CopyLoggerWithLevel(entry.Logger, level).WithFields(entry.Data) on the result (changes to these entries' fields will not affect each other).

func CustomFieldsStreamServerInterceptor

func CustomFieldsStreamServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.StreamServerInterceptor

func CustomFieldsUnaryServerInterceptor

func CustomFieldsUnaryServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.UnaryServerInterceptor

func DisableRequestID

func DisableRequestID(o *gwLogCfg)

DisableRequestID disables request-id inclusion (and generation if needed) in gw interceptor logs

func EnableAccountID

func EnableAccountID(o *gwLogCfg)

EnableAccountID is a shorthand for WithAccountID(nil)

func EnableDynamicLogLevel

func EnableDynamicLogLevel(o *gwLogCfg)

EnableDynamicLogLevel is a shorthand for WithDynamicLogLevel(true)

func GatewayLoggingInterceptor

func GatewayLoggingInterceptor(logger *logrus.Logger, opts ...GWLogOption) grpc.UnaryClientInterceptor

GatewayLoggingInterceptor handles the functions of the various toolkit interceptors offered for the grpc server, as well as the standard grpc_logrus server interceptor behavior (superset of grpc_logrus client interceptor behavior)

func GatewayLoggingSentinelInterceptor

func GatewayLoggingSentinelInterceptor() grpc.UnaryClientInterceptor

GatewayLoggingSentinelInterceptor is meant to be the last interceptor in the client interceptor chain, it sets a value left in the context by the GatewayLoggingInterceptor so that it knows whether the call makes it to the server, and thus the server will log the call, and the gateway doesn't need to.

func LogLevelInterceptor

func LogLevelInterceptor(defaultLevel logrus.Level) grpc.UnaryServerInterceptor

LogLevelInterceptor sets the level of the logger in the context to either the default or the value set in the context via grpc metadata. Also sets the custom log tag if present for pseudo-tracing purposes

func New

func New(level string) *logrus.Logger

func SentinelValueFromCtx

func SentinelValueFromCtx(ctx context.Context) (value, ok bool)

func StreamClientInterceptor

func StreamClientInterceptor(entry *logrus.Entry, opts ...Option) grpc.StreamClientInterceptor

func StreamServerInterceptor

func StreamServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.StreamServerInterceptor

func UnaryClientInterceptor

func UnaryClientInterceptor(entry *logrus.Entry, opts ...Option) grpc.UnaryClientInterceptor

func UnaryServerInterceptor

func UnaryServerInterceptor(entry *logrus.Entry, opts ...Option) grpc.UnaryServerInterceptor

Types

type CodeToLevel

type CodeToLevel func(code codes.Code) logrus.Level

CodeToLevel function defines the mapping between gRPC return codes and interceptor log level. From https://github.com/grpc-ecosystem/go-grpc-middleware/blob/06f64829ca1f521d41cd6235a7a204a6566fb0dc/logging/logrus/options.go#L57

type GWLogOption

type GWLogOption func(*gwLogCfg)

GWLogOption is a type of function that alters a gwLogCfg in the instantiation of a GatewayLoggingInterceptor

func WithAccountID

func WithAccountID(keyfunc jwt.Keyfunc) GWLogOption

WithAccountID enables the account_id field in gw interceptor logs, like the server interceptor

func WithCodeFunc

func WithCodeFunc(codeFunc grpc_logrus.CodeToLevel) GWLogOption

func WithDynamicLogLevel

func WithDynamicLogLevel(enable bool) GWLogOption

WithDynamicLogLevel enables or disables dynamic log levels like handled in the server interceptor

type Option

type Option func(*options)

func WithCustomFields

func WithCustomFields(fields []string) Option

Allows to provide custom fields for logging which are expected to be in JWT token

func WithCustomHeaders

func WithCustomHeaders(headers []string) Option

Allows to provide custom fields for logging from request headers

func WithLevels

func WithLevels(f CodeToLevel) Option

WithLevels customizes the function for mapping gRPC return codes and interceptor log level statements. From https://github.com/grpc-ecosystem/go-grpc-middleware/blob/06f64829ca1f521d41cd6235a7a204a6566fb0dc/logging/logrus/options.go#L70