slogdriver

README

slogdriver

slog based logging library optimized for Cloud Logging. This packaged inspired by Zapdriver and zerodriver.

What is this package?

This package provides simple structured logger optimized for Cloud Logging based on slog.

Requirements

Go 1.21 or above

Usage

Initialize a logger.

logger := slogdriver.New(os.Stdout, slogdriver.HandlerOptions{})

This logger is *slog.Logger. Then, write by using slog API.

logger.Info("Hello World!", slog.String("key", "value"))

GCP specific fields

If your log follows LogEntry format, you can query logs or create metrics alert easier and efficiently on GCP Cloud Logging console.

Severity

You can use slog.Level(Debug, Info, Warn, Error). And this library prepare all severities for Cloud Logging.

• DEFAULT
• DEBUG
• INFO
• NOTICE
• WARNING
• ERROR
• CRITICAL
• ALERT
• EMERGENCY

logger := slogdriver.New(os.Stdout, slogdriver.HandlerOptions{
  Level: slogdriver.LevelDefault,
})
logger.Log(context.Background(), slogdriver.LevelEmergency, "emergency msg")

HTTP request

To log HTTP related metrics and information, you can create slog.Attr with the following function.

var req *http.Request
var res *http.Response
logger.Info("Http Request finished", slogdriver.MakeHTTPAttr(req, res))

The following fields needs to be set manually:

• ServerIP
• Latency
• CacheLookup
• CacheHit
• CacheValidatedWithOriginServer
• CacheFillBytes

Using these feature, you can log HTTP related information as follows,

p := slogdriver.MakeHTTPPayload(req, res)
p.Latency = time.Since(start)
logger.Info("http finished", slogdriver.MakeHTTPAttrFromHTTPPayload(p))
// Or, you can create attr manually
logger.Info("http finished", slog.Any(slogdriver.HTTPKey, p))

Trace context

import "go.opencensus.io/trace"

// Set projectId or, you can set environment GOOGLE_CLOUD_PROJECT
logger := slogdriver.New(os.Stdout, slogdriver.HandlerOptions{ProjectID: "YOUR_PROJECT_ID"})

ctx, span := trace.StartSpan(context.Background(), "span")
defer span.End()

logger.InfoContext(ctx, "Hello World")
// got:
// {"severity":"INFO","message":"Hello World","logging.googleapis.com/trace":"projects/YOUR_PROJECT_ID/traces/00000000000000000000000000000000","logging.googleapis.com/spanId":"0000000000000000","logging.googleapis.com/trace_sampled":true}

If you use go.opencensus.io/trace, it is able to set traceId with request context like the below:

import (
	"contrib.go.opencensus.io/exporter/stackdriver/propagation"
	"go.opencensus.io/plugin/ochttp"
)

handler := &ochttp.Handler{Propagation: &propagation.HTTPFormat{}}
handler.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
	logger.InfoContext(r.Context(), "Hello World") // This log should include trace information.
})

You can see example for Cloud Run.

Labels

You can add any "labels" to your log as following:

logger.Info("", slog.Group(slogdriver.LabelKey, slog.String("label", "hoge")))

You can set common label like the follow:

logger = logger.With(slog.Group(slogdriver.LabelKey, slog.String("commonLabel", "hoge")))
logger.Info("Hello World", slog.Group(slogdriver.LabelKey, slog.String("label1", "fuga")))
// got:
// {"severity":"INFO","message":"Hello World","logging.googleapis.com/labels":{"commonLabel":"hoge","label1":"fuga"}}
logger.Warn("Hello World", slog.Group(slogdriver.LabelKey, slog.String("label2", "fuga")))
// got:
// {"severity":"WARNING","message":"Hello World","logging.googleapis.com/labels":{"commonLabel":"hoge","label2":"fuga"}}

Source Location

logger := slogdriver.New(os.Stdout, slogdriver.HandlerOptions{AddSource: true})
logger.Info("Hello World")
// {"severity":"INFO","message":"Hello World","logging.googleapis.com/sourceLocation":{"file":"/path/to/source.go","line":"12","function":"yourFunction"}}

TODO

• severity
• message
• httpRequest
• time, timestamp
• insertId
• labels
• operation
• sourceLocation
• spanId
• trace
• traceSampled

docs

https://cloud.google.com/logging/docs/structured-logging#special-payload-fields

Documentation

Overview

MIT License

Copyright (c) 2021 hirosassa

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Source:

https://github.com/hirosassa/zerodriver/blob/32567406b83903dc813682fd5f1999ebbf462f2d/http.go

Index

• Constants
• func MakeHTTPAttr(req *http.Request, res *http.Response) slog.Attr
• func MakeHTTPAttrFromHTTPPayload(p HTTPPayload) slog.Attr
• func New(w io.Writer, opts HandlerOptions) *slog.Logger
• func NewHandler(w io.Writer, opts HandlerOptions) slog.Handler
• type GAELatency
• type HTTPPayload
  • func MakeHTTPPayload(req *http.Request, res *http.Response) HTTPPayload
• type HandlerOptions
• type Latency
  • func MakeLatency(d time.Duration, isGKE bool) Latency
• type LogEntrySourceLocation

Examples

• New

Constants

const (
	LevelDefault   slog.Level = slog.LevelDebug - 2
	LevelDebug     slog.Level = slog.LevelDebug
	LevelInfo      slog.Level = slog.LevelInfo
	LevelNotice    slog.Level = slog.LevelWarn - 2
	LevelWarning   slog.Level = slog.LevelWarn
	LevelError     slog.Level = slog.LevelError
	LevelCritical  slog.Level = slog.LevelError + 2
	LevelAlert     slog.Level = slog.LevelError + 4
	LevelEmergency slog.Level = slog.LevelError + 6
)

const (
	MessageKey        = "message"
	SeverityKey       = "severity"
	HTTPKey           = "httpRequest"
	SourceLocationKey = "logging.googleapis.com/sourceLocation"
	LabelKey          = "logging.googleapis.com/labels"

	TraceKey        = "logging.googleapis.com/trace"
	SpanIDKey       = "logging.googleapis.com/spanId"
	TraceSampledKey = "logging.googleapis.com/trace_sampled"
)

Functions

func MakeHTTPAttr

func MakeHTTPAttr(req *http.Request, res *http.Response) slog.Attr

MakeHTTPAttr returns slog.Attr struct.

func MakeHTTPAttrFromHTTPPayload

func MakeHTTPAttrFromHTTPPayload(p HTTPPayload) slog.Attr

func New

func New(w io.Writer, opts HandlerOptions) *slog.Logger

Example

package main

import (
	"log/slog"
	"os"

	"github.com/kitagry/slogdriver"
)

func main() {
	logger := slogdriver.New(os.Stdout, slogdriver.HandlerOptions{AddSource: true})
	logger = logger.With(slog.Group(slogdriver.LabelKey, slog.String("commonLabel", "hoge")))
	logger.Info("Hello World", slog.Group(slogdriver.LabelKey, slog.String("specifiedLabel", "fuga")))
}

func NewHandler

func NewHandler(w io.Writer, opts HandlerOptions) slog.Handler

Types

type GAELatency

type GAELatency struct {
	Seconds int64 `json:"seconds"`
	Nanos   int32 `json:"nanos"`
}

GAELatency is the Latency for GAE and Cloud Run.

type HTTPPayload

type HTTPPayload struct {
	RequestMethod                  string  `json:"requestMethod"`
	RequestURL                     string  `json:"requestUrl"`
	RequestSize                    string  `json:"requestSize"`
	Status                         int     `json:"status"`
	ResponseSize                   string  `json:"responseSize"`
	UserAgent                      string  `json:"userAgent"`
	RemoteIP                       string  `json:"remoteIp"`
	ServerIP                       string  `json:"serverIp"`
	Referer                        string  `json:"referer"`
	Latency                        Latency `json:"latency"`
	CacheLookup                    bool    `json:"cacheLookup"`
	CacheHit                       bool    `json:"cacheHit"`
	CacheValidatedWithOriginServer bool    `json:"cacheValidatedWithOriginServer"`
	CacheFillBytes                 string  `json:"cacheFillBytes"`
	Protocol                       string  `json:"protocol"`
}

HTTPPayload is the struct consists of http request related components. Details are in following link. https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#HttpRequest

func MakeHTTPPayload

func MakeHTTPPayload(req *http.Request, res *http.Response) HTTPPayload

MakeHTTPPayload returns a HTTPPayload struct.

type HandlerOptions

type HandlerOptions struct {
	// ProjectId is Google Cloud Project ID
	// If you want to use trace_id, you should set this or set GOOGLE_CLOUD_PROJECT environment.
	// Cloud Shell and App Engine set this environment variable to the project ID, so use it if present.
	ProjectID string

	// When AddSource is true, the handler adds a ("logging.googleapis.com/sourceLocation", {"file":"path/to/file.go","line":"12"})
	// attribute to the output indicating the source code position of the log statement. AddSource is false by default
	// to skip the cost of computing this information.
	AddSource bool

	// Level reports the minimum record level that will be logged.
	// The handler discards records with lower levels.
	// If Level is nil, the handler assumes LevelInfo.
	// The handler calls Level.Level for each record processed;
	// to adjust the minimum level dynamically, use a LevelVar.
	Level slog.Leveler
}

type Latency

type Latency any

Latency is the interface of the request processing latency on the server. The format of the Latency should differ for GKE and for GAE, Cloud Run.

func MakeLatency

func MakeLatency(d time.Duration, isGKE bool) Latency

MakeLatency returns Latency based on passed time.Duration object.

type LogEntrySourceLocation

type LogEntrySourceLocation struct {
	File     string `json:"file"`
	Line     string `json:"line"`
	Function string `json:"function"`
}