slogclickhouse

README

slog-clickhouse

A ClickHouse Handler for slog Go library.

See also:

• slog-multi: slog.Handler chaining, fanout, routing, failover, load balancing...
• slog-formatter: slog attribute formatting
• slog-sampling: slog sampling policy
• slog-gin: Gin middleware for slog logger
• slog-echo: Echo middleware for slog logger
• slog-fiber: Fiber middleware for slog logger
• slog-chi: Chi middleware for slog logger
• slog-datadog: A slog handler for Datadog
• slog-rollbar: A slog handler for Rollbar
• slog-sentry: A slog handler for Sentry
• slog-syslog: A slog handler for Syslog
• slog-logstash: A slog handler for Logstash
• slog-fluentd: A slog handler for Fluentd
• slog-graylog: A slog handler for Graylog
• slog-loki: A slog handler for Loki
• slog-slack: A slog handler for Slack
• slog-telegram: A slog handler for Telegram
• slog-mattermost: A slog handler for Mattermost
• slog-microsoft-teams: A slog handler for Microsoft Teams
• slog-webhook: A slog handler for Webhook
• slog-kafka: A slog handler for Kafka
• slog-nats: A slog handler for NATS
• slog-parquet: A slog handler for Parquet + Object Storage
• slog-zap: A slog handler for Zap
• slog-zerolog: A slog handler for Zerolog
• slog-logrus: A slog handler for Logrus
• slog-channel: A slog handler for Go channels

Installation

go get github.com/smllnest/slog-clickhouse

Compatibility: go >= 1.21

Usage

Handler options

type Option struct {
	// log level (default: debug)
	Level slog.Leveler

	// ClickHouse connection
	DB       *sql.DB
	LogTable string
	Timeout  time.Duration // default: 60s

	// optional: customize clickhouse event builder
	Converter Converter

	// optional: see slog.HandlerOptions
	AddSource   bool
	ReplaceAttr func(groups []string, a slog.Attr) slog.Attr
}

Other global parameters:

slogclickhouse.SourceKey = "source"
slogclickhouse.ContextKey = "extra"
slogclickhouse.RequestKey = "request"
slogclickhouse.ErrorKeys = []string{"error", "err"}
slogclickhouse.RequestIgnoreHeaders = false

Supported attributes

The following attributes are interpreted by slogclickhouse.DefaultConverter:

Atribute name	slog.Kind	Underlying type
"error"	any	error
"request"	any	*http.Request
other attributes	*

Other attributes will be injected in extra field.

Example

You must create the table before using it:

CREATE TABLE logs (
   timestamp DateTime,
   level String,
   message String,
   attrs String
) ENGINE = MergeTree()
ORDER BY timestamp
PARTITION BY toYYYYMMDD(timestamp)

then run the below code:

package main

import (
	"log/slog"
	"testing"
	"time"

	"github.com/smallnest/slog-clickhouse"
)

func main() {
	conn := clickhouse.OpenDB(&clickhouse.Options{
		Addr: []string{"127.0.0.1:9000"},
		Auth: clickhouse.Auth{
			Database: "myapp",
			Username: "",
			Password: "",
		},
		Settings: clickhouse.Settings{
			"max_execution_time": 60,
		},
		DialTimeout:          time.Second * 30,
		Debug:                true,
		BlockBufferSize:      10,
		MaxCompressionBuffer: 10240,
	})

	if err := conn.Ping(); err != nil {
		t.Log("local clickhouse server is not running, skipping test...")
		return
	}

	conn.SetMaxIdleConns(5)
	conn.SetMaxOpenConns(10)
	conn.SetConnMaxLifetime(time.Hour)

	logger := slog.New(Option{Level: slog.LevelDebug, DB: conn, LogTable: "myapp.logs"}.NewClickHouseHandler())

	logger.Info("Hello, ClickHouse!", "key1", "value1", "key2", 2)
}

check the result in ClickHouse:

Documentation

Index

• Variables
• func DefaultConverter(addSource bool, replaceAttr func(groups []string, a slog.Attr) slog.Attr, ...) map[string]any
• type ClickHouseHandler
  • func (h *ClickHouseHandler) Enabled(_ context.Context, level slog.Level) bool
  • func (h *ClickHouseHandler) Handle(ctx context.Context, record slog.Record) error
  • func (h *ClickHouseHandler) WithAttrs(attrs []slog.Attr) slog.Handler
  • func (h *ClickHouseHandler) WithGroup(name string) slog.Handler
• type Converter
• type Option
  • func (o Option) NewClickHouseHandler() slog.Handler

Variables

var ContextKey = "extra"

var ErrorKeys = []string{"error", "err"}

var RequestIgnoreHeaders = false

var RequestKey = "request"

var SourceKey = "source"

Functions

func DefaultConverter

func DefaultConverter(addSource bool, replaceAttr func(groups []string, a slog.Attr) slog.Attr, loggerAttr []slog.Attr, groups []string, record *slog.Record) map[string]any

DefaultConverter is the default converter used by the handler.

Types

type ClickHouseHandler

type ClickHouseHandler struct {
	// contains filtered or unexported fields
}

func (*ClickHouseHandler) Enabled

func (h *ClickHouseHandler) Enabled(_ context.Context, level slog.Level) bool

func (*ClickHouseHandler) Handle

func (h *ClickHouseHandler) Handle(ctx context.Context, record slog.Record) error

func (*ClickHouseHandler) WithAttrs

func (h *ClickHouseHandler) WithAttrs(attrs []slog.Attr) slog.Handler

func (*ClickHouseHandler) WithGroup

func (h *ClickHouseHandler) WithGroup(name string) slog.Handler

type Converter

type Converter func(addSource bool, replaceAttr func(groups []string, a slog.Attr) slog.Attr, loggerAttr []slog.Attr, groups []string, record *slog.Record) map[string]any

Converter is a function that converts a log record to a map of fields.

type Option

type Option struct {
	// Hostname, optional: os.Hostname() will be used if not set
	Hostname string

	// log level (default: debug)
	Level slog.Leveler

	// ClickHouse Connection
	DB *sql.DB
	// ClickHouse Log Table
	LogTable string
	Timeout  time.Duration // default: 60s

	// optional: customize clickhouse event builder
	Converter Converter

	// optional: see slog.HandlerOptions
	AddSource   bool
	ReplaceAttr func(groups []string, a slog.Attr) slog.Attr
}

func (Option) NewClickHouseHandler

func (o Option) NewClickHouseHandler() slog.Handler