golog

package module
v0.0.0-...-349f3ea Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 6, 2023 License: MIT Imports: 16 Imported by: 16

README

golog

Travis CI Software License GoDoc Coverage Status goreport

golog，支持:

  1. 日志格式化标准
  2. 日志级别颜色
  3. 按天/大小滚动
  4. 自动压缩
  5. 自动删除
  6. 自动日志文件名
  7. logrus一行集成
  8. 日志限速
  9. 限制所有日志文件总大小

Integration with logrus and log

Use default settings:

package main

import (
	"github.com/bingoohuang/golog"
	"github.com/sirupsen/logrus"
	"log"
)

func main() {
	golog.Setup()

	log.Printf("Hello, this message is logged by std log, #%d", 1)    // default Info
	log.Printf("T! Hello, this message is logged by std log, #%d", 2) // Trace
	log.Printf("D! Hello, this message is logged by std log, #%d", 3) // Debug
	log.Printf("I! Hello, this message is logged by std log, #%d", 4) // Info
	log.Printf("W! Hello, this message is logged by std log, #%d", 5) // Warn
	log.Printf("F! Hello, this message is logged by std log, #%d", 6) // Fatal

	logrus.Tracef("Hello, this message is logged by std log, #%d", 7)
	logrus.Debugf("Hello, this message is logged by std log, #%d", 8)
	logrus.Infof("Hello, this message is logged by std log, #%d", 9)
	logrus.Warnf("Hello, this message is logged by std log, #%d", 10)
}

will output the log messages as below:

2020-12-29 08:52:43.810 [INFO ] 29490 --- [1    ] [-] logfmt.LogrusOption.Setup logrus.go:121 : log file created:~/logs/gologdemo.log
2020-12-29 08:52:43.810 [INFO ] 29490 --- [1    ] [-] main.main main.go:35 : Hello, this message is logged by std log, #1
2020-12-29 08:52:43.810 [INFO ] 29490 --- [1    ] [-] main.main main.go:38 : Hello, this message is logged by std log, #4
2020-12-29 08:52:43.810 [WARN ] 29490 --- [1    ] [-] main.main main.go:39 : Hello, this message is logged by std log, #5
2020-12-29 08:52:43.811 [FATAL] 29490 --- [1    ] [-] main.main main.go:40 : Hello, this message is logged by std log, #6
2020-12-29 08:52:43.811 [INFO ] 29490 --- [1    ] [-] main.main main.go:44 : Hello, this message is logged by std log, #8
2020-12-29 08:52:43.811 [WARN ] 29490 --- [1    ] [-] main.main main.go:45 : Hello, this message is logged by std log, #8

Customize the settings:

golog.Setup(golog.Spec("level=debug,rotate=.yyyy-MM-dd-HH,maxAge=5d,gzipAge=1d"))

Specifications

name env prerequisite default value description
level GOLOG_LEVEL - info log level to record (debug/info/warn/error)
file GOLOG_FILE - ~/logs/{bin}/{bin}.log base log file name, if root user, default log file will be /var/log/{bin}/{bin}.log
rotate GOLOG_ROTATE - .yyyy-MM-dd time rotate pattern(full pattern: yyyy-MM-dd HH:mm)[Split according to the Settings of the last bit]
maxAge GOLOG_MAXAGE - 30d max age to keep log files (unit m/h/d/w)
gzipAge GOLOG_GZIPAGE - 3d gzip aged log files (unit m/h/d/w)
maxSize GOLOG_MAXSIZE - 100M max size to rotate log files (unit K/M/K/KiB/MiB/GiB/KB/MB/GB)
totalSizeCap GOLOG_TOTALSIZECAP - 1G 用来指定所有日志文件的总大小上限，例如设置为3GB的话，那么到了这个值，就会删除旧的日志 (unit K/M/K/KiB/MiB/GiB/KB/MB/GB)
stdout GOLOG_STDOUT - false print the log to stdout at the same time or not
printColor GOLOG_PRINTCOLOR layout is empty true print color on the log level or not, only for stdout=true
printCall GOLOG_PRINTCALL layout is empty false print caller file:line or not (performance slow)
simple GOLOG_SIMPLE layout is empty false simple to print log (not print PID --- [GID] [TraceID])
layout GOLOG_LAYOUT - (empty) log line layout customization, like %t %5l %pid --- [%5gid] [%trace] %20caller : %fields %msg%n
fixstd GOLOG_FIXSTD - true improve standard log for golog format.
file
  1. If the file is an existed directory, like /var/log/, a log file will appended as /var/log/{bin}.log
  2. If the file is not a valid directory or file, the suffix '.log' will be used to distinguished as directory or logfile.

so examples:

  1. /var/log(existed dir) -> /var/log/{bin}.log
  2. /var/log/app.log(only existed dir) -> (same)
  3. /var/log/applog(existed file) -> (same)
  4. /home/bingoo/logs(not existed w/o .log) -> /home/bingoo/logs/{bin}.log
  5. /home/bingoo/logs/app.log(not existed w/ .log) -> (same)

environments

NAME DEFAULT VALUE MEANING example
GOLOG_ASYNC_QUEUE_SIZE 10000 asynchronously logging channel size 1000
GOLOG_FLUSH_LEVEL WarnLevel FLUSH WHEN LEVEL IS higher than WARN
GOLOG_DEBUG (none) Enable debug logging before golog setup on
  1. asynchronously log example: log.Printf("[LOG_ASYNC] request received %s", remote_addr)
  2. turn off log example: log.Printf("[LOG_OFF] request received %s", remote_addr)

Layout pattern

layout := `%t{HH:mm:ss.SSS} %5l{length=5} %pid --- [GID=%gid] [%trace] %caller : %fields %msg%n`
golog.Setup(golog.Spec("level=debug,rotate=.yyyy-mm-dd-HH-mm"), golog.Layout(layout))
pattern remark
%time %t %time same with %time{yyyy-MM-dd HH:mm:ss.SSS}
%level %l %level same with %level{printColor=false lowerCase=false length=0}
%pid process ID
%gid go routine ID
%08gid go routine ID, Pad with leading zeroes(width 8)
%5gid go routine ID, Pad with spaces (width 5, right justified)
%-10trace trace ID, Pad with spaces (width 10, left justified)
%caller caller information, %caller{sep=:,level=warn,skip=2}, sep defines the separator between filename and line number, level defines the lowest level to print caller information,skip prints the number of levels of parent calls.
%fields fields JSON
%context(name=watchID) context's value, where whose name is watchID for example. vars.Set("watchID", "your id")
%message %msg %m log detail message, %m{singleLine=true}, singleLine indicates whether the message should merged into a single line when there are multiple newlines in the message.
%n new line
%% escape percent sign

Demonstration

$ go get github.com/bingoohuang/golog/gologdemo
$ ADDR=":54264" gologdemo
log file created: gologdemo.log
start to listen on :54264
2020-07-18 10:59:24.179 removed by 5m0s gologdemo.log.2020-07-18-10-48.1.gz
2020-07-18 10:59:24.179 removed by 5m0s gologdemo.log.2020-07-18-10-48.2.gz
2020-07-18 10:59:24.202 log file renamed to  gologdemo.log.2020-07-18-10-59.2
2020-07-18 10:59:36.225 log file renamed to  gologdemo.log.2020-07-18-10-59.3
2020-07-18 10:59:36.226 removed by 5m0s gologdemo.log.2020-07-18-10-48.3.gz
2020-07-18 10:59:45.293 log file renamed to  gologdemo.log.2020-07-18-10-59.4
2020-07-18 10:59:45.293 removed by 5m0s gologdemo.log.2020-07-18-10-48.4.gz
2020-07-18 10:59:57.313 log file renamed to  gologdemo.log.2020-07-18-10-59.5
2020-07-18 10:59:57.313 removed by 5m0s gologdemo.log.2020-07-18-10-48.5.gz
2020-07-18 11:00:00.297 log file renamed to  gologdemo.log.2020-07-18-11-00
2020-07-18 11:00:09.360 log file renamed to  gologdemo.log.2020-07-18-11-00.1
2020-07-18 11:00:09.361 removed by 5m0s gologdemo.log.2020-07-18-10-49.gz
2020-07-18 11:00:21.391 log file renamed to  gologdemo.log.2020-07-18-11-00.2

demo log file content:

2020-07-23 12:26:33.144 [INFO ] 9019 --- 214   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":207} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / dkNTliprMVGmkfLaOPqIDEwZBHUVBeukHOmAEsTDFRsGbqcuwcnhUNOQZGyGZazNwxFfOumzuUSdnzCOvIUASPlddzWhZsjyEbhU
2020-07-23 12:26:33.144 [INFO ] 9019 --- 669   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":662} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / FMVACHSBfekZuLPiGrjPOrMfsImGEWTIcLiBbcHlTJWpuVMzhDRyvBThyOUBOllxUEPJlMGGhXhyLHZknzcNaJycUysJuBFhdQjJ
2020-07-23 12:26:33.144 [INFO ] 9019 --- 379   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":372} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / cQbUXmEQUrJXouDNlMyDBFhykLOjCaNRbwDEdTjsUlZTCIWHsycwhnGitpRDfICIwTKeGlAMPVyMWnLQXUHFIcPtLLudqiGtfkvH
2020-07-23 12:26:33.144 [INFO ] 9019 --- 119   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":112} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / uMAaQDwxDPlBEtrFiGbhyFNFJPDHNdQFcYzNvXgsrtBcjZQDCIDYBnDaasJKNVQSbaQPUmzCFRCCqppwIGGgzSIrBvFVopVvSZHL
2020-07-23 12:26:33.144 [INFO ] 9019 --- 244   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":237} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / OPBsKeflLHELLpymnBqxaGrvEyAjCStmMUAKzXUPFrJQOepHQzfARmStzkTmzWJnZmEJqEjiPQxCKhnicomhYBKiOXoyimKMPipp
2020-07-23 12:26:33.144 [INFO ] 9019 --- 368   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":361} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / vlnikHYVmbSqimicIgfpUBImaKRmPuxZnljbgbhEUONUUtAKjxsaBbTUJTdYjMVWxDkcbjWDgzUFENhfyBAHqHCLDOLjQXkYFocm
2020-07-23 12:26:33.144 [INFO ] 9019 --- 910   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":903} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / ggcnhxToaBYtsMRWJCJEHpZViQphaRTwSMhUytFjdPzKltmjHRSReYfswPSbcrCSsUybGIibQPCRavxvwKMyQoOjelNPacinPYFK
2020-07-23 12:26:33.110 [INFO ] 9019 --- 660   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":653} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / DcXihdRvktitsFKQmAGIpiFsDYslqnPQebbmQrqUGZGTdHAkGHvoUmCMiejCYEzfEriLFlcTjPiHDOxMaOdhyLyPaHyYmwqMCnok
2020-07-23 12:26:33.110 [INFO ] 9019 --- 440   [-] main.go:101          : {"contentType":"","proto":"HTTP/1.1","workerID":433} 2020-07-23 12:26:33.098 127.0.0.1:34458 GET / GkbfwhOuBvinWDBIdKrVWTbtKJsSDtCeJirgIoiUovmUALuCfHPdjYccdNZyGsWXFTlpmsBpbIJLlPVWuzuNxpcTAFcRQXGmOtjt

watch the log file rotating and gzipping and deleting watch -c "ls -tlh gologdemo.log*"

Log Creation Sequence

+------------+   Write log messages    +-------------------------+
| file=a.log +------------------------>+ a.log created           |
+------------+                         +-------------------------+    rotate=.yyyy-MM-dd
                                                                    +----------------------+
+------------+                         +-------------------------+   time goes to next day |
|            +------------------------>+~/logs/{app}.log created |                         |
+------------+                         +-------------------------+                         |
                                                                                           |
                                                                                           |
                                                                                           |
                                       +-------------------------------------+             |
           maxSize=100M                |  x.log renamed to x.log.2020-07-17  |             |
   +-----------------------------------+                                     <-------------+
   |       still current day.          |  a new file x.log recreated!        |
   |                                   +-------------------------------------+
   |
   |
   |        +------------------------------------------------+
   |        | x.log renamed to x.log.2020-07-17.1(100M)      |    reached new 100M
   +-------->                                                +-----------------------------+
            | a new file x.log recreated to current writing. |    still current day        |
            +------------------------------------------------+                             |
                                                                                           |
                                                                                           |
                                         +-------------------------------------------+     |
       day goes to 2020-07-21            | x.log renamed to x.log.2020-07-17.2(100M) |     |
   +-------------------------------------+                                           <-----+
   |          gzipAge=3d                 | a new file x.log recreated!               |
   |                                     +-------------------------------------------+
   |
   | +-----------------------------+                          +----------------------------------+
   | | x.log.2020-07-17.1.log.gz   |   day goes to 2020-7-22  | x.log.2020|07|17.1.log.gz deleted|
   +->                             +------------------------->|                                  |
     | x.log.2020-07-17.2.log.gz   |      maxAge=4d           | x.log.2020|07|17.2.log.gz deleted|
     +-----------------------------+                          +----------------------------------+

Log rate limiter

more examples

limit config examples:

  1. [L:100,15s:ignore.sync] to limit 1 message every 15 seconds or every 100 messages with "ignore.sync" as key
  2. [L:15s:ignore.sync] to limit 1 message every 15 seconds with "ignore.sync" as key
  3. [L:100,15s] to limit 1 message every 15 seconds or every 100 messages with the first two words in the message as key
  4. [L:100,0s] to limit 1 message every 100 messages with the first two words in the message as key
  5. [L:15s] to limit 1 message every 15 seconds with the first two words in the message as key
  6. [L:LimitConf1] to limit using configuration whose name is LimitConf1 registered first by golog.RegisterLimiter(golog.LimitConf{EveryTime: 200 * time.Millisecond, Key: "LimitConf1", Level: "INFO"})
golog.Setup()
log.Printf("[L:200ms] W! Hello i:%d", i) // will limit to 1 log per 200ms.
logrus.Infof("[L:200ms] Hello i:%d", i) // will limit to 1 log per 200ms.
logrus.Infof("[L:LimitConf1] Hello i:%d", i) // will limit to by registered configuration `LimitConf1`.

Help

  1. sed "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" x.log to strip color from log file.
  2. tail -F x.log even if x.log recreated.
  3. GIN framework extra logs will be printed?
    1. Use gin.New() instead of gin.Default(), Because gin.Default() has extra Logger()
    2. ginlogrus.Logger(nil, true), true indicates whether static resources are filtered
	r = gin.New()
	r.Use(ginlogrus.Logger(nil, true), gin.Recovery())

golog gin with trace ID

package main

import (
	"github.com/bingoohuang/golog"
	"github.com/bingoohuang/golog/pkg/ginlogrus"
	"github.com/gin-gonic/gin"
	"log"
)

func main() {
	gin.SetMode(gin.ReleaseMode)
	golog.Setup()

	r := gin.New()
	r.Use(ginlogrus.Logger(nil, true), gin.Recovery())

	r.GET("/ping", func(c *gin.Context) {
		log.Printf("pinged1")
		log.Printf("pinged2")
		c.JSON(200, gin.H{"message": "pong"})

		fmt.Println("context trace id:", ginlogrus.GetTraceIDGin(c))
	})
	// ...
}

2020-08-24 09:47:30.530 [INFO ] 68880 --- [24   ] [87513ae4-10d4-43f3-be5e-f8a11e636f4b] ginlogurs_test.go:21 : pinged1
2020-08-24 09:47:30.531 [INFO ] 68880 --- [24   ] [87513ae4-10d4-43f3-be5e-f8a11e636f4b] ginlogurs_test.go:22 : pinged2
2020-08-24 09:47:30.531 [INFO ] 68880 --- [24   ] [87513ae4-10d4-43f3-be5e-f8a11e636f4b] ginlogrus.go:64      : 127.0.0.1 GET /ping [200] 18  Go-http-client/1.1 (746.916µs)

Thanks to the giant shoulders:

  1. slog Lightweight, configurable, extensible logging library written in Go. 一个易于使用的，轻量级、可配置、可扩展的日志库
  2. lestrrat-go/file-rotatelogs
  3. benbjohnson/clock
  4. rifflock/lfshook A local file system hook for golang logrus logger
  5. mzky/utils 一个工具集，包括文件组件，日志组件
  6. Lumberjack writing logs to rolling files.
  7. 应用程序日志规范
  8. op/go-logging format
  9. log4j layout
  10. CLI 控制台颜色渲染工具库, 拥有简洁的使用API，支持16色，256色，RGB色彩渲染输出

Relative blogs

  1. Logging in Go: A Comparison of the Top 8 Libraries

    Package Time Objects Allocated
    zerolog 81 ns/op 0 allocs/op
    zap 193 ns/op 0 allocs/op
    zap (sugared) 227 ns/op 1 allocs/op
    slog 322 ns/op 0 allocs/op
    apex/log 19518 ns/op 53 allocs/op
    log15 19812 ns/op 70 allocs/op
    logrus 21997 ns/op 68 allocs/op

Documentation

Overview

Package golog is a convenient logging.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CheckPrivileges 

func CheckPrivileges() bool

CheckPrivileges checks root rights to use system service

func CreateLogDir 

func CreateLogDir(logPath string, logSpec *LogSpec) string

CreateLogDir creates log dir.

func DisableLogging 

func DisableLogging()

DisableLogging disable all logrus logging and standard logging.

func ExecutableInCurrentDir 

func ExecutableInCurrentDir() (bool, error)

ExecutableInCurrentDir check the exe is in the working dir.

func Printf 

func Printf(format string, v ...interface{})

Printf calls Output to print to the standard logger. Arguments are handled in the manner of fmt.Printf. If the last argument is an error, the format will be prepended with "E!" for error level if there is no level tag defined in it.

func RegisterLimiter 

func RegisterLimiter(c LimitConf)

RegisterLimiter registers a limit for the log generation frequency.

func Setup 

func Setup(fns ...SetupOptionFn) *logfmt.Result

Setup set up the logrus logger with specific configuration like guava CacheBuilderSpec. eg: "level=info,file=a.log,rotate=yyyy-MM-dd,maxAge=30d,gzipAge=3d,maxSize=100M,printColor,stdout,printCaller"

Types

type LimitConf 

type LimitConf struct {
	Key       string
	Level     string
	EveryNum  int
	EveryTime time.Duration
}

LimitConf defines the log limit configuration.

type LogSpec 

type LogSpec struct {
	Level        string        `spec:"level,info"`
	File         string        `spec:"file"`
	Rotate       spec.Layout   `spec:"rotate,.yyyy-MM-dd"`
	Stdout       string        `spec:"stdout"`
	MaxAge       time.Duration `spec:"maxAge,30d"`
	GzipAge      time.Duration `spec:"gzipAge,3d"`
	MaxSize      spec.Size     `spec:"maxSize,100M"`
	TotalSizeCap spec.Size     `spec:"totalSizeCap,1G"` // 可选，用来指定所有日志文件的总大小上限，例如设置为3GB的话，那么到了这个值，就会删除旧的日志
	PrintColor   bool          `spec:"printColor,false"`
	PrintCaller  bool          `spec:"printCall,false"`
	Simple       bool          `spec:"simple,false"`
	FixStd       bool          `spec:"fixstd,true"` // 是否增强log.Print...的输出
}

LogSpec defines the spec structure to be mapped to the log specification.

type SetupOption 

type SetupOption struct {
	Logger  *logrus.Logger
	Spec    string
	Layout  string
	LogPath string
}

SetupOption defines the options to setup.

func (SetupOption) InitiateOption 

func (o SetupOption) InitiateOption() logfmt.Option

InitiateOption initialize options.

type SetupOptionFn 

type SetupOptionFn func(*SetupOption)

SetupOptionFn is func type to option setter

func Layout 

func Layout(v string) SetupOptionFn

Layout defines the layout of log.

func LogPath 

func LogPath(v string) SetupOptionFn

LogPath defines the log path.

func Logger 

func Logger(v *logrus.Logger) SetupOptionFn

Logger defines the root logrus logger.

func Spec 

func Spec(v string) SetupOptionFn

Spec defines the specification of log.

type SetupOptionFns 

type SetupOptionFns []SetupOptionFn

SetupOptionFns is the slice of SetupOptionFn

func (SetupOptionFns) Setup 

func (fns SetupOptionFns) Setup(o *SetupOption)

Setup setups the options.

Source Files

View all Source files

Directories

Path Synopsis
cmd
golog
golog-rand
pkg
autoload
caller
clock
compress
gid
ginlogrus
hlog
homedir
httpx
local
lock
logctx
logfmt
port
randx
rotate
spec
stack
str
strftime
term
timex
typ
unmask

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.