rollingf

package module
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 26, 2023 License: Apache-2.0 Imports: 18 Imported by: 0

README

RollingF GoDoc

RollingF is an IO tool that allows you to control the rules for rolling update files.

Customize

Using RollingF you can customize four rules when rolling to update a file:

  1. Checker. Checker decides wheter trigger the rolling when write. For example, rolling the file every day or when the file size reaches 1M.
  2. Matcher. Matcher decides which files to be further processed. For example, app.log app.log.1 app.log.2 ...
  3. Filter. Filter filters files that you don't want to process in the processor. For example, some files that are too old, remove them.
  4. Processor. Processor processes the filtered files. For example, renaming the older files, rolling the new file.
+------------------+             +---------------+             +---------------+
|                  |             |               |             |               |
|  Write to file   +------------->    Checker    +------------->    rolling?   +----------N---------------+
|                  |             |               |             |               |                          |
+------------------+             +---------------+             +-------+-------+                          |
                                                                       |                                  |
                                                                       |                                  |
                                                                       Y                                  |
                                                                       |                                  |
                                                                       |                                  |
                                                                       |                      +-----------v-------------+
+------------------+                                           +-------v-------+              |                         |
|                  |                                           |               |              |                         |
|     Filter       <------------------some files---------------+    Mather     |              |  write content to file  |
|                  |                                           |               |              |                         |
+--------+---------+                                           +---------------+              |                         |
         |                                                                                    +-----------^-------------+
         |                                                                                                |
         +--------------------------------+                                                               |
         |                                |                                                               |
         |                                |                                                               |
+--------v---------+             +--------v---------+          +---------------+                          |
|                  |             |                  |          |               |                          |
| filtered files   |             | remaining files  +---------->   Processor   +-----after processed------+
|                  |             |                  |          |               |
+------------------+             +------------------+          +---------------+

Default components

All the components has the default implementions.

  • Checker
    • IntervalChecker checks whether a file should be rolled at regular intervals. If interval <= 0, it will never roll.
    • MaxSizeChecker checks whether a file should be rolled when its size exceeds maxSize.
  • Matcher
    • DefaultMatcher matches the simple file names. eg. app.log app.log.1 app.log.2 ...
    • CompressMatcher matches the compressed file names. eg. app.log app.log.1.gz app.log.2.gz ...
  • Filter
    • MaxSizeFilter filter files by size.
    • MaxAgeFilter filter files by age.
  • Processor
    • DefaultProcessor renames the files, increase the tail number of the file name.
    • Compressor compress the files.

Usage

Simple usage
    rf := rollingf.New(rollingf.NewRollConf("/tmp/any_app/any_app.log", time.Hour, 1024*1024, 30*24*time.Hour, 5))
    if rf == nil {
      return
    }

    rf.Write([]byte("simple rollingf"))

compress

    rf := rollingf.New(rollingf.NewRollConf("/tmp/any_app/any_app.log", time.Hour, 1024*1024, 30*24*time.Hour, 5)).WithOptions(rollingf.Compress(rollingf.Gzip))
    if rf == nil {
      return
    }

    rf.Write([]byte("simple rollingf"))
Customized usage
    rf := rollingf.NewC("/tmp/any_app/any_app.log")
    if rf == nil {
      return
    }
    rf = rf.
      WithChecker(rollingf.IntervalChecker(rollingf.DurOneDay)).
      WithChecker(rollingf.MaxSizeChecker(rollingf.SizeMB)). // 1M
      WithFilter(rollingf.MaxAgeFilter(30 * rollingf.DurOneDay)).
      WithFilter(rollingf.MaxBackupsFilter(5)).
      WithDefaultMatcher().
      WithDefaultProcessor()

    defer rf.Close()

    rf.Write([]byte("hello rollingf"))
Integration with standard library's log
    rf := rollingf.New(rollingf.NewRollConf("/tmp/any_app/any_app.log", time.Hour, 1024*1024, 30*24*time.Hour, 5))
    if rf == nil {
      return
    }
    log.SetOutput(rf)

    log.Println("stdlib rollingf")
Integration with zap
var Logger *zap.SugaredLogger

func init() {
    core := logCore()
    if core == nil {
      panic("init logger failed")
    }
    Logger = zap.New(core, zap.AddCaller()).Sugar()
}

func logCore() zapcore.Core {
    rf := rollingf.New(
      rollingf.NewRollConf("/tmp/any_app/any_app.log", time.Hour, 1024*1024, 30*24*time.Hour, 5),
    )
    if rf == nil {
      return nil
    }

    encoder := zapcore.NewJSONEncoder(zap.NewDevelopmentEncoderConfig())
    writers := zapcore.NewMultiWriteSyncer(zapcore.AddSync(rf), zapcore.AddSync(os.Stderr))
    level := zap.NewAtomicLevelAt(zap.DebugLevel)
    return zapcore.NewCore(encoder, writers, level)
}

Documentation

Index

Constants

View Source 
const (
	DurOneDay  = 24 * time.Hour
	DurOneWeek = 7 * 24 * time.Hour
)
View Source 
const (
	SizeKB = 1024
	SizeMB = 1024 * 1024
)

Variables

This section is empty.

Functions

func CompressMatcher added in v1.2.0 

func CompressMatcher(format CompressFormat) *regexMatcher

CompressMatcher matches the file names with the .1.gz suffix

eg. app.log app.log.1.gz app.log.2.gz ...

func Compressor added in v1.2.0 

func Compressor(format CompressFormat) *compressor

Compressor compresses and rename the files

eg. 

base: "abc.log",
return: "abc.log.1.gz"

func DefaultMatcher 

func DefaultMatcher() *regexMatcher

DefaultMatcher matches the simple file names

eg. app.log app.log.1 app.log.2 ...

func DefaultProcessor 

func DefaultProcessor() *defaultProcessor

DefaultProcessor renames the files, increase the tail number of the file name.

func IntervalChecker 

func IntervalChecker(interval time.Duration) *intervalChecker

IntervalChecker checks whether a file should be rolled at regular intervals

If interval <= 0, it will never roll.

func IsAlpha 

func IsAlpha(s string) bool

IsAlpha checks if the string contains only unicode letters.

func IsAlphanumeric 

func IsAlphanumeric(s string) bool

IsAlphanumeric checks if the string contains only Unicode letters or digits.

func IsNumeric 

func IsNumeric(s string) bool

IsNumeric Checks if the string contains only digits. A decimal point is not a digit and returns false.

func MaxAgeFilter 

func MaxAgeFilter(maxAge time.Duration) (obj *maxAgeFilter)

MaxAgeFilter filter files by age

func MaxBackupsFilter 

func MaxBackupsFilter(maxBackups int) *maxBackupsFilter

MaxSizeFilter filter files by size

func MaxSizeChecker 

func MaxSizeChecker(maxSize int64) *maxSizeChecker

MaxSizeChecker checks whether a file should be rolled when its size(bytes) exceeds maxSize

func NewRegexMatcher added in v1.2.0 

func NewRegexMatcher(suffixPattern string) *regexMatcher

func SetDebug 

func SetDebug(enabled bool)

Types

type Checker 

type Checker interface {
	Name() string
	Check(filePath string, st *Rstat) (bool, error)
}

Checker is a file checker, it checks if a file is shall be rolled.

func DefaultChecker 

func DefaultChecker(c RollCheckerConf) []Checker

type CompressFormat added in v1.2.0 

type CompressFormat string
const (
	NoCompress CompressFormat = ""

	Gzip CompressFormat = "gzip"
	Zlib CompressFormat = "zlib"
)

type Filter 

type Filter interface {
	Name() string
	Filter(input []os.DirEntry) (remains []os.DirEntry, filtered []os.DirEntry, err error)
	DealFiltered(dir string, filtered []os.DirEntry) error
}

Filter filters out the sorted-by-filename files that need to be processed

func DefaultFilter 

func DefaultFilter(c RollFilterConf) []Filter

type Matcher 

type Matcher interface {
	// Match return true if other file base name matches
	Match(other string) bool

	// Init the matcher with the file's base name
	Init(base string)
}

Matcher mathes the files for further processing

type Option added in v1.2.0 

type Option interface {
	// contains filtered or unexported methods
}

func Compress added in v1.2.0 

func Compress(format CompressFormat) Option

Compress specifies the format of the compressed file

func Lock added in v1.2.0 

func Lock(enable bool) Option

Lock enables/disables rollingf's internal lock when Write. Default the lock is enable.

type OptionFunc added in v1.3.0 

type OptionFunc func(r *Roll)

type Processor 

type Processor interface {
	// Process process the remaining files after filtering
	Process(dir string, remains []os.DirEntry) error
}

Processor processes the remaining files after filtering

type Roll 

type Roll struct {
	// contains filtered or unexported fields
}

func New 

func New(c RollConf, opts ...Option) *Roll

New roll creates a Roll with default components

func NewC 

func NewC(filePath string, opts ...Option) *Roll

NewC creates a customizable Roll

The following components need to be populated:

  • Checker
  • Mather
  • Filter
  • Processor

func (*Roll) Close 

func (r *Roll) Close() error

func (*Roll) Open 

func (r *Roll) Open() error

func (*Roll) WithChecker 

func (r *Roll) WithChecker(c ...Checker) *Roll

func (*Roll) WithDefaultChecker 

func (r *Roll) WithDefaultChecker(c RollCheckerConf) *Roll

func (*Roll) WithDefaultFilter 

func (r *Roll) WithDefaultFilter(c RollFilterConf) *Roll

func (*Roll) WithDefaultMatcher 

func (r *Roll) WithDefaultMatcher() *Roll

func (*Roll) WithDefaultProcessor 

func (r *Roll) WithDefaultProcessor() *Roll

func (*Roll) WithFilter 

func (r *Roll) WithFilter(f ...Filter) *Roll

func (*Roll) WithMatcher 

func (r *Roll) WithMatcher(m Matcher) *Roll

func (*Roll) WithOptions added in v1.2.0 

func (r *Roll) WithOptions(opts ...Option) *Roll

func (*Roll) WithProcessor 

func (r *Roll) WithProcessor(p Processor) *Roll

func (*Roll) Write 

func (r *Roll) Write(p []byte) (n int, err error)

Write writes the given bytes to the file.

  1. The rolling will be executed when trigger a Checker.
  2. Then the file will be filter out remove files by Filters.
  3. Then the filtered files will be removed.
  4. Finally the remains will be rolled.

type RollCheckerConf 

type RollCheckerConf struct {
	// interval to roll file
	Interval time.Duration

	// the max bytes to roll file
	MaxSize int64
}

type RollConf 

type RollConf struct {
	FilePath string // file's location
	RollCheckerConf
	RollFilterConf
}

func NewRollConf 

func NewRollConf(filePath string, interval time.Duration, maxSize int64, maxAge time.Duration, maxBackups int) RollConf

type RollFilterConf 

type RollFilterConf struct {
	// the max day to keep old files, only triggered when call Write()
	MaxAge time.Duration

	// the max number of old log files to retain
	MaxBackups int
}

type Rstat 

type Rstat struct {
	// contains filtered or unexported fields
}

Rstat wraps os.FileInfo with local stored information about the file.

func (*Rstat) Birthtimespec 

func (r *Rstat) Birthtimespec() (bool, syscall.Timespec)

Birthtimespec returns the file's birth time.

func (*Rstat) Checked added in v1.3.0 

func (r *Rstat) Checked() bool

func (*Rstat) FileInfo 

func (r *Rstat) FileInfo() fs.FileInfo

FileInfo returns the underlying fs.FileInfo.

func (*Rstat) IsDir 

func (r *Rstat) IsDir() bool

func (*Rstat) Lock added in v1.3.0 

func (r *Rstat) Lock()

Lock

func (*Rstat) ModTime 

func (r *Rstat) ModTime() time.Time

func (*Rstat) Mode 

func (r *Rstat) Mode() fs.FileMode

func (*Rstat) Name 

func (r *Rstat) Name() string

func (*Rstat) RLock added in v1.3.0 

func (r *Rstat) RLock()

func (*Rstat) RUnlock added in v1.3.0 

func (r *Rstat) RUnlock()

func (*Rstat) SetChecked added in v1.3.0 

func (r *Rstat) SetChecked(checked bool)

func (*Rstat) Size 

func (r *Rstat) Size() int64

func (*Rstat) String 

func (r *Rstat) String() string

func (*Rstat) Unlock added in v1.3.0 

func (r *Rstat) Unlock()

Unlock

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.