rollingwriter

README

RollingWriter

RollingWriter is an auto rotate io.Writer implementation. It can works well with logger.

Awesome Go popular log helper

New Version v2.0 is comming out! Much more Powerfull and Efficient. Try it by follow the demo

RollingWriter contains 2 separate patrs:

• Manager: decide when to rotate the file with policy. RlingPolicy give out the rolling policy

  • WithoutRolling: no rolling will happen
  • TimeRolling: rolling by time
  • VolumeRolling: rolling by file size

• IOWriter: impement the io.Writer and do the io write

  • Writer: not parallel safe writer
  • LockedWriter: parallel safe garented by lock
  • AsyncWtiter: parallel safe async writer
  • BufferWriter: merge serval write into one file.Write()

Features

• Auto rotate with multi rotate policies
• Implement go io.Writer, provide parallel safe writer
• Max remain rolling files with auto cleanup
• Easy for user to implement your manager

Benchmark

goos: darwin
goarch: amd64
pkg: github.com/arthurkiller/rollingWriter
BenchmarkWrite-4                          300000              5952 ns/op               0 B/op          0 allocs/op
BenchmarkParallelWrite-4                  200000              7846 ns/op               0 B/op          0 allocs/op
BenchmarkAsynWrite-4                      200000              7917 ns/op           16324 B/op          1 allocs/op
BenchmarkParallelAsynWrite-4              200000              8632 ns/op           12513 B/op          1 allocs/op
BenchmarkLockedWrite-4                    200000              5829 ns/op               0 B/op          0 allocs/op
BenchmarkParallelLockedWrite-4            200000              7796 ns/op               0 B/op          0 allocs/op
BenchmarkBufferWrite-4                    200000              6943 ns/op            1984 B/op          4 allocs/op
BenchmarkParallelBufferWrite-4           1000000              1026 ns/op            7129 B/op          1 allocs/op
PASS
ok      github.com/arthurkiller/rollingWriter   14.867s

Quick Start

	writer, err := rollingwriter.NewWriterFromConfig(&config)
	if err != nil {
		panic(err)
	}

	writer.Write([]byte("hello, world"))

Want more? View demo for more details.

Any suggestion or new feature inneed, please put up an issue

Documentation

Index

• Constants
• Variables
• func AsynchronousWriterErrorChan(wr RollingWriter) (chan error, error)
• func LogFilePath(c *Config) (filepath string)
• type AsynchronousWriter
  • func (w *AsynchronousWriter) Close() error
  • func (w *AsynchronousWriter) Write(b []byte) (int, error)
• type BufferWriter
  • func (w *BufferWriter) Close() error
  • func (w *BufferWriter) Write(b []byte) (int, error)
• type Config
  • func NewDefaultConfig() Config
• type LockedWriter
  • func (w *LockedWriter) Close() error
  • func (w *LockedWriter) Write(b []byte) (n int, err error)
• type Locker
  • func (l *Locker) Lock()
  • func (l *Locker) Unlock()
• type Manager
  • func NewManager(c *Config) (Manager, error)
• type Option
  • func WithAsynchronous() Option
  • func WithBuffer() Option
  • func WithBufferThershould(n int) Option
  • func WithCompress() Option
  • func WithFileName(name string) Option
  • func WithLock() Option
  • func WithLogPath(path string) Option
  • func WithMaxRemain(max int) Option
  • func WithRollingTimePattern(pattern string) Option
  • func WithRollingVolumeSize(size string) Option
  • func WithTimeTagFormat(format string) Option
  • func WithoutRollingPolicy() Option
• type RollingWriter
  • func NewWriter(ops ...Option) (RollingWriter, error)
  • func NewWriterFromConfig(c *Config) (RollingWriter, error)
  • func NewWriterFromConfigFile(path string) (RollingWriter, error)
• type Writer
  • func (w *Writer) Close() error
  • func (w *Writer) CompressFile(oldfile *os.File, cmpname string) error
  • func (w *Writer) DoRemove()
  • func (w *Writer) Reopen(file string) error
  • func (w *Writer) Write(b []byte) (int, error)

Constants

const (
	WithoutRolling = iota
	TimeRolling
	VolumeRolling
)

RollingPolicies giveout 3 policy for rolling.

Variables

var (
	// BufferSize defined the buffer size, by default 1M buffer will be allocate
	BufferSize = 0x100000
	// QueueSize defined the queue size for asynchronize write
	QueueSize = 1024
	// Precision defined the precision about the reopen operation condition
	// check duration within second
	Precision = 1
	// DefaultFileMode set the default open mode rw-r--r-- by default
	DefaultFileMode = os.FileMode(0644)
	// DefaultFileFlag set the default file flag
	DefaultFileFlag = os.O_RDWR | os.O_CREATE | os.O_APPEND

	// ErrInternal defined the internal error
	ErrInternal = errors.New("error internal")
	// ErrClosed defined write while ctx close
	ErrClosed = errors.New("error write on close")
	// ErrInvalidArgument defined the invalid argument
	ErrInvalidArgument = errors.New("error argument invalid")
)

Functions

func AsynchronousWriterErrorChan

func AsynchronousWriterErrorChan(wr RollingWriter) (chan error, error)

AsynchronousWriterErrorChan return the error channel for asyn writer

func LogFilePath

func LogFilePath(c *Config) (filepath string)

LogFilePath return the absolute path on log file

Types

type AsynchronousWriter

type AsynchronousWriter struct {
	Writer
	// contains filtered or unexported fields
}

AsynchronousWriter provide a asynchronous writer with the writer to confirm the write

func (*AsynchronousWriter) Close

func (w *AsynchronousWriter) Close() error

Close set closed and close the file once

func (*AsynchronousWriter) Write

func (w *AsynchronousWriter) Write(b []byte) (int, error)

Only when the error channel is empty, otherwise nothing will write and the last error will be return return the error channel

type BufferWriter

type BufferWriter struct {
	Writer
	// contains filtered or unexported fields
}

BufferWriter merge some write operations into one.

func (*BufferWriter) Close

func (w *BufferWriter) Close() error

Close bufferWriter flush all buffered write then close file

func (*BufferWriter) Write

func (w *BufferWriter) Write(b []byte) (int, error)

type Config

type Config struct {
	// LogPath defined the full path of log file directory.
	// there comes out 2 different log file:
	//
	// 1. the current log
	//	log file path is located here:
	//	[LogPath]/[FileName].log
	//
	// 2. the tuncated log file
	//	the tuncated log file is backup here:
	//	[LogPath]/[FileName].log.[TimeTag]
	//  if compressed true
	//	[LogPath]/[FileName].log.gz.[TimeTag]
	//
	// NOTICE: blank field will be ignored
	// By default we using '-' as separator, you can set it yourself
	TimeTagFormat string `json:"time_tag_format"`
	LogPath       string `json:"log_path"`
	FileName      string `json:"file_name"`
	// MaxRemain will auto clear the roling file list, set 0 will disable auto clean
	MaxRemain int `json:"max_remain"`

	// RollingPolicy give out the rolling policy
	// We got 3 policies(actually, 2):
	//
	//	1. WithoutRolling: no rolling will happen
	//	2. TimeRolling: rolling by time
	//	3. VolumeRolling: rolling by file size
	RollingPolicy      int    `json:"rolling_ploicy"`
	RollingTimePattern string `json:"rolling_time_pattern"`
	RollingVolumeSize  string `json:"rolling_volume_size"`

	// WriterMode in 4 modes below
	// 1. none 2. lock
	// 3. async 4. buffer
	WriterMode string `json:"writer_mode"`
	// BufferWriterThershould in Byte
	BufferWriterThershould int `json:"buffer_thershould"`
	// Compress will compress log file with gzip
	Compress bool `json:"compress"`

	// FilterEmptyBackup will not backup empty file if you set it true
	FilterEmptyBackup bool `json:"filter_empty_backup"`
}

Config give out the config for manager

func NewDefaultConfig

func NewDefaultConfig() Config

NewDefaultConfig return the default config

type LockedWriter

type LockedWriter struct {
	Writer
	sync.Mutex
}

LockedWriter provide a synchronous writer with lock write operate will be guaranteed by lock

func (*LockedWriter) Close

func (w *LockedWriter) Close() error

Close lock and close the file

func (*LockedWriter) Write

func (w *LockedWriter) Write(b []byte) (n int, err error)

type Locker

type Locker struct {
	// contains filtered or unexported fields
}

Locker is a spinlock implementation.

A Locker must not be copied after first use.

func (*Locker) Lock

func (l *Locker) Lock()

Lock will try to get thel lock and yield the processor.

func (*Locker) Unlock

func (l *Locker) Unlock()

Unlock unlocks l.

type Manager

type Manager interface {
	// Fire will return a string channel
	// while the rolling event occoured, new file name will generate
	Fire() chan string
	// Close the Manager
	Close()
}

Manager used to trigger rolling event.

func NewManager

func NewManager(c *Config) (Manager, error)

NewManager generate the Manager with config

type Option

type Option func(*Config)

Option defined config option

func WithAsynchronous

func WithAsynchronous() Option

WithAsynchronous enable the asynchronous write for writer

func WithBuffer

func WithBuffer() Option

WithBuffer will enable the buffer writer mode

func WithBufferThershould

func WithBufferThershould(n int) Option

WithBufferThershould set buffer write thershould

func WithCompress

func WithCompress() Option

WithCompress will auto compress the tuncated log file with gzip

func WithFileName

func WithFileName(name string) Option

WithFileName set the log file name

func WithLock

func WithLock() Option

WithLock will enable the lock in writer Writer will call write with the Lock to guarantee the parallel safe

func WithLogPath

func WithLogPath(path string) Option

WithLogPath set the log dir and auto create dir tree if the dir/path is not exist

func WithMaxRemain

func WithMaxRemain(max int) Option

WithMaxRemain enable the auto deletion for old file when exceed the given max value Bydefault -1 will disable the auto deletion

func WithRollingTimePattern

func WithRollingTimePattern(pattern string) Option

WithRollingTimePattern set the time rolling policy time pattern obey the Corn table style visit http://crontab.org/ for details

func WithRollingVolumeSize

func WithRollingVolumeSize(size string) Option

WithRollingVolumeSize set the rolling file truncation threshold size

func WithTimeTagFormat

func WithTimeTagFormat(format string) Option

WithTimeTagFormat set the TimeTag format string

func WithoutRollingPolicy

func WithoutRollingPolicy() Option

WithoutRolling set no rolling policy

type RollingWriter

type RollingWriter interface {
	io.Writer
	Close() error
}

RollingWriter implement the io writer

func NewWriter

func NewWriter(ops ...Option) (RollingWriter, error)

NewWriter generate the rollingWriter with given option

func NewWriterFromConfig

func NewWriterFromConfig(c *Config) (RollingWriter, error)

NewWriterFromConfig generate the rollingWriter with given config

func NewWriterFromConfigFile

func NewWriterFromConfigFile(path string) (RollingWriter, error)

NewWriterFromConfigFile generate the rollingWriter with given config file

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer provide a synchronous file writer if Lock is set true, write will be guaranteed by lock

func (*Writer) Close

func (w *Writer) Close() error

Close the file and return

func (*Writer) CompressFile

func (w *Writer) CompressFile(oldfile *os.File, cmpname string) error

CompressFile compress log file write into .gz

func (*Writer) DoRemove

func (w *Writer) DoRemove()

DoRemove will delete the oldest file

func (*Writer) Reopen

func (w *Writer) Reopen(file string) error

Reopen do the rotate, open new file and swap FD then trate the old FD

func (*Writer) Write

func (w *Writer) Write(b []byte) (int, error)