Documentation ¶
Index ¶
Constants ¶
const DefaultFingerprintSize = 1000 // bytes
const MinFingerprintSize = 16 // bytes
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct { Finder `mapstructure:",squash" yaml:",inline"` IncludeFileName bool `` /* 137-byte string literal not displayed */ IncludeFilePath bool `` /* 137-byte string literal not displayed */ IncludeFileNameResolved bool `` /* 146-byte string literal not displayed */ IncludeFilePathResolved bool `` /* 146-byte string literal not displayed */ PollInterval helper.Duration `` /* 133-byte string literal not displayed */ StartAt string `` /* 128-byte string literal not displayed */ FingerprintSize helper.ByteSize `` /* 136-byte string literal not displayed */ MaxLogSize helper.ByteSize `` /* 132-byte string literal not displayed */ MaxConcurrentFiles int `` /* 140-byte string literal not displayed */ Splitter helper.SplitterConfig `` /* 127-byte string literal not displayed */ }
Config is the configuration of a file input operator
type FileAttributes ¶
type Finder ¶
type Fingerprint ¶
type Fingerprint struct {
FirstBytes []byte
}
Fingerprint is used to identify a file A file's fingerprint is the first N bytes of the file, where N is the fingerprintSize on the file_input operator
func (Fingerprint) Copy ¶
func (f Fingerprint) Copy() *Fingerprint
Copy creates a new copy of the fingerprint
func (Fingerprint) StartsWith ¶
func (f Fingerprint) StartsWith(old *Fingerprint) bool
StartsWith returns true if the fingerprints are the same or if the new fingerprint starts with the old one This is important functionality for tracking new files, since their initial size is typically less than that of a fingerprint. As the file grows, its fingerprint is updated until it reaches a maximum size, as configured on the operator
type Input ¶
type Input struct { *zap.SugaredLogger PollInterval time.Duration SplitterConfig helper.SplitterConfig MaxLogSize int MaxConcurrentFiles int SeenPaths map[string]struct{} // contains filtered or unexported fields }
TODO rename this struct
func (*Input) NewFingerprint ¶
func (f *Input) NewFingerprint(file *os.File) (*Fingerprint, error)
NewFingerprint creates a new fingerprint from an open file
type PositionalScanner ¶
PositionalScanner is a scanner that maintains position
func NewPositionalScanner ¶
func NewPositionalScanner(r io.Reader, maxLogSize int, startOffset int64, splitFunc bufio.SplitFunc) *PositionalScanner
NewPositionalScanner creates a new positional scanner
func (*PositionalScanner) Pos ¶
func (ps *PositionalScanner) Pos() int64
Pos returns the current position of the scanner
type Reader ¶
type Reader struct { Fingerprint *Fingerprint Offset int64 *zap.SugaredLogger `json:"-"` // contains filtered or unexported fields }
Reader manages a single file
func (*Reader) InitializeOffset ¶
InitializeOffset sets the starting offset