helper

package
v0.80.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2023 License: Apache-2.0 Imports: 27 Imported by: 18

Documentation

Index

Constants

View Source 
const (
	// HTTP2xx is a special key that is represents a range from 200 to 299. Literal value is "2xx"
	HTTP2xx = "2xx"

	// HTTP3xx is a special key that is represents a range from 300 to 399. Literal value is "3xx"
	HTTP3xx = "3xx"

	// HTTP4xx is a special key that is represents a range from 400 to 499. Literal value is "4xx"
	HTTP4xx = "4xx"

	// HTTP5xx is a special key that is represents a range from 500 to 599. Literal value is "5xx"
	HTTP5xx = "5xx"
)
View Source 
const DropOnError = "drop"

DropOnError specifies an on_error mode for dropping entries after an error.

View Source 
const EpochKey = "epoch"

EpochKey is literally "epoch" and can parse seconds and/or subseconds

View Source 
const GotimeKey = "gotime"

GotimeKey is literally "gotime" and uses Golang's native time.Parse

View Source 
const NativeKey = "native" // provided for operator development

NativeKey is literally "native" and refers to Golang's native time.Time

View Source 
const SendOnError = "send"

SendOnError specifies an on_error mode for sending entries after an error.

View Source 
const StrptimeKey = "strptime"

StrptimeKey is literally "strptime", and is the default layout type

Variables

This section is empty.

Functions

func GetExprEnv 

func GetExprEnv(e *entry.Entry) map[string]interface{}

GetExprEnv returns a map of key/value pairs that can be be used to evaluate an expression

func IsNop added in v0.55.0 

func IsNop(enc string) bool

func NewLineEndSplitFunc 

func NewLineEndSplitFunc(re *regexp.Regexp, flushAtEOF bool, trimFunc trimFunc) bufio.SplitFunc

NewLineEndSplitFunc creates a bufio.SplitFunc that splits an incoming stream into tokens that end with a match to the regex pattern provided

func NewLineStartSplitFunc 

func NewLineStartSplitFunc(re *regexp.Regexp, flushAtEOF bool, trimFunc trimFunc) bufio.SplitFunc

NewLineStartSplitFunc creates a bufio.SplitFunc that splits an incoming stream into tokens that start with a match to the regex pattern provided

func NewNewlineSplitFunc 

func NewNewlineSplitFunc(enc encoding.Encoding, flushAtEOF bool, trimFunc trimFunc) (bufio.SplitFunc, error)

NewNewlineSplitFunc splits log lines by newline, just as bufio.ScanLines, but never returning an token using EOF as a terminator

func PutExprEnv 

func PutExprEnv(e map[string]interface{})

PutExprEnv adds a key/value pair that will can be used to evaluate an expression

func SplitNone 

func SplitNone(maxLogSize int) bufio.SplitFunc

SplitNone doesn't split any of the bytes, it reads in all of the bytes and returns it all at once. This is for when the encoding is nop

Types

type Attributer 

type Attributer struct {
	// contains filtered or unexported fields
}

Attributer is a helper that adds attributes to an entry

func (*Attributer) Attribute 

func (l *Attributer) Attribute(e *entry.Entry) error

Attribute will add attributes to an entry

type AttributerConfig 

type AttributerConfig struct {
	Attributes map[string]ExprStringConfig `mapstructure:"attributes"`
}

AttributerConfig is the configuration of a attributer

func NewAttributerConfig 

func NewAttributerConfig() AttributerConfig

NewAttributerConfig creates a new attributer config with default values

func (AttributerConfig) Build 

func (c AttributerConfig) Build() (Attributer, error)

Build will build a attributer from the supplied configuration

type BasicConfig 

type BasicConfig struct {
	OperatorID   string `mapstructure:"id"`
	OperatorType string `mapstructure:"type"`
}

BasicConfig provides a basic implemention for an operator config.

func NewBasicConfig 

func NewBasicConfig(operatorID, operatorType string) BasicConfig

NewBasicConfig creates a new basic config

func (BasicConfig) Build 

func (c BasicConfig) Build(logger *zap.SugaredLogger) (BasicOperator, error)

Build will build a basic operator.

func (BasicConfig) ID 

func (c BasicConfig) ID() string

ID will return the operator id.

func (*BasicConfig) SetID 

func (c *BasicConfig) SetID(id string)

SetID will Update the operator id.

func (BasicConfig) Type 

func (c BasicConfig) Type() string

Type will return the operator type.

type BasicOperator 

type BasicOperator struct {
	OperatorID   string
	OperatorType string
	*zap.SugaredLogger
}

BasicOperator provides a basic implementation of an operator.

func (*BasicOperator) ID 

func (p *BasicOperator) ID() string

ID will return the operator id.

func (*BasicOperator) Logger 

func (p *BasicOperator) Logger() *zap.SugaredLogger

Logger returns the operator's scoped logger.

func (*BasicOperator) Start 

func (p *BasicOperator) Start(_ operator.Persister) error

Start will start the operator.

func (*BasicOperator) Stop 

func (p *BasicOperator) Stop() error

Stop will stop the operator.

func (*BasicOperator) Type 

func (p *BasicOperator) Type() string

Type will return the operator type.

type ByteSize 

type ByteSize int64

func (*ByteSize) UnmarshalText 

func (h *ByteSize) UnmarshalText(text []byte) (err error)

type Encoding 

type Encoding struct {
	Encoding encoding.Encoding
	// contains filtered or unexported fields
}

func (*Encoding) Decode 

func (e *Encoding) Decode(msgBuf []byte) ([]byte, error)

Decode converts the bytes in msgBuf to utf-8 from the configured encoding

type EncodingConfig 

type EncodingConfig struct {
	Encoding string `mapstructure:"encoding,omitempty"`
}

EncodingConfig is the configuration of a Encoding helper

func NewEncodingConfig 

func NewEncodingConfig() EncodingConfig

NewBasicConfig creates a new Encoding config

func (EncodingConfig) Build 

func (c EncodingConfig) Build() (Encoding, error)

Build will build an Encoding operator.

type ExprString 

type ExprString struct {
	SubStrings []string
	SubExprs   []*vm.Program
}

An ExprString is made up of a list of string literals interleaved with expressions. len(SubStrings) == len(SubExprs) + 1

func (*ExprString) Render 

func (e *ExprString) Render(env map[string]interface{}) (string, error)

Render will render an ExprString as a string

type ExprStringConfig 

type ExprStringConfig string

ExprStringConfig is a string that represents an expression

func (ExprStringConfig) Build 

func (e ExprStringConfig) Build() (*ExprString, error)

Build creates an ExprStr string from the specified config

type Flusher 

type Flusher struct {
	// contains filtered or unexported fields
}

Flusher keeps information about flush state

func (*Flusher) Flushed 

func (f *Flusher) Flushed()

Flushed reset data length

func (*Flusher) ShouldFlush 

func (f *Flusher) ShouldFlush() bool

ShouldFlush returns true if data should be forcefully flushed

func (*Flusher) SplitFunc 

func (f *Flusher) SplitFunc(splitFunc bufio.SplitFunc) bufio.SplitFunc

func (*Flusher) UpdateDataChangeTime 

func (f *Flusher) UpdateDataChangeTime(length int)

type FlusherConfig 

type FlusherConfig struct {
	Period time.Duration `mapstructure:"force_flush_period"`
}

FlusherConfig is a configuration of Flusher helper

func NewFlusherConfig 

func NewFlusherConfig() FlusherConfig

NewFlusherConfig creates a default Flusher config

func (*FlusherConfig) Build 

func (c *FlusherConfig) Build() *Flusher

Build creates Flusher from configuration

type IPResolver 

type IPResolver struct {
	// contains filtered or unexported fields
}

func NewIPResolver added in v0.53.0 

func NewIPResolver() *IPResolver

Create new resolver

func (*IPResolver) GetHostFromIP added in v0.53.0 

func (r *IPResolver) GetHostFromIP(ip string) (host string)

GetHostFromIp returns hostname for given ip It is taken from cache if exists, otherwise lookup is performed and result is put into cache

func (*IPResolver) Stop 

func (r *IPResolver) Stop()

Stop cache invalidation

type Identifier 

type Identifier struct {
	// contains filtered or unexported fields
}

Identifier is a helper that adds values to the resource of an entry

func (*Identifier) Identify 

func (i *Identifier) Identify(e *entry.Entry) error

Identify will add values to the resource of an entry

type IdentifierConfig 

type IdentifierConfig struct {
	Resource map[string]ExprStringConfig `mapstructure:"resource"`
}

IdentifierConfig is the configuration of a resource identifier

func NewIdentifierConfig 

func NewIdentifierConfig() IdentifierConfig

NewIdentifierConfig creates a new identifier config with default values

func (IdentifierConfig) Build 

func (c IdentifierConfig) Build() (Identifier, error)

Build will build an identifier from the supplied configuration

type InputConfig 

type InputConfig struct {
	AttributerConfig `mapstructure:",squash"`
	IdentifierConfig `mapstructure:",squash"`
	WriterConfig     `mapstructure:",squash"`
}

InputConfig provides a basic implementation of an input operator config.

func NewInputConfig 

func NewInputConfig(operatorID, operatorType string) InputConfig

NewInputConfig creates a new input config with default values.

func (InputConfig) Build 

func (c InputConfig) Build(logger *zap.SugaredLogger) (InputOperator, error)

Build will build a base producer.

type InputOperator 

type InputOperator struct {
	Attributer
	Identifier
	WriterOperator
}

InputOperator provides a basic implementation of an input operator.

func (*InputOperator) CanProcess 

func (i *InputOperator) CanProcess() bool

CanProcess will always return false for an input operator.

func (*InputOperator) NewEntry 

func (i *InputOperator) NewEntry(value interface{}) (*entry.Entry, error)

NewEntry will create a new entry using the `attributes`, and `resource` configuration.

func (*InputOperator) Process 

func (i *InputOperator) Process(_ context.Context, entry *entry.Entry) error

Process will always return an error if called.

type Multiline 

type Multiline struct {
	SplitFunc bufio.SplitFunc
	Force     *Flusher
}

Multiline consists of splitFunc and variables needed to perform force flush

type MultilineConfig 

type MultilineConfig struct {
	LineStartPattern string `mapstructure:"line_start_pattern"`
	LineEndPattern   string `mapstructure:"line_end_pattern"`
}

MultilineConfig is the configuration of a multiline helper

func NewMultilineConfig 

func NewMultilineConfig() MultilineConfig

NewBasicConfig creates a new Multiline config

func (MultilineConfig) Build 

func (c MultilineConfig) Build(enc encoding.Encoding, flushAtEOF, preserveLeadingWhitespaces, preserveTrailingWhitespaces bool, force *Flusher, maxLogSize int) (bufio.SplitFunc, error)

Build will build a Multiline operator.

type OutputConfig 

type OutputConfig struct {
	BasicConfig `mapstructure:",squash"`
}

OutputConfig provides a basic implementation of an output operator config.

func NewOutputConfig 

func NewOutputConfig(operatorID, operatorType string) OutputConfig

NewOutputConfig creates a new output config

func (OutputConfig) Build 

func (c OutputConfig) Build(logger *zap.SugaredLogger) (OutputOperator, error)

Build will build an output operator.

type OutputOperator 

type OutputOperator struct {
	BasicOperator
}

OutputOperator provides a basic implementation of an output operator.

func (*OutputOperator) CanOutput 

func (o *OutputOperator) CanOutput() bool

CanOutput will always return false for an output operator.

func (*OutputOperator) CanProcess 

func (o *OutputOperator) CanProcess() bool

CanProcess will always return true for an output operator.

func (*OutputOperator) GetOutputIDs 

func (o *OutputOperator) GetOutputIDs() []string

GetOutputIDs will always return an empty array for an output ID.

func (*OutputOperator) Outputs 

func (o *OutputOperator) Outputs() []operator.Operator

Outputs will always return an empty array for an output operator.

func (*OutputOperator) SetOutputIDs 

func (o *OutputOperator) SetOutputIDs(_ []string)

SetOutputIDs will return nothing and does nothing.

func (*OutputOperator) SetOutputs 

func (o *OutputOperator) SetOutputs(_ []operator.Operator) error

SetOutputs will return an error if called.

type ParseFunction 

type ParseFunction = func(interface{}) (interface{}, error)

ParseFunction is function that parses a raw value.

type ParserConfig 

type ParserConfig struct {
	TransformerConfig `mapstructure:",squash"`
	ParseFrom         entry.Field         `mapstructure:"parse_from"`
	ParseTo           entry.RootableField `mapstructure:"parse_to"`
	BodyField         *entry.Field        `mapstructure:"body"`
	TimeParser        *TimeParser         `mapstructure:"timestamp,omitempty"`
	SeverityConfig    *SeverityConfig     `mapstructure:"severity,omitempty"`
	TraceParser       *TraceParser        `mapstructure:"trace,omitempty"`
	ScopeNameParser   *ScopeNameParser    `mapstructure:"scope_name,omitempty"`
}

ParserConfig provides the basic implementation of a parser config.

func NewParserConfig 

func NewParserConfig(operatorID, operatorType string) ParserConfig

NewParserConfig creates a new parser config with default values

func (ParserConfig) Build 

func (c ParserConfig) Build(logger *zap.SugaredLogger) (ParserOperator, error)

Build will build a parser operator.

type ParserOperator 

type ParserOperator struct {
	TransformerOperator
	ParseFrom       entry.Field
	ParseTo         entry.Field
	BodyField       *entry.Field
	TimeParser      *TimeParser
	SeverityParser  *SeverityParser
	TraceParser     *TraceParser
	ScopeNameParser *ScopeNameParser
}

ParserOperator provides a basic implementation of a parser operator.

func (*ParserOperator) ParseWith 

func (p *ParserOperator) ParseWith(ctx context.Context, entry *entry.Entry, parse ParseFunction) error

ParseWith will process an entry's field with a parser function.

func (*ParserOperator) ProcessWith 

func (p *ParserOperator) ProcessWith(ctx context.Context, entry *entry.Entry, parse ParseFunction) error

ProcessWith will run ParseWith on the entry, then forward the entry on to the next operators.

func (*ParserOperator) ProcessWithCallback 

func (p *ParserOperator) ProcessWithCallback(ctx context.Context, entry *entry.Entry, parse ParseFunction, cb func(*entry.Entry) error) error

type ScopeNameParser 

type ScopeNameParser struct {
	ParseFrom entry.Field `mapstructure:"parse_from,omitempty"`
}

ScopeNameParser is a helper that parses severity onto an entry.

func NewScopeNameParser 

func NewScopeNameParser() ScopeNameParser

NewScopeNameParser creates a new scope parser with default values

func (*ScopeNameParser) Parse 

func (p *ScopeNameParser) Parse(ent *entry.Entry) error

Parse will parse severity from a field and attach it to the entry

type SeverityConfig added in v0.55.0 

type SeverityConfig struct {
	ParseFrom *entry.Field           `mapstructure:"parse_from,omitempty"`
	Preset    string                 `mapstructure:"preset,omitempty"`
	Mapping   map[string]interface{} `mapstructure:"mapping,omitempty"`
}

SeverityConfig allows users to specify how to parse a severity from a field.

func NewSeverityConfig added in v0.55.0 

func NewSeverityConfig() SeverityConfig

NewSeverityConfig creates a new severity parser config

func (*SeverityConfig) Build added in v0.55.0 

func (c *SeverityConfig) Build(_ *zap.SugaredLogger) (SeverityParser, error)

Build builds a SeverityParser from a SeverityConfig

type SeverityParser 

type SeverityParser struct {
	ParseFrom entry.Field
	Mapping   severityMap
}

SeverityParser is a helper that parses severity onto an entry.

func (*SeverityParser) Parse 

func (p *SeverityParser) Parse(ent *entry.Entry) error

Parse will parse severity from a field and attach it to the entry

type SpanIDConfig added in v0.53.0 

type SpanIDConfig struct {
	ParseFrom *entry.Field `mapstructure:"parse_from,omitempty"`
}

type Splitter 

type Splitter struct {
	Encoding  Encoding
	SplitFunc bufio.SplitFunc
	Flusher   *Flusher
}

Splitter consolidates Flusher and dependent splitFunc

type SplitterConfig 

type SplitterConfig struct {
	EncodingConfig              EncodingConfig  `mapstructure:",squash,omitempty"`
	Flusher                     FlusherConfig   `mapstructure:",squash,omitempty"`
	Multiline                   MultilineConfig `mapstructure:"multiline,omitempty"`
	PreserveLeadingWhitespaces  bool            `mapstructure:"preserve_leading_whitespaces,omitempty"`
	PreserveTrailingWhitespaces bool            `mapstructure:"preserve_trailing_whitespaces,omitempty"`
}

SplitterConfig consolidates MultilineConfig and FlusherConfig

func NewSplitterConfig 

func NewSplitterConfig() SplitterConfig

NewSplitterConfig returns default SplitterConfig

func (*SplitterConfig) Build 

func (c *SplitterConfig) Build(flushAtEOF bool, maxLogSize int) (*Splitter, error)

Build builds Splitter struct

type TimeParser 

type TimeParser struct {
	ParseFrom  *entry.Field `mapstructure:"parse_from"`
	Layout     string       `mapstructure:"layout"`
	LayoutType string       `mapstructure:"layout_type"`
	Location   string       `mapstructure:"location"`
	// contains filtered or unexported fields
}

TimeParser is a helper that parses time onto an entry.

func NewTimeParser 

func NewTimeParser() TimeParser

NewTimeParser creates a new time parser with default values

func (*TimeParser) IsZero 

func (t *TimeParser) IsZero() bool

IsZero returns true if the TimeParser is not a valid config

func (*TimeParser) Parse 

func (t *TimeParser) Parse(entry *entry.Entry) error

Parse will parse time from a field and attach it to the entry

func (*TimeParser) Unmarshal added in v0.61.0 

func (t *TimeParser) Unmarshal(component *confmap.Conf) error

Unmarshal starting from default settings

func (*TimeParser) Validate 

func (t *TimeParser) Validate() error

Validate validates a TimeParser, and reconfigures it if necessary

type TraceFlagsConfig 

type TraceFlagsConfig struct {
	ParseFrom *entry.Field `mapstructure:"parse_from,omitempty"`
}

type TraceIDConfig added in v0.53.0 

type TraceIDConfig struct {
	ParseFrom *entry.Field `mapstructure:"parse_from,omitempty"`
}

type TraceParser 

type TraceParser struct {
	TraceID    *TraceIDConfig    `mapstructure:"trace_id,omitempty"`
	SpanID     *SpanIDConfig     `mapstructure:"span_id,omitempty"`
	TraceFlags *TraceFlagsConfig `mapstructure:"trace_flags,omitempty"`
}

TraceParser is a helper that parses trace spans (and flags) onto an entry.

func NewTraceParser 

func NewTraceParser() TraceParser

NewTraceParser creates a new trace parser with default values

func (*TraceParser) Parse 

func (t *TraceParser) Parse(entry *entry.Entry) error

Parse will parse a trace (trace_id, span_id and flags) from a field and attach it to the entry

func (*TraceParser) Validate 

func (t *TraceParser) Validate() error

Validate validates a TraceParser, and reconfigures it if necessary

type TransformFunction 

type TransformFunction = func(*entry.Entry) error

TransformFunction is function that transforms an entry.

type TransformerConfig 

type TransformerConfig struct {
	WriterConfig `mapstructure:",squash"`
	OnError      string `mapstructure:"on_error"`
	IfExpr       string `mapstructure:"if"`
}

TransformerConfig provides a basic implementation of a transformer config.

func NewTransformerConfig 

func NewTransformerConfig(operatorID, operatorType string) TransformerConfig

NewTransformerConfig creates a new transformer config with default values

func (TransformerConfig) Build 

func (c TransformerConfig) Build(logger *zap.SugaredLogger) (TransformerOperator, error)

Build will build a transformer operator.

type TransformerOperator 

type TransformerOperator struct {
	WriterOperator
	OnError string
	IfExpr  *vm.Program
}

TransformerOperator provides a basic implementation of a transformer operator.

func (*TransformerOperator) CanProcess 

func (t *TransformerOperator) CanProcess() bool

CanProcess will always return true for a transformer operator.

func (*TransformerOperator) HandleEntryError 

func (t *TransformerOperator) HandleEntryError(ctx context.Context, entry *entry.Entry, err error) error

HandleEntryError will handle an entry error using the on_error strategy.

func (*TransformerOperator) ProcessWith 

func (t *TransformerOperator) ProcessWith(ctx context.Context, entry *entry.Entry, transform TransformFunction) error

ProcessWith will process an entry with a transform function.

func (*TransformerOperator) Skip 

func (t *TransformerOperator) Skip(_ context.Context, entry *entry.Entry) (bool, error)

type WriterConfig 

type WriterConfig struct {
	BasicConfig `mapstructure:",squash"`
	OutputIDs   []string `mapstructure:"output"`
}

WriterConfig is the configuration of a writer operator.

func NewWriterConfig 

func NewWriterConfig(operatorID, operatorType string) WriterConfig

NewWriterConfig creates a new writer config

func (WriterConfig) Build 

func (c WriterConfig) Build(logger *zap.SugaredLogger) (WriterOperator, error)

Build will build a writer operator from the config.

type WriterOperator 

type WriterOperator struct {
	BasicOperator
	OutputIDs       []string
	OutputOperators []operator.Operator
}

WriterOperator is an operator that can write to other operators.

func (*WriterOperator) CanOutput 

func (w *WriterOperator) CanOutput() bool

CanOutput always returns true for a writer operator.

func (*WriterOperator) GetOutputIDs 

func (w *WriterOperator) GetOutputIDs() []string

GetOutputIDs returns the output IDs of the writer operator.

func (*WriterOperator) Outputs 

func (w *WriterOperator) Outputs() []operator.Operator

Outputs returns the outputs of the writer operator.

func (*WriterOperator) SetOutputIDs 

func (w *WriterOperator) SetOutputIDs(opIds []string)

SetOutputIDs will set the outputs of the operator.

func (*WriterOperator) SetOutputs 

func (w *WriterOperator) SetOutputs(operators []operator.Operator) error

SetOutputs will set the outputs of the operator.

func (*WriterOperator) Write 

func (w *WriterOperator) Write(ctx context.Context, e *entry.Entry)

Write will write an entry to the outputs of the operator.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.