helper

package
v0.12.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 7, 2020 License: Apache-2.0 Imports: 18 Imported by: 24

Documentation

Index

Constants

View Source
const (
	// HTTP2xx is a special key that is represents a range from 200 to 299. Literal value is "2xx"
	HTTP2xx = "2xx"

	// HTTP3xx is a special key that is represents a range from 300 to 399. Literal value is "3xx"
	HTTP3xx = "3xx"

	// HTTP4xx is a special key that is represents a range from 400 to 499. Literal value is "4xx"
	HTTP4xx = "4xx"

	// HTTP5xx is a special key that is represents a range from 500 to 599. Literal value is "5xx"
	HTTP5xx = "5xx"
)
View Source
const DropOnError = "drop"

DropOnError specifies an on_error mode for dropping entries after an error.

View Source
const EpochKey = "epoch"

EpochKey is literally "epoch" and can parse seconds and/or subseconds

View Source
const GotimeKey = "gotime"

GotimeKey is literally "gotime" and uses Golang's native time.Parse

View Source
const NativeKey = "native" // provided for operator development

NativeKey is literally "native" and refers to Golang's native time.Time

View Source
const SendOnError = "send"

SendOnError specifies an on_error mode for sending entries after an error.

View Source
const StrptimeKey = "strptime"

StrptimeKey is literally "strptime", and is the default layout type

Variables

View Source
var OffsetsBucket = []byte(`offsets`)

OffsetsBucket is the scope provided to offset persistence

Functions

func GetExprEnv

func GetExprEnv(e *entry.Entry) map[string]interface{}

GetExprEnv returns a map of key/value pairs that can be be used to evaluate an expression

func PutExprEnv

func PutExprEnv(e map[string]interface{})

PutExprEnv adds a key/value pair that will can be used to evaluate an expression

Types

type BasicConfig

type BasicConfig struct {
	OperatorID   string `json:"id"   yaml:"id"`
	OperatorType string `json:"type" yaml:"type"`
}

BasicConfig provides a basic implemention for an operator config.

func NewBasicConfig

func NewBasicConfig(operatorID, operatorType string) BasicConfig

NewBasicConfig creates a new basic config

func (BasicConfig) Build

func (c BasicConfig) Build(context operator.BuildContext) (BasicOperator, error)

Build will build a basic operator.

func (BasicConfig) ID

func (c BasicConfig) ID() string

ID will return the operator id.

func (BasicConfig) Type

func (c BasicConfig) Type() string

Type will return the operator type.

type BasicOperator

type BasicOperator struct {
	OperatorID   string
	OperatorType string
	*zap.SugaredLogger
}

BasicOperator provides a basic implementation of an operator.

func (*BasicOperator) ID

func (p *BasicOperator) ID() string

ID will return the operator id.

func (*BasicOperator) Logger

func (p *BasicOperator) Logger() *zap.SugaredLogger

Logger returns the operator's scoped logger.

func (*BasicOperator) Start

func (p *BasicOperator) Start() error

Start will start the operator.

func (*BasicOperator) Stop

func (p *BasicOperator) Stop() error

Stop will stop the operator.

func (*BasicOperator) Type

func (p *BasicOperator) Type() string

Type will return the operator type.

type Duration added in v0.10.0

type Duration struct {
	time.Duration
}

Duration is the representation of a length of time

func NewDuration added in v0.10.0

func NewDuration(t time.Duration) Duration

NewDuration creates a new duration from a time

func (Duration) MarshalJSON added in v0.10.0

func (d Duration) MarshalJSON() ([]byte, error)

MarshalJSON will marshal the duration as a json string

func (Duration) MarshalYAML added in v0.10.0

func (d Duration) MarshalYAML() (interface{}, error)

MarshalYAML will marshal the duration as a yaml string

func (*Duration) Raw added in v0.10.0

func (d *Duration) Raw() time.Duration

Raw will return the raw duration, without modification

func (*Duration) UnmarshalJSON added in v0.10.0

func (d *Duration) UnmarshalJSON(raw []byte) error

UnmarshalJSON will unmarshal json as a duration

func (*Duration) UnmarshalYAML added in v0.10.0

func (d *Duration) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML will unmarshal yaml as a duration

type ExprString

type ExprString struct {
	SubStrings []string
	SubExprs   []*vm.Program
}

An ExprString is made up of a list of string literals interleaved with expressions. len(SubStrings) == len(SubExprs) + 1

func (*ExprString) Render

func (e *ExprString) Render(env map[string]interface{}) (string, error)

Render will render an ExprString as a string

type ExprStringConfig

type ExprStringConfig string

ExprStringConfig is a string that represents an expression

func (ExprStringConfig) Build

func (e ExprStringConfig) Build() (*ExprString, error)

Build creates an ExprStr string from the specified config

type HostIdentifier

type HostIdentifier struct {
	// contains filtered or unexported fields
}

HostIdentifier is a helper that adds host related metadata to an entry's resource

func (*HostIdentifier) Identify

func (h *HostIdentifier) Identify(entry *entry.Entry)

Identify will add host related metadata to an entry's resource

type HostIdentifierConfig

type HostIdentifierConfig struct {
	IncludeHostname bool `json:"include_hostname,omitempty"     yaml:"include_hostname,omitempty"`
	IncludeIP       bool `json:"include_ip,omitempty"     yaml:"include_ip,omitempty"`
	// contains filtered or unexported fields
}

HostIdentifierConfig is the configuration of a host identifier

func NewHostIdentifierConfig

func NewHostIdentifierConfig() HostIdentifierConfig

NewHostIdentifierConfig returns a HostIdentifierConfig with default values

func (HostIdentifierConfig) Build

Build will build a host labeler from the supplied configuration

type Identifier

type Identifier struct {
	// contains filtered or unexported fields
}

Identifier is a helper that adds values to the resource of an entry

func (*Identifier) Identify

func (i *Identifier) Identify(e *entry.Entry) error

Identify will add values to the resource of an entry

type IdentifierConfig

type IdentifierConfig struct {
	Resource map[string]ExprStringConfig `json:"resource" yaml:"resource"`
}

IdentifierConfig is the configuration of a resource identifier

func NewIdentifierConfig

func NewIdentifierConfig() IdentifierConfig

NewIdentifierConfig creates a new identifier config with default values

func (IdentifierConfig) Build

func (c IdentifierConfig) Build() (Identifier, error)

Build will build an identifier from the supplied configuration

type InputConfig

type InputConfig struct {
	LabelerConfig    `yaml:",inline"`
	IdentifierConfig `yaml:",inline"`
	WriterConfig     `yaml:",inline"`
	WriteTo          entry.Field `json:"write_to" yaml:"write_to"`
}

InputConfig provides a basic implementation of an input operator config.

func NewInputConfig

func NewInputConfig(operatorID, operatorType string) InputConfig

NewInputConfig creates a new input config with default values.

func (InputConfig) Build

func (c InputConfig) Build(context operator.BuildContext) (InputOperator, error)

Build will build a base producer.

type InputOperator

type InputOperator struct {
	Labeler
	Identifier
	WriterOperator
	WriteTo entry.Field
}

InputOperator provides a basic implementation of an input operator.

func (*InputOperator) CanProcess

func (i *InputOperator) CanProcess() bool

CanProcess will always return false for an input operator.

func (*InputOperator) NewEntry

func (i *InputOperator) NewEntry(value interface{}) (*entry.Entry, error)

NewEntry will create a new entry using the `write_to`, `labels`, and `resource` configuration.

func (*InputOperator) Process

func (i *InputOperator) Process(ctx context.Context, entry *entry.Entry) error

Process will always return an error if called.

type Labeler

type Labeler struct {
	// contains filtered or unexported fields
}

Labeler is a helper that adds labels to an entry

func (*Labeler) Label

func (l *Labeler) Label(e *entry.Entry) error

Label will add labels to an entry

type LabelerConfig

type LabelerConfig struct {
	Labels map[string]ExprStringConfig `json:"labels" yaml:"labels"`
}

LabelerConfig is the configuration of a labeler

func NewLabelerConfig

func NewLabelerConfig() LabelerConfig

NewLabelerConfig creates a new labeler config with default values

func (LabelerConfig) Build

func (c LabelerConfig) Build() (Labeler, error)

Build will build a labeler from the supplied configuration

type OutputConfig

type OutputConfig struct {
	BasicConfig `mapstructure:",squash" yaml:",inline"`
}

OutputConfig provides a basic implementation of an output operator config.

func NewOutputConfig

func NewOutputConfig(operatorID, operatorType string) OutputConfig

NewOutputConfig creates a new output config

func (OutputConfig) Build

Build will build an output operator.

type OutputIDs

type OutputIDs []string

OutputIDs is a collection of operator IDs used as outputs.

func NewOutputIDsFromArray added in v0.12.2

func NewOutputIDsFromArray(array []interface{}) (OutputIDs, error)

NewOutputIDsFromArray creates a new OutputIDs object from an array

func NewOutputIDsFromInterface added in v0.12.2

func NewOutputIDsFromInterface(value interface{}) (OutputIDs, error)

NewOutputIDsFromInterface creates a new OutputIDs object from an interface

func (*OutputIDs) UnmarshalJSON

func (o *OutputIDs) UnmarshalJSON(bytes []byte) error

UnmarshalJSON will unmarshal a string or array of strings to OutputIDs.

func (*OutputIDs) UnmarshalYAML

func (o *OutputIDs) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML will unmarshal a string or array of strings to OutputIDs.

type OutputOperator

type OutputOperator struct {
	BasicOperator
}

OutputOperator provides a basic implementation of an output operator.

func (*OutputOperator) CanOutput

func (o *OutputOperator) CanOutput() bool

CanOutput will always return false for an output operator.

func (*OutputOperator) CanProcess

func (o *OutputOperator) CanProcess() bool

CanProcess will always return true for an output operator.

func (*OutputOperator) Outputs

func (o *OutputOperator) Outputs() []operator.Operator

Outputs will always return an empty array for an output operator.

func (*OutputOperator) SetOutputs

func (o *OutputOperator) SetOutputs(operators []operator.Operator) error

SetOutputs will return an error if called.

type ParseFunction

type ParseFunction = func(interface{}) (interface{}, error)

ParseFunction is function that parses a raw value.

type ParserConfig

type ParserConfig struct {
	TransformerConfig `yaml:",inline"`

	ParseFrom            entry.Field           `json:"parse_from" yaml:"parse_from"`
	ParseTo              entry.Field           `json:"parse_to"   yaml:"parse_to"`
	Preserve             bool                  `json:"preserve"   yaml:"preserve"`
	TimeParser           *TimeParser           `json:"timestamp,omitempty" yaml:"timestamp,omitempty"`
	SeverityParserConfig *SeverityParserConfig `json:"severity,omitempty" yaml:"severity,omitempty"`
}

ParserConfig provides the basic implementation of a parser config.

func NewParserConfig

func NewParserConfig(operatorID, operatorType string) ParserConfig

NewParserConfig creates a new parser config with default values

func (ParserConfig) Build

Build will build a parser operator.

type ParserOperator

type ParserOperator struct {
	TransformerOperator
	ParseFrom      entry.Field
	ParseTo        entry.Field
	Preserve       bool
	TimeParser     *TimeParser
	SeverityParser *SeverityParser
}

ParserOperator provides a basic implementation of a parser operator.

func (*ParserOperator) ProcessWith

func (p *ParserOperator) ProcessWith(ctx context.Context, entry *entry.Entry, parse ParseFunction) error

ProcessWith will process an entry with a parser function.

type Persister

type Persister interface {
	Get(string) []byte
	Set(string, []byte)
	Sync() error
	Load() error
}

Persister is a helper used to persist data

type ScopedBBoltPersister

type ScopedBBoltPersister struct {
	// contains filtered or unexported fields
}

ScopedBBoltPersister is a persister that uses a database for the backend

func NewScopedDBPersister

func NewScopedDBPersister(db database.Database, scope string) *ScopedBBoltPersister

NewScopedDBPersister returns a new ScopedBBoltPersister

func (*ScopedBBoltPersister) Get

func (p *ScopedBBoltPersister) Get(key string) []byte

Get retrieves a key from the cache

func (*ScopedBBoltPersister) Load

func (p *ScopedBBoltPersister) Load() error

Load populates the cache with the values from the database, overwriting anything currently in the cache.

func (*ScopedBBoltPersister) Set

func (p *ScopedBBoltPersister) Set(key string, val []byte)

Set saves a key in the cache

func (*ScopedBBoltPersister) Sync

func (p *ScopedBBoltPersister) Sync() error

Sync saves the cache to the backend, ensuring values are safely written to disk before returning

type SeverityParser

type SeverityParser struct {
	ParseFrom entry.Field
	Preserve  bool
	Mapping   severityMap
}

SeverityParser is a helper that parses severity onto an entry.

func (*SeverityParser) Parse

func (p *SeverityParser) Parse(ctx context.Context, ent *entry.Entry) error

Parse will parse severity from a field and attach it to the entry

type SeverityParserConfig

type SeverityParserConfig struct {
	ParseFrom *entry.Field                `json:"parse_from,omitempty" yaml:"parse_from,omitempty"`
	Preserve  bool                        `json:"preserve,omitempty"   yaml:"preserve,omitempty"`
	Preset    string                      `json:"preset,omitempty"     yaml:"preset,omitempty"`
	Mapping   map[interface{}]interface{} `json:"mapping,omitempty"    yaml:"mapping,omitempty"`
}

SeverityParserConfig allows users to specify how to parse a severity from a field.

func NewSeverityParserConfig

func NewSeverityParserConfig() SeverityParserConfig

NewSeverityParserConfig creates a new severity parser config

func (*SeverityParserConfig) Build

Build builds a SeverityParser from a SeverityParserConfig

type TimeParser

type TimeParser struct {
	ParseFrom  *entry.Field `json:"parse_from,omitempty"  yaml:"parse_from,omitempty"`
	Layout     string       `json:"layout,omitempty"      yaml:"layout,omitempty"`
	LayoutType string       `json:"layout_type,omitempty" yaml:"layout_type,omitempty"`
	Preserve   bool         `json:"preserve"              yaml:"preserve"`
}

TimeParser is a helper that parses time onto an entry.

func NewTimeParser

func NewTimeParser() TimeParser

NewTimeParser creates a new time parser with default values

func (*TimeParser) IsZero

func (t *TimeParser) IsZero() bool

IsZero returns true if the TimeParser is not a valid config

func (*TimeParser) Parse

func (t *TimeParser) Parse(ctx context.Context, entry *entry.Entry) error

Parse will parse time from a field and attach it to the entry

func (*TimeParser) Validate

func (t *TimeParser) Validate(context operator.BuildContext) error

Validate validates a TimeParser, and reconfigures it if necessary

type TransformFunction

type TransformFunction = func(*entry.Entry) (*entry.Entry, error)

TransformFunction is function that transforms an entry.

type TransformerConfig

type TransformerConfig struct {
	WriterConfig `yaml:",inline"`
	OnError      string `json:"on_error" yaml:"on_error"`
}

TransformerConfig provides a basic implementation of a transformer config.

func NewTransformerConfig

func NewTransformerConfig(operatorID, operatorType string) TransformerConfig

NewTransformerConfig creates a new transformer config with default values

func (TransformerConfig) Build

Build will build a transformer operator.

type TransformerOperator

type TransformerOperator struct {
	WriterOperator
	OnError string
}

TransformerOperator provides a basic implementation of a transformer operator.

func (*TransformerOperator) CanProcess

func (t *TransformerOperator) CanProcess() bool

CanProcess will always return true for a transformer operator.

func (*TransformerOperator) HandleEntryError

func (t *TransformerOperator) HandleEntryError(ctx context.Context, entry *entry.Entry, err error) error

HandleEntryError will handle an entry error using the on_error strategy.

func (*TransformerOperator) ProcessWith

func (t *TransformerOperator) ProcessWith(ctx context.Context, entry *entry.Entry, transform TransformFunction) error

ProcessWith will process an entry with a transform function.

type WriterConfig

type WriterConfig struct {
	BasicConfig `yaml:",inline"`
	OutputIDs   OutputIDs `json:"output" yaml:"output"`
}

WriterConfig is the configuration of a writer operator.

func NewWriterConfig

func NewWriterConfig(operatorID, operatorType string) WriterConfig

NewWriterConfig creates a new writer config

func (WriterConfig) Build

Build will build a writer operator from the config.

type WriterOperator

type WriterOperator struct {
	BasicOperator
	OutputIDs       OutputIDs
	OutputOperators []operator.Operator
}

WriterOperator is an operator that can write to other operators.

func (*WriterOperator) CanOutput

func (w *WriterOperator) CanOutput() bool

CanOutput always returns true for a writer operator.

func (*WriterOperator) Outputs

func (w *WriterOperator) Outputs() []operator.Operator

Outputs returns the outputs of the writer operator.

func (*WriterOperator) SetOutputs

func (w *WriterOperator) SetOutputs(operators []operator.Operator) error

SetOutputs will set the outputs of the operator.

func (*WriterOperator) Write

func (w *WriterOperator) Write(ctx context.Context, e *entry.Entry)

Write will write an entry to the outputs of the operator.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL