helper

package
v0.9.14 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 31, 2020 License: Apache-2.0 Imports: 17 Imported by: 24

Documentation

Index

Constants

View Source 
const (
	// HTTP2xx is a special key that is represents a range from 200 to 299. Literal value is "2xx"
	HTTP2xx = "2xx"

	// HTTP3xx is a special key that is represents a range from 300 to 399. Literal value is "3xx"
	HTTP3xx = "3xx"

	// HTTP4xx is a special key that is represents a range from 400 to 499. Literal value is "4xx"
	HTTP4xx = "4xx"

	// HTTP5xx is a special key that is represents a range from 500 to 599. Literal value is "5xx"
	HTTP5xx = "5xx"
)
View Source 
const DropOnError = "drop"

DropOnError specifies an on_error mode for dropping entries after an error.

View Source 
const EpochKey = "epoch"

EpochKey is literally "epoch" and can parse seconds and/or subseconds

View Source 
const GotimeKey = "gotime"

GotimeKey is literally "gotime" and uses Golang's native time.Parse

View Source 
const NativeKey = "native" // provided for operator development

NativeKey is literally "native" and refers to Golang's native time.Time

View Source 
const SendOnError = "send"

SendOnError specifies an on_error mode for sending entries after an error.

View Source 
const StrptimeKey = "strptime"

StrptimeKey is literally "strptime", and is the default layout type

Variables

View Source 
var OffsetsBucket = []byte(`offsets`)

OffsetsBucket is the scope provided to offset persistence

Functions

func AddNamespace 

func AddNamespace(id string, namespace string) string

AddNamespace will add a namespace to an id.

func CanNamespace 

func CanNamespace(id string, exclusions []string) bool

CanNamespace will return a boolean indicating if an id can be namespaced.

func GetExprEnv 

func GetExprEnv(e *entry.Entry) map[string]interface{}

func PutExprEnv 

func PutExprEnv(e map[string]interface{})

Types

type BasicConfig 

type BasicConfig struct {
	OperatorID   string `json:"id"   yaml:"id"`
	OperatorType string `json:"type" yaml:"type"`
}

BasicConfig provides a basic implemention for an operator config.

func NewBasicConfig 

func NewBasicConfig(operatorID, operatorType string) BasicConfig

func (BasicConfig) Build 

func (c BasicConfig) Build(context operator.BuildContext) (BasicOperator, error)

Build will build a basic operator.

func (BasicConfig) ID 

func (c BasicConfig) ID() string

ID will return the operator id.

func (*BasicConfig) SetNamespace 

func (c *BasicConfig) SetNamespace(namespace string, exclusions ...string)

SetNamespace will namespace the operator id.

func (BasicConfig) Type 

func (c BasicConfig) Type() string

Type will return the operator type.

type BasicOperator 

type BasicOperator struct {
	OperatorID   string
	OperatorType string
	*zap.SugaredLogger
}

BasicOperator provides a basic implementation of an operator.

func (*BasicOperator) ID 

func (p *BasicOperator) ID() string

ID will return the operator id.

func (*BasicOperator) Logger 

func (p *BasicOperator) Logger() *zap.SugaredLogger

Logger returns the operator's scoped logger.

func (*BasicOperator) Start 

func (p *BasicOperator) Start() error

Start will start the operator.

func (*BasicOperator) Stop 

func (p *BasicOperator) Stop() error

Stop will stop the operator.

func (*BasicOperator) Type 

func (p *BasicOperator) Type() string

Type will return the operator type.

type ExprString 

type ExprString struct {
	SubStrings []string
	SubExprs   []*vm.Program
}

An ExprString is made up of a list of string literals interleaved with expressions. len(SubStrings) == len(SubExprs) + 1

func (*ExprString) Render 

func (e *ExprString) Render(env map[string]interface{}) (string, error)

type ExprStringConfig 

type ExprStringConfig string

func (ExprStringConfig) Build 

func (e ExprStringConfig) Build() (*ExprString, error)

type HostIdentifier 

type HostIdentifier struct {
	// contains filtered or unexported fields
}

HostIdentifier is a helper that adds host related metadata to an entry's resource

func (*HostIdentifier) Identify 

func (h *HostIdentifier) Identify(entry *entry.Entry)

Identify will add host related metadata to an entry's resource

type HostIdentifierConfig 

type HostIdentifierConfig struct {
	IncludeHostname bool `json:"include_hostname,omitempty"     yaml:"include_hostname,omitempty"`
	IncludeIP       bool `json:"include_ip,omitempty"     yaml:"include_ip,omitempty"`
	// contains filtered or unexported fields
}

HostIdentifierConfig is the configuration of a host identifier

func NewHostIdentifierConfig 

func NewHostIdentifierConfig() HostIdentifierConfig

NewHostIdentifierConfig returns a HostIdentifierConfig with default values

func (HostIdentifierConfig) Build 

func (c HostIdentifierConfig) Build() (HostIdentifier, error)

Build will build a host labeler from the supplied configuration

type Identifier 

type Identifier struct {
	// contains filtered or unexported fields
}

Identifier is a helper that adds values to the resource of an entry

func (*Identifier) Identify 

func (i *Identifier) Identify(e *entry.Entry) error

Identify will add values to the resource of an entry

type IdentifierConfig 

type IdentifierConfig struct {
	Resource map[string]ExprStringConfig `json:"resource" yaml:"resource"`
}

IdentifierConfig is the configuration of a resource identifier

func NewIdentifierConfig 

func NewIdentifierConfig() IdentifierConfig

NewIdentifierConfig creates a new identifier config with default values

func (IdentifierConfig) Build 

func (c IdentifierConfig) Build() (Identifier, error)

Build will build an identifier from the supplied configuration

type InputConfig 

type InputConfig struct {
	LabelerConfig    `yaml:",inline"`
	IdentifierConfig `yaml:",inline"`
	WriterConfig     `yaml:",inline"`
	WriteTo          entry.Field `json:"write_to" yaml:"write_to"`
}

InputConfig provides a basic implementation of an input operator config.

func NewInputConfig 

func NewInputConfig(operatorID, operatorType string) InputConfig

NewInputConfig creates a new input config with default values.

func (InputConfig) Build 

func (c InputConfig) Build(context operator.BuildContext) (InputOperator, error)

Build will build a base producer.

type InputOperator 

type InputOperator struct {
	Labeler
	Identifier
	WriterOperator
	WriteTo entry.Field
}

InputOperator provides a basic implementation of an input operator.

func (*InputOperator) CanProcess 

func (i *InputOperator) CanProcess() bool

CanProcess will always return false for an input operator.

func (*InputOperator) NewEntry 

func (i *InputOperator) NewEntry(value interface{}) (*entry.Entry, error)

NewEntry will create a new entry using the `write_to`, `labels`, and `resource` configuration.

func (*InputOperator) Process 

func (i *InputOperator) Process(ctx context.Context, entry *entry.Entry) error

Process will always return an error if called.

type Labeler 

type Labeler struct {
	// contains filtered or unexported fields
}

Labeler is a helper that adds labels to an entry

func (*Labeler) Label 

func (l *Labeler) Label(e *entry.Entry) error

Label will add labels to an entry

type LabelerConfig 

type LabelerConfig struct {
	Labels map[string]ExprStringConfig `json:"labels" yaml:"labels"`
}

LabelerConfig is the configuration of a labeler

func NewLabelerConfig 

func NewLabelerConfig() LabelerConfig

NewLabelerConfig creates a new labeler config with default values

func (LabelerConfig) Build 

func (c LabelerConfig) Build() (Labeler, error)

Build will build a labeler from the supplied configuration

type OutputConfig 

type OutputConfig struct {
	BasicConfig `mapstructure:",squash" yaml:",inline"`
}

OutputConfig provides a basic implementation of an output operator config.

func NewOutputConfig 

func NewOutputConfig(operatorID, operatorType string) OutputConfig

func (OutputConfig) Build 

func (c OutputConfig) Build(context operator.BuildContext) (OutputOperator, error)

Build will build an output operator.

func (*OutputConfig) SetNamespace 

func (c *OutputConfig) SetNamespace(namespace string, exclusions ...string)

SetNamespace will namespace the id and output of the operator config.

type OutputIDs 

type OutputIDs []string

OutputIDs is a collection of operator IDs used as outputs.

func (*OutputIDs) UnmarshalJSON 

func (o *OutputIDs) UnmarshalJSON(bytes []byte) error

UnmarshalJSON will unmarshal a string or array of strings to OutputIDs.

func (*OutputIDs) UnmarshalYAML 

func (o *OutputIDs) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML will unmarshal a string or array of strings to OutputIDs.

type OutputOperator 

type OutputOperator struct {
	BasicOperator
}

OutputOperator provides a basic implementation of an output operator.

func (*OutputOperator) CanOutput 

func (o *OutputOperator) CanOutput() bool

CanOutput will always return false for an output operator.

func (*OutputOperator) CanProcess 

func (o *OutputOperator) CanProcess() bool

CanProcess will always return true for an output operator.

func (*OutputOperator) Outputs 

func (o *OutputOperator) Outputs() []operator.Operator

Outputs will always return an empty array for an output operator.

func (*OutputOperator) SetOutputs 

func (o *OutputOperator) SetOutputs(operators []operator.Operator) error

SetOutputs will return an error if called.

type ParseFunction 

type ParseFunction = func(interface{}) (interface{}, error)

ParseFunction is function that parses a raw value.

type ParserConfig 

type ParserConfig struct {
	TransformerConfig `yaml:",inline"`

	ParseFrom            entry.Field           `json:"parse_from" yaml:"parse_from"`
	ParseTo              entry.Field           `json:"parse_to"   yaml:"parse_to"`
	Preserve             bool                  `json:"preserve"   yaml:"preserve"`
	TimeParser           *TimeParser           `json:"timestamp,omitempty" yaml:"timestamp,omitempty"`
	SeverityParserConfig *SeverityParserConfig `json:"severity,omitempty" yaml:"severity,omitempty"`
}

ParserConfig provides the basic implementation of a parser config.

func NewParserConfig 

func NewParserConfig(operatorID, operatorType string) ParserConfig

func (ParserConfig) Build 

func (c ParserConfig) Build(context operator.BuildContext) (ParserOperator, error)

Build will build a parser operator.

type ParserOperator 

type ParserOperator struct {
	TransformerOperator
	ParseFrom      entry.Field
	ParseTo        entry.Field
	Preserve       bool
	TimeParser     *TimeParser
	SeverityParser *SeverityParser
}

ParserOperator provides a basic implementation of a parser operator.

func (*ParserOperator) ProcessWith 

func (p *ParserOperator) ProcessWith(ctx context.Context, entry *entry.Entry, parse ParseFunction) error

ProcessWith will process an entry with a parser function.

type Persister 

type Persister interface {
	Get(string) []byte
	Set(string, []byte)
	Sync() error
	Load() error
}

Persister is a helper used to persist data

type ScopedBBoltPersister 

type ScopedBBoltPersister struct {
	// contains filtered or unexported fields
}

ScopedBBoltPersister is a persister that uses a database for the backend

func NewScopedDBPersister 

func NewScopedDBPersister(db operator.Database, scope string) *ScopedBBoltPersister

NewScopedDBPersister returns a new ScopedBBoltPersister

func (*ScopedBBoltPersister) Get 

func (p *ScopedBBoltPersister) Get(key string) []byte

Get retrieves a key from the cache

func (*ScopedBBoltPersister) Load 

func (p *ScopedBBoltPersister) Load() error

Load populates the cache with the values from the database, overwriting anything currently in the cache.

func (*ScopedBBoltPersister) Set 

func (p *ScopedBBoltPersister) Set(key string, val []byte)

Set saves a key in the cache

func (*ScopedBBoltPersister) Sync 

func (p *ScopedBBoltPersister) Sync() error

Sync saves the cache to the backend, ensuring values are safely written to disk before returning

type SeverityParser 

type SeverityParser struct {
	ParseFrom entry.Field
	Preserve  bool
	Mapping   severityMap
}

SeverityParser is a helper that parses severity onto an entry.

func (*SeverityParser) Parse 

func (p *SeverityParser) Parse(ctx context.Context, ent *entry.Entry) error

Parse will parse severity from a field and attach it to the entry

type SeverityParserConfig 

type SeverityParserConfig struct {
	ParseFrom *entry.Field                `json:"parse_from,omitempty" yaml:"parse_from,omitempty"`
	Preserve  bool                        `json:"preserve,omitempty"   yaml:"preserve,omitempty"`
	Preset    string                      `json:"preset,omitempty"     yaml:"preset,omitempty"`
	Mapping   map[interface{}]interface{} `json:"mapping,omitempty"    yaml:"mapping,omitempty"`
}

SeverityParserConfig allows users to specify how to parse a severity from a field.

func NewSeverityParserConfig 

func NewSeverityParserConfig() SeverityParserConfig

func (*SeverityParserConfig) Build 

func (c *SeverityParserConfig) Build(context operator.BuildContext) (SeverityParser, error)

Build builds a SeverityParser from a SeverityParserConfig

type TimeParser 

type TimeParser struct {
	ParseFrom  *entry.Field `json:"parse_from,omitempty"  yaml:"parse_from,omitempty"`
	Layout     string       `json:"layout,omitempty"      yaml:"layout,omitempty"`
	LayoutType string       `json:"layout_type,omitempty" yaml:"layout_type,omitempty"`
	Preserve   bool         `json:"preserve"              yaml:"preserve"`
}

TimeParser is a helper that parses time onto an entry.

func NewTimeParser 

func NewTimeParser() TimeParser

func (*TimeParser) IsZero 

func (t *TimeParser) IsZero() bool

IsZero returns true if the TimeParser is not a valid config

func (*TimeParser) Parse 

func (t *TimeParser) Parse(ctx context.Context, entry *entry.Entry) error

Parse will parse time from a field and attach it to the entry

func (*TimeParser) Validate 

func (t *TimeParser) Validate(context operator.BuildContext) error

Validate validates a TimeParser, and reconfigures it if necessary

type TransformFunction 

type TransformFunction = func(*entry.Entry) (*entry.Entry, error)

TransformFunction is function that transforms an entry.

type TransformerConfig 

type TransformerConfig struct {
	WriterConfig `yaml:",inline"`
	OnError      string `json:"on_error" yaml:"on_error"`
}

TransformerConfig provides a basic implementation of a transformer config.

func NewTransformerConfig 

func NewTransformerConfig(operatorID, operatorType string) TransformerConfig

func (TransformerConfig) Build 

func (c TransformerConfig) Build(context operator.BuildContext) (TransformerOperator, error)

Build will build a transformer operator.

type TransformerOperator 

type TransformerOperator struct {
	WriterOperator
	OnError string
}

TransformerOperator provides a basic implementation of a transformer operator.

func (*TransformerOperator) CanProcess 

func (t *TransformerOperator) CanProcess() bool

CanProcess will always return true for a transformer operator.

func (*TransformerOperator) HandleEntryError 

func (t *TransformerOperator) HandleEntryError(ctx context.Context, entry *entry.Entry, err error) error

HandleEntryError will handle an entry error using the on_error strategy.

func (*TransformerOperator) ProcessWith 

func (t *TransformerOperator) ProcessWith(ctx context.Context, entry *entry.Entry, transform TransformFunction) error

ProcessWith will process an entry with a transform function.

type WriterConfig 

type WriterConfig struct {
	BasicConfig `yaml:",inline"`
	OutputIDs   OutputIDs `json:"output" yaml:"output"`
}

WriterConfig is the configuration of a writer operator.

func NewWriterConfig 

func NewWriterConfig(operatorID, operatorType string) WriterConfig

func (WriterConfig) Build 

func (c WriterConfig) Build(context operator.BuildContext) (WriterOperator, error)

Build will build a writer operator from the config.

func (*WriterConfig) SetNamespace 

func (c *WriterConfig) SetNamespace(namespace string, exclusions ...string)

SetNamespace will namespace the output ids of the writer.

type WriterOperator 

type WriterOperator struct {
	BasicOperator
	OutputIDs       OutputIDs
	OutputOperators []operator.Operator
}

WriterOperator is an operator that can write to other operators.

func (*WriterOperator) CanOutput 

func (w *WriterOperator) CanOutput() bool

CanOutput always returns true for a writer operator.

func (*WriterOperator) Outputs 

func (w *WriterOperator) Outputs() []operator.Operator

Outputs returns the outputs of the writer operator.

func (*WriterOperator) SetOutputs 

func (w *WriterOperator) SetOutputs(operators []operator.Operator) error

SetOutputs will set the outputs of the operator.

func (*WriterOperator) Write 

func (w *WriterOperator) Write(ctx context.Context, e *entry.Entry)

Write will write an entry to the outputs of the operator.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.