beat

Documentation

Index

• Variables
• type Beat
• type BeatConfig
• type Beater
• type Client
• type ClientConfig
• type ClientEventer
• type Creator
• type Event
  • func (e *Event) Delete(key string) error
  • func (e *Event) GetValue(key string) (interface{}, error)
  • func (e *Event) PutValue(key string, v interface{}) (interface{}, error)
• type Info
• type Pipeline
• type PipelineACKHandler
• type Processor
• type ProcessorList
• type PublishMode
• type SetupMLCallback

Variables

var GracefulExit = errors.New("graceful exit")

GracefulExit is an error that signals to exit with a code of 0.

Types

type Beat

type Beat struct {
	Info      Info     // beat metadata.
	Publisher Pipeline // Publisher pipeline

	SetupMLCallback SetupMLCallback // setup callback for ML job configs
	InSetupCmd      bool            // this is set to true when the `setup` command is called

	// XXX: remove Config from public interface.
	//      It's currently used by filebeat modules to setup the Ingest Node
	//      pipeline and ML jobs.
	Config *BeatConfig // Common Beat configuration data.

	BeatConfig *common.Config // The beat's own configuration section
}

Beat contains the basic beat data and the publisher client used to publish events.

type BeatConfig

type BeatConfig struct {
	// output/publishing related configurations
	Output common.ConfigNamespace `config:"output"`
}

BeatConfig struct contains the basic configuration of every beat

type Beater

type Beater interface {
	// The main event loop. This method should block until signalled to stop by an
	// invocation of the Stop() method.
	Run(b *Beat) error

	// Stop is invoked to signal that the Run method should finish its execution.
	// It will be invoked at most once.
	Stop()
}

Beater is the interface that must be implemented by every Beat. A Beater provides the main Run-loop and a Stop method to break the Run-loop. Instantiation and Configuration is normally provided by a Beat-`Creator`.

Once the beat is fully configured, the Run() method is invoked. The Run()-method implements the beat its run-loop. Once the Run()-method returns, the beat shuts down.

The Stop() method is invoked the first time (and only the first time) a shutdown signal is received. The Stop()-method normally will stop the Run()-loop, such that the beat can gracefully shutdown.

type Client

type Client interface {
	Publish(Event)
	PublishAll([]Event)
	Close() error
}

Client holds a connection to the beats publisher pipeline

type ClientConfig

type ClientConfig struct {
	PublishMode PublishMode

	// EventMetadata configures additional fields/tags to be added to published events.
	EventMetadata common.EventMetadata

	// Meta provides additional meta data to be added to the Meta field in the beat.Event
	// structure.
	Meta common.MapStr

	// Fields provides additional 'global' fields to be added to every event
	Fields common.MapStr

	// DynamicFields provides additional fields to be added to every event, supporting live updates
	DynamicFields *common.MapStrPointer

	// Processors passes additional processor to the client, to be executed before
	// the pipeline processors.
	Processor ProcessorList

	// WaitClose sets the maximum duration to wait on ACK, if client still has events
	// active non-acknowledged events in the publisher pipeline.
	// WaitClose is only effective if one of ACKCount, ACKEvents and ACKLastEvents
	// is configured
	WaitClose time.Duration

	// Events configures callbacks for common client callbacks
	Events ClientEventer

	// ACKCount reports the number of published events recently acknowledged
	// by the pipeline.
	ACKCount func(int)

	// ACKEvents reports the events private data of recently acknowledged events.
	// Note: The slice passed must be copied if the events are to be processed
	//       after the handler returns.
	ACKEvents func([]interface{})

	// ACKLastEvent reports the last ACKed event out of a batch of ACKed events only.
	// Only the events 'Private' field will be reported.
	ACKLastEvent func(interface{})
}

ClientConfig defines common configuration options one can pass to Pipeline.ConnectWith to control the clients behavior and provide ACK support.

type ClientEventer

type ClientEventer interface {
	Closing() // Closing indicates the client is being shutdown next
	Closed()  // Closed indicates the client being fully shutdown

	Published()             // event has been successfully forwarded to the publisher pipeline
	FilteredOut(Event)      // event has been filtered out/dropped by processors
	DroppedOnPublish(Event) // event has been dropped, while waiting for the queue
}

ClientEventer provides access to internal client events.

type Creator

type Creator func(*Beat, *common.Config) (Beater, error)

Creator initializes and configures a new Beater instance used to execute the beat's run-loop.

type Event

type Event struct {
	Timestamp time.Time
	Meta      common.MapStr
	Fields    common.MapStr
	Private   interface{} // for beats private use
}

Event is the common event format shared by all beats. Every event must have a timestamp and provide encodable Fields in `Fields`. The `Meta`-fields can be used to pass additional meta-data to the outputs. Output can optionally publish a subset of Meta, or ignore Meta.

func (*Event) Delete

func (e *Event) Delete(key string) error

func (*Event) GetValue

func (e *Event) GetValue(key string) (interface{}, error)

func (*Event) PutValue

func (e *Event) PutValue(key string, v interface{}) (interface{}, error)

type Info

type Info struct {
	Beat        string    // The actual beat's name
	IndexPrefix string    // The beat's index prefix in Elasticsearch.
	Version     string    // The beat version. Defaults to the libbeat version when an implementation does not set a version
	Name        string    // configured beat name
	Hostname    string    // hostname
	UUID        uuid.UUID // ID assigned to beat instance
}

Info stores a beats instance meta data.

type Pipeline

type Pipeline interface {
	Connect() (Client, error)
	ConnectWith(ClientConfig) (Client, error)
	SetACKHandler(PipelineACKHandler) error
}

type PipelineACKHandler

type PipelineACKHandler struct {
	// ACKCount reports the number of published events recently acknowledged
	// by the pipeline.
	ACKCount func(int)

	// ACKEvents reports the events recently acknowledged by the pipeline.
	// Only the events 'Private' field will be reported.
	ACKEvents func([]interface{})

	// ACKLastEvent reports the last ACKed event per pipeline client.
	// Only the events 'Private' field will be reported.
	ACKLastEvents func([]interface{})
}

PipelineACKHandler configures some pipeline-wide event ACK handler.

type Processor

type Processor interface {
	String() string // print full processor description
	Run(in *Event) (event *Event, err error)
}

Processor defines the minimal required interface for processor, that can be registered with the publisher pipeline.

type ProcessorList

type ProcessorList interface {
	All() []Processor
}

type PublishMode

type PublishMode uint8

PublishMode enum sets some requirements on the client connection to the beats publisher pipeline

const (
	// DefaultGuarantees are up to the pipeline configuration, as configured by the
	// operator.
	DefaultGuarantees PublishMode = iota

	// GuaranteedSend ensures events are retried until acknowledged by the output.
	// Normally guaranteed sending should be used with some client ACK-handling
	// to update state keeping track of the sending status.
	GuaranteedSend

	// DropIfFull drops an event to be send if the pipeline is currently full.
	// This ensures a beats internals can continue processing if the pipeline has
	// filled up. Useful if an event stream must be processed to keep internal
	// state up-to-date.
	DropIfFull
)

type SetupMLCallback

type SetupMLCallback func(*Beat) error

SetupMLCallback can be used by the Beat to register MachineLearning configurations for the enabled modules.