mb

package
v0.0.0-...-be347a3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 6, 2019 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package mb (short for Metricbeat) contains the public interfaces that are used to implement Modules and their associated MetricSets.

Index

Examples

Constants

View Source 
const (
	// TimestampKey is the key used in events created by MetricSets to add their
	// own timestamp to an event. If a timestamp is not specified then the that
	// the fetch started will be used.
	TimestampKey string = "@timestamp"

	// ModuleDataKey is the key used in events created by MetricSets to add data
	// to an event that is common to the module. The data must be a
	// common.MapStr and when the final event is built the object will be stored
	// in the event under a key that is the module name.
	ModuleDataKey string = "_module"

	// NamespaceKey is used to define a different namespace for the metricset
	// This is useful for dynamic metricsets or metricsets which do not
	// put the name under the same name as the package. This is for example
	// the case in elasticsearch `node_stats` which puts the data under `node.stats`.
	NamespaceKey string = "_namespace"

	// RTTKey is used by a MetricSet to specify the round trip time (RTT), or
	// total amount of time, taken to collect the information in the event. The
	// data must be of type time.Duration otherwise the value is ignored.
	RTTKey string = "_rtt"
)

Variables

View Source 
var (
	// ErrEmptyConfig indicates that modules configuration list is nil or empty.
	ErrEmptyConfig = errors.New("one or more modules must be configured")

	// ErrAllModulesDisabled indicates that all modules are disabled. At least
	// one module must be enabled.
	ErrAllModulesDisabled = errors.New("all modules are disabled")

	// ErrModuleDisabled indicates a disabled module has been tried to instantiate.
	ErrModuleDisabled = errors.New("disabled module")
)
View Source 
var DefaultModuleFactory = func(base BaseModule) (Module, error) {
	return &base, nil
}

DefaultModuleFactory returns the given BaseModule and never returns an error. If a MetricSets are registered without an associated ModuleFactory, then the DefaultModuleFactory will be used to instantiate a Module.

View Source 
var Registry = NewRegister()

Registry is the singleton Register instance where all ModuleFactory's and MetricSetFactory's should be registered.

Functions

func AddMetricSetInfo 

func AddMetricSetInfo(module, metricset string, event *Event)

AddMetricSetInfo is an EventModifier that adds information about the MetricSet that generated the event. It will always add the metricset and module names. And it will add the host, period (in milliseconds), and duration (round-trip time in nanoseconds) values if they are non-zero values. 

{
  "event": {
    "dataset": "apache.status",
    "duration": 115,
    "module": "apache"
  },
  "service": {
    "address": "127.0.0.1",
  },
  "metricset": {
    "name": "status",
    "period": 10000
  }
}

func NewModule 

func NewModule(config *common.Config, r *Register) (Module, []MetricSet, error)

NewModule builds a new Module and its associated MetricSets based on the provided configuration data. config contains config data (the data will be unpacked into ModuleConfig structs). r is the Register where the ModuleFactory's and MetricSetFactory's will be obtained from. This method returns a Module and its configured MetricSets or an error.

Types

type BaseMetricSet 

type BaseMetricSet struct {
	// contains filtered or unexported fields
}

BaseMetricSet implements the MetricSet interface.

The BaseMetricSet type can be embedded into another struct to satisfy the MetricSet interface requirements, leaving only the Fetch() method to be implemented to have a complete MetricSet implementation.

func (*BaseMetricSet) FullyQualifiedName 

func (b *BaseMetricSet) FullyQualifiedName() string

FullyQualifiedName returns the complete name of the MetricSet, including the name of the module.

func (*BaseMetricSet) GoString 

func (b *BaseMetricSet) GoString() string

func (*BaseMetricSet) Host 

func (b *BaseMetricSet) Host() string

Host returns the hostname or other module specific value that identifies a specific host or service instance from which to collect metrics.

func (*BaseMetricSet) HostData 

func (b *BaseMetricSet) HostData() HostData

HostData returns the parsed host data.

func (*BaseMetricSet) ID 

func (b *BaseMetricSet) ID() string

ID returns the unique ID of the MetricSet.

func (*BaseMetricSet) Logger 

func (b *BaseMetricSet) Logger() *logp.Logger

Logger returns the logger.

func (*BaseMetricSet) Metrics 

func (b *BaseMetricSet) Metrics() *monitoring.Registry

Metrics returns the metrics registry.

func (*BaseMetricSet) Module 

func (b *BaseMetricSet) Module() Module

Module returns the parent Module for the MetricSet.

func (*BaseMetricSet) Name 

func (b *BaseMetricSet) Name() string

Name returns the name of the MetricSet. It should not include the name of the module.

func (*BaseMetricSet) Registration 

func (b *BaseMetricSet) Registration() MetricSetRegistration

Registration returns the parameters that were used when the MetricSet was registered with the registry.

func (*BaseMetricSet) String 

func (b *BaseMetricSet) String() string

type BaseModule 

type BaseModule struct {
	// contains filtered or unexported fields
}

BaseModule implements the Module interface.

When a Module needs to store additional data or provide methods to its MetricSets, it can embed this type into another struct to satisfy the Module interface requirements.

func (*BaseModule) Config 

func (m *BaseModule) Config() ModuleConfig

Config returns the ModuleConfig used to create the Module.

func (*BaseModule) GoString 

func (m *BaseModule) GoString() string

func (*BaseModule) Name 

func (m *BaseModule) Name() string

Name returns the name of the Module.

func (*BaseModule) String 

func (m *BaseModule) String() string

func (*BaseModule) UnpackConfig 

func (m *BaseModule) UnpackConfig(to interface{}) error

UnpackConfig unpacks the raw module config to the given object.

type Closer 

type Closer interface {
	Close() error
}

Closer is an optional interface that a MetricSet can implement in order to cleanup any resources it has open at shutdown.

type Event 

type Event struct {
	RootFields      common.MapStr // Fields that will be added to the root of the event.
	ModuleFields    common.MapStr // Fields that will be namespaced under [module].
	MetricSetFields common.MapStr // Fields that will be namespaced under [module].[metricset].

	Index     string        // Index name prefix. If set overwrites the default prefix.
	ID        string        // ID of event. If set, overwrites the default ID.
	Namespace string        // Fully qualified namespace to use for MetricSetFields.
	Timestamp time.Time     // Timestamp when the event data was collected.
	Error     error         // Error that occurred while collecting the event data.
	Host      string        // Host from which the data was collected.
	Service   string        // Service type
	Took      time.Duration // Amount of time it took to collect the event data.
	Period    time.Duration // Period that is set to retrieve the events

	DisableTimeSeries bool // true if the event doesn't contain timeseries data
}

Event contains the data generated by a MetricSet.

func TransformMapStrToEvent 

func TransformMapStrToEvent(module string, m common.MapStr, err error) Event

TransformMapStrToEvent transforms a common.MapStr produced by MetricSet (like any MetricSet that does not natively produce a mb.Event). It accounts for the special key names and routes the data stored under those keys to the correct location in the event.

func (*Event) BeatEvent 

func (e *Event) BeatEvent(module, metricSet string, modifiers ...EventModifier) beat.Event

BeatEvent returns a new beat.Event containing the data this Event. It does mutate the underlying data in the Event.

type EventFetcher 

type EventFetcher interface {
	MetricSet
	Fetch() (common.MapStr, error)
}

EventFetcher is a MetricSet that returns a single event when collecting data. Use ReportingMetricSet for new MetricSet implementations.

type EventModifier 

type EventModifier func(module, metricset string, event *Event)

EventModifier is a function that can modifies an Event. This is typically used to apply transformations to an Event as it is converted to a beat.Event. An example is AddMetricSetInfo.

type EventsFetcher 

type EventsFetcher interface {
	MetricSet
	Fetch() ([]common.MapStr, error)
}

EventsFetcher is a MetricSet that returns a multiple events when collecting data. Use ReportingMetricSet for new MetricSet implementations.

type HostData 

type HostData struct {
	URI          string // The full URI that should be used in connections.
	SanitizedURI string // A sanitized version of the URI without credentials.

	Host     string // The host and possibly port.
	User     string // Username
	Password string // Password
}

HostData contains values parsed from the 'host' configuration. Other configuration data like protocols, usernames, and passwords may also be used to construct this HostData data.

func (HostData) GoString 

func (h HostData) GoString() string

func (HostData) String 

func (h HostData) String() string

type HostParser 

type HostParser func(module Module, host string) (HostData, error)

HostParser is a function that parses a host value from the configuration and returns a HostData object. The module is provided in case additional configuration values are required to parse and build the HostData object. An error should be returned if the host or configuration is invalid.

type LightMetricSet 

type LightMetricSet struct {
	Name    string
	Module  string
	Default bool `config:"default"`
	Input   struct {
		Module    string      `config:"module" validate:"required"`
		MetricSet string      `config:"metricset" validate:"required"`
		Defaults  interface{} `config:"defaults"`
	} `config:"input" validate:"required"`
}

LightMetricSet contains the definition of a non-registered metric set

func (*LightMetricSet) Registration 

func (m *LightMetricSet) Registration(r *Register) (MetricSetRegistration, error)

Registration obtains a metric set registration for this light metric set, this registration contains a metric set factory that reprocess metric set creation taking into account the light metric set defaults

type MetricSet 

type MetricSet interface {
	ID() string     // Unique ID identifying a running MetricSet.
	Name() string   // Name returns the name of the MetricSet.
	Module() Module // Module returns the parent Module for the MetricSet.
	Host() string   // Host returns a hostname or other module specific value
	// that identifies a specific host or service instance from which to collect
	// metrics.
	HostData() HostData                  // HostData returns the parsed host data.
	Registration() MetricSetRegistration // Params used in registration.
	Metrics() *monitoring.Registry       // MetricSet specific metrics
	Logger() *logp.Logger                // MetricSet specific logger
}

MetricSet is the common interface for all MetricSet implementations. In addition to this interface, all MetricSets must implement either EventFetcher or EventsFetcher (but not both).

type MetricSetFactory 

type MetricSetFactory func(base BaseMetricSet) (MetricSet, error)

MetricSetFactory accepts a BaseMetricSet and returns a MetricSet. If there was an error creating the MetricSet then an error will be returned. The returned MetricSet must also implement either EventFetcher or EventsFetcher (but not both).

type MetricSetOption 

type MetricSetOption func(info *MetricSetRegistration)

MetricSetOption sets an option for a MetricSetFactory that is being registered.

func DefaultMetricSet 

func DefaultMetricSet() MetricSetOption

DefaultMetricSet specifies that the MetricSetFactory will be the default when no MetricSet names are specified in the configuration.

func WithHostParser 

func WithHostParser(p HostParser) MetricSetOption

WithHostParser specifies the HostParser that should be used with the MetricSet.

func WithNamespace 

func WithNamespace(namespace string) MetricSetOption

WithNamespace specifies the fully qualified namespace under which MetricSet data will be added. If no namespace is specified then [module].[metricset] will be used.

type MetricSetRegistration 

type MetricSetRegistration struct {
	Name    string
	Factory MetricSetFactory

	// Options
	IsDefault  bool
	HostParser HostParser
	Namespace  string
}

MetricSetRegistration contains the parameters that were used to register a MetricSet.

type Module 

type Module interface {
	Name() string                      // Name returns the name of the Module.
	Config() ModuleConfig              // Config returns the ModuleConfig used to create the Module.
	UnpackConfig(to interface{}) error // UnpackConfig unpacks the raw module config to the given object.
}

Module is the common interface for all Module implementations.

type ModuleConfig 

type ModuleConfig struct {
	Hosts       []string      `config:"hosts"`
	Period      time.Duration `config:"period"     validate:"positive"`
	Timeout     time.Duration `config:"timeout"    validate:"positive"`
	Module      string        `config:"module"     validate:"required"`
	MetricSets  []string      `config:"metricsets"`
	Enabled     bool          `config:"enabled"`
	Raw         bool          `config:"raw"`
	Query       QueryParams   `config:"query"`
	ServiceName string        `config:"service.name"`
}

ModuleConfig is the base configuration data for all Modules.

The Raw config option is used to enable raw fields in a metricset. This means the metricset fetches not only the predefined fields but add alls raw data under the raw namespace to the event.

func DefaultModuleConfig 

func DefaultModuleConfig() ModuleConfig

DefaultModuleConfig returns a ModuleConfig with the default values populated.

func (ModuleConfig) GoString 

func (c ModuleConfig) GoString() string

func (ModuleConfig) String 

func (c ModuleConfig) String() string

type ModuleFactory 

type ModuleFactory func(base BaseModule) (Module, error)

ModuleFactory accepts a BaseModule and returns a Module. If there was an error creating the Module then an error will be returned.

Example

ExampleModuleFactory demonstrates how to register a custom ModuleFactory and unpack additional configuration data. 

// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.

package main

import (
	"github.com/elastic/beats/metricbeat/mb"
)

func init() {
	// Register the ModuleFactory function for the "example" module.
	if err := mb.Registry.AddModule("example", NewModule); err != nil {
		panic(err)
	}
}

type Module struct {
	mb.BaseModule
	Protocol string
}

func NewModule(base mb.BaseModule) (mb.Module, error) {
	// Unpack additional configuration options.
	config := struct {
		Protocol string `config:"protocol"`
	}{
		Protocol: "udp",
	}
	if err := base.UnpackConfig(&config); err != nil {
		return nil, err
	}

	return &Module{BaseModule: base, Protocol: config.Protocol}, nil
}

// ExampleModuleFactory demonstrates how to register a custom ModuleFactory
// and unpack additional configuration data.
func main() {}

Output:

type ModulesSource 

type ModulesSource interface {
	Modules() ([]string, error)
	HasModule(module string) bool
	MetricSets(module string) ([]string, error)
	DefaultMetricSets(module string) ([]string, error)
	HasMetricSet(module, name string) bool
	MetricSetRegistration(r *Register, module, name string) (MetricSetRegistration, error)
	String() string
}

ModulesSource contains a source of non-registered modules

type PushMetricSet deprecated 

type PushMetricSet interface {
	MetricSet
	Run(r PushReporter)
}

PushMetricSet is a MetricSet that pushes events (rather than pulling them periodically via a Fetch callback). Run is invoked to start the event subscription and it should block until the MetricSet is ready to stop or the PushReporter's done channel is closed.

Deprecated: Use PushMetricSetV2.

type PushMetricSetV2 

type PushMetricSetV2 interface {
	MetricSet
	Run(r PushReporterV2)
}

PushMetricSetV2 is a MetricSet that pushes events (rather than pulling them periodically via a Fetch callback). Run is invoked to start the event subscription and it should block until the MetricSet is ready to stop or the PushReporterV2's done channel is closed.

type PushMetricSetV2WithContext 

type PushMetricSetV2WithContext interface {
	MetricSet
	Run(ctx context.Context, r ReporterV2)
}

PushMetricSetV2WithContext is a MetricSet that pushes events (rather than pulling them periodically via a Fetch callback). Run is invoked to start the event subscription and it should block until the MetricSet is ready to stop or the context is closed.

type PushReporter deprecated 

type PushReporter interface {
	Reporter

	// Done returns a channel that's closed when work done on behalf of this
	// reporter should be canceled.
	Done() <-chan struct{}
}

PushReporter is used by a MetricSet to report events, errors, or errors with metadata. It provides a done channel used to signal that reporter should stop.

Deprecated: Use PushReporterV2.

type PushReporterV2 

type PushReporterV2 interface {
	ReporterV2

	// Done returns a channel that's closed when work done on behalf of this
	// reporter should be canceled.
	Done() <-chan struct{}
}

PushReporterV2 is used by a MetricSet to report events, errors, or errors with metadata. It provides a done channel used to signal that reporter should stop.

type QueryParams 

type QueryParams map[string]interface{}

QueryParams is a convenient map[string]interface{} wrapper to implement the String interface which returns the values in common query params format (key=value&key2=value2) which is the way that the url package expects this params (without the initial '?')

func (QueryParams) String 

func (q QueryParams) String() (s string)

String returns the values in common query params format (key=value&key2=value2) which is the way that the url package expects this params (without the initial '?')

type Register 

type Register struct {
	// contains filtered or unexported fields
}

Register contains the factory functions for creating new Modules and new MetricSets. Registers are thread safe for concurrent usage.

func NewRegister 

func NewRegister() *Register

NewRegister creates and returns a new Register.

func (*Register) AddMetricSet 

func (r *Register) AddMetricSet(module string, name string, factory MetricSetFactory, hostParser ...HostParser) error

AddMetricSet registers a new MetricSetFactory. Optionally it accepts a single HostParser function for parsing the 'host' configuration data. An error is returned if any parameter is empty or nil or if a factory has already been registered under the name.

Use MustAddMetricSet for new code.

func (*Register) AddModule 

func (r *Register) AddModule(name string, factory ModuleFactory) error

AddModule registers a new ModuleFactory. An error is returned if the name is empty, factory is nil, or if a factory has already been registered under the name.

func (*Register) DefaultMetricSets 

func (r *Register) DefaultMetricSets(module string) ([]string, error)

DefaultMetricSets returns the names of the default MetricSets for a module. An error is returned if no default MetricSet is declared or the module does not exist.

func (*Register) MetricSets 

func (r *Register) MetricSets(module string) []string

MetricSets returns the list of MetricSets registered for a given module

func (*Register) Modules 

func (r *Register) Modules() []string

Modules returns the list of module names that are registered

func (*Register) MustAddMetricSet 

func (r *Register) MustAddMetricSet(module, name string, factory MetricSetFactory, options ...MetricSetOption)

MustAddMetricSet registers a new MetricSetFactory. It panics if any parameter is empty or nil OR if a factory has already been registered under this name.

func (*Register) SetSecondarySource 

func (r *Register) SetSecondarySource(source ModulesSource)

SetSecondarySource sets an additional source of modules

func (*Register) String 

func (r *Register) String() string

String return a string representation of the registered ModuleFactory's and MetricSetFactory's.

type Reporter deprecated 

type Reporter interface {
	Event(event common.MapStr) bool               // Event reports a single successful event.
	ErrorWith(err error, meta common.MapStr) bool // ErrorWith reports a single error event with the additional metadata.
	Error(err error) bool                         // Error reports a single error event.
}

Reporter is used by a MetricSet to report events, errors, or errors with metadata. The methods return false if and only if publishing failed because the MetricSet is being closed.

Deprecated: Use ReporterV2.

type ReporterV2 

type ReporterV2 interface {
	Event(event Event) bool // Event reports a single successful event.
	Error(err error) bool
}

ReporterV2 is used by a MetricSet to report Events. The methods return false if and only if publishing failed because the MetricSet is being closed.

type ReportingMetricSet deprecated 

type ReportingMetricSet interface {
	MetricSet
	Fetch(r Reporter)
}

ReportingMetricSet is a MetricSet that reports events or errors through the Reporter interface. Fetch is called periodically to collect events.

Deprecated: Use ReportingMetricSetV2.

Example

ExampleReportingMetricSet demonstrates how to register a MetricSetFactory and implement a ReportingMetricSet. 

// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.

package main

import (
	"fmt"

	"github.com/elastic/beats/libbeat/common"
	"github.com/elastic/beats/metricbeat/mb"
	"github.com/elastic/beats/metricbeat/mb/parse"
)

var hostParser = parse.URLHostParserBuilder{
	DefaultScheme: "http",
}.Build()

func init() {
	// Register the MetricSetFactory function for the "status" MetricSet.
	mb.Registry.MustAddMetricSet("someapp", "status", NewMetricSet,
		mb.WithHostParser(hostParser),
	)
}

type MetricSet struct {
	mb.BaseMetricSet
}

func NewMetricSet(base mb.BaseMetricSet) (mb.MetricSet, error) {
	fmt.Println("someapp-status url=", base.HostData().SanitizedURI)
	return &MetricSet{BaseMetricSet: base}, nil
}

// Fetch will be called periodically by the framework.
func (ms *MetricSet) Fetch(report mb.Reporter) {
	// Fetch data from the host at ms.HostData().URI and return the data.
	data, err := common.MapStr{
		"some_metric":          18.0,
		"answer_to_everything": 42,
	}, error(nil)
	if err != nil {
		// Report an error if it occurs.
		report.Error(err)
		return
	}

	// Otherwise report the collected data.
	report.Event(data)
}

// ExampleReportingMetricSet demonstrates how to register a MetricSetFactory
// and implement a ReportingMetricSet.
func main() {}

Output:

type ReportingMetricSetV2 

type ReportingMetricSetV2 interface {
	MetricSet
	Fetch(r ReporterV2)
}

ReportingMetricSetV2 is a MetricSet that reports events or errors through the ReporterV2 interface. Fetch is called periodically to collect events.

type ReportingMetricSetV2Error 

type ReportingMetricSetV2Error interface {
	MetricSet
	Fetch(r ReporterV2) error
}

ReportingMetricSetV2Error is a MetricSet that reports events or errors through the ReporterV2 interface. Fetch is called periodically to collect events.

type ReportingMetricSetV2WithContext 

type ReportingMetricSetV2WithContext interface {
	MetricSet
	Fetch(ctx context.Context, r ReporterV2) error
}

ReportingMetricSetV2WithContext is a MetricSet that reports events or errors through the ReporterV2 interface. Fetch is called periodically to collect events.

Source Files

View all Source files

Directories

Path Synopsis
Package module contains the low-level utilities for running Metricbeat modules and metricsets.
 Package module contains the low-level utilities for running Metricbeat modules and metricsets.
Package testing provides utility functions for testing Module and MetricSet implementations.
 Package testing provides utility functions for testing Module and MetricSet implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.