README
¶
Beats - The Lightweight Shippers of the Elastic Stack
The Beats are lightweight data shippers, written in Go, that you install on your servers to capture all sorts of operational data (think of logs, metrics, or network packet data). The Beats send the operational data to Elasticsearch, either directly or via Logstash, so it can be visualized with Kibana.
By "lightweight", we mean that Beats have a small installation footprint, use limited system resources, and have no runtime dependencies.
This repository contains libbeat, our Go framework for creating Beats, and all the officially supported Beats:
| Beat | Description |
|---|---|
| Filebeat | Tails and ships log files |
| Heartbeat | Ping remote services for availability |
| Metricbeat | Fetches sets of metrics from the operating system and services |
| Packetbeat | Monitors the network and applications by sniffing packets |
| Winlogbeat | Fetches and ships Windows Event logs |
In addition to the above Beats, which are officially supported by Elastic, the community has created a set of other Beats that make use of libbeat but live outside of this Github repository. We maintain a list of community Beats here.
Documentation and Getting Started
You can find the documentation and getting started guides for each of the Beats on the elastic.co site:
Getting Help
If you need help or hit an issue, please start by opening a topic on our discuss forums. Please note that we reserve GitHub tickets for confirmed bugs and enhancement requests.
Downloads
You can download pre-compiled Beats binaries, as well as packages for the supported platforms, from this page.
Contributing
We'd love working with you! You can help make the Beats better in many ways: report issues, help us reproduce issues, fix bugs, add functionality, or even create your own Beat.
Please start by reading our CONTRIBUTING file.
If you are creating a new Beat, you don't need to submit the code to this repository. You can simply start working in a new repository and make use of the libbeat packages, by following our developer guide. After you have a working prototype, open a pull request to add your Beat to the list of community Beats.
Building Beats from the Source
See our CONTRIBUTING file for information about setting up your dev environment to build Beats from the source.
Directories
¶
| Path | Synopsis |
|---|---|
|
harvester
Package harvester harvests different inputs for new information.
|
Package harvester harvests different inputs for new information. |
|
harvester/reader
Package reader provides interface and struct to read messages and report them to a harvester
|
Package reader provides interface and struct to read messages and report them to a harvester |
|
look
Package look defines common formatters for fields/types to be used when generating custom events.
|
Package look defines common formatters for fields/types to be used when generating custom events. |
|
beat
Package beat provides the functions required to manage the life-cycle of a Beat.
|
Package beat provides the functions required to manage the life-cycle of a Beat. |
|
common/dtfmt
Package dtfmt provides time formatter support with pattern syntax mostly similar to joda DateTimeFormat.
|
Package dtfmt provides time formatter support with pattern syntax mostly similar to joda DateTimeFormat. |
|
common/streambuf
Package streambuf provides helpers for buffering multiple packet payloads and some general parsing functions.
|
Package streambuf provides helpers for buffering multiple packet payloads and some general parsing functions. |
|
ml-importer
Package mlimporter contains code for loading Elastic X-Pack Machine Learning job configurations.
|
Package mlimporter contains code for loading Elastic X-Pack Machine Learning job configurations. |
|
outputs/mode
Package mode defines and implents output strategies with failover or load balancing modes for use by output plugins.
|
Package mode defines and implents output strategies with failover or load balancing modes for use by output plugins. |
|
paths
Package paths provides a common way to handle paths configuration for all Beats.
|
Package paths provides a common way to handle paths configuration for all Beats. |
|
Package metricbeat contains the entrypoint to Metricbeat which is a lightweight data shipper for operating system and service metrics.
|
Package metricbeat contains the entrypoint to Metricbeat which is a lightweight data shipper for operating system and service metrics. |
|
beater
Package beater provides the implementation of the libbeat Beater interface for Metricbeat and functions for running Metricbeat Modules on their own.
|
Package beater provides the implementation of the libbeat Beater interface for Metricbeat and functions for running Metricbeat Modules on their own. |
|
include
Package include imports all Module and MetricSet packages so that they register their factories with the global registry.
|
Package include imports all Module and MetricSet packages so that they register their factories with the global registry. |
|
mb
Package mb (short for Metricbeat) contains the public interfaces that are used to implement Modules and their associated MetricSets.
|
Package mb (short for Metricbeat) contains the public interfaces that are used to implement Modules and their associated MetricSets. |
|
mb/testing
Package testing provides utility functions for testing Module and MetricSet implementations.
|
Package testing provides utility functions for testing Module and MetricSet implementations. |
|
module
Package module contains Metricbeat modules and their MetricSet implementations.
|
Package module contains Metricbeat modules and their MetricSet implementations. |
|
module/apache
Package apache is Metricbeat module for the Apache HTTPD server.
|
Package apache is Metricbeat module for the Apache HTTPD server. |
|
module/apache/status
Package status reads Apache HTTPD server status from the mod_status module.
|
Package status reads Apache HTTPD server status from the mod_status module. |
|
module/ceph
Package ceph is a Metricbeat module that contains MetricSets.
|
Package ceph is a Metricbeat module that contains MetricSets. |
|
module/couchbase
Package couchbase is a Metricbeat module that contains MetricSets.
|
Package couchbase is a Metricbeat module that contains MetricSets. |
|
module/docker
Package docker is a Metricbeat module that contains MetricSets.
|
Package docker is a Metricbeat module that contains MetricSets. |
|
module/haproxy
Package haproxy is a Metricbeat module that contains MetricSets.
|
Package haproxy is a Metricbeat module that contains MetricSets. |
|
module/jolokia
Package jolokia is a Metricbeat module that contains MetricSets.
|
Package jolokia is a Metricbeat module that contains MetricSets. |
|
module/kafka
Package kafka is a Metricbeat module that contains MetricSets.
|
Package kafka is a Metricbeat module that contains MetricSets. |
|
module/mongodb
Package mongodb is a Metricbeat module that contains MetricSets.
|
Package mongodb is a Metricbeat module that contains MetricSets. |
|
module/mysql
Package mysql is Metricbeat module for MySQL server.
|
Package mysql is Metricbeat module for MySQL server. |
|
module/mysql/status
Package status fetches MySQL server status metrics.
|
Package status fetches MySQL server status metrics. |
|
module/nginx
Package nginx is Metricbeat module for the Nginx reverse proxy.
|
Package nginx is Metricbeat module for the Nginx reverse proxy. |
|
module/nginx/stubstatus
Package stubstatus reads server status from nginx host under /server-status, ngx_http_stub_status_module is required.
|
Package stubstatus reads server status from nginx host under /server-status, ngx_http_stub_status_module is required. |
|
module/php_fpm
Package php_fpm is a Metricbeat module that contains MetricSets.
|
Package php_fpm is a Metricbeat module that contains MetricSets. |
|
module/postgresql
Package postgresql is a Metricbeat module that contains MetricSets.
|
Package postgresql is a Metricbeat module that contains MetricSets. |
|
module/prometheus
Package prometheus is a Metricbeat module that contains MetricSets.
|
Package prometheus is a Metricbeat module that contains MetricSets. |
|
module/redis
Package redis contains shared Redis functionality for the metric sets
|
Package redis contains shared Redis functionality for the metric sets |
|
module/redis/info
Package info fetches Redis server information and statistics using the Redis INFO command.
|
Package info fetches Redis server information and statistics using the Redis INFO command. |
|
module/system
Package system is a Metricbeat module that contains MetricSets that collect system level information like CPU and memory stats.
|
Package system is a Metricbeat module that contains MetricSets that collect system level information like CPU and memory stats. |
|
module/system/core
Package core collects cpu core metrics from the host OS.
|
Package core collects cpu core metrics from the host OS. |
|
module/system/cpu
Package cpu collects CPU metrics from the host OS.
|
Package cpu collects CPU metrics from the host OS. |
|
module/system/diskio
Package diskio fetches disk IO metrics from the OS.
|
Package diskio fetches disk IO metrics from the OS. |
|
module/system/filesystem
Package filesystem provides a MetricSet implementation that fetches metrics for each of the mounted file systems.
|
Package filesystem provides a MetricSet implementation that fetches metrics for each of the mounted file systems. |
|
module/system/fsstat
Package fsstat provides a MetricSet for fetching aggregated filesystem stats.
|
Package fsstat provides a MetricSet for fetching aggregated filesystem stats. |
|
module/system/load
Package load collects system load metrics from the host OS.
|
Package load collects system load metrics from the host OS. |
|
module/system/memory
Package memory collects memory metrics from the host OS.
|
Package memory collects memory metrics from the host OS. |
|
module/system/network
Package network provides network IO metrics from the OS.
|
Package network provides network IO metrics from the OS. |
|
module/system/process
Package process collects metrics about the running processes using information from the operating system.
|
Package process collects metrics about the running processes using information from the operating system. |
|
module/system/socket
Package socket captures active sockets and the processes that own them.
|
Package socket captures active sockets and the processes that own them. |
|
module/zookeeper
Package zookeeper is a Metricbeat module for ZooKeeper servers.
|
Package zookeeper is a Metricbeat module for ZooKeeper servers. |
|
module/zookeeper/mntr
Package mntr fetches metrics from ZooKeeper by using the mntr command which was added to ZooKeeper in version 3.4.0.
|
Package mntr fetches metrics from ZooKeeper by using the mntr command which was added to ZooKeeper in version 3.4.0. |
|
schema
Package schema contains helper utilities to parse, convert, rename, and reorganize data from the format returned by the various monitored systems to the data model used by Metricbeat.
|
Package schema contains helper utilities to parse, convert, rename, and reorganize data from the format returned by the various monitored systems to the data model used by Metricbeat. |
|
schema/mapstriface
Package mapstriface contains utilities for transforming map[string]interface{} objects into metricbeat events.
|
Package mapstriface contains utilities for transforming map[string]interface{} objects into metricbeat events. |
|
schema/mapstrstr
Package mapstrstr contains utilities for transforming map[string]string objects into metricbeat events.
|
Package mapstrstr contains utilities for transforming map[string]string objects into metricbeat events. |
|
include
Package include imports all protos packages so that they register with the global registry.
|
Package include imports all protos packages so that they register with the global registry. |
|
protos/applayer
Package applayer provides common definitions with common fields for use with application layer protocols among beats.
|
Package applayer provides common definitions with common fields for use with application layer protocols among beats. |
|
protos/dns
Package dns provides support for parsing DNS messages and reporting the results.
|
Package dns provides support for parsing DNS messages and reporting the results. |
|
Package winlogbeat contains the entrypoint to Winlogbeat which is a lightweight data shipper for Windows event logs.
|
Package winlogbeat contains the entrypoint to Winlogbeat which is a lightweight data shipper for Windows event logs. |
|
beater
Package beater provides the implementation of the libbeat Beater interface for Winlogbeat.
|
Package beater provides the implementation of the libbeat Beater interface for Winlogbeat. |
|
checkpoint
Package checkpoint persists event log state information to disk so that event log monitoring can resume from the last read event in the case of a restart or unexpected interruption.
|
Package checkpoint persists event log state information to disk so that event log monitoring can resume from the last read event in the case of a restart or unexpected interruption. |
|
config
Package config provides the winlogbeat specific configuration options.
|
Package config provides the winlogbeat specific configuration options. |
|
eventlog
Package eventlog provides the means for reading event logs from Windows.
|
Package eventlog provides the means for reading event logs from Windows. |
|
sys
Package sys provides common data structures and utilties functions that are used by the subpackages for interfacing with the system level APIs to collect event log records from Windows.
|
Package sys provides common data structures and utilties functions that are used by the subpackages for interfacing with the system level APIs to collect event log records from Windows. |
|
sys/eventlogging
Package eventlogging provides access to the Event Logging API that was designed for applications that run on the Windows Server 2003, Windows XP, or Windows 2000 operating system.
|
Package eventlogging provides access to the Event Logging API that was designed for applications that run on the Windows Server 2003, Windows XP, or Windows 2000 operating system. |
|
sys/wineventlog
Package wineventlog provides access to the Windows Event Log API used in all versions of Windows since Vista (i.e.
|
Package wineventlog provides access to the Windows Event Log API used in all versions of Windows since Vista (i.e. |
Click to show internal directories.
Click to hide internal directories.