README
¶
Kubernetes plugin
It reads Kubernetes logs and also adds pod meta-information. Also, it joins split logs into a single event.
Source log file should be named in the following format:
[pod-name]_[namespace]_[container-name]-[container-id].log
E.g. my_pod-1566485760-trtrq_my-namespace_my-container-4e0301b633eaa2bfdcafdeba59ba0c72a3815911a6a820bf273534b0f32d98e0.log
An information which plugin adds:
k8s_node– node name where pod is running;k8s_node_label_*– node labels;k8s_pod– pod name;k8s_namespace– pod namespace name;k8s_container– pod container name;k8s_label_*– pod labels.
⚠ Use add_file_name plugin if you want to add filename to events.
Example:
pipelines:
example_k8s_pipeline:
input:
type: k8s
offsets_file: /data/offsets.yaml
file_config: // customize file plugin
persistence_mode: sync
read_buffer_size: 2048
Config params
split_event_size int default=1000000
Docker splits long logs by 16kb chunks. The plugin joins them back, but if an event is longer than this value in bytes, it will be split after all.
Due to the optimization process it's not a strict rule. Events may be split even if they won't exceed the limit.
allowed_pod_labels []string
If set, it defines which pod labels to add to the event, others will be ignored.
allowed_node_labels []string
If set, it defines which node labels to add to the event, others will be ignored.
only_node bool default=false
Skips retrieving Kubernetes meta information using Kubernetes API and adds only k8s_node field.
watching_dir string default=/var/log/containers
Kubernetes dir with container logs. It's like watching_dir parameter from file plugin config.
offsets_file string required
The filename to store offsets of processed files. It's like offsets_file parameter from file plugin config.
file_config file.Config
Under the hood this plugin uses file plugin to collect logs from files. So you can change any file plugin config parameter using file_config section. Check out an example.
meta cfg.MetaTemplates
K8sMeta params
Add meta information to an event (look at Meta params) Use go-template syntax
Built-in meta params
k8s_pod: {{ .pod_name }}
k8s_namespace: {{ .namespace_name }}
k8s_container: {{ .container }}
k8s_container_id: {{ .container_id }}
Example: component: '{{ index .pod.Labels "component" | default .k8s_container }}'
Meta params
pod_name - string
namespace - string
container_name - string
container_id - string
pod - k8s.io/api/core/v1.Pod
Generated using insane-doc
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
// > @3@4@5@6
// >
// > Docker splits long logs by 16kb chunks. The plugin joins them back, but if an event is longer than this value in bytes, it will be split after all.
// > > Due to the optimization process it's not a strict rule. Events may be split even if they won't exceed the limit.
SplitEventSize int `json:"split_event_size" default:"1000000"` // *
// > @3@4@5@6
// >
// > If set, it defines which pod labels to add to the event, others will be ignored.
AllowedPodLabels []string `json:"allowed_pod_labels" slice:"true"` // *
AllowedPodLabels_ map[string]bool
// > @3@4@5@6
// >
// > If set, it defines which node labels to add to the event, others will be ignored.
AllowedNodeLabels []string `json:"allowed_node_labels" slice:"true"` // *
AllowedNodeLabels_ map[string]bool
// > @3@4@5@6
// >
// > Skips retrieving Kubernetes meta information using Kubernetes API and adds only `k8s_node` field.
OnlyNode bool `json:"only_node" default:"false"` // *
// > @3@4@5@6
// >
// > Kubernetes dir with container logs. It's like `watching_dir` parameter from [file plugin](/plugin/input/file/README.md) config.
WatchingDir string `json:"watching_dir" default:"/var/log/containers"` // *
WatchingDir_ string
// > @3@4@5@6
// >
// > The filename to store offsets of processed files. It's like `offsets_file` parameter from [file plugin](/plugin/input/file/README.md) config.
OffsetsFile string `json:"offsets_file" required:"true"` // *
// > @3@4@5@6
// >
// > Under the hood this plugin uses [file plugin](/plugin/input/file/README.md) to collect logs from files. So you can change any [file plugin](/plugin/input/file/README.md) config parameter using `file_config` section. Check out an example.
FileConfig file.Config `json:"file_config" child:"true"` // *
// > @3@4@5@6
// >
// > K8sMeta params
// >
// > Add meta information to an event (look at Meta params)
// > Use [go-template](https://pkg.go.dev/text/template) syntax
// >
// > Built-in meta params
// >
// > `k8s_pod`: `{{ .pod_name }}`
// >
// > `k8s_namespace`: `{{ .namespace_name }}`
// >
// > `k8s_container`: `{{ .container }}`
// >
// > `k8s_container_id`: `{{ .container_id }}`
// >
// > Example: “`component: '{{ index .pod.Labels "component" | default .k8s_container }}'“`
K8sMeta cfg.MetaTemplates `json:"meta"` // *
}
type MultilineAction ¶
type MultilineAction struct {
// contains filtered or unexported fields
}
func (*MultilineAction) Do ¶
func (p *MultilineAction) Do(event *pipeline.Event) pipeline.ActionResult
func (*MultilineAction) Start ¶
func (p *MultilineAction) Start(config pipeline.AnyConfig, params *pipeline.ActionPluginParams)
func (*MultilineAction) Stop ¶
func (p *MultilineAction) Stop()
Source Files
Directories