sourceprocessor

README

Source Processor

The sourceprocessor adds _source and other tags related to Sumo Logic metadata taxonomy.

It leverages data tagged by k8sprocessor and must be after it in the processing chain. It has certain expectations on the label names used by k8sprocessor which might be configured below.

Config

• collector (default = ""): name of the collector, put in collector tag
• source (default = "traces"): name of the source, put in _source tag
• source_name (default = "%{namespace}.%{pod}.%{container}"): _sourceName template
• source_category (default = "%{namespace}/%{pod_name}"): _sourceCategory template
• source_category_prefix (default = "kubernetes/"): prefix added before each _sourceCategory value
• source_category_replace_dash (default = "/"): character which all dashes (-) are being replaced to

Filtering section

NOTE: The filtering is done on the resource level attributes.

• exclude (default = empty): a mapping of field names to exclusion regexes for those particular fields. Whenever a value under particular field matches a corresponding regex, the processed entry is dropped.

  NOTE:

  When systemd related filtering is taking place (exclude contains an entry for _SYSTEMD_UNIT) then whenever the processed record contains _HOSTNAME attribute it will be added to the resulting record under host key.

Keys section (must match k8sprocessor config)

• annotation_prefix (default = "pod_annotation_"): prefix which allows to find given annotation; it is used for including/excluding pods, among other attributes
• pod_template_hash_key (default = "pod_labels_pod-template-hash"): attribute where pod template hash is found (used for pod extraction)
• pod_name_key (default = "pod_name"): attribute where name portion of the pod is stored during enrichment
• namespace_key (default = "namespace"): attribute where namespace name is found
• pod_key (default = "pod"): attribute where pod full name is found
• container_key (default = "container"): attribute where container name is found
• source_host_key (default = "source_host"): attribute where source host is found

Name translation and template keys

The key names provided as namespace, pod, pod_name, container in templates for source_category or source_nameare replaced with the key name provided in namespace_key, pod_key, pod_name_key, container_key respectively.

For example, when default template for source_category is being used (%{namespace}/%{pod_name}) and namespace_key=k8s.namespace.name, the resource has attributes:

k8s.namespace.name: my-namespace
pod_name: some-name

Then the _source_category will contain: my-namespace/some-name

Example config

processors:
  source:
    collector: "mycollector"
    source_name: "%{namespace}.%{pod}.%{container}"
    source_category: "%{namespace}/%{pod_name}"
    source_category_prefix: "kubernetes/"
    source_category_replace_dash: "/"
    exclude:
      namespace: "kube-system"
      pod: "custom-pod-.*"

Documentation

Overview

Package sourceprocessor implements a processor for tagging _source* fields

Index

• func NewFactory() component.ProcessorFactory
• func SafeEncodeString(s string) string
• type Config

Functions

func NewFactory

func NewFactory() component.ProcessorFactory

NewFactory returns a new factory for the Span processor.

func SafeEncodeString

func SafeEncodeString(s string) string

SafeEncodeString converts the pod_template_hash to an alphanumeric string using the same logic Kubernetes uses at https://github.com/kubernetes/apimachinery/blob/18a5ff3097b4b189511742e39151a153ee16988b/pkg/util/rand/rand.go#L119

Types

type Config

type Config struct {
	*config.ProcessorSettings `mapstructure:"-"`

	Collector                 string `mapstructure:"collector"`
	Source                    string `mapstructure:"source"`
	SourceName                string `mapstructure:"source_name"`
	SourceCategory            string `mapstructure:"source_category"`
	SourceCategoryPrefix      string `mapstructure:"source_category_prefix"`
	SourceCategoryReplaceDash string `mapstructure:"source_category_replace_dash"`
	ExcludeNamespaceRegex     string `mapstructure:"exclude_namespace_regex"`
	ExcludePodRegex           string `mapstructure:"exclude_pod_regex"`
	ExcludeContainerRegex     string `mapstructure:"exclude_container_regex"`
	ExcludeHostRegex          string `mapstructure:"exclude_host_regex"`

	AnnotationPrefix   string `mapstructure:"annotation_prefix"`
	ContainerKey       string `mapstructure:"container_key"`
	NamespaceKey       string `mapstructure:"namespace_key"`
	PodKey             string `mapstructure:"pod_key"`
	PodIDKey           string `mapstructure:"pod_id_key"`
	PodNameKey         string `mapstructure:"pod_name_key"`
	PodTemplateHashKey string `mapstructure:"pod_template_hash_key"`
	SourceHostKey      string `mapstructure:"source_host_key"`
}

Config defines configuration for Source processor.