redactionbykeyprocessor

README

Redaction By Key processor

Status
Stability	[alpha]
Supported pipeline types	traces
Distributions	contrib

This processor deletes span attributes that don't match a list of allowed span attributes. It also masks span attribute values that match a blocked value list. Span attributes that aren't on the allowed list are removed before any value checks are done.

Use Cases

Typical use-cases:

• Prevent sensitive fields from accidentally leaking into traces
• Ensure compliance with legal, privacy, or security requirements

For example:

• EU General Data Protection Regulation (GDPR) prohibits the transfer of any personal data like birthdates, addresses, or ip addresses across borders without explicit consent from the data subject. Popular trace aggregation services are located in US, not in EU. You can use the redaction processor to scrub personal data from your data.
• PRC legislation prohibits the transfer of geographic coordinates outside of the PRC. Popular trace aggregation services are located in US, not in the PRC. You can use the redaction processor to scrub geographic coordinates from your data.
• Payment Card Industry (PCI) Data Security Standards prohibit logging certain things or storing them unencrypted. You can use the redaction processor to scrub them from your traces.

The above is written by an engineer, not a lawyer. The redaction processor is intended as one line of defence rather than the only compliance measure in place.

Processor Configuration

Please refer to config.go for the config spec.

Examples:

processors:
  redactionbykey:
    # allow_all_keys is a flag which when set to true, which can disables the
    # allowed_keys list. The list of blocked_values is applied regardless. If
    # you just want to block values, set this to true.
    allow_all_keys: false
    # allowed_keys is a list of span attribute keys that are allowed to pass
    # through. The list is designed to fail closed. If allowed_keys is empty,
    # no span attributes are allowed and all span attributes are removed. To
    # allow all keys, set allow_all_keys to true. To allow the span attributes
    # you know are good, add them to the list.
    allowed_keys:
      - description
      - group
      - id
      - name
    # blocked_values is a list of regular expressions for blocking values of
    # allowed span attributes. Values that match are masked
    blocked_values:
      - "4[0-9]{12}(?:[0-9]{3})?" ## Visa credit card number
      - "(5[1-5][0-9]{14})"       ## MasterCard number
    blocked_values_by_key:
      - key: http.url
        regex: ".*:.*(@)"
    # summary controls the verbosity level of the diagnostic attributes that
    # the processor adds to the spans when it redacts or masks other
    # attributes. In some contexts a list of redacted attributes leaks
    # information, while it is valuable when integrating and testing a new
    # configuration. Possible values:
    # - `debug` includes both redacted key counts and names in the summary
    # - `info` includes just the redacted key counts in the summary
    # - `silent` omits the summary attributes
    summary: debug

Refer to config.yaml for how to fit the configuration into an OpenTelemetry Collector pipeline definition.

Only span attributes included on the list of allowed keys list are retained. If allowed_keys is empty, then no span attributes are allowed. All span attributes are removed in that case. To keep all span attributes, you should explicitly set allow_all_keys to true.

blocked_values applies to the values of the allowed keys. If the value of an allowed key matches the regular expression for a blocked value, the matching part of the value is then masked with a fixed length of asterisks.

For example, if notes is on the list of allowed keys, then the notes span attribute is retained. However, if there is a value such as a credit card number in the notes field that matched a regular expression on the list of blocked values, then that value is masked.

Documentation

Index

• func NewFactory() processor.Factory
• type BlockedValueByKey
• type Config

Functions

func NewFactory

func NewFactory() processor.Factory

NewFactory creates a factory for the redaction processor.

Types

type BlockedValueByKey

type BlockedValueByKey struct {
	Key   string `mapstructure:"key"`
	Regex string `mapstructure:"regex"`
}

type Config

type Config struct {
	// AllowAllKeys is a flag to allow all span attribute keys. Setting this
	// to true disables the AllowedKeys list. The list of BlockedValues is
	// applied regardless. If you just want to block values, set this to true.
	AllowAllKeys bool `mapstructure:"allow_all_keys"`

	// AllowedKeys is a list of allowed span attribute keys. Span attributes
	// not on the list are removed. The list fails closed if it's empty. To
	// allow all keys, you should explicitly set AllowAllKeys
	AllowedKeys []string `mapstructure:"allowed_keys"`

	// BlockedValues is a list of regular expressions for blocking values of
	// allowed span attributes. Values that match are masked
	BlockedValues []string `mapstructure:"blocked_values"`

	// BlockedValues is a list of regular expressions for blocking values of
	// allowed span attributes. Values that match are masked
	BlockedValuesByKey []BlockedValueByKey `mapstructure:"blocked_values_by_key"`

	// Summary controls the verbosity level of the diagnostic attributes that
	// the processor adds to the spans when it redacts or masks other
	// attributes. In some contexts a list of redacted attributes leaks
	// information, while it is valuable when integrating and testing a new
	// configuration. Possible values are `debug`, `info`, and `silent`.
	Summary string `mapstructure:"summary"`
}