scrubber

Documentation

Overview

Package scrubber implements support for cleaning sensitive information out of strings and files.

Compatibility

This module's API is not yet stable, and may change incompatibly from version to version.

Index

• Variables
• func AddDefaultReplacers(scrubber *Scrubber)
• func AddStrippedKeys(strippedKeys []string)
• func HideKeyExceptLastFiveChars(key string) string
• func ScrubBytes(file []byte) ([]byte, error)
• func ScrubDataObj(data *interface{})
• func ScrubFile(filePath string) ([]byte, error)
• func ScrubJSON(data []byte) ([]byte, error)
• func ScrubJSONString(data string) (string, error)
• func ScrubLine(url string) string
• func ScrubString(data string) (string, error)
• func ScrubYaml(data []byte) ([]byte, error)
• func ScrubYamlString(data string) (string, error)
• type Replacer
• type ReplacerKind
• type Scrubber
  • func New() *Scrubber
  • func NewWithDefaults() *Scrubber
  • func (c *Scrubber) AddReplacer(kind ReplacerKind, replacer Replacer)
  • func (c *Scrubber) ScrubBytes(data []byte) ([]byte, error)
  • func (c *Scrubber) ScrubDataObj(data *interface{})
  • func (c *Scrubber) ScrubFile(filePath string) ([]byte, error)
  • func (c *Scrubber) ScrubJSON(input []byte) ([]byte, error)
  • func (c *Scrubber) ScrubLine(message string) string
  • func (c *Scrubber) ScrubYaml(input []byte) ([]byte, error)

Variables

var (
	// DefaultScrubber is the scrubber used by the package-level cleaning functions.
	//
	// It includes a set of agent-specific replacers.  It can scrub DataDog App
	// and API keys, passwords from URLs, and multi-line PEM-formatted TLS keys and
	// certificates.  It contains special handling for YAML-like content (with
	// lines of the form "key: value") and can scrub passwords, tokens, and SNMP
	// community strings in such content.
	//
	// See default.go for details of these replacers.
	DefaultScrubber = &Scrubber{}
)

Functions

func AddDefaultReplacers

func AddDefaultReplacers(scrubber *Scrubber)

AddDefaultReplacers to a scrubber. This is called automatically for DefaultScrubber, but can be used to initialize other, custom scrubbers with the default replacers.

func AddStrippedKeys

func AddStrippedKeys(strippedKeys []string)

AddStrippedKeys adds to the set of YAML keys that will be recognized and have their values stripped. This modifies the DefaultScrubber directly and be added to any created scrubbers.

func HideKeyExceptLastFiveChars

func HideKeyExceptLastFiveChars(key string) string

HideKeyExceptLastFiveChars replaces all characters in the key with "*", except for the last 5 characters. If the key is an unrecognized length, replace all of it with the default string of "*"s instead.

func ScrubBytes

func ScrubBytes(file []byte) ([]byte, error)

ScrubBytes scrubs credentials from the given slice of bytes, using the default scrubber.

func ScrubDataObj

func ScrubDataObj(data *interface{})

ScrubDataObj scrubs credentials from the data interface by recursively walking over all the nodes

func ScrubFile

func ScrubFile(filePath string) ([]byte, error)

ScrubFile scrubs credentials from the given file, using the default scrubber.

func ScrubJSON

func ScrubJSON(data []byte) ([]byte, error)

ScrubJSON scrubs credentials from the given JSON by loading the data and scrubbing the object instead of the serialized string, using the default scrubber.

func ScrubJSONString

func ScrubJSONString(data string) (string, error)

ScrubJSONString scrubs credentials from the given JSON string by loading the data and scrubbing the object instead of the serialized string, using the default scrubber.

func ScrubLine

func ScrubLine(url string) string

ScrubLine scrubs credentials from a single line of text, using the default scrubber. It can be safely applied to URLs or to strings containing URLs. It does not run multi-line replacers, and should not be used on multi-line inputs.

func ScrubString

func ScrubString(data string) (string, error)

ScrubString scrubs credentials from the given string, using the default scrubber.

func ScrubYaml

func ScrubYaml(data []byte) ([]byte, error)

ScrubYaml scrubs credentials from the given YAML by loading the data and scrubbing the object instead of the serialized string, using the default scrubber.

func ScrubYamlString

func ScrubYamlString(data string) (string, error)

ScrubYamlString scrubs credentials from the given YAML string by loading the data and scrubbing the object instead of the serialized string, using the default scrubber.

Types

type Replacer

type Replacer struct {
	// Regex must match the sensitive information
	Regex *regexp.Regexp
	// YAMLKeyRegex matches the key of sensitive information in a dict/map. This is used when iterating over a
	// map[string]interface{} to scrub data for all matching key before being serialized.
	YAMLKeyRegex *regexp.Regexp
	// ProcessValue is a callback to be executed when YAMLKeyRegex matches the key of a map/dict in a YAML object. The
	// value is passed to the function and replaced by the returned interface. This is useful to produce custom
	// scrubbing. Example: keeping the last 5 digit of an api key.
	ProcessValue func(data interface{}) interface{}
	// Hints, if given, are strings which must also be present in the text for the regexp to match.
	// Especially in single-line replacers, this can be used to limit the contexts where an otherwise
	// very broad Regex is actually replaced.
	Hints []string
	// Repl is the text to replace the substring matching Regex.  It can use the regexp package's
	// replacement characters ($1, etc.) (see regexp#Regexp.ReplaceAll).
	Repl []byte
	// ReplFunc, if set, is called with the matched bytes (see regexp#Regexp.ReplaceAllFunc). Only
	// one of Repl and ReplFunc should be set.
	ReplFunc func(b []byte) []byte
}

Replacer represents a replacement of sensitive information with a "clean" version.

type ReplacerKind

type ReplacerKind int

ReplacerKind modifies how a Replacer is applied

const (
	// SingleLine indicates to Cleaner#AddReplacer that the replacer applies to
	// single lines.
	SingleLine ReplacerKind = iota
	// MultiLine indicates to Cleaner#AddReplacer that the replacer applies to
	// entire multiline text values.
	MultiLine
)

type Scrubber

type Scrubber struct {
	// contains filtered or unexported fields
}

Scrubber implements support for cleaning sensitive information out of strings and files. Its intended use is to "clean" data before it is logged or transmitted to a remote system, so that the meaning of the data remains clear without disclosing any sensitive information.

Scrubber works by applying a set of replacers, in order. It first applies all SingleLine replacers to each non-comment, non-blank line of the input.

Comments and blank lines are omitted. Comments are considered to begin with `#`.

It then applies all MultiLine replacers to the entire text of the input.

func New

func New() *Scrubber

New creates a new scrubber with no replacers installed.

func NewWithDefaults

func NewWithDefaults() *Scrubber

NewWithDefaults creates a new scrubber with the default replacers installed.

func (*Scrubber) AddReplacer

func (c *Scrubber) AddReplacer(kind ReplacerKind, replacer Replacer)

AddReplacer adds a replacer of the given kind to the scrubber.

func (*Scrubber) ScrubBytes

func (c *Scrubber) ScrubBytes(data []byte) ([]byte, error)

ScrubBytes scrubs credentials from slice of bytes

func (*Scrubber) ScrubDataObj

func (c *Scrubber) ScrubDataObj(data *interface{})

ScrubDataObj scrubs credentials from the data interface by recursively walking over all the nodes

func (*Scrubber) ScrubFile

func (c *Scrubber) ScrubFile(filePath string) ([]byte, error)

ScrubFile scrubs credentials from file given by pathname

func (*Scrubber) ScrubJSON

func (c *Scrubber) ScrubJSON(input []byte) ([]byte, error)

ScrubJSON scrubs credentials from the given json by loading the data and scrubbing the object instead of the serialized string.

func (*Scrubber) ScrubLine

func (c *Scrubber) ScrubLine(message string) string

ScrubLine scrubs credentials from a single line of text. It can be safely applied to URLs or to strings containing URLs. It does not run multi-line replacers, and should not be used on multi-line inputs.

func (*Scrubber) ScrubYaml

func (c *Scrubber) ScrubYaml(input []byte) ([]byte, error)

ScrubYaml scrubs credentials from the given YAML by loading the data and scrubbing the object instead of the serialized string.