scrubber

Documentation

Overview

Package scrubber implements support for cleaning sensitive information out of strings and files.

Compatibility

This module's API is not yet stable, and may change incompatibly from version to version.

Index

• Variables
• func AddDefaultReplacers(scrubber *Scrubber)
• func AddStrippedKeys(strippedKeys []string)
• func ScrubBytes(file []byte) ([]byte, error)
• func ScrubFile(filePath string) ([]byte, error)
• func ScrubLine(url string) string
• type Replacer
• type ReplacerKind
• type Scrubber
  • func New() *Scrubber
  • func (c *Scrubber) AddReplacer(kind ReplacerKind, replacer Replacer)
  • func (c *Scrubber) NewWriter(path string, perms os.FileMode) (*Writer, error)
  • func (c *Scrubber) ScrubBytes(file []byte) ([]byte, error)
  • func (c *Scrubber) ScrubFile(filePath string) ([]byte, error)
  • func (c *Scrubber) ScrubLine(message string) string
• type Writer
  • func NewWriter(path string, perms os.FileMode) (*Writer, error)
  • func (f *Writer) Close() error
  • func (f *Writer) Flush() error
  • func (f *Writer) Write(p []byte) (int, error)
  • func (f *Writer) WriteFromFile(filePath string) (int, error)
• Bugs

Variables

var DefaultScrubber = &Scrubber{}

DefaultScrubber is the scrubber used by the package-level cleaning functions.

It includes a set of agent-specific replacers. It can scrub DataDog App and API keys, passwords from URLs, and multi-line PEM-formatted TLS keys and certificates. It contains special handling for YAML-like content (with lines of the form "key: value") and can scrub passwords, tokens, and SNMP community strings in such content.

See default.go for details of these replacers.

Functions

func AddDefaultReplacers

func AddDefaultReplacers(scrubber *Scrubber)

AddDefaultReplacers to a scrubber. This is called automatically for DefaultScrubber, but can be used to initialize other, custom scrubbers with the default replacers.

func AddStrippedKeys

func AddStrippedKeys(strippedKeys []string)

AddStrippedKeys adds to the set of YAML keys that will be recognized and have their values stripped. This modifies the DefaultScrubber directly.

func ScrubBytes

func ScrubBytes(file []byte) ([]byte, error)

ScrubBytes scrubs credentials from the given slice of bytes, using the default scrubber.

func ScrubFile

func ScrubFile(filePath string) ([]byte, error)

ScrubFile scrubs credentials from the given file, using the default scrubber.

func ScrubLine

func ScrubLine(url string) string

ScrubLine scrubs credentials from a single line of text, using the default scrubber. It can be safely applied to URLs or to strings containing URLs. It does not run multi-line replacers, and should not be used on multi-line inputs.

Types

type Replacer

type Replacer struct {
	// Regex must match the sensitive information
	Regex *regexp.Regexp
	// Hints, if given, are strings which must also be present in the text for the regexp to match.
	// Especially in single-line replacers, this can be used to limit the contexts where an otherwise
	// very broad Regex is actually replaced.
	Hints []string
	// Repl is the text to replace the substring matching Regex.  It can use the regexp package's
	// replacement characters ($1, etc.) (see regexp#Regexp.ReplaceAll).
	Repl []byte
	// ReplFunc, if set, is called with the matched bytes (see regexp#Regexp.ReplaceAllFunc). Only
	// one of Repl and ReplFunc should be set.
	ReplFunc func(b []byte) []byte
}

Replacer represents a replacement of sensitive information with a "clean" version.

type ReplacerKind

type ReplacerKind int

ReplacerKind modifies how a Replacer is applied

const (
	// SingleLine indicates to Cleaner#AddReplacer that the replacer applies to
	// single lines.
	SingleLine ReplacerKind = iota
	// MultiLine indicates to Cleaner#AddReplacer that the replacer applies to
	// entire multiline text values.
	MultiLine
)

type Scrubber

type Scrubber struct {
	// contains filtered or unexported fields
}

Scrubber implements support for cleaning sensitive information out of strings and files. Its intended use is to "clean" data before it is logged or transmitted to a remote system, so that the meaning of the data remains clear without disclosing any sensitive information.

Scrubber works by applying a set of replacers, in order. It first applies all SingleLine replacers to each non-comment, non-blank line of the input.

Comments and blank lines are omitted. Comments are considered to begin with `#`.

It then applies all MultiLine replacers to the entire text of the input.

func New

func New() *Scrubber

New creates a new scrubber with no replacers installed.

func (*Scrubber) AddReplacer

func (c *Scrubber) AddReplacer(kind ReplacerKind, replacer Replacer)

AddReplacer adds a replacer of the given kind to the scrubber.

func (*Scrubber) NewWriter

func (c *Scrubber) NewWriter(path string, perms os.FileMode) (*Writer, error)

NewWriter creates a new Writer tied to this scrubber. The writer will write scrubbed data to the given file path with the given permissions.

func (*Scrubber) ScrubBytes

func (c *Scrubber) ScrubBytes(file []byte) ([]byte, error)

ScrubBytes scrubs credentials from slice of bytes

func (*Scrubber) ScrubFile

func (c *Scrubber) ScrubFile(filePath string) ([]byte, error)

ScrubFile scrubs credentials from file given by pathname

func (*Scrubber) ScrubLine

func (c *Scrubber) ScrubLine(message string) string

ScrubLine scrubs credentials from a single line of text. It can be safely applied to URLs or to strings containing URLs. It does not run multi-line replacers, and should not be used on multi-line inputs.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer is an io.Writer implementation that scrubs content before writing to a target file.

func NewWriter

func NewWriter(path string, perms os.FileMode) (*Writer, error)

NewWriter instantiates a Writer to the given file path with the given permissions, using the default scrubber.

func (*Writer) Close

func (f *Writer) Close() error

Close closes the underlying file, if buffered previously flushes the contents

func (*Writer) Flush

func (f *Writer) Flush() error

Flush if this is a buffered writer, it flushes the buffer, otherwise NOP

func (*Writer) Write

func (f *Writer) Write(p []byte) (int, error)

Write writes the scrubbed byte stream, applying all replacers and credential cleanup to target

func (*Writer) WriteFromFile

func (f *Writer) WriteFromFile(filePath string) (int, error)

WriteFromFile will read contents from file and write them scrubbed to target. If the file does not exist, this returns an error.

Notes

Bugs

• Writer applies scrubbing to each "chunk" of data independently. If a sensitive value spans two chunks, it will not be matched by a replacer and thus not scrubbed.