obfuscate

Documentation

Overview

Package obfuscate implements quantizing and obfuscating of tags and resources for a set of spans matching a certain criteria.

Index

• Constants
• type Config
• type Enablable
• type HTTPConfig
• type JSONConfig
• type Logger
• type ObfuscatedQuery
  • func (oq *ObfuscatedQuery) Cost() int64
• type Obfuscator
  • func NewObfuscator(cfg *Config) *Obfuscator
  • func (o *Obfuscator) Obfuscate(typ, q string) (*ObfuscatedQuery, error)
  • func (o *Obfuscator) ObfuscateSQLExecPlan(jsonPlan string, normalize bool) (string, error)
  • func (o *Obfuscator) ObfuscateSQLString(in string) (*ObfuscatedQuery, error)
  • func (o *Obfuscator) ObfuscateStatsGroup(typ, q string) string
  • func (*Obfuscator) QuantizeRedisString(query string) string
  • func (o *Obfuscator) SQLLiteralEscapes() bool
  • func (o *Obfuscator) SetSQLLiteralEscapes(ok bool)
  • func (o *Obfuscator) Stop()
• type SQLConfig
• type SQLTokenizer
  • func NewSQLTokenizer(sql string, literalEscapes bool, cfg SQLConfig) *SQLTokenizer
  • func (tkn *SQLTokenizer) Err() error
  • func (tkn *SQLTokenizer) Reset(in string)
  • func (tkn *SQLTokenizer) Scan() (TokenKind, []byte)
  • func (tkn *SQLTokenizer) SeenEscape() bool
• type SyntaxError
  • func (e *SyntaxError) Error() string
• type TokenKind
  • func (k TokenKind) String() string

Constants

const (
	LexError = TokenKind(57346) + iota

	ID
	Limit
	Null
	String
	DoubleQuotedString
	Number
	BooleanLiteral
	ValueArg
	ListArg
	Comment
	Variable
	Savepoint
	PreparedStatement
	EscapeSequence
	NullSafeEqual
	LE
	GE
	NE
	As
	From
	Update
	Insert
	Into
	Join
	TableName
	ColonCast

	// FilteredGroupable specifies that the given token has been discarded by one of the
	// token filters and that it is groupable together with consecutive FilteredGroupable
	// tokens.
	FilteredGroupable

	// FilteredGroupableParenthesis is a parenthesis marked as filtered groupable. It is the
	// beginning of either a group of values ('(') or a nested query. We track is as
	// a special case for when it may start a nested query as opposed to just another
	// value group to be obfuscated.
	FilteredGroupableParenthesis

	// Filtered specifies that the token is a comma and was discarded by one
	// of the filters.
	Filtered

	// FilteredBracketedIdentifier specifies that we are currently discarding
	// a bracketed identifier (MSSQL).
	// See issue https://github.com/DataDog/datadog-trace-agent/issues/475.
	FilteredBracketedIdentifier
)

list of available tokens; this list has been reduced because we don't need a full-fledged tokenizer to implement a Lexer

const EndChar = unicode.MaxRune + 1

EndChar is used to signal that the scanner has finished reading the query. This happens when there are no more characters left in the query or when invalid encoding is discovered. EndChar is an invalid rune value that can not be found in any valid string.

Types

type Config

type Config struct {
	// ES holds the obfuscation configuration for ElasticSearch bodies.
	ES JSONConfig `mapstructure:"elasticsearch"`

	// Mongo holds the obfuscation configuration for MongoDB queries.
	Mongo JSONConfig `mapstructure:"mongodb"`

	// SQLExecPlan holds the obfuscation configuration for SQL Exec Plans. This is strictly for safety related obfuscation,
	// not normalization. Normalization of exec plans is configured in SQLExecPlanNormalize.
	SQLExecPlan JSONConfig `mapstructure:"sql_exec_plan"`

	// SQLExecPlanNormalize holds the normalization configuration for SQL Exec Plans.
	SQLExecPlanNormalize JSONConfig `mapstructure:"sql_exec_plan_normalize"`

	// SQL specifies additonal SQL configuration options.
	SQL SQLConfig `mapstructure:"-"`

	// HTTP holds the obfuscation settings for HTTP URLs.
	HTTP HTTPConfig `mapstructure:"http"`

	// RemoveStackTraces specifies whether stack traces should be removed.
	// More specifically "error.stack" tag values will be cleared.
	RemoveStackTraces bool `mapstructure:"remove_stack_traces"`

	// Redis holds the configuration for obfuscating the "redis.raw_command" tag
	// for spans of type "redis".
	Redis Enablable `mapstructure:"redis"`

	// Memcached holds the configuration for obfuscating the "memcached.command" tag
	// for spans of type "memcached".
	Memcached Enablable `mapstructure:"memcached"`

	// Statsd specifies the statsd client to use when reporting metrics.
	Statsd statsd.ClientInterface

	// ErrorLogger specifies the logger to use when logging errors.
	Log Logger
}

Config holds the configuration for obfuscating sensitive data for various span types.

type Enablable

type Enablable struct {
	Enabled bool `mapstructure:"enabled"`
}

Enablable can represent any option that has an "enabled" boolean sub-field.

type HTTPConfig

type HTTPConfig struct {
	// RemoveQueryStrings determines query strings to be removed from HTTP URLs.
	RemoveQueryString bool `mapstructure:"remove_query_string" json:"remove_query_string"`

	// RemovePathDigits determines digits in path segments to be obfuscated.
	RemovePathDigits bool `mapstructure:"remove_paths_with_digits" json:"remove_path_digits"`
}

HTTPConfig holds the configuration settings for HTTP obfuscation.

type JSONConfig

type JSONConfig struct {
	// Enabled will specify whether obfuscation should be enabled.
	Enabled bool `mapstructure:"enabled"`

	// KeepValues will specify a set of keys for which their values will
	// not be obfuscated.
	KeepValues []string `mapstructure:"keep_values"`

	// ObfuscateSQLValues will specify a set of keys for which their values
	// will be passed through SQL obfuscation
	ObfuscateSQLValues []string `mapstructure:"obfuscate_sql_values"`
}

JSONConfig holds the obfuscation configuration for sensitive data found in JSON objects.

type Logger

type Logger interface {
	Errorf(format string, params ...interface{}) error
	Debugf(format string, params ...interface{})
}

Logger ...

type ObfuscatedQuery

type ObfuscatedQuery struct {
	Query     string // the obfuscated SQL query
	TablesCSV string // comma-separated list of tables that the query addresses
}

ObfuscatedQuery specifies information about an obfuscated SQL query.

func (*ObfuscatedQuery) Cost

func (oq *ObfuscatedQuery) Cost() int64

Cost returns the number of bytes needed to store all the fields of this ObfuscatedQuery.

type Obfuscator

type Obfuscator struct {
	// contains filtered or unexported fields
}

Obfuscator quantizes and obfuscates spans. The obfuscator is not safe for concurrent use.

func NewObfuscator

func NewObfuscator(cfg *Config) *Obfuscator

NewObfuscator creates a new obfuscator

func (*Obfuscator) Obfuscate

func (o *Obfuscator) Obfuscate(typ, q string) (*ObfuscatedQuery, error)

Obfuscate may obfuscate span's properties based on its type and on the Obfuscator's configuration.

func (*Obfuscator) ObfuscateSQLExecPlan

func (o *Obfuscator) ObfuscateSQLExecPlan(jsonPlan string, normalize bool) (string, error)

ObfuscateSQLExecPlan obfuscates query conditions in the provided JSON encoded execution plan. If normalize=True, then cost and row estimates are also obfuscated away.

func (*Obfuscator) ObfuscateSQLString

func (o *Obfuscator) ObfuscateSQLString(in string) (*ObfuscatedQuery, error)

ObfuscateSQLString quantizes and obfuscates the given input SQL query string. Quantization removes some elements such as comments and aliases and obfuscation attempts to hide sensitive information in strings and numbers by redacting them.

func (*Obfuscator) ObfuscateStatsGroup

func (o *Obfuscator) ObfuscateStatsGroup(typ, q string) string

ObfuscateStatsGroup obfuscates the given stats bucket group.

func (*Obfuscator) QuantizeRedisString

func (*Obfuscator) QuantizeRedisString(query string) string

QuantizeRedisString returns a quantized version of a Redis query.

TODO(gbbr): Refactor this method to use the tokenizer and remove "compactWhitespaces". This method is buggy when commands contain quoted strings with newlines.

func (*Obfuscator) SQLLiteralEscapes

func (o *Obfuscator) SQLLiteralEscapes() bool

SQLLiteralEscapes reports whether escape characters should be treated literally by the SQL obfuscator.

func (*Obfuscator) SetSQLLiteralEscapes

func (o *Obfuscator) SetSQLLiteralEscapes(ok bool)

SetSQLLiteralEscapes sets whether or not escape characters should be treated literally by the SQL obfuscator.

func (*Obfuscator) Stop

func (o *Obfuscator) Stop()

Stop cleans up after a finished Obfuscator.

type SQLConfig

type SQLConfig struct {
	// Cache reports whether SQL query obfuscation result caching should be enabled.
	Cache bool

	// TableNames enables adding SQL table names unto spans in the "sql.tables" tag.
	TableNames bool

	// QuantizeTables enables quantiation of table names.
	QuantizeTables bool
}

SQLConfig specifies the configuration for SQL obfuscation.

type SQLTokenizer

type SQLTokenizer struct {
	// contains filtered or unexported fields
}

SQLTokenizer is the struct used to generate SQL tokens for the parser.

func NewSQLTokenizer

func NewSQLTokenizer(sql string, literalEscapes bool, cfg SQLConfig) *SQLTokenizer

NewSQLTokenizer creates a new SQLTokenizer for the given SQL string. The literalEscapes argument specifies whether escape characters should be treated literally or as such.

func (*SQLTokenizer) Err

func (tkn *SQLTokenizer) Err() error

Err returns the last error that the tokenizer encountered, or nil.

func (*SQLTokenizer) Reset

func (tkn *SQLTokenizer) Reset(in string)

Reset the underlying buffer and positions

func (*SQLTokenizer) Scan

func (tkn *SQLTokenizer) Scan() (TokenKind, []byte)

Scan scans the tokenizer for the next token and returns the token type and the token buffer.

func (*SQLTokenizer) SeenEscape

func (tkn *SQLTokenizer) SeenEscape() bool

SeenEscape returns whether or not this tokenizer has seen an escape character within a scanned string

type SyntaxError

type SyntaxError struct {
	Offset int64 // error occurred after reading Offset bytes
	// contains filtered or unexported fields
}

A SyntaxError is a description of a JSON syntax error.

func (*SyntaxError) Error

func (e *SyntaxError) Error() string

type TokenKind

type TokenKind uint32

TokenKind specifies the type of the token being scanned. It may be one of the defined constants below or in some cases the actual rune itself.

func (TokenKind) String

func (k TokenKind) String() string