rere

README

rere

Redact sensitive fields through an allow or deny list of field and key names

What problem does rere solve?

rere allows you to redact sensitive fields. This is useful when you want to log a data structure but need to redact sensitive fields like passwords, keys, or other sensitive information and do not have control over the struct definition.

rere can be used for structs you've defined yourself, but there are better approaches to tagging fields as secret in those cases. rere is best used when you need to redact data from structs you don't control.

rere does not try to be intelligent about which fields or values to redact. It takes an opinionated stance to redact only string and []byte field and key values based on a provided allow or deny list of field and key names.

Install

Install rere by running:

go get github.com/dustinspecker/rere

Usage

rere has two functions it supports, RedactWithAllowList and RedactWithDenyList.

RedactWithAllowList will redact every string and []byte it finds in a provided array, map, slice, or struct. It will not redact any field or key names that are provided in an allow list.

RedactWithDenyList will skip redacting every string and []byte it finds in a provided array, map, slice, or struct. It will redact any field or key names are are provided in a deny list.

Both of these functions create a deep copy to avoid modifying the original value.

It is strongly recommended to only use RedactWithAllowList since it will prevent accidental leaks of sensitive information. Please read the warning in Example using a deny list for more information.

Example using an allow list

An example of typical usage of rere.RedactWithAllowList is:

func ExampleRedactWithAllowList() {
 // RedactWithAllowList will redact string and byte slice/array field values for field names not found in allow list
 type user struct {
  Username string
  Password string
  Key      []byte
  IsAdmin  bool
  Groups   []string
 }

 testUser := user{
  Username: "dustin",
  Password: "super secret",
  Key:      []byte("another secret"),
  IsAdmin:  true,
  Groups:   []string{"users"},
 }
 // RedactWithAllowList redacts all strings and []byte by default
 defaultRedactedUser := rere.RedactWithAllowList(testUser, nil)
 fmt.Printf("default redacted value: %+v\n", defaultRedactedUser)

 // allowList is matched against case insensitively
 allowList := []string{"username", "groups"}
 redactedUser := rere.RedactWithAllowList(testUser, allowList)
 fmt.Printf("redacted value with allow list: %+v\n", redactedUser)

 // RedactWithAllowList will not modify the original value - perfect for logging
 // RedactWithAllowList does not require a pointer. This is just to help further exemplify the point
 // that the original value is left unchanged.
 redactedUserPointer := rere.RedactWithAllowList(&testUser, allowList)
 fmt.Printf("redacted pointer value: %+v\n", *redactedUserPointer)
 fmt.Printf("original value left unchanged: %+v\n", testUser)

 //nolint:lll // ignore long line length for example output
 // Output: default redacted value: {Username:REDACTED Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[REDACTED]}
 // redacted value with allow list: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
 // redacted pointer value: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
 // original value left unchanged: {Username:dustin Password:super secret Key:[97 110 111 116 104 101 114 32 115 101 99 114 101 116] IsAdmin:true Groups:[users]}
}

Example using a deny list

An example of typical usage of rere.RedactWithDenyList is:

func ExampleRedactWithDenyList() {
 // RedactWithDenyList will redact string and byte slice/array field values for field names found in deny list
 type user struct {
  Username string
  Password string
  Key      []byte
  IsAdmin  bool
  Groups   []string
 }

 testUser := user{
  Username: "dustin",
  Password: "super secret",
  Key:      []byte("another secret"),
  IsAdmin:  true,
  Groups:   []string{"users"},
 }
 // RedactWithDenyList redacts nothing by default
 defaultRedactedUser := rere.RedactWithDenyList(testUser, nil)
 fmt.Printf("default denied value: %+v\n", defaultRedactedUser)

 // denyList is matched against case insensitively
 denyList := []string{"password", "Key"}
 redactedUser := rere.RedactWithDenyList(testUser, denyList)
 fmt.Printf("redacted value with deny list: %+v\n", redactedUser)

 // RedactWithDenyList will not modify the original value - perfect for logging
 // RedactWithDenyList does not require a pointer. This is just to help further exemplify the point
 // that the original value is left unchanged.
 redactedUserPointer := rere.RedactWithDenyList(&testUser, denyList)
 fmt.Printf("redacted pointer value: %+v\n", *redactedUserPointer)
 fmt.Printf("original value left unchanged: %+v\n", testUser)

 //nolint:lll // ignore long line length for example output
 // Output: default denied value: {Username:dustin Password:super secret Key:[97 110 111 116 104 101 114 32 115 101 99 114 101 116] IsAdmin:true Groups:[users]}
 // redacted value with deny list: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
 // redacted pointer value: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
 // original value left unchanged: {Username:dustin Password:super secret Key:[97 110 111 116 104 101 114 32 115 101 99 114 101 116] IsAdmin:true Groups:[users]}
}

NOTE: It is STRONGLY discouraged to use RedactWithDenyList in production code, as it is easy to accidentally miss redacting sensitive information.

Example: a struct in v1 has a field name of "Password". In v2, a new field name of "PrivateKey" is added and code is migrated from using "Password" to "PrivateKey". If the deny list is not updated, then the new field, "PrivateKey", will not be redacted.

RedactWithAllowList is recommended for production code, as it is more explicit about what fields are not redacted. In the previous example, the "PrivateKey" field would be redacted if it is not in the allow list. If a new field like "Organization" is added in v2, but forgotten in the allow list, then the worse case is that the "Organization" field is redacted by accident, which is less severe than leaking a "PrivateKey" field.

More examples

More examples can be found in examples_test.go.

How does rere work?

rere redacts values by the following process:

1. Create a deep copy of the input value
2. Traverse through any pointers to retrieve actual element value
3. Iterate and recurse through the element's struct fields, map keys, and slice/array elements
4. Use reflection to redact any field or key values that are string or []byte
   1. For RedactWithAllowList, if a field or key name is found in the allow list (case insensitive), then the value is left unchanged
   2. For RedactWithDenyList, if a field or key name is not found in the deny list (case insensitive), then the value is left unchanged

Why the name rere?

rere is a play on Git's rerere. The original name for this project was rerere where it stood for "recurse, reflect, redact" (the process of how rere works). This package's exported functions, RedactWithAllowList and RedactWithDenyList, would result in a stutter, so the package name was shortened to rere.

Also, I have a speech impediment, struggle to pronounce "re", and am my own worst enemy.

License

MIT

Documentation

Overview

Package rere redacts sensitive fields through an allow or deny list of field and key names

Index

• func RedactWithAllowList[T any](value T, allowList []string) T
• func RedactWithDenyList[T any](value T, denyList []string) T

Examples

• RedactWithAllowList
• RedactWithDenyList

Functions

func RedactWithAllowList

func RedactWithAllowList[T any](value T, allowList []string) T

RedactWithAllowList by default redacts all string and []byte field and key values found in the provided value. If a field or key name is in the allow list then it will not be redacted.

String fields are redacted with "REDACTED". Byte slice fields are redacted with []byte("REDACTED"). Empty string and byte slice fields are not redacted to make it easier to troubleshoot empty values.

RedactWithAllowList will create a deep copy of the provided value, so the original value is not modified.

RedactWithAllowList will loop through elements in slices and arrays to redact using above approach.

If RedactWithAllowList is directly provided a string or []byte value then it will redact the value with "REDACTED", regardless of the allow list. If a field or key value is a []string then the slice will be redacted if the field or key name does not appear in the allow list.

Example

package main

import (
	"fmt"

	"github.com/dustinspecker/rere"
)

func main() {
	// RedactWithAllowList will redact string and byte slice/array field values for field names not found in allow list
	type user struct {
		Username string
		Password string
		Key      []byte
		IsAdmin  bool
		Groups   []string
	}

	testUser := user{
		Username: "dustin",
		Password: "super secret",
		Key:      []byte("another secret"),
		IsAdmin:  true,
		Groups:   []string{"users"},
	}
	// RedactWithAllowList redacts all strings and []byte by default
	defaultRedactedUser := rere.RedactWithAllowList(testUser, nil)
	fmt.Printf("default redacted value: %+v\n", defaultRedactedUser)

	// allowList is matched against case insensitively
	allowList := []string{"username", "groups"}
	redactedUser := rere.RedactWithAllowList(testUser, allowList)
	fmt.Printf("redacted value with allow list: %+v\n", redactedUser)

	// RedactWithAllowList will not modify the original value - perfect for logging
	// RedactWithAllowList does not require a pointer. This is just to help further exemplify the point
	// that the original value is left unchanged.
	redactedUserPointer := rere.RedactWithAllowList(&testUser, allowList)
	fmt.Printf("redacted pointer value: %+v\n", *redactedUserPointer)
	fmt.Printf("original value left unchanged: %+v\n", testUser)

}

Output:

default redacted value: {Username:REDACTED Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[REDACTED]}
redacted value with allow list: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
redacted pointer value: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
original value left unchanged: {Username:dustin Password:super secret Key:[97 110 111 116 104 101 114 32 115 101 99 114 101 116] IsAdmin:true Groups:[users]}

func RedactWithDenyList

func RedactWithDenyList[T any](value T, denyList []string) T

RedactWithDenyList by default leaves all string and []byte field and key values found in the provided value as-is. If a field or key name is in the deny list then it will be redacted.

String fields are redacted with "REDACTED". Byte slice fields are redacted with []byte("REDACTED"). Empty string and byte slice fields are not redacted to make it easier to troubleshoot empty values.

RedactWithDenyList will create a deep copy of the provided value, so the original value is not modified.

RedactWithDenyList will loop through elements in slices and arrays to redact using above approach.

If RedactWithDenyList is directly provided a string or []byte value then it will not redact the value, regardless of the deny list. If a field or key value is a []string then the slice will be redacted if the field or key name does appear in the deny list.

NOTE: It is *STRONGLY* discouraged to use RedactWithDenyList in production code, as it is easy to accidentally miss redacting sensitive information. Example: a struct in v1 has a field name of "Password". In v2, a new field name of "PrivateKey" is added and code is migrated from using "Password" to "PrivateKey". If the deny list is not updated, then the new field, "PrivateKey", will not be redacted.

RedactWithAllowList is recommended for production code, as it is more explicit about what fields are not redacted. In the above example, the "PrivateKey" field would be redacted if it is not in the allow list. If a new field like "Organization" is added in v2, but forgotten in the allow list, then the worse case is that the "Organization" field is not redacted, which is less severe than leaking a "PrivateKey" field.

Example

package main

import (
	"fmt"

	"github.com/dustinspecker/rere"
)

func main() {
	// RedactWithDenyList will redact string and byte slice/array field values for field names found in deny list
	type user struct {
		Username string
		Password string
		Key      []byte
		IsAdmin  bool
		Groups   []string
	}

	testUser := user{
		Username: "dustin",
		Password: "super secret",
		Key:      []byte("another secret"),
		IsAdmin:  true,
		Groups:   []string{"users"},
	}
	// RedactWithDenyList redacts nothing by default
	defaultRedactedUser := rere.RedactWithDenyList(testUser, nil)
	fmt.Printf("default denied value: %+v\n", defaultRedactedUser)

	// denyList is matched against case insensitively
	denyList := []string{"password", "Key"}
	redactedUser := rere.RedactWithDenyList(testUser, denyList)
	fmt.Printf("redacted value with deny list: %+v\n", redactedUser)

	// RedactWithDenyList will not modify the original value - perfect for logging
	// RedactWithDenyList does not require a pointer. This is just to help further exemplify the point
	// that the original value is left unchanged.
	redactedUserPointer := rere.RedactWithDenyList(&testUser, denyList)
	fmt.Printf("redacted pointer value: %+v\n", *redactedUserPointer)
	fmt.Printf("original value left unchanged: %+v\n", testUser)

}

Output:

default denied value: {Username:dustin Password:super secret Key:[97 110 111 116 104 101 114 32 115 101 99 114 101 116] IsAdmin:true Groups:[users]}
redacted value with deny list: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
redacted pointer value: {Username:dustin Password:REDACTED Key:[82 69 68 65 67 84 69 68] IsAdmin:true Groups:[users]}
original value left unchanged: {Username:dustin Password:super secret Key:[97 110 111 116 104 101 114 32 115 101 99 114 101 116] IsAdmin:true Groups:[users]}