secrets

package
v0.0.0-...-a70aae3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 14, 2024 License: Apache-2.0 Imports: 41 Imported by: 16

Documentation

Overview

Package secrets provides a secrets store based on Google Secret Manager.

It supports stored and auto-generated secrets.

Stored secrets are predefined blobs or texts (e.g. passwords or serialized Tink keysets) that are placed in the Google Secret Manager and then read back via this package. How secrets are written is outside the scope of this package, it just reads them.

Auto-generated secrets are random high-entropy strings with no inherent structure. They are usually derived via a key derivation function from their name and some root secret read from Google Secret Manager. They are useful when calculating HMACs, as salts, etc.

Primary Tink AEAD key

This package exposes functionality to encrypt/decrypt blobs using Tink AEAD implementation. The reference to the encryption key in Google Secret Manager is supplied via `-primary-tink-aead-key` flag. This key must be explicitly initialized first.

Create a new Google Secret Manager secret (with no values) named `tink-aead-primary` in the cloud project that the server is running in (referred to as `<cloud-project>` below). Make sure the server account has "Secret Manager Secret Accessor" role in it. These steps are usually done using Terraform.

Initialize the key value by using https://go.chromium.org/luci/server/cmd/secret-tool tool: 

cd server/cmd/secret-tool
go run main.go create sm://<cloud-project>/tink-aead-primary -secret-type tink-aes256-gcm

This secret now contains a serialized Tink keyset with the primary encryption key. Pass `-primary-tink-aead-key sm://tink-aead-primary` to the server binary to tell it to use this key.

If necessary the key can be rotated using the same `secret-tool` tool: 

cd server/cmd/secret-tool
go run main.go rotation-begin sm://<cloud-project>/tink-aead-primary
# wait several hours for the new key to propagate into all caches
# confirm by looking at /chrome/infra/secrets/gsm/version metric
go run main.go rotation-end sm://<cloud-project>/tink-aead-primary

This will add a new active key to the keyset. It will be used to encrypt new plain text values, but the old key will still be recognized when decrypting existing cipher texts until the next rotation occurs: at most one old key version is retained currently.

The root secret

The root secret is used to derive auto-generated secrets using HKDF. The reference to the root secret key in Google Secret Manager is supplied via `-root-secret` flag. This key must be explicitly initialized first.

Create a new Google Secret Manager secret (with no values) named `root-secret` in the cloud project that the server is running in (referred to as `<cloud-project>` below). Make sure the server account has "Secret Manager Secret Accessor" role in it. These steps are usually done using Terraform.

Next create a new secret version with a randomly generated blob: 

cd server/cmd/secret-tool
go run main.go create sm://<cloud-project>/root-secret -secret-type random-bytes-32

Pass `-root-secret sm://root-secret` to the server binary to tell it to use this key.

To rotate the key use the `secret-tool` again: 

cd server/cmd/secret-tool
go run main.go rotation-begin sm://<cloud-project>/root-secret
# wait several hours for the new key to propagate into all caches
go run main.go rotation-end sm://<cloud-project>/root-secret

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoSuchSecret indicates the store can't find the requested secret.
	ErrNoSuchSecret = errors.New("secret not found")
	// ErrNoStoreConfigured indicates there's no Store in the context.
	ErrNoStoreConfigured = errors.New("secrets.Store is not in the context")
)
View Source 
var ErrNoPrimaryAEAD = errors.New("the primary AEAD primitive is not configured")

ErrNoPrimaryAEAD indicates the context doesn't have a primary AEAD primitive installed.

For production code it usually happens if `-primary-tink-aead-key` flag wasn't set.

For test code, it happens if the test context wasn't prepared correctly. See GeneratePrimaryTinkAEADForTest for generating a random key for tests.

View Source 
var ModuleName = module.RegisterName("go.chromium.org/luci/server/secrets")

ModuleName can be used to refer to this module when declaring dependencies.

Functions

func AddRotationHandler 

func AddRotationHandler(ctx context.Context, name string, cb RotationHandler) error

AddRotationHandler registers a callback called when the secret is updated.

If the context doesn't have Store set, returns ErrNoStoreConfigured.

func Decrypt 

func Decrypt(ctx context.Context, ciphertext, additionalData []byte) ([]byte, error)

Decrypt decrypts `ciphertext` with `additionalData` as additional authenticated data using the primary tink AEAD primitive in the context.

See PrimaryTinkAEAD for caveats.

Returns ErrNoPrimaryAEAD if there's no primary AEAD primitive in the context.

func Encrypt 

func Encrypt(ctx context.Context, plaintext, additionalData []byte) ([]byte, error)

Encrypt encrypts `plaintext` with `additionalData` as additional authenticated data using the primary tink AEAD primitive in the context.

See PrimaryTinkAEAD for caveats.

Returns ErrNoPrimaryAEAD if there's no primary AEAD primitive in the context.

func GeneratePrimaryTinkAEADForTest 

func GeneratePrimaryTinkAEADForTest(ctx context.Context) context.Context

GeneratePrimaryTinkAEADForTest generates a new key and sets it as primary.

Must be used only in tests.

func NewModule 

func NewModule(opts *ModuleOptions) module.Module

NewModule returns a server module that adds a secret store backed by Google Secret Manager to the global server context.

func NewModuleFromFlags 

func NewModuleFromFlags() module.Module

NewModuleFromFlags is a variant of NewModule that initializes options through command line flags.

Calling this function registers flags in flag.CommandLine. They are usually parsed in server.Main(...).

func URLSafeDecrypt 

func URLSafeDecrypt(ctx context.Context, encryptedString string, additionalData []byte) ([]byte, error)

URLSafeDecrypt decrypts the plaintext from the string generated by URLSafeEncrypt with the same additional data.

func URLSafeEncrypt 

func URLSafeEncrypt(ctx context.Context, plaintext, additionalData []byte) (string, error)

URLSafeEncrypt encrypts plaintext to an encrypted, URL safe string.

func Use 

func Use(ctx context.Context, s Store) context.Context

Use installs a Store implementation into the context.

Types

type AEADHandle 

type AEADHandle struct {
	// contains filtered or unexported fields
}

AEADHandle implements tink.AEAD by delegating to an atomically updated tink.AEAD primitive stored inside.

The value stored inside is updated whenever the underlying secret with the keyset is rotated. This can happen at any time, even between two consecutive Encrypt calls. Use Unwrap to get an immutable copy of the current tink.AEAD primitive.

func LoadTinkAEAD 

func LoadTinkAEAD(ctx context.Context, secretName string) (*AEADHandle, error)

LoadTinkAEAD loads a tink AEAD key from the given secret, subscribing to its rotation.

Returns a handle that points to a tink.AEAD primitive updated atomically when the underlying keys are rotated. You can either use the handle directly as a tink.AEAD primitive itself (in which case its underlying keyset may be changing between individual Encrypt/Decrypt operations), or get the current immutable tink.AEAD primitive via Unwrap() method. The latter is useful if you depend on Encrypt/Decrypt operations to not spontaneously change keys or you are calling them in a tight loop.

If the context has a process cache initialized (true for contexts in production code), loaded keys are cached there, i.e. calling LoadTinkAEAD twice with the same `secretName` value will return the exact same object. If the context doesn't have a process cache (happens in tests), LoadTinkAEAD constructs a new handle each time it is called.

The returned AEADHandle is always valid. If a new value of the rotated secret is malformed, the handle will retain its old keyset. Once loaded, it never "spoils".

func PrimaryTinkAEAD 

func PrimaryTinkAEAD(ctx context.Context) *AEADHandle

PrimaryTinkAEAD returns a handle pointing to an AEAD primitive to use by default in the process.

See https://pkg.go.dev/github.com/google/tink/go/tink#AEAD for the description of the AEAD primitive.

Make sure to append a context string to `additionalData` when calling Encrypt/Decrypt to guarantee that a cipher text generated in one context isn't unexpectedly reused in another context. Failure to do so can lead to compromises.

In production the keyset behind PrimaryTinkAEAD is specified via the `-primary-tink-aead-key` flag. This flag is optional. PrimaryTinkAEAD will return nil if the `-primary-tink-aead-key` flag was omitted. Code that depends on a presence of an AEAD implementation must check that the return value of PrimaryTinkAEAD is not nil during startup.

Tests can use GeneratePrimaryTinkAEADForTest to prepare a context with some randomly generated key.

func (*AEADHandle) Decrypt 

func (h *AEADHandle) Decrypt(ciphertext, additionalData []byte) ([]byte, error)

Decrypt is part of tink.AEAD interface.

func (*AEADHandle) Encrypt 

func (h *AEADHandle) Encrypt(plaintext, additionalData []byte) ([]byte, error)

Encrypt is part of tink.AEAD interface.

func (*AEADHandle) Unwrap 

func (h *AEADHandle) Unwrap() tink.AEAD

Unwrap returns the current immutable tink.AEAD pointed to by the handle.

Useful if you depend on Encrypt/Decrypt operations to not spontaneously change keys or you are calling them in a tight loop.

Do not retain the returned tink.AEAD for long. It will become essentially incorrect when the underlying keyset is rotated.

type DerivedStore 

type DerivedStore struct {
	// contains filtered or unexported fields
}

DerivedStore implements Store by deriving secrets from some single root secret using HKDF.

Caches all derived secrets internally forever. Assumes the set of possible key names is limited.

func NewDerivedStore 

func NewDerivedStore(root Secret) *DerivedStore

NewDerivedStore returns a store that derives secrets from the given root key.

func (*DerivedStore) AddRotationHandler 

func (d *DerivedStore) AddRotationHandler(ctx context.Context, name string, cb RotationHandler) error

AddRotationHandler is not implemented.

func (*DerivedStore) RandomSecret 

func (d *DerivedStore) RandomSecret(ctx context.Context, name string) (Secret, error)

RandomSecret returns a generated secret given its name.

func (*DerivedStore) SetRoot 

func (d *DerivedStore) SetRoot(root Secret)

SetRoot replaces the root key used to derive secrets.

func (*DerivedStore) StoredSecret 

func (d *DerivedStore) StoredSecret(ctx context.Context, name string) (Secret, error)

StoredSecret returns an error, since DerivedStore always derives secrets.

type ModuleOptions 

type ModuleOptions struct {
	// RootSecret points to the root secret used to derive random secrets.
	//
	// In production it should be a reference to a Google Secret Manager secret
	// (in a form "sm://<project>/<secret>" or just "sm://<secret>" to fetch it
	// from the current project).
	//
	// In non-production environments it can be a literal secret value in a form
	// "devsecret://<base64-encoded secret>" or "devsecret-text://<secret>". If
	// omitted in a non-production environment, some phony hardcoded value is
	// used.
	//
	// When using Google Secret Manager, the secret version "latest" is used to
	// get the current value of the root secret, and a single immediately
	// preceding previous version (if it is still enabled) is used to get the
	// previous version of the root secret. This allows graceful rotation of
	// random secrets.
	RootSecret string

	// PrimaryTinkAEADKey is the secret name with the JSON-serialized clear text
	// Tink AEAD keyset to use for AEAD operations by default via PrimaryTinkAEAD.
	//
	// It is optional. If unset, PrimaryTinkAEAD will return nil. Code that
	// depends on a presence of an AEAD implementation must check that the return
	// value of PrimaryTinkAEAD is not nil during startup.
	PrimaryTinkAEADKey string
}

ModuleOptions contain configuration of the secrets server module.

func (*ModuleOptions) Register 

func (o *ModuleOptions) Register(f *flag.FlagSet)

Register registers the command line flags.

type RotationHandler 

type RotationHandler func(context.Context, Secret)

RotationHandler is called from an internal goroutine after the store fetches a new version of a stored secret.

type Secret 

type Secret struct {
	Active  []byte   // current value of the secret, always set
	Passive [][]byte // optional list of other values, in no particular order
}

Secret represents multiple versions of some secret blob.

There's a current version (which is always set) that should be used for all kinds of operations: active (like encryption, signing, etc) and passive (like decryption, checking signatures, etc).

And there's zero or more other versions that should be used only for passive operations. Other versions contain previous or future values of the secret. They are important for implementing graceful rotation of the secret.

func RandomSecret 

func RandomSecret(ctx context.Context, name string) (Secret, error)

RandomSecret returns a random secret using Store in the context.

If the context doesn't have Store set, returns ErrNoStoreConfigured.

func StoredSecret 

func StoredSecret(ctx context.Context, name string) (Secret, error)

StoredSecret returns a stored secret using Store in the context.

If the context doesn't have Store set, returns ErrNoStoreConfigured.

func (Secret) Blobs 

func (s Secret) Blobs() [][]byte

Blobs returns the active version and all passive versions as one array.

func (Secret) Equal 

func (s Secret) Equal(a Secret) bool

Equal returns true if secrets are equal.

Does *not* run in constant time. Shouldn't be used in a cryptographic context due to susceptibility to timing attacks.

type SecretManagerStore 

type SecretManagerStore struct {
	// CloudProject is used for loading secrets of the form "sm://<name>".
	CloudProject string
	// AccessSecretVersion is an RPC to fetch the secret from the Secret Manager.
	AccessSecretVersion func(context.Context, *secretmanagerpb.AccessSecretVersionRequest, ...gax.CallOption) (*secretmanagerpb.AccessSecretVersionResponse, error)
	// contains filtered or unexported fields
}

SecretManagerStore implements Store using Google Secret Manager.

Stored secrets are fetched directly from Google Secret Manager. Random secrets are derived from a root secret using HKDF via DerivedStore.

func (*SecretManagerStore) AddRotationHandler 

func (sm *SecretManagerStore) AddRotationHandler(ctx context.Context, name string, cb RotationHandler) error

AddRotationHandler registers a callback which is called when the stored secret is updated.

The handler is called from an internal goroutine and receives a context passed to MaintenanceLoop. If multiple handlers for the same secret are registered, they are called in order of their registration one by one.

func (*SecretManagerStore) LoadRootSecret 

func (sm *SecretManagerStore) LoadRootSecret(ctx context.Context, rootSecret string) error

LoadRootSecret loads the root secret used to generate random secrets.

See StoredSecret for the format of the root secret.

func (*SecretManagerStore) MaintenanceLoop 

func (sm *SecretManagerStore) MaintenanceLoop(ctx context.Context)

MaintenanceLoop runs a loop that periodically rereads secrets.

It exits on context cancellation. Logs errors inside.

func (*SecretManagerStore) RandomSecret 

func (sm *SecretManagerStore) RandomSecret(ctx context.Context, name string) (Secret, error)

RandomSecret returns a random secret given its name.

func (*SecretManagerStore) ReportMetrics 

func (sm *SecretManagerStore) ReportMetrics(ctx context.Context)

ReportMetrics is called on each metrics flush to populate metrics.

func (*SecretManagerStore) SetRandomSecretsStore 

func (sm *SecretManagerStore) SetRandomSecretsStore(s Store)

SetRandomSecretsStore changes the store used for RandomSecret(...).

Can be used instead of LoadRootSecret to hook up a custom implementation.

func (*SecretManagerStore) StoredSecret 

func (sm *SecretManagerStore) StoredSecret(ctx context.Context, name string) (Secret, error)

StoredSecret returns a stored secret given its name.

Value of `name` should have form:

  • `sm://<project>/<secret>`: a concrete secret in Google Secret Manager.
  • `sm://<secret>`: same as `sm://<CloudProject>/<secret>`.
  • `devsecret://<base64-encoded secret>`: return this concrete secret.
  • `devsecret-gen://tink/aead`: generate a new secret of the Tink AEAD.
  • `devsecret-text://<string>`: return this concrete secret.

Caches secrets loaded from Google Secret Manager in memory and sets up a periodic background task to update the cached values to facilitate graceful rotation.

Calls to StoredSecret return the latest value from this local cache and thus are fast. To be notified about changes to the secret as soon as they are detected use AddRotationHandler.

type Store 

type Store interface {
	// RandomSecret returns a random secret given its name.
	//
	// The store will auto-generate the secret if necessary. Its value is
	// a random high-entropy blob.
	RandomSecret(ctx context.Context, name string) (Secret, error)

	// StoredSecret returns a previously stored secret given its name.
	//
	// How it was stored depends on the concrete implementation of the Store. The
	// difference from RandomSecret is that the Store will never try to
	// auto-generate such secret if it is missing and will return ErrNoSuchSecret
	// instead.
	StoredSecret(ctx context.Context, name string) (Secret, error)

	// AddRotationHandler registers a callback called when the secret is updated.
	//
	// Useful when a value of StoredSecret(...) is used to derive something else.
	// The callback allows the store to notify the consumer of the secret when
	// it changes.
	AddRotationHandler(ctx context.Context, name string, cb RotationHandler) error
}

Store knows how to retrieve or autogenerate a secret given its name.

See SecretManagerStore for a concrete implementation usually used in production.

func CurrentStore 

func CurrentStore(ctx context.Context) Store

CurrentStore returns a store installed in the context or nil.

Source Files

View all Source files

Directories

Path Synopsis
Package testsecrets provides a dumb in-memory secret store to use in unit tests.
 Package testsecrets provides a dumb in-memory secret store to use in unit tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.