v1

README

Dapr encryption scheme v1: dapr.io/enc/v1

This document contains the reference for the Dapr encryption scheme v1, identified by the header dapr.io/enc/v1.

The Dapr encryption scheme is optimized for processing data as a stream. Data is chunked into multiple parts which are encrypted independently. This allows us to return data to callers as a stream, even when decrypting messages, being confident that we are not flushing unverified data to the client.

Sources: The encryption scheme that Dapr uses is heavily inspired by the Tink wire format (from the Tink library maintained by Google), as well as by Filippo Valsorda's age, and Minio's DARE.

Key

Each message is encrypted with a 256-bit symmetric File Key (FK) that is randomly generated by Dapr for each new message. The key must be generated as 32 byte of output from a CSPRNG (such as Go's crypto/rand.Reader) and must not be reused for other files.

The FK is wrapped using a key stored in a key vault (Key Encryption Key (KEK)) by Dapr. The result of the wrapping operation is the Wrapped File Key (WFK). The algorithm used depends on the type of the KEK as well as the algorithms supported by the component:

• For symmetric keys:
  • AES-KW with 256-bit keys (RFC 3394): A256KW
    • Because the File Key is 256-bit long, only 256-bit wrapping keys can be used
  • AES-CBC with keys 128-bit, 192-bit, and 256-bit: A128CBC-NOPAD, A192CBC-NOPAD, A256CBC-NOPAD
    • These don't use PKCS#7 padding because the File Key is 256-bit so it's a multiple of the AES block size.
• For RSA keys:
  • RSA OAEP with SHA-256: RSA-OAEP-256
    • Dapr doesn't impose limitations on the size of the key, and any key bigger than 1024 bits should work; however, 4096-bit keys are strongly recommended.

Other key wrapping algorithms can be implemented in the future.

Ciphertext format

The ciphertext is formatted as:

header || binary_payload

Header

The header is human-readable and contains 3 items, each terminated by a line feed (0x0A) character:

1. Name and version of the encryption scheme used. In this version of the spec, this is always dapr.io/enc/v1.
2. The manifest, which is a JSON object, compacted with all the unnecessary whitespaces removed.
3. The MAC for the header, base64-encoded.

Base64 encoding follows RFC 4648 §4 ("standard" format, with padding included)

For example:

dapr.io/enc/v1
{"k":"mykey","kw":1,"wfk":"hGYjwDpWEXEymSTFZ95zgX8krElb3Gqyls67R8zJA3k=","cph":1,"np":"Y3J5cHRvIQ=="}
pBDKLrhAWL7IAvDKBV/v7lmbTG6AEZbf3srUN0Pnn30=

Manifest

The second line in the header is the manifest, which is a JSON object.

It's important that the manifest is in a "compact" format that does not contain newline characters.

Note that per the JSON spec (RFC 8259), newline characters within a string are safely encoded as the string \n, which would not interpreted as newlines while scanning the manifest. The spec also defines what constitutes "unnecessary whitespace".

Its corresponding Go struct is:

type Manifest struct {
	// Name of the key that can be used to decrypt the message.
	// This is optional, and if specified can be in the format `key` or `key/version`.
	KeyName string `json:"k,omitempty"`
	// ID of the wrapping algorithm used.
	// 0x01 = A256KW
	// 0x02 = A128CBC-NOPAD
	// 0x03 = A192CBC-NOPAD
	// 0x04 = A256CBC-NOPAD
	// 0x05 = RSA-OAEP-256
	KeyWrappingAlgorithm int `json:"kw"`
	// The Wrapped File Key
	WFK []byte `json:"wfk"`
	// ID of the cipher used.
	// 0x01 = AES-GCM
	// 0x02 = ChaCha20-Poly1305
	Cipher int `json:"cph"`
	// Random sequence of 7 bytes generated by a CSPRNG.
	NoncePrefix []byte `json:"np"`
}

• KeyName is the name of the key that can be used to decrypt the message.
  Usually this is the same as the name of the key used to encrypt the message, but when asymmetric ciphers are used, it could be different.
  Including a KeyName in the manifest is not required, but when it's present, it's used as the default value for the key name while decrypting the document (however, users can override this value by passing a custom one while decrypting the document).
• Cipher indicates the cipher used to encrypt the actual data, and it must be an AEAD symmetric cipher.
  • Dapr will choose AES-GCM as cipher by default.
  • ChaCha20-Poly1305 is offered as an option for users that work with hardware that doesn't support AES-NI (such as Raspberry Pi), and needs to be enabled explicitly.
  • Other AEAD ciphers can be supported in the future if needed.

MAC

The third and final line of the plaintext header is the MAC for the header, which is the HMAC-SHA-256 hash computed over the previous 2 lines (including the final newline character).

The HMAC key is derived from the (plain-text) File Key with HKDF-SHA-256:

mac-key = HKDF-SHA-256(ikm = file key, salt = empty, info = "header")
MAC = HMAC-SHA-256(key = mac-key, message = first 2 lines of the header, including the trailing newline character)

HKDF-SHA-256 is a key derivation function based on HMAC with SHA-256. See RFC 5869 ("HMAC-based Extract-and-Expand Key Derivation Function (HKDF)"). Being based on HMAC, it's not vulnerable to length-extension attacks, so we do not consider using SHA-512 and truncating the output to 256-bits necessary.

Note that there's one newline character (0x0A) at the end of the MAC, which concludes the header.

Because each JSON encoder could produce a slightly different output, when verifying the manifest the MAC should be computed on the exact manifest string as included in the header. Verifiers should not re-encode the message as JSON themselves.

Binary payload

The binary payload begins immediately after the header (after the 3rd newline character) and it includes each segment of data encrypted:

segment_0 || segment_1 || ... || segment_k

Segments

The plaintext is chunked into segments of 64KB (65,536 bytes) each; the last segment may be shorter. Segments must never be empty, unless the entire file is empty.

Because segments are 64KB each, and we can have up to 2^32 segments, the maximum size of the encrypted message is 256TB.

Each segment of plaintext is encrypted independently and stored together with its authentication tag at the end:

encrypted_chunk || tag

Tag size is 16 bytes for AES-GCM and ChaCha20-Poly1305, so each encrypted segment has an overhead of 16 bytes.

Segments are encrypted with a Payload Key (PK) that is derived from the plain-text File Key and the nonce prefix:

payload-key = HKDF-SHA-256(ikm = file key, salt = nonce prefix, info = "payload")

Each segment is encrypted using a different 12-byte nonce:

nonce_prefix || i || last_segment

Where:

• nonce_prefix (7 bytes) is the nonce prefix from the header.
• i (4 bytes) is the sequence number, as a 32-bit unsigned integer counter, encoded as big-endian. The first segment has sequence number 0, and it increases.
• last_segment (1 byte) is 0x01 if this is the last segment, or 0x00 otherwise.

Documentation

Index

• Constants
• Variables
• func Decrypt(in io.Reader, opts DecryptOptions) (io.Reader, error)
• func Encrypt(in io.Reader, opts EncryptOptions) (io.Reader, error)
• type Cipher
  • func NewCipherFromID(id int) (Cipher, error)
  • func (c Cipher) ID() int
  • func (c Cipher) MarshalJSON() ([]byte, error)
  • func (c *Cipher) UnmarshalJSON(dataB []byte) error
  • func (c Cipher) Validate() (Cipher, error)
• type DecryptOptions
• type EncryptOptions
• type KeyAlgorithm
  • func NewKeyAlgorithmFromID(id int) (KeyAlgorithm, error)
  • func (a KeyAlgorithm) ID() int
  • func (a KeyAlgorithm) MarshalJSON() ([]byte, error)
  • func (a *KeyAlgorithm) UnmarshalJSON(dataB []byte) error
  • func (a KeyAlgorithm) Validate() (KeyAlgorithm, error)
• type Manifest
  • func (m *Manifest) Validate() (err error)
• type UnwrapKeyFn
• type WrapKeyFn

Constants

const (
	// SchemeName is the name of the encryption scheme.
	SchemeName = "dapr.io/enc/v1"

	// Size of each segment in the encrypted message.
	// Each segment is exactly 64KB in length, except the last one which could be shorter.
	SegmentSize = 64 << 10

	// Overhead of each segment in bytes.
	// This is equivalent to the size of the authentication tag for AES-GCM and ChaCha20-Poly1305.
	SegmentOverhead = 16

	// Length of the nonce prefix.
	NoncePrefixLength = 7
)

Variables

var (
	// Error returned when trying to decrypt a document whose manifest does not contain a key name, and the caller did not provide an explicit key name.
	ErrDecryptionKeyMissing = errors.New("document's manifest does not contain a key name, and no key name was provided")

	// Error returned when the signature of the document could not be validated.
	ErrDecryptionSignature = errors.New("failed to validate the document's signature")

	// Error returned when the deryption fails.
	// Most commonly this happens when a segment has been tampered with.
	ErrDecryptionFailed = errors.New("failed to decrypt segment")
)

var BufPool = sync.Pool{
	New: func() any {
		const bufSize = SegmentSize + SegmentOverhead + 1

		b := make([]byte, bufSize)
		return &b
	},
}

BufPool is a sync.Pool that returns buffers of SegmentSize+SegmentOverhead, plus one extra byte

Functions

func Decrypt

func Decrypt(in io.Reader, opts DecryptOptions) (io.Reader, error)

Decrypt a document using the `dapr.io/enc/v1` scheme The ciphertext is read from the `in` stream and written to the returned stream

func Encrypt

func Encrypt(in io.Reader, opts EncryptOptions) (io.Reader, error)

Encrypt a document using the `dapr.io/enc/v1` scheme. The plaintext is read from the `in` stream and written to the returned stream.

Types

type Cipher

type Cipher string

Cipher used to encrypt the file.

const (
	CipherAESGCM           Cipher = "AES-GCM"
	CipherChaCha20Poly1305 Cipher = "CHACHA20-POLY1305"
)

func NewCipherFromID

func NewCipherFromID(id int) (Cipher, error)

NewCipherFromID returns a Cipher from its ID.

func (Cipher) ID

func (c Cipher) ID() int

ID returns the numeric ID for the cipher.

func (Cipher) MarshalJSON

func (c Cipher) MarshalJSON() ([]byte, error)

MarhsalJSON implements json.Marshaler.

func (*Cipher) UnmarshalJSON

func (c *Cipher) UnmarshalJSON(dataB []byte) error

UnmarshalJSON implements json.Unmarshaler.

func (Cipher) Validate

func (c Cipher) Validate() (Cipher, error)

Validate the passed cipher and resolves aliases.

type DecryptOptions

type DecryptOptions struct {
	// Function that is invoked to unwrap the key
	UnwrapKeyFn UnwrapKeyFn
	// If set, uses this value as key name rather than the one included in the manifest
	KeyName string
}

DecryptOptions contains the options passed to the Decrypt method

type EncryptOptions

type EncryptOptions struct {
	// Function that is invoked to wrap the key
	WrapKeyFn WrapKeyFn
	// Algorithm used to wrap the file key
	// This must be one of the supported KeyAlgorithm constants, and must be usable by the kind of key provided
	Algorithm KeyAlgorithm
	// Name of the key to use
	KeyName string
	// Name of the key to include as decryption key
	// If empty, uses KeyName
	DecryptionKeyName string
	// If true, does not include the key name in the manifest
	OmitKeyName bool
	// Cipher used to encrypt the data
	// If nil, defaults to AES-GCM
	Cipher *Cipher
}

EncryptOptions contains the options passed to the Encrypt method

type KeyAlgorithm

type KeyAlgorithm string

Algorithm used to wrap the file key.

const (
	KeyAlgorithmAES256KW   KeyAlgorithm = "A256KW"
	KeyAlgorithmAES128CBC  KeyAlgorithm = "A128CBC-NOPAD"
	KeyAlgorithmAES192CBC  KeyAlgorithm = "A192CBC-NOPAD"
	KeyAlgorithmAES256CBC  KeyAlgorithm = "A256CBC-NOPAD"
	KeyAlgorithmRSAOAEP256 KeyAlgorithm = "RSA-OAEP-256"

	KeyAlgorithmAES KeyAlgorithm = "AES" // Alias for A256KW
	KeyAlgorithmRSA KeyAlgorithm = "RSA" // Alias for RSA-OAEP-256

)

func NewKeyAlgorithmFromID

func NewKeyAlgorithmFromID(id int) (KeyAlgorithm, error)

NewKeyAlgorithmFromID returns a KeyAlgorithm from its ID.

func (KeyAlgorithm) ID

func (a KeyAlgorithm) ID() int

ID returns the numeric ID for the algorithm.

func (KeyAlgorithm) MarshalJSON

func (a KeyAlgorithm) MarshalJSON() ([]byte, error)

MarhsalJSON implements json.Marshaler.

func (*KeyAlgorithm) UnmarshalJSON

func (a *KeyAlgorithm) UnmarshalJSON(dataB []byte) error

UnmarshalJSON implements json.Unmarshaler.

func (KeyAlgorithm) Validate

func (a KeyAlgorithm) Validate() (KeyAlgorithm, error)

Validate the passed algorithm and resolves aliases.

type Manifest

type Manifest struct {
	// Name of the key that can be used to decrypt the message.
	// This is optional, and if specified can be in the format `key` or `key/version`.
	KeyName string `json:"k,omitempty"`
	// ID of the wrapping algorithm used.
	KeyWrappingAlgorithm KeyAlgorithm `json:"kw"`
	// The Wrapped File Key.
	WFK []byte `json:"wfk"`
	// ID of the cipher used.
	Cipher Cipher `json:"cph"`
	// Random sequence of 7 bytes generated by a CSPRNG
	NoncePrefix []byte `json:"np"`
}

Manifest contains the properties for the clear-text manifest which is added at the beginning of the encrypted document.

func (*Manifest) Validate

func (m *Manifest) Validate() (err error)

Validate the object and returns no error if everything is fine. It also resolves aliases for the key algorithm and cipher.

type UnwrapKeyFn

type UnwrapKeyFn = func(wrappedKey []byte, algorithm string, keyName string, nonce []byte, tag []byte) (plaintextKey []byte, err error)

Signature of the method that unwraps keys. This does not accept a context, which needs to be provided by the caller of the Decrypt method inside the lambda.

type WrapKeyFn

type WrapKeyFn = func(plaintextKey []byte, algorithm string, keyName string, nonce []byte) (wrappedKey []byte, tag []byte, err error)

Signature of the method that wraps keys. This does not accept a context, which needs to be provided by the caller of the Encrypt method inside the lambda.