silent

package module
v0.0.0-...-82cf267 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 30, 2024 License: MIT Imports: 8 Imported by: 0

README

Silent GoDoc

Silent is a Go library designed for transparent data encryption at rest in SQL, NoSQL databases, and beyond. It eliminates boilerplate code, allowing you to manage sensitive data with minimal changes to your application. True to its name, it operates silently, making your code less verbose and more secure.

⚠️ Warning: This is a working prototype, but it's still in early development stage, and should not be used in production. While core ideas will remain the same, design, API and data format is not yet stable and may change in future updates.

Key features

  • Zero boilerplate: configure encryption once and use it everywhere
  • Pluggable Crypter interface for custom encryption strategies
  • Built-in crypter that supports key rotation and powered by the encryption-at-rest library from MinIO
  • HashiCorp Vault Adapter: Integration with HashiCorp Vault encryption service (coming soon)
  • Support for SQL databases, JSON serialization, and more formats (BSON and others coming soon)

Installation

go get github.com/destel/silent

Basic usage

Using silent to encrypt columns in your database is incredibly simple. Just use silent.EncryptedValue instead of regular strings in your models, then work with your models as usual. Silent handles the encryption and decryption seamlessly behind the scenes.

// Use EncryptedValue in your models
type User struct {
    Username string
    Token    silent.EncryptedValue
}

user := User{
    Username: "john_doe",
    Token:    silent.EncryptedValue("some token"),
}

// Work with your models as usual
res, err := db.ExecContext(ctx, `INSERT INTO users (username, token) VALUES (?, ?)`, user.Username, user.Token)

Before you start using silent, you'll need to initialize it once at the start of your application:

// Create a Crypter instance
var crypter silent.Crypter = ... // Initialize crypter

// Bind the crypter to the EncryptedValue type
silent.BindCrypterTo[silent.EncryptedValue](crypter)

And that's it! With just a few lines of code, you can add encryption at rest to your database fields, ensuring the security of sensitive data without the hassle of complex setup or extensive code changes.

Full runnable example

Design philosophy

Library is built upon three core concepts that work together to provide a simple and flexible way to encrypt and decrypt sensitive data:

  • Crypter Interface
    • Defines the Encrypt and Decrypt methods
    • Allows for custom encryption strategies
  • EncryptedValue type
    • Transparently handles encryption and decryption
    • Abstracts away the complexity of working with encrypted data
  • BindCrypterTo function
    • Binds a crypter instance to an EncryptedValue type

MultiKeyCrypter

Silent ships with a built-in MultiKeyCrypter that provides a secure and flexible encryption solution. It supports multiple encryption keys and seamless key rotation, making it easy to maintain the security of your encrypted data over time.

Features
  • Support for multiple encryption keys
  • Zero downtime key rotation
  • Powered by the MinIO encryption-at-rest library, ensuring strong security
  • Bypass mode for easy testing and debugging in development environments
Usage

To use MultiKeyCrypter, simply create an instance, then add your encryption keys with unique IDs. MultiKeyCrypter uses the last added key for encryption and automatically selects the appropriate key for decryption based on the key ID embedded in the encrypted data

crypter := silent.MultiKeyCrypter{}
crypter.AddKey(1, []byte("your-encryption-key-1")) // never hardcode keys in production
crypter.AddKey(2, []byte("your-encryption-key-2"))

silent.BindCrypterTo[silent.EncryptedValue](&crypter)

To rotate keys, simply add a new key with a unique identifier, without removing the old keys:

crypter.AddKey(3, []byte("your-new-encryption-key"))
Bypass mode

MultiKeyCrypter also supports a bypass mode, which is useful for testing and debugging in development environments. In bypass mode data is prefixed with '#' instead of being encrypted, making it readable in plain text. Decryption is still performed as usual, allowing to work with both encrypted and plain text data in the same environment.

crypter := silent.MultiKeyCrypter{}
crypter.AddKey(1, []byte("your-encryption-key"))
crypter.Bypass = true

silent.BindCrypterTo[silent.EncryptedValue](&crypter)
Best practices
  • Never hardcode encryption keys in your code
  • Use a secure key management system to store and manage your keys
  • Rotate your encryption keys regularly

Coming soon: HashiCorp Vault crypter

First stable release will have a built-in adapter for HashiCorp Vault encryption service.

Beyond SQL: encrypted data everywhere

Silent's EncryptedValue is not limited to SQL databases. It seamlessly integrates with various storage systems and formats, ensuring that your sensitive data remains encrypted across your entire application stack.

JSON serialization

EncryptedValue is automatically encrypted when serialized to JSON, making it effortless to secure your data in JSON-based storage systems. This includes:

  • NoSQL databases that use JSON for communication (e.g., CouchDB, Firebase)
  • REST APIs
  • JSON files
  • Message queues
  • Caches

Simply use EncryptedValue in your structs, and Silent will handle the encryption and decryption transparently whenever the data is serialized or deserialized.

Full runnable example

type User struct {
    Username string                `json:"username"`
    Token    silent.EncryptedValue `json:"token"`
}
Coming soon: BSON and more

I'm actively working on expanding Silent's support for more formats and storage systems. The first stable release will include support for BSON serialization, used in MongoDB.

Advanced usage: creating custom EncryptedValue types

In some scenarios, you may need to use different encryption strategies or keys for different parts of your application. For example, you might have a requirement to encrypt sensitive user data with a specific encryption algorithm, while system-level data needs to be encrypted with a different algorithm or key. Silent provides the flexibility to handle such cases by allowing you to create custom EncryptedValue types, each bound to its own crypter instance.

Silent uses a type factory pattern to create custom EncryptedValue types. To ensure that each custom EncryptedValue type is unique, Silent employs a trick of creating distinct dummy types that are not used later in the code.

// Define a type
type dummyType struct{} // this won't be used in your code
type CustomEncryptedValue = silent.EncryptedValueFactory[dummyType]

// Bind it to its own crypter
silent.BindCrypterTo[CustomEncryptedValue](customCrypter)

Now, you can use different encryption strategies or keys for different parts of your application.

type User struct {
    Username string
    Token    silent.EncryptedValue
}

type Admin struct {
    Username string
    Token    CustomEncryptedValue
}

Documentation

Overview

Package silent is a Go library designed for transparent data encryption at rest in SQL, NoSQL databases, and beyond. It eliminates boilerplate code, allowing you to manage sensitive data with minimal changes to your application.

Example (DatabaseEncryptAndDecrypt)

This example showcases how to automatically encrypt and decrypt the token column of the users table in the database. The tokens are encrypted before storing them in the database and decrypted when retrieving the user data. Additionally, the example demonstrates that database is actually encrypted by reading the users again and scanning the token column as []byte. 

package main

import (
	"database/sql"
	"encoding/base64"
	"fmt"

	"github.com/destel/silent"
	_ "github.com/proullon/ramsql/driver"
)

type User struct {
	Username string                `json:"username"`
	Token    silent.EncryptedValue `json:"token"`
}

// RawUser is a helper type to read users from the database without decrypting the token column.
// It serves to demonstrate that the token column is indeed encrypted in the database.
type RawUser struct {
	Username string
	Token    []byte
}

func main() {
	db, err := initDB()
	if err != nil {
		fmt.Println("failed to init db:", err)
		return
	}

	// Initialize the crypter and bind it to the EncryptedValue type
	crypter := silent.MultiKeyCrypter{}
	crypter.AddKey(0x1, mustDecodeBase64("Qpk1tvmH8nAljiKyyDaGJXRH82ZjWtEX+2PR50sB5WU="))

	silent.BindCrypterTo[silent.EncryptedValue](&crypter)

	// Prepare some users
	alice := User{
		Username: "alice",
		Token:    silent.EncryptedValue("some token"),
	}

	bob := User{
		Username: "bob",
		Token:    silent.EncryptedValue("another token"),
	}

	// Save encrypted users to DB
	if err := saveUser(db, &alice, &bob); err != nil {
		fmt.Println("failed to save users:", err)
		return
	}

	// Read the users back. They will be automatically decrypted
	rows, err := db.Query("SELECT username, token FROM users")
	if err != nil {
		fmt.Println("failed to fetch users:", err)
		return
	}

	users, err := scanAllRows(rows, func(rows *sql.Rows) (*User, error) {
		var u User
		err := rows.Scan(&u.Username, &u.Token)
		return &u, err
	})
	if err != nil {
		fmt.Println("failed to scan users:", err)
		return
	}

	fmt.Println("Decrypted users:")
	for _, u := range users {
		fmt.Printf("%+v\n", u)
	}
	fmt.Println("")

	// Now read the same users again but without decrypting the token column.
	// Scan into RawUser type for this
	rows, err = db.Query("SELECT username, token FROM users")
	if err != nil {
		fmt.Println("failed to fetch users:", err)
		return
	}

	encryptedUsers, err := scanAllRows(rows, func(rows *sql.Rows) (*RawUser, error) {
		var u RawUser
		err := rows.Scan(&u.Username, &u.Token)
		return &u, err
	})
	if err != nil {
		fmt.Println("failed to scan users:", err)
		return
	}

	fmt.Println("Encrypted users:")
	for _, u := range encryptedUsers {
		fmt.Printf("%+v\n", u)
	}
}

func mustDecodeBase64(s string) []byte {
	res, err := base64.StdEncoding.DecodeString(s)
	if err != nil {
		panic(err)
	}
	return res
}

func initDB() (*sql.DB, error) {
	db, err := sql.Open("ramsql", "testdb")
	if err != nil {
		return nil, err
	}

	_, err = db.Exec("CREATE TABLE users (username VARCHAR(255), token VARBINARY(255), PRIMARY KEY (username))")
	if err != nil {
		return nil, err
	}

	return db, nil
}

func saveUser(db *sql.DB, users ...*User) error {
	for _, u := range users {
		_, err := db.Exec(`INSERT INTO users (username, token) VALUES (?, ?)`, u.Username, u.Token)
		if err != nil {
			return err
		}

	}
	return nil
}

// scanAllRows is a generic helper function that scans all rows from a sql.Rows object into a slice of T
func scanAllRows[T any](rows *sql.Rows, f func(*sql.Rows) (T, error)) ([]T, error) {
	defer rows.Close()

	var res []T
	for rows.Next() {
		v, err := f(rows)
		if err != nil {
			return nil, err
		}

		res = append(res, v)
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	return res, nil
}

Output:
Example (JsonEncryptAndDecrypt)

This example illustrates how to automatically encrypt and decrypt the token field of the User struct when marshaling and unmarshaling JSON data 

package main

import (
	"encoding/base64"
	"encoding/json"
	"fmt"

	"github.com/destel/silent"
	_ "github.com/proullon/ramsql/driver"
)

type User struct {
	Username string                `json:"username"`
	Token    silent.EncryptedValue `json:"token"`
}

func main() {
	// Initialize the crypter and bind it to the EncryptedValue type
	crypter := silent.MultiKeyCrypter{}
	crypter.AddKey(0x1, mustDecodeBase64("Qpk1tvmH8nAljiKyyDaGJXRH82ZjWtEX+2PR50sB5WU="))

	silent.BindCrypterTo[silent.EncryptedValue](&crypter)

	// Marshal some users to JSON to demonstrate how the token field is automatically encrypted
	users := []User{
		{
			Username: "alice",
			Token:    silent.EncryptedValue("some token"),
		},
		{
			Username: "bob",
			Token:    silent.EncryptedValue("another token"),
		},
	}

	j, err := json.MarshalIndent(users, "", "  ")
	if err != nil {
		fmt.Println("failed to marshal users:", err)
		return
	}

	// Print the encrypted JSON
	fmt.Println("Encrypted JSON:")
	fmt.Println(string(j))
	fmt.Println("")

	// Unmarshal the JSON back to demonstrate how the token field is automatically decrypted
	var decryptedUsers []User
	if err := json.Unmarshal(j, &decryptedUsers); err != nil {
		fmt.Println("failed to unmarshal users:", err)
		return
	}

	// Print the decrypted users
	fmt.Println("Decrypted users:")
	for _, u := range decryptedUsers {
		fmt.Printf("%+v\n", u)
	}
}

func mustDecodeBase64(s string) []byte {
	res, err := base64.StdEncoding.DecodeString(s)
	if err != nil {
		panic(err)
	}
	return res
}

Output:

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	ErrUnsupportedVersion = errors.New("unsupported version")
	ErrUnknownKey         = errors.New("unknown key id")
)

Functions

func BindCrypterTo 

func BindCrypterTo[F EncryptedValueFactory[T], T any](c Crypter)

BindCrypterTo binds a crypter instance to a specific EncryptedValue type. Example usage: 

BindCrypterTo[silent.EncryptedValue](&crypter)

Types

type Crypter 

type Crypter interface {
	Encrypt(data []byte) ([]byte, error)
	Decrypt(data []byte) ([]byte, error)
}

Crypter is an interface that can be implemented to provide a custom encryption strategy

type EncryptedValue 

type EncryptedValue = EncryptedValueFactory[dummy]

EncryptedValue is a built-in type that is automatically encrypted when written to, and decrypted when read from, the database.

type EncryptedValueFactory 

type EncryptedValueFactory[T any] []byte

EncryptedValueFactory is a generic type factory for creating custom EncryptedValue types. To define a new EncryptedValue type, create a unique dummy type and use it as the generic parameter: 

type dummy1 struct{} // this won't be used in your code
type MyEncryptedValue = EncryptedValueFactory[dummy1]

func (EncryptedValueFactory[T]) MarshalJSON 

func (v EncryptedValueFactory[T]) MarshalJSON() ([]byte, error)

MarshalJSON encrypts the value and marshals it into JSON format.

  • If the value is empty, it is marshalled as a JSON representation of an empty string ("").
  • If the encrypted data forms a valid UTF-8 string, it is marshaled as a string prefixed with '#'.
  • Otherwise, the data is marshaled as a base64-encoded string.

func (*EncryptedValueFactory[T]) Scan 

func (v *EncryptedValueFactory[T]) Scan(value interface{}) error

Scan is a sql.Scanner implementation. It decrypts the value from the database.

func (EncryptedValueFactory[T]) String 

func (v EncryptedValueFactory[T]) String() string

String returns a string representation of the EncryptedValue

func (*EncryptedValueFactory[T]) UnmarshalJSON 

func (v *EncryptedValueFactory[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON decrypts the value from JSON.

func (EncryptedValueFactory[T]) Value 

func (v EncryptedValueFactory[T]) Value() (driver.Value, error)

Value is a driver.Valuer implementation. It encrypts the value and returns a byte slice suitable for database storage.

type MultiKeyCrypter 

type MultiKeyCrypter struct {

	// Bypass be set to true to bypass the encryption and keep the values human-readable.
	// In bypass mode, the data is prefixed with a '#' character.
	Bypass bool
	// contains filtered or unexported fields
}

MultiKeyCrypter is a Crypter implementation that supports multiple encryption keys and seamless key rotation. It uses the last added key for encryption and automatically selects the appropriate key for decryption based on the key ID embedded in the encrypted data. This design simplifies adding new keys, while maintaining compatibility with previously used keys.

func (*MultiKeyCrypter) AddKey 

func (s *MultiKeyCrypter) AddKey(keyID uint32, key []byte)

AddKey adds a new key to the crypter. The keyID must be unique and the key must be at least 32 bytes long.

func (*MultiKeyCrypter) Decrypt 

func (s *MultiKeyCrypter) Decrypt(data []byte) ([]byte, error)

Decrypt decrypts the data. The key is automatically selected based on the key ID embedded in the data.

func (*MultiKeyCrypter) DecryptReader 

func (s *MultiKeyCrypter) DecryptReader(r io.Reader) (io.Reader, error)

DecryptReader is a streaming version of [Decrypt].

func (*MultiKeyCrypter) Encrypt 

func (s *MultiKeyCrypter) Encrypt(data []byte) ([]byte, error)

Encrypt encrypts the data using the last added key. Encrypted data will contain the key ID and the encrypted data.

func (*MultiKeyCrypter) EncryptWriter 

func (s *MultiKeyCrypter) EncryptWriter(w io.Writer) (io.WriteCloser, error)

EncryptWriter is a streaming version of [Encrypt].

func (*MultiKeyCrypter) EncryptedSize 

func (s *MultiKeyCrypter) EncryptedSize(dataSize int) (int, error)

EncryptedSize returns the size of the encrypted data.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.