chestnut

package module
v0.0.0-...-015e77e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2022 License: MIT Imports: 12 Imported by: 0

README

🌰  Chestnut (fork)

Vibe Check Go Report Card GitHub go.mod Go version GitHub

Support the original author:

Buy OP A Coffee

Chestnut is encrypted storage for Go. The goal was an easy to use encrypted store with helpful features that was quick to set up, but highly flexible.

Chestnut is written in pure go and designed not to have strong opinions about things like storage, compression, hashing, secrets, or encryption. Chestnut is a storage chest, and not a datastore itself. As such, Chestnut must be backed by a storage solution.

Currently, Chestnut supports BBolt and NutsDB as backing storage.

This fork adds support for Bitcask wrapped in a provided layer of abstraction.

Table of Contents

Getting Started

Installing

To start using Chestnut, install Go (version 1.11+) and run go get:

$ go get -u github.com/yunginnanet/chestnut
Importing Chestnut

To use Chestnut as an encrypted store, import as:

import (
  "github.com/yunginnanet/chestnut"
  "github.com/yunginnanet/chestnut/encryptor/aes"
  "github.com/yunginnanet/chestnut/encryptor/crypto"
  "github.com/yunginnanet/chestnut/storage/nuts"
)

// use nutsdb for storage
store := nuts.NewStore(path)

// use AES256-CFB for encryption
opt := chestnut.WithAES(crypto.Key256, aes.CFB, mySecret)

cn := chestnut.NewChestnut(store, opt)
if err := cn.Open(); err != nil {
    return err
}

defer cn.Close()
Requirements

Chestnut has two requirements:

  1. Storage that supports the storage.Storage interface (with a lightweight adapter).
  2. Encryption which supports the crypto.Encryptor interface.

Storage

Chestnut will work seamlessly with any storage solution (or adapter) that supports thestorage.Storage interface.

Built-in

Currently, Chestnut has built-in support for BBolt and NutsDB.

BBolt

https://github.com/etcd-io/bbolt Chestnut has built-in support for using BBolt as a backing store.

To use bbolt for a backing store you can import Chestnut's bolt package and call bolt.NewStore():

import "github.com/yunginnanet/chestnut/storage/bolt"

//use or create a bbolt backing store at path
store := bolt.NewStore(path)

// use bbolt for the storage chest
cn := chestnut.NewChestnut(store, ...)
NutsDB

https://github.com/xujiajun/nutsdb
Chestnut has built-in support for using NutsDB as a backing store.

To use nutsDB for a backing store you can import Chestnut's nuts package and call nuts.NewStore():

import "github.com/yunginnanet/chestnut/storage/nuts"

//use or create a nutsdb backing store at path
store := nuts.NewStore(path)

// use nutsdb for the storage chest
cn := chestnut.NewChestnut(store, ...)
Planned

Other K/V stores like LevelDB.

GORM (probably not) Gorm is an ORM, so while it's not a datastore per se, it could be adapted to support sparse encryption and would mean automatic support for databases like mysql, sqlite, etc. However, most (if not all) of those DBs already support built-in encryption, so w/o some compelling use-case that's not already covered I don't see a lot of value-add.

Encryption

Chestnut supports several flavors of AES out of the box:

  • AES128-CFB, AES192-CFB, and AES256-CFB
  • AES128-CTR, AES192-CTR, and AES256-CTR
  • AES128-GCM, AES192-GCM, and AES256-GCM

You can add AES encryption to Chestnut by passing the chestnut.WithAES() option:

opt := chestnut.WithAES(crypto.Key256, aes.CFB, mySecret)
AES256-CTR

For encryption we recommend using AES256-CTR. We chose AES256-CTR based in part on this helpful analysis from Shawn Wang, PostgreSQL Database Core.

Custom Encryption

Chestnut supports drop-in custom encryption. A struct that supports the crypto.Encryptor interface can be used with the chestnut.WithEncryptor() option.

Supporting crypto.Encryptor interface is straightforward and mainly consists of vending the following two methods:

// Encrypt returns data encrypted with the secret.
Encrypt(plaintext []byte) (ciphertext []byte, err error)

// Decrypt returns data decrypted with the secret.
Decrypt(ciphertext []byte) (plaintext []byte, err error)
Chained Encryption

Chestnut also supports chained encryption which allows data to be arbitrarily transformed by a chain of Encryptors in a FIFO order.

A chain of crypto.Encryptors can be passed to Chestnut with the chestnut.WithEncryptorChain option:

opt := chestnut.WithEncryptorChain(
    encryptor.NewAESEncryptor(crypto.Key128, aes.CFB, secret1),
    encryptor.NewAESEncryptor(crypto.Key192, aes.CTR, secret2),
    encryptor.NewAESEncryptor(crypto.Key256, aes.GCM, secret3),
)

or by using a crypto.ChainEncryptor with the chestnut.WithEncryptor option:

encryptors := []crypto.Encryptor{
    encryptor.NewAESEncryptor(crypto.Key128, aes.CFB, secret1),
    encryptor.NewAESEncryptor(crypto.Key192, aes.CTR, secret2),
    encryptor.NewAESEncryptor(crypto.Key256, aes.GCM, secret3),
}
chain := crypto.NewChainEncryptor(encryptors...)
opt := chestnut.WithEncryptor(chain)

If you use both the chestnut.WithEncryptor and the chestnut.WithEncryptorChain options, the crypto.Encryptor from chestnut.WithEncryptor will be prepended* to the chain.

Sparse Encryption

Chestnut supports the sparse encryption of structs.

Sparse encryption is a transparent feature of saving structs with Chestnut.Save(), Chestnut.Load(), and Chestnut.Sparse(); or structs that support the value.Keyed interface with Chestnut.SaveKeyed(), Chestnut.LoadKeyed(), and Chestnut.SparseKeyed().

What is "sparse" encryption?

With sparse encryption, only struct fields marked as secure will be encrypted. The remaining "plaintext" fields are encoded and stored separately.

This allows you to load a "sparse" copy of the struct by calling Chestnut.Sparse() or Chestnut.SparseKeyed() (if you have a value.Keyed value) and examine the plaintext fields without the overhead of decryption. When a sparse struct is loaded, the contents of struct fields marked as secure are replaced by empty values.

Enabling Sparse Encryption

Chestnut uses struct tags to indicate which specific struct fields should be encrypted. To enable sparse encryption for a struct, add the secure tag option to the JSON tag of at least one struct field:

SecretKey string `json:",secure"` // 'secure' option (bare minimum)

like so:

type MySparseStruct struct {
    SecretValue string `json:"secret_value,secure"` // <-- add 'secure' here
    PublicValue string `json:"public_value"`
}
Using Sparse Encryption

Structs can be sparsely encrypted by calling Chestnut.Save(), or if the struct supports the value.Keyed interface, Chestnut.SaveKeyed(). Chestnut will automatically detect the secure tag and do the rest.

If no secure fields are found, Chestnut will encrypt the entire struct.

sparseObj := &MySparseStruct{
    SecretValue: "this is a secret",
    PublicValue: "this is public",
}

err := cn.Save("my-namespace",  []byte("my-key"), sparseObj)

When MySparseStruct is saved, Chestnut will detect the secure struct field and only encrypt those fields. Any remaining fields will be encoded as plaintext. In the case of MySparseStructthis means that SecretValuewill be encrypted prior to being encoded, and PublicValuewill not be encrypted.

Sparse Loading

A sparse struct can be loaded by calling Chestnut.Sparse(), or if the struct supports the value.Keyed interface, Chestnut.SparseKeyed(). When these methods are called to load a sparsely encrypted struct, a partially decoded struct will be returned, but the no decryption will occur. Secure fields will instead be replaced by empty values.

sparseObj := &MySparseStruct{}

err := cn.Sparse("my-namespace",  []byte("my-key"), sparseObj)

Examining the struct will reveal that the secure fields were replaced with empty values, and not decrypted.

*MySparseStruct{
    SecretValue: ""
    PublicValue: "this is public"
}

Only sparsely encrypted structs can be sparsely loaded
If Chestnut.Sparse() or Chestnut.SparseKeyed() is called on a struct that was not sparsely encrypted, the fully decrypted struct will be returned.

Decryption

A sparsely encrypted struct can be fully decrypted by calling Chestnut.Load(), or if the struct supports the value.Keyed interface, Chestnut.LoadKeyed(). When any of those methods are called on a sparsely encrypted struct, a fully decrpted copy of the struct is returned.

Secrets

Chestnut secrets are handled through the crypto.Secret interface. The crypto.Secret interface is designed to provide a high degree of flexibility around how you store, retrieve, and manage the secrets you use for encryption.

While Chestnut currently only comes with AES symmetric key encryption, the crypto.Secret interface can easily be adapted to support other forms of encryption like a private key-based crypto.Encryptor.

Chestnut currently provides three basic immplementations of the crypto.Secret interface which should cover most cases.

TextSecret

crypto.TextSecret provides a lightweight wrapper around a plaintext string.

textSecret := crypto.NewTextSecret("a-secret")
ManagedSecret

crypto.ManagedSecret provides a unique ID alongside a plaintext string secret. You can use this id to securely track the secret if you use external vaults or functionality like rollover.

managedSecret := crypto.NewManagedSecret("my-secret-id", "a-secret")
SecureSecret

crypto.SecureSecret provides a unique id for a secret alongside an openSecret() callback which returns a byte representation of the secret for encryption and decryption on SecureSecret.Open(). When crypto.SecureSecret calls openSecret() it will pass a copy of itself as a crypto.Secret. This allows for remote loading of the secret based on its id, or using a secure in-memory storage solution for the secret like memguarded.

openSecret := func(s crypto.Secret) []byte {
	// fetch the secret 
    mySecret := getMySecretFromTheVault(s.ID())
    return mySecret
}
secureSecret := crypto.NewSecureSecret("my-secret-id", openSecret)

Compression

Chestnut supports compression of the encoded data. Compression takes place prior to encryption.

Compression can be enabled through the chestnut.WithCompression option and passing it a supported compression format:

opt := chestnut.WithCompression(compress.Zstd)

Data compressed while chestnut.WithCompression is active with a supported compression format will continue to be correctly decompressed when read even if compression is no longer active (i.e. chestnut.WithCompression is no longer being used). This is not true with custom compression. Data compressed using custom compression cannot be decompressed if that custom compression is disabled.

Zstandard

Chestnut currently supports Zstandard compression out of the box with the compress.Zstd format option. To enable Zstandard compression, call chestnut.WithCompression passing compress.Zstd as the compression format:

opt := chestnut.WithCompression(compress.Zstd)

Please Note: I have no affiliation with Facebook (past or present) and just liked this compression format.

Custom Compression

If you wish to supply your own compression routines you can do so easily with the chestnut.WithCompressors option:

opt := chestnut.WithCompressors(myCompressorFn, myDecompressorFn)

Your two custom compression functions, a compressor compress.CompressorFunc, and a decompressor compress.DecompressorFunc must have the following format:

Compressor(data []byte) (compressed []byte, err error)

Decompressor(compressed []byte) (data []byte, err error)
Compression + Sparse Encryption

Enabling compression will not affect sparse encryption. Sparsely encrypted values compress their secure and plaintext encodings independently.

Operations

Chestnut supports all basic CRUD operations with a few extras.

All WRITE operations: Chestnut.Put(), Chestnut.Save(), & Chestnut.SaveKeyed(), will encrypt data prior to it being stored.

All READ operations: Chestnut.Get(), Chestnut.Load(), & Chestnut.LoadKeyed(), will decrypt data prior to it being returned.

All SPARSE operations: Chestnut.Sparse(), & Chestnut.SparseKeyed(), will not decrypt data prior to it being returned.

In all cases no record of the plaintext data is kept
(even with DebugLevel logging enabled).

Basic Operations
Put

To save an encrypted value to a namespaced key in the storage chest, use the Chestnut.Put() function:

err := cn.Put("my-namespace", []byte("my-key"), []byte("plaintext"))

This will set the value of the "my-key" key to the encrypted ciphertext of "plaintext" in the my-namespace namespace. If a namespace does not exist, it will be automatically created.

If the key already exists, and the storage chest was initialized with the chestnut.OverwritesForbidden option, this call will fail with ErrForbidden.

To retrieve this value, we can use the Chestnut.Get() function:

Get

To retrieve a decrypted value from a namespaced key in the storage chest, we can use the Chestnut.Get() function:

plaintext, err := cn.Get("my-namespace", []byte("my-key"))
Delete

Use the Chestnut.Delete() function to delete a key from the store.

err := cn.Delete("my-namespace", []byte("my-key"))
Struct Operations

Chestnut provides several functions for working directly with structs. In addition to handling the marshalling, encoding and encryption of structs for you, these functions provide automatic support for the Chestnut struct field tag options secure and hash. SEE: Struct Field Tags for more detail.

Save

To encrypt and save a struct to the store we can use the Chestnut.Save() function:

err := cn.Save("my-namespace", []byte("my-key"), myStruct)

Chestnut will marshal and encrypt the encoded byte representation. If the struct supports the secure struct field tag option on one of its fields, Chestnut will automatically sparsely encrypt the struct. Other supported struct field tag options will also be applied.

Load

To retrieve the fully decrypted struct, we can use the Chestnut.Load() function:

err := cn.Load("my-namespace", []byte("my-key"), &myStruct)
Sparse

Chestnut.Sparse() loads the struct at key and returns the sparsely decoded result. Unlike Chestnut.Load(), it does not decrypt the encoded struct and secure fields are replaced with empty values. To retrieve a sparse value, we can use the Chestnut.Sparse() function:

err := cn.Sparse("my-namespace", []byte("my-key"), &myStruct)

If the struct was not saved as a sparsely encoded struct this has no effect and is equivalent to calling Chestnut.Load(). Structs must have been saved with secure fields to be loaded as sparse structs by Chestnut.Sparse().

When a sparse struct is returned, any fields marked as secure will be decoded
as nil or empty values. For more information, please see the section on sparse encryption.

Keyed Operations

Chestnut provides several convenience functions for working with struct values that support the value.Keyed interface. Keyed values can supply their own namespace and keys via calls to Keyed.Namespace() and Keyed.Keys(), respectively. Internally these functions are equivalent to calling Chestnut.Save(), Chestnut.Load(), and Chestnut.Sparse() with an explicit namespace and key.

SaveKeyed

To encrypt and store a struct that implements the value.Keyed interface to the store we can use the Chestnut.SaveKeyed() function:

err := cn.SaveKeyed(myKeyedStruct)

To save a keyed struct with Chestnut.SaveKeyed(), the struct must be initialized with namespace and key you want to save it to prior to calling Chestnut.LoadKeyed() in order to satisfy the value.Keyed interface:

ko := MyKeyedValue{ name: "my-namespase", key: "my key"}
err := cn.SaveKeyed(&ko)

For more information, please see Chestnut.Save()

LoadKeyed

To retrieve the fully decrypted struct that implements the value.Keyed interface, we can use the Chestnut.LoadKeyed() function:

err := cn.LoadKeyed(&myKeyedStruct)

To load a keyed struct with Chestnut.LoadKeyed(), the struct must be initialized with namespace and key you want to retrieve prior to calling Chestnut.LoadKeyed() in order to satisfy the value.Keyed interface:

ko := MyKeyedValue{ name: "my-namespase", key: "my key"}
err := cn.LoadKeyed(&ko)

For more information, please see Chestnut.Load()

SparseKeyed

To retrieve a sparsely encrypted struct that implements the value.Keyed interface, we can use the Chestnut.SparseKeyed() function:

err := cn.SparseKeyed(&myKeyedStruct)

To load a sparse keyed struct with Chestnut.SparseKeyed(), the struct must be initialized with namespace and key you want to retrieve prior to calling Chestnut.SparseKeyed() in order to satisfy the value.Keyed interface:

ko := MyKeyedValue{ name: "my-namespase", key: "my key"}
err := cn.SparseKeyed(&ko)

For more information, please see Chestnut.Sparse()

Extra Operations

Chestnut supports a few additional functions that you might find helpful. In the future more may be added assuming they can be reasonably supported by the storage.Storage interface and generally make sense to do so. If there is a specific function you'd like to see added, please feel free to open an issue request to discuss.

Has

You can check to see if a key exists by calling Chestnut.Has(). If the key is found, it will return true, otherwise false. If an error occured, Chestnut.Has() will return false along with the error.

has, err := cn.Has("my-namespace", []byte("my-key"))
List

To get a list of all the keys for a namespace you can call Chestnut.List():

keys, err := cn.List("my-namespace")
ListAll

To get a mapped list of all keys in the store organized by namespace you can call Chestnut.ListAll():

keymap, err := cn.ListAll()
Export

To export the storage chest to another path you can call Chestnut.Export():

err := cn.Export("/a/path/someplace")

Chestnut cannot be exported to its current location. If you call Chestnut.Export() and pass the path to Chestnut's current location an error will be returned.

Struct Field Tags

Chestnut currently supports two extensions to the `json` struct field tag as options: secure when added to the `json` tag, secure marks the field for sparse encryption. hash when added to the `json` tag, hash marks a string field for hashing.

These options will be automatically detected and applied when the struct is saved with Chestnut.Save(), or Chestnut.SaveKeyed().

NOTE: The order in which the tag options appear is unimportant.

// these are equivalent

`json:"my_value,secure,omitempty"`

`json:"my_value,omitempty,secure"`

In the future, Chestnut will also support its own struct field tag. `cn`.

Secure

When the secure option is added to a `json` struct field tag, the struct field is marked for sparse encryption. If Chestnut detects a secure option on a struct field tag, only those fields marked with secure will be encrypted. If no secure fields are found, Chestnut will encrypt the entire struct.

To mark a struct field as secure, just add secure as an option to a `json` struct field tag (like omitempty). The following are some examples of how the secure option can be added to the `json` struct field tag:

type MySecureStruct struct {
    ValueA     int      `json:",secure"`           // *will* be encrypted
    ValueB     struct{} `json:"value_b,secure"`    // *will* be encrypted
    ValueC     string   `json:",omitempty,secure"` // *will* be encrypted
    PlaintextA string                              // will *not* be encrypted
    PlaintextB int      `json:""`                  // will *not* be encrypted
    PlaintextC int      `json:"-"`                 // will *not* be encrypted
    privateA   int      `json:",secure"`           // will *not* be encrypted
}

Fields marked with secure are encrypted hierarchically, meaning if you have:

package main

type MyStructA struct {
	ValueA string `json:"value_a,secure"`    // *will* be encrypted
}

type MyStructB struct {
	MyStructA                                // will *not* be encrypted
	ValueB    string `json:"value_b"`        // will *not* be encrypted
}

type MyStructC struct {
	MyStructA                                // will *not* be encrypted
	ValueC    string `json:"value_c"`        // will *not* be encrypted
}

type MyStructD struct {
	ValueD string    `json:"value_d,secure"` // *will* be encrypted
	Embed1 MyStructA                         // will *not* be encrypted
	Embed2 MyStructB                         // will *not* be encrypted
	Embed3 MyStructB `json:"embed_3,secure"` // *will* be encrypted
}

var myStruct = &MyStructD{
	ValueD: "foo",
	Embed1: MyStructA{
		ValueA: "bar",
	},
	Embed2: MyStructB{
		MyStructA: MyStructA{
			ValueA: "quack",
		},
		ValueB: "baz",
	},
	Embed3: MyStructB{
		MyStructA: MyStructA{
			ValueA: "foobar",
		},
		ValueB: "bonk",
	},
}

myStruct will be encrypted by Chestnut as:

*MyStructD {
  ValueD: ****
  Embed1: main.MyStructA{
      ValueA: ****
  },
  Embed2: main.MyStructB{
      MyStructA: main.MyStructA{
      	ValueA: ****
      },
      ValueB: ****
  },
  Embed3: ****
}

where '****' represents an encrypted value.

Please see Sparse Encryption for more information.

Hash

When the hash option is added to a `json` struct field tag of a string field, the string field is marked for hashing. If Chestnut detects a hash option on a string field, the string value of the field will be replaced with its hash.

If the hash option is applied to a struct field that is not type string, it is ignored.

To hash a string field of a struct, just add hash as an option to a `json` struct field tag (like omitempty). The following are some examples of how the hash option can be added to the `json` struct field tag:

type MyHashStruct struct {
    ValueA     string   `json:",hash"`           // *will* be hashed
    ValueB     string   `json:"value_b,hash"`    // *will* be hashed
    ValueC     string   `json:",omitempty,hash"` // *will* be hashed
    ValueD     string   `json:",hash,omitempty"` // *will* be hashed
    ...
    Count      int      `json:"count,hash"`      // will *not* be hashed
}

Taking the above struct as an example:

var myHashStruct = &MyHashStruct {
    ValueA: "value a",
    ValueB  "value b",
    ValueC  "value c",
    ValueD  "value d",
    ...
    Count   42,
}

myHashStruct will be encoded as:

*main.MyHashStruct {
    ValueA: "sha256:BEBA1D9847D6E595D8DD6832DEE5432916C6F7AE438BC9A99C5BAFDD0E93793E"
    ValueB  "sha256:2A53D83488A34E898436908A7064276859FFF56F69D16E2F61573057EDEBFB64"
    ValueC  "sha256:80C92DE321A1CB8AEA3025890DE39A5BAA95A91DF022B10E1A71BEABB8BCC1BE"
    ValueD  "sha256:8080378350428FABDE1724D9B920D613B8920A71151F1A9AF37EB4AF43628AE4"
	...
    Count   42
}
SHA256

Chestnut currently supports SHA256 for hashing. In the future the hash option may be extending to include an algorithm name e.g.:

`json:"some_field,hash=sha3-256"`

in which which case hash would continue to default to sha256.

Hash Prefix

Hashed struct fields will have the algorithm used to hash the value pre-pended to the hash string:

sha256:BEBA1D9847D6E595D8DD6832DEE5432916C6F7AE438BC9A99C5BAFDD0E93793E

Chestnut uses the [hash alogorithm name]: prefix to know that it has already hashed the value, and it should hash it again when the struct is saved.

IMPORTANT! Changing or removing the hash prefix will cause Chestnut to rehash the value of the struct field the next time the struct is saved.

Multiple Tags

Chestnut supports the combining of tag options. You are free to mark a struct field as both secure and hash:

type MyCombinedStruct struct {
    ValueA     string   `json:"value_a,secure,hash"` // will be hashed *AND* encrypted 
    ...
}

As with other tag options, the order in which they appear is unimportant.

However, the order in which Chestnut applies them is fixed. A field marked with both secure and hash will be first be hashed, and then encrypted. This order of operations cannot be changed for obvious reasons.

Disable Overwrites

Chestnut supports the disabling of overwrites via the chestnut.OverwritesForbidden option.

cn := chestnut.NewChestnut(store, encryptor chestnut.OverwritesForbidden())

When this option is set, once a value has been saved to a namespaced key, successive calls to save a value to the same key will fail with ErrForbidden.

The key must be explicitly deleted before a new call to save a value for the same key will succeed.

Keystore

Chestnut includes an implementation of IPFS compliant keystore which can be found here.

Importing Keystore

Using the Keystore is straight forward:

package main

import (
	"github.com/yunginnanet/chestnut"
	"github.com/yunginnanet/chestnut/encryptor/aes"
	"github.com/yunginnanet/chestnut/encryptor/crypto"
	"github.com/yunginnanet/chestnut/keystore"
	"github.com/yunginnanet/chestnut/storage/nuts"
)

// use nutsdb
store := nuts.NewStore(path)

// use a simple text secret
textSecret := crypto.TextSecret("i-am-a-good-secret")

// use AES256-CFB encryption
opt := chestnut.WithAES(crypto.Key256, aes.CFB, textSecret)

// open the keystore with nutsdb and the aes encryptor
ks := keystore.NewKeystore(store, opt)
if err := ks.Open(); err != nil {
    return err
}

A complete example of the Chestnut Keystore can be found here.

Important Note
package main

import  (
    "github.com/ipfs/go-ipfs/keystore"
    "github.com/libp2p/go-libp2p-core/crypto"
)

If you want to work with the Keystore, please make make sure you are importing go-ipfs and go-libp2p-core, and NOT importing go-ipfs-keystore and go-libp2p-crypto — which are DEPRECATED, out of date, a/o archived, etc.

Logging

Chestnut supports logging via the log.Logger interface and the chestnut.WithLogger() option. The log.Logger interface conforms to Logrus, Zap, and the standard Go logger (with the log.Std adapter), for example:

opt := chestnut.WithLogger(myLogger)
Logrus Logger

Chestnut supports Logrus logger.

*logrus.Logger and *logrus.Entry both work with Chestnut's log.Logger interface:

logger := logrus.New() // *logrus.Logger

opt := chestnut.WithLogger(logger)

or

logger := logrus.New()
logger = logger.WithField("hello", "world") // *logrus.Entry

opt := chestnut.WithLogger(logger)

In addition to the chestnut.WithLogger() option, you can use the convenience option, chestnut.WithLogrusLogger():

opt := chestnut.WithLogrusLogger(log.InfoLevel)

chestnut.WithLogrusLogger() will return a new *logrus.Entry set to the normalized log level you requested. This is equivalent to calling logrus.New() followed by logrus.SetLevel().

Zap Logger

Chestnut supports Zap logger.

*zap.SugaredLogger works with Chestnut's log.Logger interface:

logger :=zap.NewProduction().Sugar() // *zap.SugaredLogger

opt := chestnut.WithLogger(logger)

In addition to the chestnut.WithLogger() option, you can use the convenience option, chestnut.WithZapLogger():

opt := chestnut.WithZapLogger(log.InfoLevel)

chestnut.WithZapLogger() will return a new *zap.SugaredLogger set to the normalized log level you requested. This is equivalent to calling zap.NewProduction(), followed by zap.Core().Enabled(), and finally, zap.Sugar().

Standard Logger

Chestnut supports the Go standard logger.

We provide a lightweight wrapper for Go's standard logger *log.Logger which supports Chestnut's log.Logger interface.

import (
    "log"
    "os"

    "github.com/yunginnanet/chestnut"
    cnlog "github.com/yunginnanet/chestnut/log"
)

logger := cnlog.NewStdLogger(log.InfoLevel os.Stderr, "", log.LstdFlags)
opt := chestnut.WithLogger(logger)

chestnut.WithStdLogger() will return a new *log.stdLogger set to the normalized log level you requested. This is equivalent to calling log.NewStdLogger() with the specified log level.

Storage

Lastly, the Chestnut stores also accept matching options for logging: storage.WithLogger, storage.WithLogrusLogger, storage.WithZapLogger, and storage.WithStdLogger.

// enable logging 
opt := storage.WithLogger(myLogger)
// use nutsdb
store := nuts.NewStore(path, opt)

Enabling logging for the backing store was intentionally kept separate for additional flexibility. This allows you to log Chestnut operations without automatically incurring the noise of store operations and vice versa.

Examples

Run any example with make <example-dir>

$ make sparse

Known Issues

  • Because we use JSON encoding for structs, time.Time will lose resolution when encoded:

    IN:  2021-01-09 17:29:36.349522 -0800 PST m=+0.002746093
OUT: 2021-01-09 17:29:36.349522 -0800 PST

Misc

JSON encoding

We use the jsoniter JSON encoder internally.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var DefaultChestOptions = ChestOptions{
	// contains filtered or unexported fields
}

DefaultChestOptions represents the recommended default ChestOptions for a store.

View Source 
var ErrForbidden = errors.New("forbidden")

ErrForbidden the storage operation is forbidden

Functions

This section is empty.

Types

type ChestOption 

type ChestOption interface {
	// contains filtered or unexported methods
}

A ChestOption sets options such as encryptors, key rolling, and other parameters, etc.

func OverwritesForbidden 

func OverwritesForbidden() ChestOption

OverwritesForbidden prevents the store from overwriting existing data.

func WithAES 

func WithAES(keyLen crypto.KeyLen, mode crypto.Mode, secret crypto.Secret) ChestOption

WithAES is a convenience that returns a ChestOption which sets the encryptor to be an AESEncryptor initialized with a key length, cipher mode, and Secret.

func WithCompression 

func WithCompression(format compress.Format) ChestOption

WithCompression instructs the storage chest to compress data using the this compression format before committing it. Compression this way is self-contained, meaning changes only effect data going forward. Previously saved data, compressed or uncompressed, will be transparently retrieved regardless of a change to this setting.

func WithCompressors 

func WithCompressors(c compress.CompressorFunc, d compress.DecompressorFunc) ChestOption

WithCompressors instructs the storage chest to compress/decompress data with these compressor functions before committing it. If this option is set, WithCompression is ignored.

func WithEncryptor 

func WithEncryptor(e crypto.Encryptor) ChestOption

WithEncryptor returns a ChestOption that specifies the encryptor to use.

func WithEncryptorChain 

func WithEncryptorChain(encryptors ...crypto.Encryptor) ChestOption

WithEncryptorChain returns a ChestOption that specifies an encryptor chain. for encrypted stores. The first encryptor will be the outer most, while the last encryptor will be the inner most wrapper around the real call. All encryptors added by this method will be chained. If a single encryptor has also been set, it will be *prepended* to the encryptor chain, making it the outer most encryptor in the encryptor chain.

func WithLogger 

func WithLogger(l log.Logger) ChestOption

WithLogger returns a StoreOption which sets the logger to use for the encrypted store.

func WithStdLogger 

func WithStdLogger(lvl log.Level) ChestOption

WithStdLogger is a convenience that returns a StoreOption for a standard err logger.

func WithZerologLogger 

func WithZerologLogger(lvl log.Level) ChestOption

WithZerologLogger is a convenience that returns a StoreOption for a default Zerolog logger.

type ChestOptions 

type ChestOptions struct {
	// contains filtered or unexported fields
}

ChestOptions provides a default implementation for common options for a secure store.

type Chestnut 

type Chestnut struct {
	// contains filtered or unexported fields
}

Chestnut is used to manage an encrypted store. It provides additional features such as chained encryption, independently secured secrets, sparse encryption, and hashing. For more detail, SEE: https://git.tcp.direct/kayos/chestnut-bitcask/blob/master/README.md

func NewChestnut 

func NewChestnut(store storage.Storage, opt ...ChestOption) *Chestnut

NewChestnut is used to create a new chestnut encrypted store.

func (*Chestnut) CanPut 

func (cn *Chestnut) CanPut(name string, key []byte) error

CanPut returns nil if writing to key is ok. If overwrites are disabled and the key exists, ErrForbidden is returned.

func (*Chestnut) Close 

func (cn *Chestnut) Close() error

Close the storage chest

func (*Chestnut) Delete 

func (cn *Chestnut) Delete(name string, key []byte) error

Delete removes a key from the storage chest.

func (*Chestnut) Export 

func (cn *Chestnut) Export(path string) error

Export saves a copy of the storage chest to directory at path.

func (*Chestnut) Get 

func (cn *Chestnut) Get(name string, key []byte) ([]byte, error)

Get decrypts the ciphertext at key and returns the plaintext.

func (*Chestnut) Has 

func (cn *Chestnut) Has(name string, key []byte) (bool, error)

Has checks for a key in the storage chest. Has returns true if the key is found, otherwise false.

func (*Chestnut) List 

func (cn *Chestnut) List(namespace string) ([][]byte, error)

List returns a list of keys in the namespace.

func (*Chestnut) Load 

func (cn *Chestnut) Load(name string, key []byte, v interface{}) error

Load decrypts the struct at key and returns the decoded result in v.

func (*Chestnut) LoadKeyed 

func (cn *Chestnut) LoadKeyed(v value.Keyed) error

LoadKeyed decrypts the keyed value and returns the result.

func (*Chestnut) Logger 

func (cn *Chestnut) Logger() log.Logger

Logger gets a copy of the logger from the storage chest's options

func (*Chestnut) Open 

func (cn *Chestnut) Open() error

Open the storage chest

func (*Chestnut) Put 

func (cn *Chestnut) Put(name string, key []byte, plaintext []byte) error

Put encrypts the plaintext and stores it at key.

func (*Chestnut) Save 

func (cn *Chestnut) Save(name string, key []byte, v interface{}) error

Save encrypts the struct in v and stores the encoded result at key.

func (*Chestnut) SaveKeyed 

func (cn *Chestnut) SaveKeyed(v value.Keyed) error

SaveKeyed encrypts the keyed value and stores the result.

func (*Chestnut) SetLogger 

func (cn *Chestnut) SetLogger(l log.Logger)

SetLogger sets the logger for the storage chest

func (*Chestnut) Sparse 

func (cn *Chestnut) Sparse(name string, key []byte, v interface{}) error

Sparse loads the struct at key and returns the sparsely decoded result in v. Unlike Load, it does not decrypt the encoded struct and secure fields are replaced with empty values. If the struct was not saved as a sparsely encoded struct this has no effect and is equivalent to calling Load. Structs must have been saved with secure fields to be loaded as sparse structs by Sparse.

func (*Chestnut) SparseKeyed 

func (cn *Chestnut) SparseKeyed(v value.Keyed) error

SparseKeyed sparsely loads the keyed value and returns the result. Unlike LoadKeyed, it does not decrypt the keyed value. SEE: Sparse.

type EmptyChestOption 

type EmptyChestOption struct{}

EmptyChestOption does not alter the encrypted store's configuration. It can be embedded in another structure to build custom options.

Source Files

View all Source files

Directories

Path Synopsis
encoding
compress
compress/zstd
json
json/encoders
json/encoders/hash
json/encoders/lookup
json/encoders/secure
json/packager
tags
aes
crypto
examples
hash
keystore
sparse
bitcask
bolt
nuts
store_test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.