ekv

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 6, 2023 License: BSD-2-Clause Imports: 17 Imported by: 11

README

Encrypted KV Store

pipeline status coverage report

EKV is a directory and file-based encrypted key value storage library with metadata protection written in golang. It is intended for use in mobile and desktop applications where one may want to transfer protected files to a new device while protecting the nature of the information of what is stored in addition to the contents.

Features:

  1. Both the key and the contents behind the key are protected on disk.
  2. A best-effort approach is used to store and flush changes to disk.
  3. Thread-safety within the same EKV instance.

EKV is not a secured memory enclave. Data is protected when stored to disk, not RAM. Please see Memguard and similar projects if that is what you need.

EKV requires a cryptographically secure random number generator. We recommend Elixxir's fastRNG.

EKV is released under the simplified BSD License.

Known Limitations and Roadmap

EKV has several known limitations at this time:

  1. The code is currently in beta and has not been audited.
  2. The password to open and close the store is a string that can be dumped from memory. We would like to improve that by storing the password in a secured memory enclave (e.g., Memguard).
  3. EKV protects keys and contents, it doesn't protect the size of those files or the number of unique keys being stored in the database. We would like to include controls for EKV users to hide that information by setting a block size for files and adding a number of fake files to the directory.
  4. Users are currently limited to the number of files the operating system can support in a single directory.
  5. The underlying file system must support hex encoded 256 bit file names.

General Usage

EKV implements the following interface:

type KeyValue interface {
	Set(key string, objectToStore Marshaler) error
	Get(key string, loadIntoThisObject Unmarshaler) error
	Delete(key string) error
	SetInterface(key string, objectToSTore interface{}) error
	GetInterface(key string, v interface{}) error
}

EKV works with any object that implements the following functions:

  1. Marhsaler: func Marshal() []byte
  2. Unmarshaler: func Unmarshal ([]byte) error

For example, we can make a "MarshalableString" type:

// This is a simple marshalable object
type MarshalableString struct {
	S string
}

func (s *MarshalableString) Marshal() []byte {
	return []byte(s.S)
}

func (s *MarshalableString) Unmarshal(d []byte) error {
	s.S = string(d)
	return nil
}

To load and store to the EKV with this type:

import (
	...
	"crypto/rand"
	"gitlab.com/elixxir/ekv"
)

func main() {
	kvstore, err := ekv.NewFilestoreWithNonceGenerator("somedirectory",
		"Some Password", rand.Reader)
	if err != nil {
		// Print/handle could not create or open error ...
	}

	i := &MarshalableString{
		S: "TheValue",
	}
	err = f.Set("SomeKey", i)
	if err != nil {
		// Print/handle could not write error ...
	}

	s := &MarshalableString{}
	err = f.Get("SomeKey", s)
	if err != nil {
		// Print/handle could not read error
	}
	if s.S == "Hi" {
		// Always true
	}
}
Generic Interfaces (JSON Encoding)

You can also leverage the default JSON Marshalling using GetInterface and SetInterface as follows:

	err = f.SetInterface("SomeKey", i)
	if err != nil {
		// write error
	}

	s = &MarshalableString{}
	err = f.GetInterface("SomeKey", s)
	if err != nil {
		// read error
	}
	if s.S == "Hi" {
		// Always true
	}
Deleting Data

To delete, use Delete, which will also remove the file corresponding to the key:

	err = f.Delete("SomeKey")
	if err != nil {
		// Could not delete
	}
Detecting if a key exists:

To detect if a key exists you can use the Exists function on the error returned by Get and GetInterface:

	err = f.GetInterface("SomeKey", s)
	if !ekv.Exists(err) {
		// Does not exist...
	}

Cryptographic Primitives

All cryptographic code is located in crypto.go.

To create keys, EKV uses the construct:

  • H(H(password)||H(keyname))

The keyname is the name of the key and password is the password or passphrase used to generate the key. EKV uses the 256bit blake2b hash.

Code:

func hashStringWithPassword(data, password string) []byte {
	dHash := blake2b.Sum256([]byte(data))
	pHash := blake2b.Sum256([]byte(password))
	s := append(pHash[:], dHash[:]...)
	h := blake2b.Sum256(s)
	return h[:]
}

To encrypt files, EKV uses ChaCha20Poly1305 with a randomly generated nonce. The cryptographically secure pseudo-random number generator must be provided by the user:

func initChaCha20Poly1305(password string) cipher.AEAD {
	pwHash := blake2b.Sum256([]byte(password))
	chaCipher, err := chacha20poly1305.NewX(pwHash[:])
	if err != nil {
		panic(fmt.Sprintf("Could not init XChaCha20Poly1305 mode: %s",
			err.Error()))
	}
	return chaCipher
}

func encrypt(data []byte, password string, csprng io.Reader) []byte {
	chaCipher := initChaCha20Poly1305(password)
	nonce := make([]byte, chaCipher.NonceSize())
	if _, err := io.ReadFull(csprng, nonce); err != nil {
		panic(fmt.Sprintf("Could not generate nonce: %s", err.Error()))
	}
	ciphertext := chaCipher.Seal(nonce, nonce, data, nil)
	return ciphertext
}

func decrypt(data []byte, password string) ([]byte, error) {
	chaCipher := initChaCha20Poly1305(password)
	nonceLen := chaCipher.NonceSize()
	nonce, ciphertext := data[:nonceLen], data[nonceLen:]
	plaintext, err := chaCipher.Open(nil, nonce, ciphertext, nil)
	if err != nil {
		return nil, errors.Wrap(err, "Cannot decrypt with password!")
	}
	return plaintext, nil
}

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Exists 

func Exists(err error) bool

Exists determines if the error message is known to report the key does not exist. Returns true if the error does not specify or it is nil and false otherwise.

Types

type Extender added in v0.4.0 

type Extender interface {
	// Extend can be used to add more keys to the current transaction
	// if an error is returned, abort and return it
	Extend(keys []string) (map[string]Operable, error)
	// IsClosed returns true if the current transaction is in scope
	// will always be true if inside the execution of the transaction
	IsClosed() bool
}

type Filestore 

type Filestore struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Filestore implements an ekv by reading and writing to files in a directory.

func NewFilestore 

func NewFilestore(basedir, password string) (*Filestore, error)

NewFilestore returns an initialized filestore object or an error if it can't read and write to the directory/.ekv.1/2 file. Note that this file is not used other than to verify read/write capabilities on the directory.

func NewFilestoreWithNonceGenerator 

func NewFilestoreWithNonceGenerator(basedir, password string,
	csprng io.Reader) (*Filestore, error)

NewFilestoreWithNonceGenerator returns an initialized filestore object that uses a custom RNG for Nonce generation.

func (*Filestore) Close 

func (f *Filestore) Close()

Close is equivalent to nil'ing out the Filestore object. This function is in place for the future when we add secure memory storage for keys.

func (*Filestore) Delete 

func (f *Filestore) Delete(key string) error

Delete the value for the given key per [KeyValue.Delete]

func (*Filestore) Get 

func (f *Filestore) Get(key string, loadIntoThisObject Unmarshaler) error

Get the value for the given key per [KeyValue.Get]

func (*Filestore) GetBytes added in v0.2.2 

func (f *Filestore) GetBytes(key string) ([]byte, error)

GetBytes implements [KeyValue.GetBytes]

func (*Filestore) GetInterface 

func (f *Filestore) GetInterface(key string, v interface{}) error

GetInterface uses json to encode and get data per [KeyValue.GetInterface]

func (*Filestore) Set 

func (f *Filestore) Set(key string, objectToStore Marshaler) error

Set the value for the given key per [KeyValue.Set]

func (*Filestore) SetBytes added in v0.2.2 

func (f *Filestore) SetBytes(key string, data []byte) error

SetBytes implements [KeyValue.SetBytes]

func (*Filestore) SetInterface 

func (f *Filestore) SetInterface(key string, objectToStore interface{}) error

SetInterface uses json to encode and set data per [KeyValue.SetInterface]

func (*Filestore) SetNonceGenerator 

func (f *Filestore) SetNonceGenerator(csprng io.Reader)

SetNonceGenerator sets the cryptographically secure pseudo-random number generator (csprng) used during encryption to generate nonces.

func (*Filestore) Transaction added in v0.3.0 

func (f *Filestore) Transaction(op TransactionOperation, keys ...string) error

Transaction implements [KeyValue.Transaction]

type KeyValue 

type KeyValue interface {
	// Set stores using an object that can marshal itself.
	Set(key string, objectToStore Marshaler) error
	// Get loads into an object that can unmarshal itself.
	Get(key string, loadIntoThisObject Unmarshaler) error
	// Delete destroys a key.
	Delete(key string) error
	// SetInterface uses a JSON encoder to store an interface object.
	SetInterface(key string, objectToSTore interface{}) error
	// GetInterface uses a JSON decode to load an interface object.
	GetInterface(key string, v interface{}) error
	// SetBytes stores raw bytes.
	SetBytes(key string, data []byte) error
	// GetBytes loads raw bytes.
	GetBytes(key string) ([]byte, error)
	// Transaction locks a set of keys while they are being mutated and
	// allows the function to operate on them exclusively.
	// More keys can be added to the transaction, but they must only be operated
	// on in conjunction with the previously locked keys otherwise deadlocks can
	// occur
	// If the op returns an error, the operation will be aborted.
	Transaction(op TransactionOperation, keys ...string) error
}

KeyValue is the interface that ekv implements. Simple functions are provided for objects that can Marshal and Unmarshal themselves, and an interface version of these is provided which should use JSON or another generic object encoding system.

type Marshaler 

type Marshaler interface {
	Marshal() []byte
}

Marshaler interface defines objects which can "Marshal" themselves into a byte slice. This should produce a byte slice that can be used to fully reconstruct the object later.

type Memstore 

type Memstore struct {
	// contains filtered or unexported fields
}

Memstore is an unencrypted memory-based map that implements the KeyValue interface.

func MakeMemstore added in v0.1.7 

func MakeMemstore() *Memstore

MakeMemstore returns a new Memstore with a newly initialised a new map.

func (*Memstore) Delete 

func (m *Memstore) Delete(key string) error

Delete removes the value from the store per [KeyValue.Delete]

func (*Memstore) Get 

func (m *Memstore) Get(key string, loadIntoThisObject Unmarshaler) error

Get implements [KeyValue.Get]

func (*Memstore) GetBytes added in v0.2.2 

func (m *Memstore) GetBytes(key string) ([]byte, error)

SetBytes implements [KeyValue.GetBytes]

func (*Memstore) GetInterface 

func (m *Memstore) GetInterface(key string, objectToLoad interface{}) error

GetInterface gets the value using a JSON encoder per [KeyValue.GetInterface]

func (*Memstore) Set 

func (m *Memstore) Set(key string, objectToStore Marshaler) error

Set stores the value if there's no serialization error per [KeyValue.Set]

func (*Memstore) SetBytes added in v0.2.2 

func (m *Memstore) SetBytes(key string, data []byte) error

SetBytes implements [KeyValue.SetBytes]

func (*Memstore) SetInterface 

func (m *Memstore) SetInterface(key string, objectToStore interface{}) error

SetInterface sets the value using a JSON encoder per [KeyValue.SetInterface]

func (*Memstore) Transaction added in v0.3.0 

func (m *Memstore) Transaction(op TransactionOperation, keys ...string) error

Transaction implements [KeyValue.Transaction]

type Operable added in v0.4.0 

type Operable interface {
	// Key returns the key this interface is operating on
	Key() string
	// Exists returns if the file currently exists
	// will panic if the current transaction isn't in scope
	Exists() bool
	// Delete deletes the file at the key and destroy it.
	// will panic if the current transaction isn't in scope
	Delete()
	// Set stores raw bytes.
	// will panic if the current transaction isn't in scope
	Set(data []byte)
	// Get loads raw bytes.
	// will panic if the current transaction isn't in scope
	Get() ([]byte, bool)
	// Flush executes the operation and returns an error if the operation
	// failed. It will set the operable to closed as well.
	// if flush is not called, it will be called by the handler
	Flush() error
	// IsClosed returns true if the current transaction is in scope
	// will always be true if inside the execution of the transaction
	IsClosed() bool
}

Operable describes edits to a single key inside a transaction

type OperableOps added in v0.4.0 

type OperableOps uint8

type TransactionOperation added in v0.3.0 

type TransactionOperation func(files map[string]Operable, ext Extender) error

type Unmarshaler 

type Unmarshaler interface {
	Unmarshal([]byte) error
}

Unmarshaler interface defines objects which can be initialized by a byte slice. An error should be returned if the object cannot be decoded or, optionally, when Unmarshal is called against a pre-initialized object.

Source Files

View all Source files

Directories

Path Synopsis
Package portableOS contains global OS functions that can be overwritten to be used with other filesystems not supported by the os package, such as wasm.
 Package portableOS contains global OS functions that can be overwritten to be used with other filesystems not supported by the os package, such as wasm.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.