bits

Documentation

Index

• Constants
• Variables
• func Get(kr KeyReader, cw ChunkWriter, conf Config) error
• func Move(kr KeyReader, kw KeyWriter, conf Config) error
• func Put(cr ChunkReader, kw KeyWriter, conf Config) error
• type ChunkReader
• type ChunkWriter
• type Config
  • func DefaultConf(secret Secret) (conf Config, err error)
• type K
  • func DecodeKey(b []byte) (k K, err error)
  • func (k K) String() string
• type KeyHash
• type KeyIndex
• type KeyReader
• type KeyWriter
• type RemoteStore
• type Secret
  • func DecodeSecret(b []byte) (s Secret, err error)
  • func GenerateSecret() (s Secret, err error)
  • func (s Secret) Encode() (b []byte)
  • func (s Secret) Pol() (p chunker.Pol)
  • func (s Secret) String() string
• type Store
• type StoreMap
  • func (sm StoreMap) GetSrcs() (stores []Store)
  • func (sm StoreMap) MoveDst() (store Store, err error)
  • func (sm StoreMap) MoveSrc() (store Store, err error)
  • func (sm StoreMap) PutDst() (store Store, err error)

Constants

const (
	//KeySize describes the byte size of each chunk key
	KeySize = 32
)

const (
	//SecretSize describes the byte size of the AES-GCM shared secret
	SecretSize = 32
)

Variables

var (
	//ErrNoSuchKey is returned when a given key could not be found
	ErrNoSuchKey = errors.New("no such key")
)

var ZeroSecret = Secret{}

ZeroSecret is secret with only 0x00 bytes, it can be used to check if a secret is empty

Functions

func Get

func Get(kr KeyReader, cw ChunkWriter, conf Config) error

Get will read and decrypt chunks for keys provided by the key reader and write each chunk's contents to chunk writer 'cw' in order of key appearance. Chunks are fetched concurrently (locally or remote) but are guaranteed to arrive in order to writer 'cw' for assembly in the original format

func Move

func Move(kr KeyReader, kw KeyWriter, conf Config) error

Move will attempt to move all keys read from key reader 'kr' to the (remote)store configured in Config 'conf' and outputs pushed keys to key writer 'kw'.

func Put

func Put(cr ChunkReader, kw KeyWriter, conf Config) error

Put reads chunks from a chunk reader and stores each chunk encrypted under a content-based key 'k' in the configured store. Compute intensive operations are run concurrently but keys are guaranteed to arrive at 'kw' in order, i.e: key of the first chunk will be writtern first

Types

type ChunkReader

type ChunkReader interface {
	Read() ([]byte, error)
}

ChunkReader allows reading one piece of input at a time

type ChunkWriter

type ChunkWriter interface {
	io.Writer
}

ChunkWriter accepts chunks for writing

type Config

type Config struct {
	AEAD    cipher.AEAD
	KeyHash KeyHash

	PutConcurrency  int
	MoveConcurrency int
	GetConcurrency  int

	Stores StoreMap

	Index KeyIndex
}

Config describes how the library's Split, Join and Push behaves

func DefaultConf

func DefaultConf(secret Secret) (conf Config, err error)

DefaultConf sets up sensible configs

type K

type K [32]byte

K is the key of a single chunk it is both used to store each piece as well as to encrypt it

func DecodeKey

func DecodeKey(b []byte) (k K, err error)

DecodeKey attempts to read a key from a byteslice

func (K) String

func (k K) String() string

Returns the key encoded such that it is human readable, with only ASCII characters but is yet space efficient.

type KeyHash

type KeyHash func([]byte) K

KeyHash turns a arbitrary sized chunk into content-based key

type KeyIndex

type KeyIndex interface {
	KeyWriter
	Has(k K) bool
}

KeyIndex holds information about just the chunk keys

type KeyReader

type KeyReader interface {
	Reset()
	Read() (K, error)
}

KeyReader will return keys while calling

type KeyWriter

type KeyWriter interface {
	Write(k K) error
}

KeyWriter is called when a key is outputted

type RemoteStore

type RemoteStore interface {
	Index(KeyWriter) error
	Store
}

RemoteStore stores chunks at a distant location such that an indexing mechanism is economic to prevent movement of chunks that are already present

type Secret

type Secret [SecretSize]byte

Secret is the 32 byte key that scopes the deduplication and facilitates end-to-end encryption. The first 8 bytes are always a polynomial that can be used for CBC the rest should always be randomly generated bytes

func DecodeSecret

func DecodeSecret(b []byte) (s Secret, err error)

DecodeSecret attempts to decode a secret from a byte slice

func GenerateSecret

func GenerateSecret() (s Secret, err error)

GenerateSecret will create a new secret with a random (irreducable) polynomial and random bytes to fill up the space.

func (Secret) Encode

func (s Secret) Encode() (b []byte)

Encode a secret into a byte slice

func (Secret) Pol

func (s Secret) Pol() (p chunker.Pol)

Pol returns the first 8 bytes of the secret as a polynomial

func (Secret) String

func (s Secret) String() string

String implements the Stringer interface

type Store

type Store interface {

	//will do nothing if exists, must be atomic
	Put(k K, chunk []byte) error

	//returns os.NotExist if the chunk doesnt exist
	Get(k K) (chunk []byte, err error)
}

Store holds chunks and is expected to have such a low latency that checking existence before put call is economic per key

type StoreMap

type StoreMap map[string]Store

StoreMap hols our store configurations and allows retrieval of stores for various purposes thoughout our code

func (StoreMap) GetSrcs

func (sm StoreMap) GetSrcs() (stores []Store)

GetSrcs returns an ordered list of stores for getting chunks for the current store configuration. this can be a default or explictely overwitten by the user @TODO store by 'localness' or 'likelyhood of having chunks'

func (StoreMap) MoveDst

func (sm StoreMap) MoveDst() (store Store, err error)

MoveDst returns a store to which chunks will be moved for the current store configration. this can be the default or overwitten by user preference

func (StoreMap) MoveSrc

func (sm StoreMap) MoveSrc() (store Store, err error)

MoveSrc returns a store from which chunks will be moved from in the current store configuration. this can either be the default or the store overwritten by user configuration

func (StoreMap) PutDst

func (sm StoreMap) PutDst() (store Store, err error)

PutDst returns a store that can be used for putting chunks in the current store configuration, this can either be the default or overwitten by user configuration