ecc

Documentation

Overview

Package ecc implements common support for error correction in sharded blob providers

Index

• Constants
• func CreateAlgorithm(opts *Options) (encryption.Encryptor, error)
• func CreateEncryptor(p Parameters) (encryption.Encryptor, error)
• func RegisterAlgorithm(name string, createFunc CreateECCFunc)
• func SupportedAlgorithms() []string
• type CreateECCFunc
• type Options
• type Parameters
• type ReedSolomonCrcECC
  • func (r *ReedSolomonCrcECC) Decrypt(input gather.Bytes, _ []byte, output *gather.WriteBuffer) error
  • func (r *ReedSolomonCrcECC) Encrypt(input gather.Bytes, _ []byte, output *gather.WriteBuffer) error
  • func (r *ReedSolomonCrcECC) Overhead() int

Constants

const (
	// AlgorithmReedSolomonWithCrc32 is the name of an implemented algorithm.
	AlgorithmReedSolomonWithCrc32 = "REED-SOLOMON-CRC32"
)

const DefaultAlgorithm = AlgorithmReedSolomonWithCrc32

DefaultAlgorithm is the name of the default ecc algorithm.

Functions

func CreateAlgorithm

func CreateAlgorithm(opts *Options) (encryption.Encryptor, error)

CreateAlgorithm returns new encryption.Encryptor with error correction.

func CreateEncryptor

func CreateEncryptor(p Parameters) (encryption.Encryptor, error)

CreateEncryptor returns new encryption.Encryptor with error correction.

func RegisterAlgorithm

func RegisterAlgorithm(name string, createFunc CreateECCFunc)

RegisterAlgorithm registers new ecc algorithm.

func SupportedAlgorithms

func SupportedAlgorithms() []string

SupportedAlgorithms returns the names of the supported ecc methods.

Types

type CreateECCFunc

type CreateECCFunc func(opts *Options) (encryption.Encryptor, error)

CreateECCFunc creates an ECC for given parameters.

type Options

type Options struct {
	// Algorithm name to be used. Leave empty to disable error correction.
	Algorithm string `json:"algorithm,omitempty"`

	// OverheadPercent is how much more space can be used for ECC, in percentage.
	// Between 0 and 100. 0 means disable ECC.
	OverheadPercent int `json:"overheadPercent,omitempty"`

	// MaxShardSize represents the max shard size before splitting in blocks.
	// Use 0 to compute based on file size.
	MaxShardSize int `json:"maxShardSize,omitempty"`

	// Only set to true during benchmark tests
	DeleteFirstShardForTests bool
}

Options represents the configuration for all ECC algorithms.

type Parameters

type Parameters interface {
	GetECCAlgorithm() string
	GetECCOverheadPercent() int
}

Parameters encapsulates all ECC parameters.

type ReedSolomonCrcECC

type ReedSolomonCrcECC struct {
	Options
	DataShards            int
	ParityShards          int
	ThresholdParityInput  int
	ThresholdParityOutput int
	ThresholdBlocksInput  int
	ThresholdBlocksOutput int
	// contains filtered or unexported fields
}

ReedSolomonCrcECC implements Reed-Solomon error codes with CRC32 error detection.

func (*ReedSolomonCrcECC) Decrypt

func (r *ReedSolomonCrcECC) Decrypt(input gather.Bytes, _ []byte, output *gather.WriteBuffer) error

Decrypt corrects the data from input based on the ECC data. See Encrypt comments for a description of the layout.

func (*ReedSolomonCrcECC) Encrypt

func (r *ReedSolomonCrcECC) Encrypt(input gather.Bytes, _ []byte, output *gather.WriteBuffer) error

Encrypt creates ECC for the bytes in input. The bytes in output are stored in with the layout: ([CRC32][Parity shard])+ ([CRC32][Data shard])+ With one detail: the length of the original data is prepended to the data itself, so that we can know it when storing also the padding. All shards must be of the same size, so it may be needed to pad the input data. The parity data comes first so we can avoid storing the padding needed for the data shards, and instead compute the padded size based on the input length. All parity shards are always stored.

func (*ReedSolomonCrcECC) Overhead

func (r *ReedSolomonCrcECC) Overhead() int

Overhead should not be called. It's just implemented because it is in the interface.