erasure

README

erasure

A simple erasure code API.

Erasure provides a simple way of taking data and cutting it into shards in such a way that only a subset of the shards are required to recover the original data.

Documentation

Overview

Package erasure provides a common API for various erasure codes. It adds optional checksumming.

Index

• type Code
• type Reader
  • func NewReader(c Code, readers ...io.Reader) *Reader
  • func (r *Reader) Read(p []byte) (int, error)
• type Writer
  • func NewWriter(c Code) *Writer
  • func (w *Writer) Close() error
  • func (w *Writer) Readers() []io.Reader
  • func (w *Writer) Write(p []byte) (int, error)

Types

type Code

type Code interface {
	// Encode accepts a byte slice and returns a slice of sharded data.
	Encode([]byte) ([][]byte, error)

	// Decode accepts a slice of (possibly nil) shards and returns the original
	// byte slice.
	Decode([][]byte) ([]byte, error)

	// Shards returns the total number of data + parity shards employed by this
	// erasure code.
	Shards() int
}

Code is a simple abstraction on top of a specific erasure code implementation.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader reconstructs a data stream.

func NewReader

func NewReader(c Code, readers ...io.Reader) *Reader

NewReader creates a new reader. The Code passed should be identical to the Code that created the shards, and each reader should point to one of those shards. Readers can be given in any order.

func (*Reader) Read

func (r *Reader) Read(p []byte) (int, error)

type Writer

type Writer struct {
	// ChunkSize is the number of bytes that will be used for each parity
	// calculation.  If this is less than 1, 1e7 (~10MB) is used.
	ChunkSize int
	// contains filtered or unexported fields
}

Writer implements the io.Writer interface.

func NewWriter

func NewWriter(c Code) *Writer

NewWriter creates a Writer implementing the given erasure code.

func (*Writer) Close

func (w *Writer) Close() error

func (*Writer) Readers

func (w *Writer) Readers() []io.Reader

Readers returns Code.Shards() readers. Each reader corresponds to one shard from the erasure code. Readers must be read concurrently with writes.

func (*Writer) Write

func (w *Writer) Write(p []byte) (int, error)