Documentation
¶
Overview ¶
Package compressio provides parallel compression and decompression, as well as optional SHA-256 hashing. It also provides another storage variant (nocompressio) that does not compress data but tracks its integrity.
The stream format is defined as follows.
/------------------------------------------------------\ | chunk size (4-bytes) | +------------------------------------------------------+ | (optional) hash (32-bytes) | +------------------------------------------------------+ | compressed data size (4-bytes) | +------------------------------------------------------+ | compressed data | +------------------------------------------------------+ | (optional) hash (32-bytes) | +------------------------------------------------------+ | compressed data size (4-bytes) | +------------------------------------------------------+ | ...... | \------------------------------------------------------/
where each subsequent hash is calculated from the following items in order
compressed data compressed data size previous hash
so the stream integrity cannot be compromised by switching and mixing compressed chunks.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrHashMismatch = errors.New("hash mismatch")
ErrHashMismatch is returned if the hash does not match.
Functions ¶
This section is empty.
Types ¶
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader is a compressed reader.
type SimpleReader ¶
type SimpleReader struct {
// contains filtered or unexported fields
}
SimpleReader is a reader from uncompressed image.
func NewSimpleReader ¶
func NewSimpleReader(in io.Reader, key []byte) (*SimpleReader, error)
NewSimpleReader returns a new (uncompressed) reader. If key is non-nil, the data stream is assumed to contain expected hash values. See package comments for details.
func (*SimpleReader) Read ¶
func (r *SimpleReader) Read(p []byte) (int, error)
Read implements io.Reader.Read.
func (*SimpleReader) ReadByte ¶
func (r *SimpleReader) ReadByte() (byte, error)
ReadByte implements wire.Reader.ReadByte.
type SimpleWriter ¶
type SimpleWriter struct {
// contains filtered or unexported fields
}
SimpleWriter is a writer that does not compress.
func NewSimpleWriter ¶
func NewSimpleWriter(out io.Writer, key []byte) (*SimpleWriter, error)
NewSimpleWriter returns a new non-compressing writer. If key is non-nil, hash values are generated and written out for compressed bytes. See package comments for details.
func (*SimpleWriter) Write ¶
func (w *SimpleWriter) Write(p []byte) (int, error)
Write implements io.Writer.Write.
func (*SimpleWriter) WriteByte ¶
func (w *SimpleWriter) WriteByte(b byte) error
WriteByte implements wire.Writer.WriteByte.
Note that this implementation is necessary on the object itself, as an interface-based dispatch cannot tell whether the array backing the slice escapes, therefore the all bytes written will generate an escape.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer is a compressed writer.
func NewWriter ¶
NewWriter returns a new compressed writer. If key is non-nil, hash values are generated and written out for compressed bytes. See package comments for details.
The recommended chunkSize is on the order of 1M. Extra memory may be buffered (in the form of read-ahead, or buffered writes), and is limited to O(chunkSize * [1+GOMAXPROCS]).
Source Files