lz4

README

lz4

LZ4 compression and decompression in pure Go.

Usage

import "github.com/pierrec/lz4"

Description

Package lz4 implements reading and writing lz4 compressed data (a frame), as specified in http://fastcompression.blogspot.fr/2013/04/lz4-streaming-format-final.html. Although the block level compression and decompression functions are exposed and are fully compatible with the lz4 block format definition, they are low level and should not be used directly.

For a complete description of an lz4 compressed block, see: http://fastcompression.blogspot.fr/2011/05/lz4-explained.html

See https://github.com/Cyan4973/lz4 for the reference C implementation.

Documentation

Overview

Package lz4 implements reading and writing lz4 compressed data (a frame), as specified in http://fastcompression.blogspot.fr/2013/04/lz4-streaming-format-final.html.

Although the block level compression and decompression functions are exposed and are fully compatible with the lz4 block format definition, they are low level and should not be used directly. For a complete description of an lz4 compressed block, see: http://fastcompression.blogspot.fr/2011/05/lz4-explained.html

See https://github.com/Cyan4973/lz4 for the reference C implementation.

Index

• Constants
• Variables
• func CompressBlock(src, dst []byte, hashTable []int) (di int, err error)
• func CompressBlockBound(n int) int
• func CompressBlockHC(src, dst []byte, depth int) (di int, err error)
• func UncompressBlock(src, dst []byte) (si int, err error)
• type Header
  • func (h Header) String() string
• type Reader
  • func NewReader(src io.Reader) *Reader
  • func (z *Reader) Read(buf []byte) (int, error)
  • func (z *Reader) Reset(r io.Reader)
• type Writer
  • func NewWriter(dst io.Writer) *Writer
  • func (z *Writer) Close() error
  • func (z *Writer) Flush() error
  • func (z *Writer) Reset(w io.Writer)
  • func (z *Writer) Write(buf []byte) (int, error)

Constants

const (
	// Extension is the LZ4 frame file name extension
	Extension = ".lz4"
	// Version is the LZ4 frame format version
	Version = 1
)

Variables

var (
	// ErrInvalidSourceShortBuffer is returned by UncompressBlock or CompressBLock when a compressed
	// block is corrupted or the destination buffer is not large enough for the uncompressed data.
	ErrInvalidSourceShortBuffer = errors.New("lz4: invalid source or destination buffer too short")
	// ErrInvalid is returned when reading an invalid LZ4 archive.
	ErrInvalid = errors.New("lz4: bad magic number")
)

Functions

func CompressBlock

func CompressBlock(src, dst []byte, hashTable []int) (di int, err error)

CompressBlock compresses the source buffer into the destination one. This is the fast version of LZ4 compression and also the default one. The size of hashTable must be at least 64Kb.

The size of the compressed data is returned. If it is 0 and no error, then the data is incompressible.

An error is returned if the destination buffer is too small.

func CompressBlockBound

func CompressBlockBound(n int) int

CompressBlockBound returns the maximum size of a given buffer of size n, when not compressible.

func CompressBlockHC

func CompressBlockHC(src, dst []byte, depth int) (di int, err error)

CompressBlockHC compresses the source buffer src into the destination dst with max search depth (use 0 or negative value for no max).

CompressBlockHC compression ratio is better than CompressBlock but it is also slower.

The size of the compressed data is returned. If it is 0 and no error, then the data is not compressible.

An error is returned if the destination buffer is too small.

func UncompressBlock

func UncompressBlock(src, dst []byte) (si int, err error)

UncompressBlock uncompresses the source buffer into the destination one, and returns the uncompressed size.

The destination buffer must be sized appropriately.

An error is returned if the source data is invalid or the destination buffer is too small.

Types

type Header

type Header struct {
	BlockChecksum    bool   // Compressed blocks checksum flag.
	NoChecksum       bool   // Frame checksum flag.
	BlockMaxSize     int    // Size of the uncompressed data block (one of [64KB, 256KB, 1MB, 4MB]). Default=4MB.
	Size             uint64 // Frame total size. It is _not_ computed by the Writer.
	CompressionLevel int    // Compression level (higher is better, use 0 for fastest compression).
	// contains filtered or unexported fields
}

Header describes the various flags that can be set on a Writer or obtained from a Reader. The default values match those of the LZ4 frame format definition (http://fastcompression.blogspot.com/2013/04/lz4-streaming-format-final.html).

NB. in a Reader, in case of concatenated frames, the Header values may change between Read() calls. It is the caller responsibility to check them if necessary.

func (Header) String

func (h Header) String() string

type Reader

type Reader struct {
	Header
	// contains filtered or unexported fields
}

Reader implements the LZ4 frame decoder. The Header is set after the first call to Read(). The Header may change between Read() calls in case of concatenated frames.

func NewReader

func NewReader(src io.Reader) *Reader

NewReader returns a new LZ4 frame decoder. No access to the underlying io.Reader is performed.

func (*Reader) Read

func (z *Reader) Read(buf []byte) (int, error)

Read decompresses data from the underlying source into the supplied buffer.

Since there can be multiple streams concatenated, Header values may change between calls to Read(). If that is the case, no data is actually read from the underlying io.Reader, to allow for potential input buffer resizing.

func (*Reader) Reset

func (z *Reader) Reset(r io.Reader)

Reset discards the Reader's state and makes it equivalent to the result of its original state from NewReader, but reading from r instead. This permits reusing a Reader rather than allocating a new one.

type Writer

type Writer struct {
	Header
	// contains filtered or unexported fields
}

Writer implements the LZ4 frame encoder.

func NewWriter

func NewWriter(dst io.Writer) *Writer

NewWriter returns a new LZ4 frame encoder. No access to the underlying io.Writer is performed. The supplied Header is checked at the first Write. It is ok to change it before the first Write but then not until a Reset() is performed.

func (*Writer) Close

func (z *Writer) Close() error

Close closes the Writer, flushing any unwritten data to the underlying io.Writer, but does not close the underlying io.Writer.

func (*Writer) Flush

func (z *Writer) Flush() error

Flush flushes any pending compressed data to the underlying writer. Flush does not return until the data has been written. If the underlying writer returns an error, Flush returns that error.

func (*Writer) Reset

func (z *Writer) Reset(w io.Writer)

Reset clears the state of the Writer z such that it is equivalent to its initial state from NewWriter, but instead writing to w. No access to the underlying io.Writer is performed.

func (*Writer) Write

func (z *Writer) Write(buf []byte) (int, error)

Write compresses data from the supplied buffer into the underlying io.Writer. Write does not return until the data has been written.