lzma

Documentation

Overview

Package lzma supports the decoding and encoding of LZMA streams. Reader and Writer support the classic LZMA format. Reader2 and Writer2 support the decoding and encoding of LZMA2 streams.

The package is written completely in Go and doesn't rely on any external library.

Example (Reader)

f, err := os.Open("fox.lzma")
if err != nil {
	log.Fatal(err)
}
// no need for defer; Fatal calls os.Exit(1) that doesn't execute deferred functions
r, err := NewReader(bufio.NewReader(f))
if err != nil {
	log.Fatal(err)
}
_, err = io.Copy(os.Stdout, r)
if err != nil {
	log.Fatal(err)
}
if err := f.Close(); err != nil {
	log.Fatal(err)
}

Output:

The quick brown fox jumps over the lazy dog.

Example (Writer)

The example uses the buffered reader and writer from package bufio.

pr, pw := io.Pipe()
go func() {
	bw := bufio.NewWriter(pw)
	w, err := NewWriter(bw)
	if err != nil {
		log.Fatal(err)
	}
	input := []byte("The quick brown fox jumps over the lazy dog.")
	if _, err = w.Write(input); err != nil {
		log.Fatal(err)
	}
	if err = w.Close(); err != nil {
		log.Fatal(err)
	}
	// reader waits for the data
	if err = bw.Flush(); err != nil {
		log.Fatal(err)
	}
}()
r, err := NewReader(pr)
if err != nil {
	log.Fatal(err)
}
_, err = io.Copy(os.Stdout, r)
if err != nil {
	log.Fatal(err)
}

Output:

The quick brown fox jumps over the lazy dog.

Index

• Constants
• Variables
• func ByteReader(r io.Reader) io.ByteReader
• func DecodeDictCap(c byte) (n int64, err error)
• func EncodeDictCap(n int64) byte
• func ValidHeader(data []byte) bool
• type LimitedByteWriter
  • func (l *LimitedByteWriter) WriteByte(c byte) error
• type MatchAlgorithm
  • func (a MatchAlgorithm) String() string
• type Properties
  • func PropertiesForCode(code byte) (p Properties, err error)
  • func (p Properties) Code() byte
  • func (p *Properties) String() string
• type Reader
  • func NewReader(lzma io.Reader) (r *Reader, err error)
  • func (r *Reader) EOSMarker() bool
  • func (r *Reader) Read(p []byte) (n int, err error)
• type Reader2
  • func NewReader2(lzma2 io.Reader) (r *Reader2, err error)
  • func (r *Reader2) EOS() bool
  • func (r *Reader2) Read(p []byte) (n int, err error)
• type Reader2Config
  • func (c Reader2Config) NewReader2(lzma2 io.Reader) (r *Reader2, err error)
  • func (c *Reader2Config) Verify() error
• type ReaderConfig
  • func (c ReaderConfig) NewReader(lzma io.Reader) (r *Reader, err error)
  • func (c *ReaderConfig) Verify() error
• type Writer
  • func NewWriter(lzma io.Writer) (w *Writer, err error)
  • func (w *Writer) Close() error
  • func (w *Writer) Write(p []byte) (n int, err error)
• type Writer2
  • func NewWriter2(lzma2 io.Writer) (w *Writer2, err error)
  • func (w *Writer2) Close() error
  • func (w *Writer2) Flush() error
  • func (w *Writer2) Write(p []byte) (n int, err error)
• type Writer2Config
  • func (c Writer2Config) NewWriter2(lzma2 io.Writer) (w *Writer2, err error)
  • func (c *Writer2Config) Verify() error
• type WriterConfig
  • func (c WriterConfig) NewWriter(lzma io.Writer) (w *Writer, err error)
  • func (c *WriterConfig) Verify() error

Examples

• Package (Reader)
• Package (Writer)

Constants

const (
	MinDictCap = 1 << 12
	MaxDictCap = 1<<32 - 1
)

MinDictCap and MaxDictCap provide the range of supported dictionary capacities.

const HeaderLen = 13

HeaderLen provides the length of the LZMA file header.

Variables

var ErrLimit = errors.New("limit reached")

ErrLimit indicates that the limit of the LimitedByteWriter has been reached.

var ErrNoSpace = errors.New("insufficient space")

ErrNoSpace indicates that there is insufficient space for the Write operation.

Functions

func ByteReader

func ByteReader(r io.Reader) io.ByteReader

ByteReader converts an io.Reader into an io.ByteReader.

func DecodeDictCap

func DecodeDictCap(c byte) (n int64, err error)

DecodeDictCap decodes the encoded dictionary capacity. The function returns an error if the code is out of range.

func EncodeDictCap

func EncodeDictCap(n int64) byte

EncodeDictCap encodes a dictionary capacity. The function returns the code for the capacity that is greater or equal n. If n exceeds the maximum support dictionary capacity, the maximum value is returned.

func ValidHeader

func ValidHeader(data []byte) bool

ValidHeader checks for a valid LZMA file header. It allows only dictionary sizes of 2^n or 2^n+2^(n-1) with n >= 10 or 2^32-1. If there is an explicit size it must not exceed 256 GiB. The length of the data argument must be HeaderLen.

Types

type LimitedByteWriter

type LimitedByteWriter struct {
	BW io.ByteWriter
	N  int64
}

LimitedByteWriter provides a byte writer that can be written until a limit is reached. The field N provides the number of remaining bytes.

func (*LimitedByteWriter) WriteByte

func (l *LimitedByteWriter) WriteByte(c byte) error

WriteByte writes a single byte to the limited byte writer. It returns ErrLimit if the limit has been reached. If the byte is successfully written the field N of the LimitedByteWriter will be decremented by one.

type MatchAlgorithm

type MatchAlgorithm byte

MatchAlgorithm identifies an algorithm to find matches in the dictionary.

const (
	HashTable4 MatchAlgorithm = iota
	BinaryTree
)

Supported matcher algorithms.

func (MatchAlgorithm) String

func (a MatchAlgorithm) String() string

String returns a string representation of the Matcher.

type Properties

type Properties struct {
	LC int
	LP int
	PB int
}

Properties contains the parameters LC, LP and PB. The parameter LC defines the number of literal context bits; parameter LP the number of literal position bits and PB the number of position bits.

func PropertiesForCode

func PropertiesForCode(code byte) (p Properties, err error)

PropertiesForCode converts a properties code byte into a Properties value.

func (Properties) Code

func (p Properties) Code() byte

Code converts the properties to a byte. The function assumes that the properties components are all in range.

func (*Properties) String

func (p *Properties) String() string

String returns the properties in a string representation.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader provides a reader for LZMA files or streams.

func NewReader

func NewReader(lzma io.Reader) (r *Reader, err error)

NewReader creates a new reader for an LZMA stream using the classic format. NewReader reads and checks the header of the LZMA stream.

func (*Reader) EOSMarker

func (r *Reader) EOSMarker() bool

EOSMarker indicates that an EOS marker has been encountered.

func (*Reader) Read

func (r *Reader) Read(p []byte) (n int, err error)

Read returns uncompressed data.

type Reader2

type Reader2 struct {
	// contains filtered or unexported fields
}

Reader2 supports the reading of LZMA2 chunk sequences. Note that the first chunk should have a dictionary reset and the first compressed chunk a properties reset. The chunk sequence may not be terminated by an end-of-stream chunk.

func NewReader2

func NewReader2(lzma2 io.Reader) (r *Reader2, err error)

NewReader2 creates a reader for an LZMA2 chunk sequence.

func (*Reader2) EOS

func (r *Reader2) EOS() bool

EOS returns whether the LZMA2 stream has been terminated by an end-of-stream chunk.

func (*Reader2) Read

func (r *Reader2) Read(p []byte) (n int, err error)

Read reads data from the LZMA2 chunk sequence.

type Reader2Config

type Reader2Config struct {
	DictCap int
}

Reader2Config stores the parameters for the LZMA2 reader. format.

func (Reader2Config) NewReader2

func (c Reader2Config) NewReader2(lzma2 io.Reader) (r *Reader2, err error)

NewReader2 creates an LZMA2 reader using the given configuration.

func (*Reader2Config) Verify

func (c *Reader2Config) Verify() error

Verify checks the reader configuration for errors. Zero configuration values will be replaced by default values.

type ReaderConfig

type ReaderConfig struct {
	DictCap int
}

ReaderConfig stores the parameters for the reader of the classic LZMA format.

func (ReaderConfig) NewReader

func (c ReaderConfig) NewReader(lzma io.Reader) (r *Reader, err error)

NewReader creates a new reader for an LZMA stream in the classic format. The function reads and verifies the the header of the LZMA stream.

func (*ReaderConfig) Verify

func (c *ReaderConfig) Verify() error

Verify checks the reader configuration for errors. Zero values will be replaced by default values.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer writes an LZMA stream in the classic format.

func NewWriter

func NewWriter(lzma io.Writer) (w *Writer, err error)

NewWriter creates a new LZMA writer using the classic format. The function writes the header to the underlying stream.

func (*Writer) Close

func (w *Writer) Close() error

Close closes the writer stream. It ensures that all data from the buffer will be compressed and the LZMA stream will be finished.

func (*Writer) Write

func (w *Writer) Write(p []byte) (n int, err error)

Write puts data into the Writer.

type Writer2

type Writer2 struct {
	// contains filtered or unexported fields
}

Writer2 supports the creation of an LZMA2 stream. But note that written data is buffered, so call Flush or Close to write data to the underlying writer. The Close method writes the end-of-stream marker to the stream. So you may be able to concatenate the output of two writers as long the output of the first writer has only been flushed but not closed.

Any change to the fields Properties, DictCap must be done before the first call to Write, Flush or Close.

func NewWriter2

func NewWriter2(lzma2 io.Writer) (w *Writer2, err error)

NewWriter2 creates an LZMA2 chunk sequence writer with the default parameters and options.

func (*Writer2) Close

func (w *Writer2) Close() error

Close terminates the LZMA2 stream with an EOS chunk.

func (*Writer2) Flush

func (w *Writer2) Flush() error

Flush writes all buffered data out to the underlying stream. This could result in multiple chunks to be created.

func (*Writer2) Write

func (w *Writer2) Write(p []byte) (n int, err error)

Writes data to LZMA2 stream. Note that written data will be buffered. Use Flush or Close to ensure that data is written to the underlying writer.

type Writer2Config

type Writer2Config struct {
	// The properties for the encoding. If the it is nil the value
	// {LC: 3, LP: 0, PB: 2} will be chosen.
	Properties *Properties
	// The capacity of the dictionary. If DictCap is zero, the value
	// 8 MiB will be chosen.
	DictCap int
	// Size of the lookahead buffer; value 0 indicates default size
	// 4096
	BufSize int
	// Match algorithm
	Matcher MatchAlgorithm
}

Writer2Config is used to create a Writer2 using parameters.

func (Writer2Config) NewWriter2

func (c Writer2Config) NewWriter2(lzma2 io.Writer) (w *Writer2, err error)

NewWriter2 creates a new LZMA2 writer using the given configuration.

func (*Writer2Config) Verify

func (c *Writer2Config) Verify() error

Verify checks the Writer2Config for correctness. Zero values will be replaced by default values.

type WriterConfig

type WriterConfig struct {
	// Properties for the encoding. If the it is nil the value
	// {LC: 3, LP: 0, PB: 2} will be chosen.
	Properties *Properties
	// The capacity of the dictionary. If DictCap is zero, the value
	// 8 MiB will be chosen.
	DictCap int
	// Size of the lookahead buffer; value 0 indicates default size
	// 4096
	BufSize int
	// Match algorithm
	Matcher MatchAlgorithm
	// SizeInHeader indicates that the header will contain an
	// explicit size.
	SizeInHeader bool
	// Size of the data to be encoded. A positive value will imply
	// than an explicit size will be set in the header.
	Size int64
	// EOSMarker requests whether the EOSMarker needs to be written.
	// If no explicit size is been given the EOSMarker will be
	// set automatically.
	EOSMarker bool
}

WriterConfig defines the configuration parameter for a writer.

func (WriterConfig) NewWriter

func (c WriterConfig) NewWriter(lzma io.Writer) (w *Writer, err error)

NewWriter creates a new LZMA writer for the classic format. The method will write the header to the underlying stream.

func (*WriterConfig) Verify

func (c *WriterConfig) Verify() error

Verify checks WriterConfig for errors. Verify will replace zero values with default values.