Documentation ¶
Index ¶
- Constants
- func DecodeAlphabet(ibs kanzi.InputBitStream, alphabet []int) (int, error)
- func EncodeAlphabet(obs kanzi.OutputBitStream, alphabet []int) (int, error)
- func GetName(entropyType uint32) string
- func GetType(entropyName string) uint32
- func NewEntropyDecoder(ibs kanzi.InputBitStream, ctx map[string]interface{}, entropyType uint32) (kanzi.EntropyDecoder, error)
- func NewEntropyEncoder(obs kanzi.OutputBitStream, ctx map[string]interface{}, entropyType uint32) (kanzi.EntropyEncoder, error)
- func NormalizeFrequencies(freqs []int, alphabet []int, totalFreq, scale int) (int, error)
- func ReadVarInt(bs kanzi.InputBitStream) uint32
- func WriteVarInt(bs kanzi.OutputBitStream, value uint32) int
- type ANSRangeDecoder
- type ANSRangeEncoder
- type AdaptiveProbMap
- type BinaryEntropyDecoder
- type BinaryEntropyEncoder
- type CMPredictor
- type ExpGolombDecoder
- type ExpGolombEncoder
- type FPAQDecoder
- type FPAQEncoder
- type FastLogisticAdaptiveProbMap
- type HuffmanDecoder
- type HuffmanEncoder
- type LinearAdaptiveProbMap
- type LogisticAdaptiveProbMap
- type NullEntropyDecoder
- type NullEntropyEncoder
- type RangeDecoder
- type RangeEncoder
- type RiceGolombDecoder
- type RiceGolombEncoder
- type TPAQMixer
- type TPAQPredictor
Constants ¶
const ( NONE_TYPE = uint32(0) // No compression HUFFMAN_TYPE = uint32(1) // Huffman FPAQ_TYPE = uint32(2) // Fast PAQ (order 0) PAQ_TYPE = uint32(3) // Obsolete RANGE_TYPE = uint32(4) // Range ANS0_TYPE = uint32(5) // Asymmetric Numerical System order 0 CM_TYPE = uint32(6) // Context Model TPAQ_TYPE = uint32(7) // Tangelo PAQ ANS1_TYPE = uint32(8) // Asymmetric Numerical System order 1 TPAQX_TYPE = uint32(9) // Tangelo PAQ Extra )
const (
// INCOMPRESSIBLE_THRESHOLD Any block with entropy*1024 greater than this threshold is considered incompressible
INCOMPRESSIBLE_THRESHOLD = 973
)
Variables ¶
This section is empty.
Functions ¶
func DecodeAlphabet ¶
func DecodeAlphabet(ibs kanzi.InputBitStream, alphabet []int) (int, error)
DecodeAlphabet reads the alphabet from the bitstream and return the number of symbols read or an error
func EncodeAlphabet ¶
func EncodeAlphabet(obs kanzi.OutputBitStream, alphabet []int) (int, error)
EncodeAlphabet writes the alphabet to the bitstream and return the number of symbols written or an error. alphabet must be sorted in increasing order alphabet size must be a power of 2 up to 256
func NewEntropyDecoder ¶
func NewEntropyDecoder(ibs kanzi.InputBitStream, ctx map[string]interface{}, entropyType uint32) (kanzi.EntropyDecoder, error)
NewEntropyDecoder creates a new entropy decoder using the provided type and bitstream
func NewEntropyEncoder ¶
func NewEntropyEncoder(obs kanzi.OutputBitStream, ctx map[string]interface{}, entropyType uint32) (kanzi.EntropyEncoder, error)
NewEntropyEncoder creates a new entropy encoder using the provided type and bitstream
func NormalizeFrequencies ¶
NormalizeFrequencies scales the frequencies so that their sum equals 'scale'. Returns the size of the alphabet or an error. The alphabet and freqs parameters are updated.
func ReadVarInt ¶
func ReadVarInt(bs kanzi.InputBitStream) uint32
ReadVarInt reads a VarInt from the bitstream and returns it as an uint32.
func WriteVarInt ¶
func WriteVarInt(bs kanzi.OutputBitStream, value uint32) int
WriteVarInt writes the provided value to the bitstream as a VarInt. Returns the number of bytes written.
Types ¶
type ANSRangeDecoder ¶
type ANSRangeDecoder struct {
// contains filtered or unexported fields
}
ANSRangeDecoder Asymmetric Numeral System Decoder
func NewANSRangeDecoder ¶
func NewANSRangeDecoder(bs kanzi.InputBitStream, args ...uint) (*ANSRangeDecoder, error)
NewANSRangeDecoder creates an instance of ANS decoder. The chunk size indicates how many bytes are encoded (per block) before resetting the frequency stats. Since the number of args is variable, this function can be called like this: NewANSRangeDecoder(bs) or NewANSRangeDecoder(bs, 0, 16384, 12) Arguments are order and chunk size chunkSize = 0 means 'use input buffer length' during decoding
func (*ANSRangeDecoder) BitStream ¶
func (this *ANSRangeDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*ANSRangeDecoder) Dispose ¶
func (this *ANSRangeDecoder) Dispose()
Dispose this implementation does nothing
type ANSRangeEncoder ¶
type ANSRangeEncoder struct {
// contains filtered or unexported fields
}
ANSRangeEncoder Asymmetric Numeral System Encoder
func NewANSRangeEncoder ¶
func NewANSRangeEncoder(bs kanzi.OutputBitStream, args ...uint) (*ANSRangeEncoder, error)
NewANSRangeEncoder creates an instance of ANS encoder. The chunk size indicates how many bytes are encoded (per block) before resetting the frequency stats. 0 means that frequencies calculated at the beginning of the block apply to the whole block Since the number of args is variable, this function can be called like this: NewANSRangeEncoder(bs) or NewANSRangeEncoder(bs, 0, 16384, 12) Arguments are order, chunk size and log range. chunkSize = 0 means 'use input buffer length' during decoding
func (*ANSRangeEncoder) BitStream ¶
func (this *ANSRangeEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*ANSRangeEncoder) Dispose ¶
func (this *ANSRangeEncoder) Dispose()
Dispose this implementation does nothing
type AdaptiveProbMap ¶
type AdaptiveProbMap struct {
// contains filtered or unexported fields
}
AdaptiveProbMap maps a probability and a context to a new probability that the next bit will be 1. After each guess, it updates its state to improve future predictions.
type BinaryEntropyDecoder ¶
type BinaryEntropyDecoder struct {
// contains filtered or unexported fields
}
BinaryEntropyDecoder entropy decoder based on arithmetic coding and using an external probability predictor.
func NewBinaryEntropyDecoder ¶
func NewBinaryEntropyDecoder(bs kanzi.InputBitStream, predictor kanzi.Predictor) (*BinaryEntropyDecoder, error)
NewBinaryEntropyDecoder creates an instance of BinaryEntropyDecoder using the given predictor to predict the probability of the next bit to be one. It outputs to the given OutputBitstream
func (*BinaryEntropyDecoder) BitStream ¶
func (this *BinaryEntropyDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*BinaryEntropyDecoder) DecodeBit ¶
func (this *BinaryEntropyDecoder) DecodeBit(pred int) byte
DecodeBit decodes one bit from the bitstream using arithmetic coding and the probability predictor provided at creation time.
func (*BinaryEntropyDecoder) DecodeByte ¶
func (this *BinaryEntropyDecoder) DecodeByte() byte
DecodeByte decodes the given value from the bitstream bit by bit
func (*BinaryEntropyDecoder) Dispose ¶
func (this *BinaryEntropyDecoder) Dispose()
Dispose must be called before getting rid of the entropy decoder This implementation does nothing.
func (*BinaryEntropyDecoder) Read ¶
func (this *BinaryEntropyDecoder) Read(block []byte) (int, error)
Read decodes data from the bitstream and return it in the provided buffer. Return the number of bytes read from the bitstream. Splits big blocks into chunks and decode the chunks byte by byte sequentially from the bitstream.
type BinaryEntropyEncoder ¶
type BinaryEntropyEncoder struct {
// contains filtered or unexported fields
}
BinaryEntropyEncoder entropy encoder based on arithmetic coding and using an external probability predictor.
func NewBinaryEntropyEncoder ¶
func NewBinaryEntropyEncoder(bs kanzi.OutputBitStream, predictor kanzi.Predictor) (*BinaryEntropyEncoder, error)
NewBinaryEntropyEncoder creates an instance of BinaryEntropyEncoder using the given predictor to predict the probability of the next bit to be one. It outputs to the given OutputBitstream
func (*BinaryEntropyEncoder) BitStream ¶
func (this *BinaryEntropyEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*BinaryEntropyEncoder) Dispose ¶
func (this *BinaryEntropyEncoder) Dispose()
Dispose must be called before getting rid of the entropy encoder This idempotent implmentation writes the last buffered bits into the bitstream.
func (*BinaryEntropyEncoder) EncodeBit ¶
func (this *BinaryEntropyEncoder) EncodeBit(bit byte, pred int)
EncodeBit encodes one bit into the bitstream using arithmetic coding and the probability predictor provided at creation time.
func (*BinaryEntropyEncoder) EncodeByte ¶
func (this *BinaryEntropyEncoder) EncodeByte(val byte)
EncodeByte encodes the given value into the bitstream bit by bit
func (*BinaryEntropyEncoder) Write ¶
func (this *BinaryEntropyEncoder) Write(block []byte) (int, error)
Write encodes the data provided into the bitstream. Return the number of byte written to the bitstream. Splits big blocks into chunks and encode the chunks byte by byte sequentially into the bitstream.
type CMPredictor ¶
type CMPredictor struct {
// contains filtered or unexported fields
}
CMPredictor context model predictor based on BCM by Ilya Muravyov. See https://github.com/encode84/bcm
func NewCMPredictor ¶
func NewCMPredictor() (*CMPredictor, error)
NewCMPredictor creates a new instance of CMPredictor
func (*CMPredictor) Get ¶
func (this *CMPredictor) Get() int
Get returns the value representing the probability of the next bit being 1 in the [0..4095] range. The probability is computed from the internal bit counters.
func (*CMPredictor) Update ¶
func (this *CMPredictor) Update(bit byte)
Update updates the probability model based on the internal bit counters
type ExpGolombDecoder ¶
type ExpGolombDecoder struct {
// contains filtered or unexported fields
}
ExpGolombDecoder Exponential Golomb Entropy Decoder
func NewExpGolombDecoder ¶
func NewExpGolombDecoder(bs kanzi.InputBitStream, sgn bool) (*ExpGolombDecoder, error)
NewExpGolombDecoder creates a new instance of ExpGolombDecoder If sgn is true, values from the bitstream will be decoded as signed (int8)
func (*ExpGolombDecoder) BitStream ¶
func (this *ExpGolombDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*ExpGolombDecoder) DecodeByte ¶
func (this *ExpGolombDecoder) DecodeByte() byte
DecodeByte decodes one byte from the bitstream If the decoder is sign aware, the returned value is an int8 cast to a byte
func (*ExpGolombDecoder) Dispose ¶
func (this *ExpGolombDecoder) Dispose()
Dispose this implementation does nothing
func (*ExpGolombDecoder) Read ¶
func (this *ExpGolombDecoder) Read(block []byte) (int, error)
Read decodes data from the bitstream and return it in the provided buffer. Return the number of bytes read from the bitstream
func (*ExpGolombDecoder) Signed ¶
func (this *ExpGolombDecoder) Signed() bool
Signed returns true if this decoder is sign aware
type ExpGolombEncoder ¶
type ExpGolombEncoder struct {
// contains filtered or unexported fields
}
ExpGolombEncoder Exponential Golomb Entropy Encoder
func NewExpGolombEncoder ¶
func NewExpGolombEncoder(bs kanzi.OutputBitStream, sgn bool) (*ExpGolombEncoder, error)
NewExpGolombEncoder creates a new instance of ExpGolombEncoder If sgn is true, values will be encoded as signed (int8) in the bitstream. Using a sign improves compression ratio for distributions centered on 0 (E.G. Gaussian) Example: -1 is better compressed as -1 (1 followed by '-') than as 255
func (*ExpGolombEncoder) BitStream ¶
func (this *ExpGolombEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*ExpGolombEncoder) Dispose ¶
func (this *ExpGolombEncoder) Dispose()
Dispose this implementation does nothing
func (*ExpGolombEncoder) EncodeByte ¶
func (this *ExpGolombEncoder) EncodeByte(val byte)
EncodeByte encodes the given value into the bitstream
func (*ExpGolombEncoder) Signed ¶
func (this *ExpGolombEncoder) Signed() bool
Signed returns true if this encoder is sign aware
type FPAQDecoder ¶ added in v1.8.0
type FPAQDecoder struct {
// contains filtered or unexported fields
}
FPAQDecoder entropy decoder derived from fpaq0r by Matt Mahoney & Alexander Ratushnyak. See http://mattmahoney.net/dc/#fpaq0. Simple (and fast) adaptive entropy bit decoder
func NewFPAQDecoder ¶ added in v1.8.0
func NewFPAQDecoder(bs kanzi.InputBitStream) (*FPAQDecoder, error)
NewFPAQDecoder creates an instance of FPAQDecoder
func (*FPAQDecoder) BitStream ¶ added in v1.8.0
func (this *FPAQDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*FPAQDecoder) Dispose ¶ added in v1.8.0
func (this *FPAQDecoder) Dispose()
Dispose must be called before getting rid of the entropy decoder This implementation does nothing.
func (*FPAQDecoder) Read ¶ added in v1.8.0
func (this *FPAQDecoder) Read(block []byte) (int, error)
Read decodes data from the bitstream and return it in the provided buffer. Return the number of bytes read from the bitstream. Splits big blocks into chunks and decode the chunks byte by byte sequentially from the bitstream.
type FPAQEncoder ¶ added in v1.8.0
type FPAQEncoder struct {
// contains filtered or unexported fields
}
FPAQEncoder entropy encoder derived from fpaq0r by Matt Mahoney & Alexander Ratushnyak. See http://mattmahoney.net/dc/#fpaq0. Simple (and fast) adaptive entropy bit encoder
func NewFPAQEncoder ¶ added in v1.8.0
func NewFPAQEncoder(bs kanzi.OutputBitStream) (*FPAQEncoder, error)
NewFPAQEncoder creates an instance of FPAQEncoder
func (*FPAQEncoder) BitStream ¶ added in v1.8.0
func (this *FPAQEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*FPAQEncoder) Dispose ¶ added in v1.8.0
func (this *FPAQEncoder) Dispose()
Dispose must be called before getting rid of the entropy encoder This idempotent implmentation writes the last buffered bits into the bitstream.
func (*FPAQEncoder) Write ¶ added in v1.8.0
func (this *FPAQEncoder) Write(block []byte) (int, error)
Write encodes the data provided into the bitstream. Return the number of byte written to the bitstream. Splits big blocks into chunks and encode the chunks byte by byte sequentially into the bitstream.
type FastLogisticAdaptiveProbMap ¶
type FastLogisticAdaptiveProbMap AdaptiveProbMap
FastLogisticAdaptiveProbMap is similar to LogisticAdaptiveProbMap but works faster at the expense of some accuracy
type HuffmanDecoder ¶
type HuffmanDecoder struct {
// contains filtered or unexported fields
}
HuffmanDecoder Implementation of a static Huffman decoder. Uses tables to decode symbols
func NewHuffmanDecoder ¶
func NewHuffmanDecoder(bs kanzi.InputBitStream, args ...uint) (*HuffmanDecoder, error)
NewHuffmanDecoder creates an instance of HuffmanDecoder. Since the number of args is variable, this function can be called like this: NewHuffmanDecoder(bs) or NewHuffmanDecoder(bs, 16384) (the second argument being the chunk size)
func (*HuffmanDecoder) BitStream ¶
func (this *HuffmanDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*HuffmanDecoder) Dispose ¶
func (this *HuffmanDecoder) Dispose()
Dispose this implementation does nothing
type HuffmanEncoder ¶
type HuffmanEncoder struct {
// contains filtered or unexported fields
}
HuffmanEncoder Implementation of a static Huffman encoder. Uses in place generation of canonical codes instead of a tree
func NewHuffmanEncoder ¶
func NewHuffmanEncoder(bs kanzi.OutputBitStream, args ...uint) (*HuffmanEncoder, error)
NewHuffmanEncoder creates an instance of HuffmanEncoder. Since the number of args is variable, this function can be called like this: NewHuffmanEncoder(bs) or NewHuffmanEncoder(bs, 16384) (the second argument being the chunk size)
func (*HuffmanEncoder) BitStream ¶
func (this *HuffmanEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*HuffmanEncoder) Dispose ¶
func (this *HuffmanEncoder) Dispose()
Dispose this implementation does nothing
type LinearAdaptiveProbMap ¶
type LinearAdaptiveProbMap AdaptiveProbMap
LinearAdaptiveProbMap maps a probability and a context into a new probability using linear interpolation of probabilities
type LogisticAdaptiveProbMap ¶
type LogisticAdaptiveProbMap AdaptiveProbMap
LogisticAdaptiveProbMap maps a probability and a context into a new probability using interpolation in the logistic domain
type NullEntropyDecoder ¶
type NullEntropyDecoder struct {
// contains filtered or unexported fields
}
NullEntropyDecoder pass through entropy decoder (reads the input bytes directly from the bitstream)
func NewNullEntropyDecoder ¶
func NewNullEntropyDecoder(bs kanzi.InputBitStream) (*NullEntropyDecoder, error)
NewNullEntropyDecoder creates a new instance of NullEntropyDecoder
func (*NullEntropyDecoder) BitStream ¶
func (this *NullEntropyDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*NullEntropyDecoder) Dispose ¶
func (this *NullEntropyDecoder) Dispose()
Dispose this implementation does nothing
type NullEntropyEncoder ¶
type NullEntropyEncoder struct {
// contains filtered or unexported fields
}
NullEntropyEncoder pass through entropy encoder (writes the input bytes directly to the bitstream)
func NewNullEntropyEncoder ¶
func NewNullEntropyEncoder(bs kanzi.OutputBitStream) (*NullEntropyEncoder, error)
NewNullEntropyEncoder creates a new instance of NullEntropyEncoder
func (*NullEntropyEncoder) BitStream ¶
func (this *NullEntropyEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*NullEntropyEncoder) Dispose ¶
func (this *NullEntropyEncoder) Dispose()
Dispose this implementation does nothing
type RangeDecoder ¶
type RangeDecoder struct {
// contains filtered or unexported fields
}
RangeDecoder Order 0 Range Entropy Decoder
func NewRangeDecoder ¶
func NewRangeDecoder(bs kanzi.InputBitStream, args ...uint) (*RangeDecoder, error)
NewRangeDecoder creates a new instance of RangeDecoder The given arguments are either empty or containing a chunk size. EG: call NewRangeDecoder(bs) or NewRangeDecoder(bs, 16384) The default chunk size is 65536 bytes.
func (*RangeDecoder) BitStream ¶
func (this *RangeDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*RangeDecoder) Dispose ¶
func (this *RangeDecoder) Dispose()
Dispose this implementation does nothing
type RangeEncoder ¶
type RangeEncoder struct {
// contains filtered or unexported fields
}
RangeEncoder a Order 0 Range Entropy Encoder
func NewRangeEncoder ¶
func NewRangeEncoder(bs kanzi.OutputBitStream, args ...uint) (*RangeEncoder, error)
NewRangeEncoder creates a new instance of RangeEncoder The given arguments are either empty or containing a chunk size and a log range (to specify the precision of the encoding). EG: call NewRangeEncoder(bs) or NewRangeEncoder(bs, 16384, 14) The default chunk size is 65536 bytes.
func (*RangeEncoder) BitStream ¶
func (this *RangeEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*RangeEncoder) Dispose ¶
func (this *RangeEncoder) Dispose()
Dispose this implementation does nothing
type RiceGolombDecoder ¶
type RiceGolombDecoder struct {
// contains filtered or unexported fields
}
RiceGolombDecoder Exponential Golomb Entropy Decoder
func NewRiceGolombDecoder ¶
func NewRiceGolombDecoder(bs kanzi.InputBitStream, sgn bool, logBase uint) (*RiceGolombDecoder, error)
NewRiceGolombDecoder creates a new instance of ExpGolombDecoder If sgn is true, values from the bitstream will be decoded as signed (int8)
func (*RiceGolombDecoder) BitStream ¶
func (this *RiceGolombDecoder) BitStream() kanzi.InputBitStream
BitStream returns the underlying bitstream
func (*RiceGolombDecoder) DecodeByte ¶
func (this *RiceGolombDecoder) DecodeByte() byte
DecodeByte decodes one byte from the bitstream If the decoder is sign aware, the returned value is an int8 cast to a byte
func (*RiceGolombDecoder) Dispose ¶
func (this *RiceGolombDecoder) Dispose()
Dispose this implementation does nothing
func (*RiceGolombDecoder) Read ¶
func (this *RiceGolombDecoder) Read(block []byte) (int, error)
Read decodes data from the bitstream and return it in the provided buffer. Return the number of bytes read from the bitstream
func (*RiceGolombDecoder) Signed ¶
func (this *RiceGolombDecoder) Signed() bool
Signed returns true if this decoder is sign aware
type RiceGolombEncoder ¶
type RiceGolombEncoder struct {
// contains filtered or unexported fields
}
RiceGolombEncoder a Rice Golomb Entropy Encoder
func NewRiceGolombEncoder ¶
func NewRiceGolombEncoder(bs kanzi.OutputBitStream, sgn bool, logBase uint) (*RiceGolombEncoder, error)
NewRiceGolombEncoder creates a new instance of RiceGolombEncoder If sgn is true, values will be encoded as signed (int8) in the bitstream. Using a sign improves compression ratio for distributions centered on 0 (E.G. Gaussian) Example: -1 is better compressed as -1 (1 followed by '-') than as 255
func (*RiceGolombEncoder) BitStream ¶
func (this *RiceGolombEncoder) BitStream() kanzi.OutputBitStream
BitStream returns the underlying bitstream
func (*RiceGolombEncoder) Dispose ¶
func (this *RiceGolombEncoder) Dispose()
Dispose this implementation does nothing
func (*RiceGolombEncoder) EncodeByte ¶
func (this *RiceGolombEncoder) EncodeByte(val byte)
EncodeByte encodes the given value into the bitstream
func (*RiceGolombEncoder) Signed ¶
func (this *RiceGolombEncoder) Signed() bool
Signed returns true if this encoder is sign aware
type TPAQMixer ¶
type TPAQMixer struct {
// contains filtered or unexported fields
}
TPAQMixer a mixer that combines models using neural networks with 8 inputs.
type TPAQPredictor ¶
type TPAQPredictor struct {
// contains filtered or unexported fields
}
TPAQPredictor bit predictor for binary entropy codecs. It uses a mixer to combine initial predictions derived for several local contexts and a secondary symbol estimation to improve the prediction from the mixer. Initially based on Tangelo 2.4 (by Jan Ondrus). PAQ8 is written by Matt Mahoney. See http://encode.su/threads/1738-TANGELO-new-compressor-(derived-from-PAQ8-FP8)
func NewTPAQPredictor ¶
func NewTPAQPredictor(ctx *map[string]interface{}) (*TPAQPredictor, error)
NewTPAQPredictor creates a new instance of TPAQPredictor using the provided map of options to select the sizes of internal structures.
func (*TPAQPredictor) Get ¶
func (this *TPAQPredictor) Get() int
Get returns the value representing the probability of the next bit being 1 (in the [0..4095] range).
func (*TPAQPredictor) Update ¶
func (this *TPAQPredictor) Update(bit byte)
Update updates the internal probability model based on the observed bit