encoding

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 9, 2024 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package encoding implements encoding/decoding data points.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BytesToInt64 added in v0.6.0 

func BytesToInt64(src []byte) int64

BytesToInt64 converts the first eight bytes of the given byte slice to an int64.

func BytesToInt64List added in v0.6.0 

func BytesToInt64List(dst []int64, src []byte, mt EncodeType, firstValue int64, itemsCount int) ([]int64, error)

BytesToInt64List decodes bytes into a list of int64.

func BytesToUint16 added in v0.6.0 

func BytesToUint16(src []byte) uint16

BytesToUint16 converts the first two bytes of the given byte slice to a uint16.

func BytesToUint32 added in v0.6.0 

func BytesToUint32(src []byte) uint32

BytesToUint32 converts the first four bytes of the given byte slice to a uint32.

func BytesToUint64 added in v0.6.0 

func BytesToUint64(src []byte) uint64

BytesToUint64 converts the first eight bytes of the given byte slice to a uint64.

func BytesToVarInt64 added in v0.6.0 

func BytesToVarInt64(src []byte) ([]byte, int64, error)

BytesToVarInt64 converts the first bytes of the given byte slice to an int64. It uses variable-length encoding.

func BytesToVarInt64List added in v0.6.0 

func BytesToVarInt64List(dst []int64, src []byte) ([]byte, error)

BytesToVarInt64List converts the first bytes of the given byte slice to an int64s. It uses variable-length encoding.

func BytesToVarUint64 added in v0.6.0 

func BytesToVarUint64(src []byte) ([]byte, uint64, error)

BytesToVarUint64 converts the first bytes of the given byte slice to a uint64. It uses variable-length encoding.

func BytesToVarUint64s added in v0.6.0 

func BytesToVarUint64s(dst []uint64, src []byte) ([]byte, error)

BytesToVarUint64s converts the first bytes of the given byte slice to a uint64s. It uses variable-length encoding.

func DecodeBytes added in v0.6.0 

func DecodeBytes(src []byte) ([]byte, []byte, error)

DecodeBytes decodes a string from src.

func EncodeBytes added in v0.6.0 

func EncodeBytes(dst, b []byte) []byte

EncodeBytes encodes a string into dst.

func EncodeBytesBlock added in v0.6.0 

func EncodeBytesBlock(dst []byte, a [][]byte) []byte

EncodeBytesBlock encodes a block of strings into dst.

func Int64ToBytes added in v0.6.0 

func Int64ToBytes(dst []byte, v int64) []byte

Int64ToBytes appends the bytes of the given int64 to the given byte slice.

func ReleaseInt64List added in v0.6.0 

func ReleaseInt64List(is *Int64List)

ReleaseInt64List releases the given list of int64. The list may be put into a pool for reuse.

func ReleaseUint64List added in v0.6.0 

func ReleaseUint64List(is *Uint64List)

ReleaseUint64List releases the given list of uint64. The list may be put into a pool for reuse.

func Uint16ToBytes added in v0.6.0 

func Uint16ToBytes(dst []byte, u uint16) []byte

Uint16ToBytes appends the bytes of the given uint16 to the given byte slice.

func Uint32ToBytes added in v0.6.0 

func Uint32ToBytes(dst []byte, u uint32) []byte

Uint32ToBytes appends the bytes of the given uint32 to the given byte slice.

func Uint64ToBytes added in v0.6.0 

func Uint64ToBytes(dst []byte, u uint64) []byte

Uint64ToBytes appends the bytes of the given uint64 to the given byte slice.

func VarInt64ListToBytes added in v0.6.0 

func VarInt64ListToBytes(dst []byte, vs []int64) []byte

VarInt64ListToBytes appends the bytes of the given int64s to the given byte slice. It uses variable-length encoding.

func VarInt64ToBytes added in v0.6.0 

func VarInt64ToBytes(dst []byte, v int64) []byte

VarInt64ToBytes appends the bytes of the given int64 to the given byte slice. It uses variable-length encoding.

func VarUint64ToBytes added in v0.6.0 

func VarUint64ToBytes(dst []byte, u uint64) []byte

VarUint64ToBytes appends the bytes of the given uint64 to the given byte slice. It uses variable-length encoding.

func VarUint64sToBytes added in v0.6.0 

func VarUint64sToBytes(dst []byte, us []uint64) []byte

VarUint64sToBytes appends the bytes of the given uint64s to the given byte slice. It uses variable-length encoding.

Types

type BufferWriter added in v0.3.0 

type BufferWriter interface {
	Write(data []byte) (n int, err error)
	WriteByte(b byte) error
	Bytes() []byte
}

BufferWriter allows writing a variable-sized buffer of bytes.

type BytesBlockDecoder added in v0.6.0 

type BytesBlockDecoder struct {
	// contains filtered or unexported fields
}

BytesBlockDecoder decodes a block of strings from src. It reuses the underlying buffer for storing the decoded strings.

func (*BytesBlockDecoder) Decode added in v0.6.0 

func (bbd *BytesBlockDecoder) Decode(dst [][]byte, src []byte, itemsCount uint64) ([][]byte, error)

Decode decodes a block of strings from src.

func (*BytesBlockDecoder) Reset added in v0.6.0 

func (bbd *BytesBlockDecoder) Reset()

Reset resets the decoder.

type EncodeType added in v0.6.0 

type EncodeType byte

EncodeType indicates the encoding type of a series. 

const (
	EncodeTypeUnknown EncodeType = iota
	EncodeTypeConst
	EncodeTypeDeltaConst
	EncodeTypeDelta
	EncodeTypeDeltaOfDelta
	EncodeTypeXOR
)

EncodeType constants.

func Int64ListToBytes added in v0.6.0 

func Int64ListToBytes(dst []byte, a []int64) (result []byte, mt EncodeType, firstValue int64)

Int64ListToBytes encodes a list of int64 into bytes.

type Int64List added in v0.6.0 

type Int64List struct {
	L []int64
}

Int64List is a list of int64.

func GenerateInt64List added in v0.6.0 

func GenerateInt64List(size int) *Int64List

GenerateInt64List generates a list of int64 with the given size. The returned list may be from a pool and should be released after use.

type Packer added in v0.3.0 

type Packer struct {
	// contains filtered or unexported fields
}

Packer writes data into a buffer.

func NewPacker added in v0.3.0 

func NewPacker(buf BufferWriter) *Packer

NewPacker returns a new Writer.

func (*Packer) Bytes added in v0.3.0 

func (w *Packer) Bytes() []byte

Bytes outputs the data in the buffer.

func (*Packer) PutUint16 added in v0.3.0 

func (w *Packer) PutUint16(v uint16)

PutUint16 puts uint16 data into the buffer.

func (*Packer) PutUint32 added in v0.3.0 

func (w *Packer) PutUint32(v uint32)

PutUint32 puts uint32 data into the buffer.

func (*Packer) PutUint64 added in v0.3.0 

func (w *Packer) PutUint64(v uint64)

PutUint64 puts uint64 data into the buffer.

func (*Packer) Write added in v0.3.0 

func (w *Packer) Write(p []byte)

Write binaries to the buffer.

type ParseInterval 

type ParseInterval = func(key []byte) time.Duration

ParseInterval parses the interval rule from the key in a kv pair.

type Reader added in v0.3.0 

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads bits from buffer.

func NewReader added in v0.3.0 

func NewReader(in io.ByteReader) *Reader

NewReader crate bit reader.

func (*Reader) ReadBits added in v0.3.0 

func (r *Reader) ReadBits(numBits int) (uint64, error)

ReadBits read number of bits.

func (*Reader) ReadBool added in v0.3.0 

func (r *Reader) ReadBool() (bool, error)

ReadBool reads a bit, 1 returns true, 0 returns false.

func (*Reader) ReadByte added in v0.3.0 

func (r *Reader) ReadByte() (byte, error)

ReadByte reads a byte.

func (*Reader) Reset added in v0.3.0 

func (r *Reader) Reset()

Reset resets the reader to read from a new slice.

type SeriesDecoder 

type SeriesDecoder interface {
	// Decode the time series data
	Decode(key, data []byte) error
	// Len denotes the size of iterator
	Len() int
	// IsFull returns whether the encoded data reached its capacity
	IsFull() bool
	// Get the data point by its time
	Get(ts uint64) ([]byte, error)
	// Iterator returns a SeriesIterator
	Iterator() SeriesIterator
	// Range returns the start and end time of this series
	Range() (start, end uint64)
}

SeriesDecoder decodes encoded time series data.

type SeriesDecoderPool 

type SeriesDecoderPool interface {
	Get(metadata []byte) SeriesDecoder
	Put(encoder SeriesDecoder)
}

SeriesDecoderPool allows putting and getting SeriesDecoder.

func NewDecoderPool added in v0.3.0 

func NewDecoderPool(name string, size int, fn ParseInterval) SeriesDecoderPool

NewDecoderPool returns a SeriesDecoderPool which provides int-based xor decoders.

type SeriesEncoder 

type SeriesEncoder interface {
	// Append a data point
	Append(ts uint64, value []byte)
	// IsFull returns whether the encoded data reached its capacity
	IsFull() bool
	// Reset the underlying buffer
	Reset(key []byte, buffer BufferWriter)
	// Encode the time series data point to a binary
	Encode() error
	// StartTime indicates the first entry's time
	StartTime() uint64
}

SeriesEncoder encodes time series data point.

type SeriesEncoderPool 

type SeriesEncoderPool interface {
	Get(metadata []byte, buffer BufferWriter) SeriesEncoder
	Put(encoder SeriesEncoder)
}

SeriesEncoderPool allows putting and getting SeriesEncoder.

func NewEncoderPool added in v0.3.0 

func NewEncoderPool(name string, size int, fn ParseInterval) SeriesEncoderPool

NewEncoderPool returns a SeriesEncoderPool which provides int-based xor encoders.

type SeriesIterator 

type SeriesIterator interface {
	// Next scroll the cursor to the next
	Next() bool
	// Val returns the value of the current data point
	Val() []byte
	// Time returns the time of the current data point
	Time() uint64
	// Error might return an error indicates a decode failure
	Error() error
}

SeriesIterator iterates time series data.

type Uint64List added in v0.6.0 

type Uint64List struct {
	L []uint64
}

Uint64List is a list of uint64.

func GenerateUint64List added in v0.6.0 

func GenerateUint64List(size int) *Uint64List

GenerateUint64List generates a list of uint64 with the given size. The returned list may be from a pool and should be released after use.

type Writer added in v0.3.0 

type Writer struct {
	// contains filtered or unexported fields
}

Writer writes bits to an io.BufferWriter.

func NewWriter added in v0.3.0 

func NewWriter() *Writer

NewWriter create bit writer.

func (*Writer) Flush added in v0.3.0 

func (w *Writer) Flush()

Flush flushes the currently in-process byte.

func (*Writer) Reset added in v0.3.0 

func (w *Writer) Reset(buffer BufferWriter)

Reset writes to a new writer.

func (*Writer) WriteBits added in v0.3.0 

func (w *Writer) WriteBits(u uint64, numBits int)

WriteBits writes number of bits.

func (*Writer) WriteBool added in v0.3.0 

func (w *Writer) WriteBool(b bool)

WriteBool writes a boolean value true: 1 false: 0

func (*Writer) WriteByte added in v0.3.0 

func (w *Writer) WriteByte(b byte) error

WriteByte write a byte.

type XORDecoder 

type XORDecoder struct {
	// contains filtered or unexported fields
}

XORDecoder decodes buffer to uint64 values using xor compress.

func NewXORDecoder 

func NewXORDecoder(br *Reader) *XORDecoder

NewXORDecoder create zstdDecoder decompress buffer using xor.

func (*XORDecoder) Err 

func (d *XORDecoder) Err() error

Err returns error raised in Next().

func (*XORDecoder) Next 

func (d *XORDecoder) Next() bool

Next return if zstdDecoder has value in buffer using xor, do uncompress logic in next method, data format reference zstdEncoder format.

func (*XORDecoder) Reset 

func (d *XORDecoder) Reset()

Reset resets the underlying buffer to decode.

func (*XORDecoder) Value 

func (d *XORDecoder) Value() uint64

Value returns uint64 from buffer.

type XOREncoder 

type XOREncoder struct {
	// contains filtered or unexported fields
}

XOREncoder intends to compress uint64 data https://www.vldb.org/pvldb/vol8/p1816-teller.pdf

func NewXOREncoder 

func NewXOREncoder(bw *Writer) *XOREncoder

NewXOREncoder creates xor zstdEncoder for compressing uint64 data.

func (*XOREncoder) Write 

func (e *XOREncoder) Write(val uint64)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.