codec

package
v1.1.0-beta.0...-5637eaf Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2024 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source 
const IntHandleFlag = intFlag

IntHandleFlag is only used to encode int handle key.

View Source 
const (
	NilFlag byte = 0
)

First byte in the encoded value which specifies the encoding type.

Variables

This section is empty.

Functions

func ConvertByCollation 

func ConvertByCollation(raw []byte, tp *types.FieldType) []byte

ConvertByCollation converts these bytes according to its collation.

func ConvertByCollationStr 

func ConvertByCollationStr(str string, tp *types.FieldType) string

ConvertByCollationStr converts this string according to its collation.

func CutColumnID 

func CutColumnID(b []byte) (remain []byte, n int64, err error)

CutColumnID cuts the column ID from b. It will return the remains as byte slice and column ID

func CutOne 

func CutOne(b []byte) (data []byte, remain []byte, err error)

CutOne cuts the first encoded value from b. It will return the first encoded item and the remains as byte slice.

func Decode 

func Decode(b []byte, size int) ([]types.Datum, error)

Decode decodes values from a byte slice generated with EncodeKey or EncodeValue before. size is the size of decoded datum slice.

func DecodeAsDateTime 

func DecodeAsDateTime(b []byte, tp byte, loc *time.Location) (remain []byte, d types.Datum, err error)

DecodeAsDateTime decodes on datum from []byte of `KindMysqlTime`.

func DecodeAsFloat32 

func DecodeAsFloat32(b []byte, tp byte) (remain []byte, d types.Datum, err error)

DecodeAsFloat32 decodes value for mysql.TypeFloat

func DecodeBytes 

func DecodeBytes(b []byte, buf []byte) ([]byte, []byte, error)

DecodeBytes decodes bytes which is encoded by EncodeBytes before, returns the leftover bytes and decoded value if no error. `buf` is used to buffer data to avoid the cost of makeslice in decodeBytes when DecodeBytes is called by Decoder.DecodeOne.

func DecodeBytesDesc 

func DecodeBytesDesc(b []byte, buf []byte) ([]byte, []byte, error)

DecodeBytesDesc decodes bytes which is encoded by EncodeBytesDesc before, returns the leftover bytes and decoded value if no error.

func DecodeCmpUintToInt 

func DecodeCmpUintToInt(u uint64) int64

DecodeCmpUintToInt decodes the u that encoded by EncodeIntToCmpUint

func DecodeCompactBytes 

func DecodeCompactBytes(b []byte) ([]byte, []byte, error)

DecodeCompactBytes decodes bytes which is encoded by EncodeCompactBytes before.

func DecodeComparableUvarint 

func DecodeComparableUvarint(b []byte) ([]byte, uint64, error)

DecodeComparableUvarint decodes mem-comparable uvarint.

func DecodeComparableVarint 

func DecodeComparableVarint(b []byte) ([]byte, int64, error)

DecodeComparableVarint decodes mem-comparable varint.

func DecodeDecimal 

func DecodeDecimal(b []byte) ([]byte, *types.MyDecimal, int, int, error)

DecodeDecimal decodes bytes to decimal.

func DecodeFloat 

func DecodeFloat(b []byte) ([]byte, float64, error)

DecodeFloat decodes a float from a byte slice generated with EncodeFloat before.

func DecodeFloatDesc 

func DecodeFloatDesc(b []byte) ([]byte, float64, error)

DecodeFloatDesc decodes a float from a byte slice generated with EncodeFloatDesc before.

func DecodeInt 

func DecodeInt(b []byte) ([]byte, int64, error)

DecodeInt decodes value encoded by EncodeInt before. It returns the leftover un-decoded slice, decoded value if no error.

func DecodeIntDesc 

func DecodeIntDesc(b []byte) ([]byte, int64, error)

DecodeIntDesc decodes value encoded by EncodeInt before. It returns the leftover un-decoded slice, decoded value if no error.

func DecodeOne 

func DecodeOne(b []byte) (remain []byte, d types.Datum, err error)

DecodeOne decodes on datum from a byte slice generated with EncodeKey or EncodeValue.

func DecodeRange 

func DecodeRange(b []byte, size int, idxColumnTypes []byte, loc *time.Location) ([]types.Datum, []byte, error)

DecodeRange decodes the range values from a byte slice that generated by EncodeKey. It handles some special values like `MinNotNull` and `MaxValueDatum`. loc can be nil and only used in when the corresponding type is `mysql.TypeTimestamp`.

func DecodeUint 

func DecodeUint(b []byte) ([]byte, uint64, error)

DecodeUint decodes value encoded by EncodeUint before. It returns the leftover un-decoded slice, decoded value if no error.

func DecodeUintDesc 

func DecodeUintDesc(b []byte) ([]byte, uint64, error)

DecodeUintDesc decodes value encoded by EncodeInt before. It returns the leftover un-decoded slice, decoded value if no error.

func DecodeUvarint 

func DecodeUvarint(b []byte) ([]byte, uint64, error)

DecodeUvarint decodes value encoded by EncodeUvarint before. It returns the leftover un-decoded slice, decoded value if no error.

func DecodeVarint 

func DecodeVarint(b []byte) ([]byte, int64, error)

DecodeVarint decodes value encoded by EncodeVarint before. It returns the leftover un-decoded slice, decoded value if no error.

func EncodeBytes 

func EncodeBytes(b []byte, data []byte) []byte

EncodeBytes guarantees the encoded value is in ascending order for comparison, encoding with the following rule: 

[group1][marker1]...[groupN][markerN]
group is 8 bytes slice which is padding with 0.
marker is `0xFF - padding 0 count`

For example: 

[] -> [0, 0, 0, 0, 0, 0, 0, 0, 247]
[1, 2, 3] -> [1, 2, 3, 0, 0, 0, 0, 0, 250]
[1, 2, 3, 0] -> [1, 2, 3, 0, 0, 0, 0, 0, 251]
[1, 2, 3, 4, 5, 6, 7, 8] -> [1, 2, 3, 4, 5, 6, 7, 8, 255, 0, 0, 0, 0, 0, 0, 0, 0, 247]

Refer: https://github.com/facebook/mysql-5.6/wiki/MyRocks-record-format#memcomparable-format

func EncodeBytesDesc 

func EncodeBytesDesc(b []byte, data []byte) []byte

EncodeBytesDesc first encodes bytes using EncodeBytes, then bitwise reverses encoded value to guarantee the encoded value is in descending order for comparison.

func EncodeBytesExt 

func EncodeBytesExt(b []byte, data []byte, isRawKv bool) []byte

EncodeBytesExt is an extension of `EncodeBytes`, which will not encode for `isRawKv = true` but just append `data` to `b`.

func EncodeCompactBytes 

func EncodeCompactBytes(b []byte, data []byte) []byte

EncodeCompactBytes joins bytes with its length into a byte slice. It is more efficient in both space and time compare to EncodeBytes. Note that the encoded result is not memcomparable.

func EncodeComparableUvarint 

func EncodeComparableUvarint(b []byte, v uint64) []byte

EncodeComparableUvarint encodes uint64 into mem-comparable bytes.

func EncodeComparableVarint 

func EncodeComparableVarint(b []byte, v int64) []byte

EncodeComparableVarint encodes an int64 to a mem-comparable bytes.

func EncodeDecimal 

func EncodeDecimal(b []byte, dec *types.MyDecimal, precision, frac int) ([]byte, error)

EncodeDecimal encodes a decimal into a byte slice which can be sorted lexicographically later.

func EncodeFloat 

func EncodeFloat(b []byte, v float64) []byte

EncodeFloat encodes a float v into a byte slice which can be sorted lexicographically later. EncodeFloat guarantees that the encoded value is in ascending order for comparison.

func EncodeFloatDesc 

func EncodeFloatDesc(b []byte, v float64) []byte

EncodeFloatDesc encodes a float v into a byte slice which can be sorted lexicographically later. EncodeFloatDesc guarantees that the encoded value is in descending order for comparison.

func EncodeInt 

func EncodeInt(b []byte, v int64) []byte

EncodeInt appends the encoded value to slice b and returns the appended slice. EncodeInt guarantees that the encoded value is in ascending order for comparison.

func EncodeIntDesc 

func EncodeIntDesc(b []byte, v int64) []byte

EncodeIntDesc appends the encoded value to slice b and returns the appended slice. EncodeIntDesc guarantees that the encoded value is in descending order for comparison.

func EncodeIntToCmpUint 

func EncodeIntToCmpUint(v int64) uint64

EncodeIntToCmpUint make int v to comparable uint type

func EncodeKey 

func EncodeKey(loc *time.Location, b []byte, v ...types.Datum) ([]byte, error)

EncodeKey appends the encoded values to byte slice b, returns the appended slice. It guarantees the encoded value is in ascending order for comparison. For decimal type, datum must set datum's length and frac.

func EncodeMySQLTime 

func EncodeMySQLTime(loc *time.Location, t types.Time, tp byte, b []byte) (_ []byte, err error)

EncodeMySQLTime encodes datum of `KindMysqlTime` to []byte.

func EncodeUint 

func EncodeUint(b []byte, v uint64) []byte

EncodeUint appends the encoded value to slice b and returns the appended slice. EncodeUint guarantees that the encoded value is in ascending order for comparison.

func EncodeUintDesc 

func EncodeUintDesc(b []byte, v uint64) []byte

EncodeUintDesc appends the encoded value to slice b and returns the appended slice. EncodeUintDesc guarantees that the encoded value is in descending order for comparison.

func EncodeUvarint 

func EncodeUvarint(b []byte, v uint64) []byte

EncodeUvarint appends the encoded value to slice b and returns the appended slice. Note that the encoded result is not memcomparable.

func EncodeValue 

func EncodeValue(loc *time.Location, b []byte, v ...types.Datum) ([]byte, error)

EncodeValue appends the encoded values to byte slice b, returning the appended slice. It does not guarantee the order for comparison.

func EncodeVarint 

func EncodeVarint(b []byte, v int64) []byte

EncodeVarint appends the encoded value to slice b and returns the appended slice. Note that the encoded result is not memcomparable.

func EncodedBytesLength 

func EncodedBytesLength(dataLen int) int

EncodedBytesLength returns the length of data after encoded

func EqualChunkRow 

func EqualChunkRow(typeCtx types.Context,
	row1 chunk.Row, allTypes1 []*types.FieldType, colIdx1 []int,
	row2 chunk.Row, allTypes2 []*types.FieldType, colIdx2 []int,
) (bool, error)

EqualChunkRow returns a boolean reporting whether row1 and row2 with their types and column index are logically equal.

func EstimateValueSize 

func EstimateValueSize(typeCtx types.Context, val types.Datum) (int, error)

EstimateValueSize uses to estimate the value size of the encoded values.

func Hash64 

func Hash64(h base.Hasher, d *types.Datum)

Hash64 is for datum hash64 calculation.

func HashChunkColumns 

func HashChunkColumns(typeCtx types.Context, h []hash.Hash64, chk *chunk.Chunk, tp *types.FieldType, colIdx int, buf []byte, isNull []bool) (err error)

HashChunkColumns writes the encoded value of each row's column, which of index `colIdx`, to h.

func HashChunkRow 

func HashChunkRow(typeCtx types.Context, w io.Writer, row chunk.Row, allTypes []*types.FieldType, colIdx []int, buf []byte) (err error)

HashChunkRow writes the encoded values to w. If two rows are logically equal, it will generate the same bytes.

func HashChunkSelected 

func HashChunkSelected(typeCtx types.Context, h []hash.Hash64, chk *chunk.Chunk, tp *types.FieldType, colIdx int, buf []byte,
	isNull, sel []bool, ignoreNull bool) (err error)

HashChunkSelected writes the encoded value of selected row's column, which of index `colIdx`, to h. sel indicates which rows are selected. If it is nil, all rows are selected.

func HashCode 

func HashCode(b []byte, d types.Datum) []byte

HashCode encodes a Datum into a unique byte slice. It is mostly the same as EncodeValue, but it doesn't contain truncation or verification logic in order to make the encoding lossless.

func HashGroupKey 

func HashGroupKey(loc *time.Location, n int, col *chunk.Column, buf [][]byte, ft *types.FieldType) ([][]byte, error)

HashGroupKey encodes each row of this column and append encoded data into buf. Only use in the aggregate executor.

func SerializeKeys 

func SerializeKeys(typeCtx types.Context, chk *chunk.Chunk, tp *types.FieldType, colIdx int, usedRows []int, filterVector []bool, nullVector []bool, serializeMode SerializeMode, serializedKeysVector [][]byte) (err error)

SerializeKeys is used in join

func SetRawValues 

func SetRawValues(data []byte, values []types.Datum) error

SetRawValues set raw datum values from a row data.

Types

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder is used to decode value to chunk.

func NewDecoder 

func NewDecoder(chk *chunk.Chunk, timezone *time.Location) *Decoder

NewDecoder creates a Decoder.

func (*Decoder) DecodeOne 

func (decoder *Decoder) DecodeOne(b []byte, colIdx int, ft *types.FieldType) (remain []byte, err error)

DecodeOne decodes one value to chunk and returns the remained bytes.

type SerializeMode 

type SerializeMode int

SerializeMode is for some special cases during serialize key 

const (
	// Normal means serialize in the normal way
	Normal SerializeMode = iota
	// NeedSignFlag when serialize integer column, if the join key is <signed, signed> or <unsigned, unsigned>,
	// the unsigned flag can be ignored, if the join key is <unsigned, signed> or <signed, unsigned>
	// the unsigned flag can not be ignored, if the unsigned flag can not be ignored, the key can not be inlined
	NeedSignFlag
	// KeepVarColumnLength when serialize var-length column, whether record the column length or not. If the join key only contains one var-length
	// column, and the key is not inlined, then no need to record the column length, otherwise, always need to record the column length
	KeepVarColumnLength
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.