encoding

package
v0.0.0-...-5695764 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 24, 2016 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// IntMin is chosen such that the range of int tags does not overlap the
	// ascii character set that is frequently used in testing.
	IntMin = 0x80

	// IntMax is the maximum int tag value.
	IntMax = 0xfd
)
View Source 
const (
	// EncodedDurationMaxLen is the largest number of bytes used when encoding a
	// Duration.
	EncodedDurationMaxLen = 1 + 3*binary.MaxVarintLen64 // 3 varints are encoded.
	// BytesDescMarker is exported for testing.
	BytesDescMarker = bytesDescMarker
)
View Source 
const NoColumnID uint32 = 0

NoColumnID is a sentinel for the EncodeFooValue methods representing an invalid column id.

View Source 
const NonsortingUvarintMaxLen = 10

NonsortingUvarintMaxLen is the maximum length of an EncodeNonsortingUvarint encoded value.

View Source 
const NonsortingVarintMaxLen = binary.MaxVarintLen64

NonsortingVarintMaxLen is the maximum length of an EncodeNonsortingVarint encoded value.

Variables

This section is empty.

Functions

func DecodeBoolValue 

func DecodeBoolValue(buf []byte) (remaining []byte, b bool, err error)

DecodeBoolValue decodes a value encoded by EncodeBoolValue.

func DecodeBytesAscending 

func DecodeBytesAscending(b []byte, r []byte) ([]byte, []byte, error)

DecodeBytesAscending decodes a []byte value from the input buffer which was encoded using EncodeBytesAscending. The decoded bytes are appended to r. The remainder of the input buffer and the decoded []byte are returned.

func DecodeBytesDescending 

func DecodeBytesDescending(b []byte, r []byte) ([]byte, []byte, error)

DecodeBytesDescending decodes a []byte value from the input buffer which was encoded using EncodeBytesDescending. The decoded bytes are appended to r. The remainder of the input buffer and the decoded []byte are returned.

func DecodeBytesValue 

func DecodeBytesValue(b []byte) (remaining []byte, data []byte, err error)

DecodeBytesValue decodes a value encoded by EncodeBytesValue.

func DecodeDecimalAscending 

func DecodeDecimalAscending(buf []byte, tmp []byte) ([]byte, *inf.Dec, error)

DecodeDecimalAscending returns the remaining byte slice after decoding and the decoded decimal from buf.

func DecodeDecimalDescending 

func DecodeDecimalDescending(buf []byte, tmp []byte) ([]byte, *inf.Dec, error)

DecodeDecimalDescending decodes decimals encoded with EncodeDecimalDescending.

func DecodeDecimalValue 

func DecodeDecimalValue(b []byte) (remaining []byte, d *inf.Dec, err error)

DecodeDecimalValue decodes a value encoded by EncodeDecimalValue.

func DecodeDurationAscending 

func DecodeDurationAscending(b []byte) ([]byte, duration.Duration, error)

DecodeDurationAscending decodes a duration.Duration value which was encoded using EncodeDurationAscending. The remainder of the input buffer and the decoded duration.Duration are returned.

func DecodeDurationDescending 

func DecodeDurationDescending(b []byte) ([]byte, duration.Duration, error)

DecodeDurationDescending is the descending version of DecodeDurationAscending.

func DecodeDurationValue 

func DecodeDurationValue(b []byte) (remaining []byte, d duration.Duration, err error)

DecodeDurationValue decodes a value encoded by EncodeDurationValue.

func DecodeFloatAscending 

func DecodeFloatAscending(buf []byte) ([]byte, float64, error)

DecodeFloatAscending returns the remaining byte slice after decoding and the decoded float64 from buf.

func DecodeFloatDescending 

func DecodeFloatDescending(buf []byte) ([]byte, float64, error)

DecodeFloatDescending decodes floats encoded with EncodeFloatDescending.

func DecodeFloatValue 

func DecodeFloatValue(b []byte) (remaining []byte, f float64, err error)

DecodeFloatValue decodes a value encoded by EncodeFloatValue.

func DecodeIfNotNull 

func DecodeIfNotNull(b []byte) ([]byte, bool)

DecodeIfNotNull decodes a not-NULL value from the input buffer. If the input buffer contains a not-NULL marker at the start of the buffer then it is removed from the buffer and true is returned for the second result. Otherwise, the buffer is returned unchanged and false is returned for the second result. Note that the not-NULL marker is identical to the empty string encoding, so do not use this routine where it is necessary to distinguish not-NULL from the empty string. This function handles both ascendingly and descendingly encoded NULLs.

func DecodeIfNull 

func DecodeIfNull(b []byte) ([]byte, bool)

DecodeIfNull decodes a NULL value from the input buffer. If the input buffer contains a null at the start of the buffer then it is removed from the buffer and true is returned for the second result. Otherwise, the buffer is returned unchanged and false is returned for the second result. Since the NULL value encoding is guaranteed to never occur as the prefix for the EncodeVarint, EncodeFloat, EncodeBytes and EncodeString encodings, it is safe to call DecodeIfNull on their encoded values. This function handles both ascendingly and descendingly encoded NULLs.

func DecodeIntValue 

func DecodeIntValue(b []byte) (remaining []byte, i int64, err error)

DecodeIntValue decodes a value encoded by EncodeIntValue.

func DecodeNonsortingDecimal 

func DecodeNonsortingDecimal(buf []byte, tmp []byte) (*inf.Dec, error)

DecodeNonsortingDecimal returns the decoded decimal from buf encoded with EncodeNonsortingDecimal. buf is assumed to contain only the encoded decimal, as the function does not know from the encoding itself what the length of the encoded value is.

func DecodeNonsortingUvarint 

func DecodeNonsortingUvarint(buf []byte) (remaining []byte, length int, value uint64, err error)

DecodeNonsortingUvarint decodes a value encoded by EncodeNonsortingUvarint. It returns the length of the encoded varint and value.

func DecodeNonsortingVarint 

func DecodeNonsortingVarint(b []byte) (remaining []byte, length int, value int64, err error)

DecodeNonsortingVarint decodes a value encoded by EncodeNonsortingVarint. It returns the length of the encoded varint and value.

func DecodeTimeAscending 

func DecodeTimeAscending(b []byte) ([]byte, time.Time, error)

DecodeTimeAscending decodes a time.Time value which was encoded using EncodeTime. The remainder of the input buffer and the decoded time.Time are returned.

func DecodeTimeDescending 

func DecodeTimeDescending(b []byte) ([]byte, time.Time, error)

DecodeTimeDescending is the descending version of DecodeTimeAscending.

func DecodeTimeValue 

func DecodeTimeValue(b []byte) (remaining []byte, t time.Time, err error)

DecodeTimeValue decodes a value encoded by EncodeTimeValue.

func DecodeUint32Ascending 

func DecodeUint32Ascending(b []byte) ([]byte, uint32, error)

DecodeUint32Ascending decodes a uint32 from the input buffer, treating the input as a big-endian 4 byte uint32 representation. The remainder of the input buffer and the decoded uint32 are returned.

func DecodeUint32Descending 

func DecodeUint32Descending(b []byte) ([]byte, uint32, error)

DecodeUint32Descending decodes a uint32 value which was encoded using EncodeUint32Descending.

func DecodeUint64Ascending 

func DecodeUint64Ascending(b []byte) ([]byte, uint64, error)

DecodeUint64Ascending decodes a uint64 from the input buffer, treating the input as a big-endian 8 byte uint64 representation. The remainder of the input buffer and the decoded uint64 are returned.

func DecodeUint64Descending 

func DecodeUint64Descending(b []byte) ([]byte, uint64, error)

DecodeUint64Descending decodes a uint64 value which was encoded using EncodeUint64Descending.

func DecodeUnsafeStringAscending 

func DecodeUnsafeStringAscending(b []byte, r []byte) ([]byte, string, error)

DecodeUnsafeStringAscending decodes a string value from the input buffer which was encoded using EncodeString or EncodeBytes. The r []byte is used as a temporary buffer in order to avoid memory allocations. The remainder of the input buffer and the decoded string are returned. Note that the returned string may share storage with the input buffer.

func DecodeUnsafeStringDescending 

func DecodeUnsafeStringDescending(b []byte, r []byte) ([]byte, string, error)

DecodeUnsafeStringDescending decodes a string value from the input buffer which was encoded using EncodeStringDescending or EncodeBytesDescending. The r []byte is used as a temporary buffer in order to avoid memory allocations. The remainder of the input buffer and the decoded string are returned. Note that the returned string may share storage with the input buffer.

func DecodeUvarintAscending 

func DecodeUvarintAscending(b []byte) ([]byte, uint64, error)

DecodeUvarintAscending decodes a varint encoded uint64 from the input buffer. The remainder of the input buffer and the decoded uint64 are returned.

func DecodeUvarintDescending 

func DecodeUvarintDescending(b []byte) ([]byte, uint64, error)

DecodeUvarintDescending decodes a uint64 value which was encoded using EncodeUvarintDescending.

func DecodeVarintAscending 

func DecodeVarintAscending(b []byte) ([]byte, int64, error)

DecodeVarintAscending decodes a value encoded by EncodeVaringAscending.

func DecodeVarintDescending 

func DecodeVarintDescending(b []byte) ([]byte, int64, error)

DecodeVarintDescending decodes a uint64 value which was encoded using EncodeVarintDescending.

func EncLenUvarintAscending 

func EncLenUvarintAscending(v uint64) int

EncLenUvarintAscending returns the encoding length for EncodeUvarintAscending without actually encoding.

func EncLenUvarintDescending 

func EncLenUvarintDescending(v uint64) int

EncLenUvarintDescending returns the encoding length for EncodeUvarintDescending without actually encoding.

func EncodeBoolValue 

func EncodeBoolValue(appendTo []byte, colID uint32, b bool) []byte

EncodeBoolValue encodes a bool value, appends it to the supplied buffer, and returns the final buffer.

func EncodeBytesAscending 

func EncodeBytesAscending(b []byte, data []byte) []byte

EncodeBytesAscending encodes the []byte value using an escape-based encoding. The encoded value is terminated with the sequence "\x00\x01" which is guaranteed to not occur elsewhere in the encoded value. The encoded bytes are append to the supplied buffer and the resulting buffer is returned.

func EncodeBytesDescending 

func EncodeBytesDescending(b []byte, data []byte) []byte

EncodeBytesDescending encodes the []byte value using an escape-based encoding and then inverts (ones complement) the result so that it sorts in reverse order, from larger to smaller lexicographically.

func EncodeBytesValue 

func EncodeBytesValue(appendTo []byte, colID uint32, data []byte) []byte

EncodeBytesValue encodes a byte array value, appends it to the supplied buffer, and returns the final buffer.

func EncodeDecimalAscending 

func EncodeDecimalAscending(appendTo []byte, d *inf.Dec) []byte

EncodeDecimalAscending returns the resulting byte slice with the encoded decimal appended to the given buffer.

Values are classified as large, medium, or small according to the value of E. If E is 11 or more, the value is large. For E between 0 and 10, the value is medium. For E less than zero, the value is small.

Large positive values are encoded as a single byte 0x34 followed by E as a varint and then M. Medium positive values are a single byte of 0x29+E followed by M. Small positive values are encoded as a single byte 0x28 followed by a descending varint encoding for -E followed by M.

Small negative values are encoded as a single byte 0x26 followed by -E as a varint and then the ones-complement of M. Medium negative values are encoded as a byte 0x25-E followed by the ones-complement of M. Large negative values consist of the single byte 0x1a followed by a descending varint encoding of E followed by the ones-complement of M.

func EncodeDecimalDescending 

func EncodeDecimalDescending(appendTo []byte, d *inf.Dec) []byte

EncodeDecimalDescending is the descending version of EncodeDecimalAscending.

func EncodeDecimalValue 

func EncodeDecimalValue(appendTo []byte, colID uint32, d *inf.Dec) []byte

EncodeDecimalValue encodes an inf.Dec value, appends it to the supplied buffer, and returns the final buffer.

func EncodeDurationAscending 

func EncodeDurationAscending(b []byte, d duration.Duration) ([]byte, error)

EncodeDurationAscending encodes a duration.Duration value, appends it to the supplied buffer, and returns the final buffer. The encoding is guaranteed to be ordered such that if t1.Compare(t2) < 0 (or = 0 or > 0) then bytes.Compare will order them the same way after encoding.

func EncodeDurationDescending 

func EncodeDurationDescending(b []byte, d duration.Duration) ([]byte, error)

EncodeDurationDescending is the descending version of EncodeDurationAscending.

func EncodeDurationValue 

func EncodeDurationValue(appendTo []byte, colID uint32, d duration.Duration) []byte

EncodeDurationValue encodes a duration.Duration value, appends it to the supplied buffer, and returns the final buffer.

func EncodeFloatAscending 

func EncodeFloatAscending(b []byte, f float64) []byte

EncodeFloatAscending returns the resulting byte slice with the encoded float64 appended to b. The encoded format for a float64 value f is, for positive f, the encoding of the 64 bits (in IEEE 754 format) re-interpreted as an int64 and encoded using EncodeUint64Ascending. For negative f, we keep the sign bit and invert all other bits, encoding this value using EncodeUint64Descending. This approach was inspired by in github.com/google/orderedcode/orderedcode.go.

One of five single-byte prefix tags are appended to the front of the encoding. These tags enforce logical ordering of keys for both ascending and descending encoding directions. The tags split the encoded floats into five categories: - NaN for an ascending encoding direction - Negative valued floats - Zero (positive and negative) - Positive valued floats - NaN for a descending encoding direction This ordering ensures that NaNs are always sorted first in either encoding direction, and that after them a logical ordering is followed.

func EncodeFloatDescending 

func EncodeFloatDescending(b []byte, f float64) []byte

EncodeFloatDescending is the descending version of EncodeFloatAscending.

func EncodeFloatValue 

func EncodeFloatValue(appendTo []byte, colID uint32, f float64) []byte

EncodeFloatValue encodes a float value, appends it to the supplied buffer, and returns the final buffer.

func EncodeIntValue 

func EncodeIntValue(appendTo []byte, colID uint32, i int64) []byte

EncodeIntValue encodes an int value, appends it to the supplied buffer, and returns the final buffer.

func EncodeNonsortingDecimal 

func EncodeNonsortingDecimal(b []byte, d *inf.Dec) []byte

EncodeNonsortingDecimal returns the resulting byte slice with the encoded decimal appended to b. The encoding is limited compared to standard encodings in this package in that

  • It will not sort lexicographically
  • It does not encode its length or terminate itself, so decoding functions must be provided the exact encoded bytes

The encoding assumes that any number can be written as ±0.xyz... * 10^exp, where xyz is a digit string, x != 0, and the last decimal in xyz is also not 0.

The encoding uses its first byte to split decimals into 7 distinct ordered groups (no NaN or Infinity support yet). The groups can be seen in encoding.go's const definition. Following this, the absolute value of the exponent of the decimal (as defined above) is encoded as an unsigned varint. Second, the absolute value of the digit string is added as a big-endian byte slice.

All together, the encoding looks like: 

<marker><uvarint exponent><big-endian encoded big.Int>.

The markers are shared with the sorting decimal encoding as follows: 

decimalNaN              -> decimalNaN
decimalNegativeInfinity -> decimalNegativeInfinity
decimalNegLarge         -> decimalNegValPosExp
decimalNegMedium        -> decimalNegValZeroExp
decimalNegSmall         -> decimalNegValNegExp
decimalZero             -> decimalZero
decimalPosSmall         -> decimalPosValNegExp
decimalPosMedium        -> decimalPosValZeroExp
decimalPosLarge         -> decimalPosValPosExp
decimalInfinity         -> decimalInfinity
decimalNaNDesc          -> decimalNaNDesc

func EncodeNonsortingUvarint 

func EncodeNonsortingUvarint(appendTo []byte, x uint64) []byte

EncodeNonsortingUvarint encodes a uint64, appends it to the supplied buffer, and returns the final buffer. The encoding used is similar to encoding/binary, but with the most significant bits first:

  • Unsigned integers are serialized 7 bits at a time, starting with the most significant bits.
  • The most significant bit (msb) in each output byte indicates if there is a continuation byte (msb = 1).

func EncodeNonsortingVarint 

func EncodeNonsortingVarint(appendTo []byte, x int64) []byte

EncodeNonsortingVarint encodes an int value using encoding/binary, appends it to the supplied buffer, and returns the final buffer.

func EncodeNotNullAscending 

func EncodeNotNullAscending(b []byte) []byte

EncodeNotNullAscending encodes a value that is larger than the NULL marker encoded by EncodeNull but less than any encoded value returned by EncodeVarint, EncodeFloat, EncodeBytes or EncodeString.

func EncodeNotNullDescending 

func EncodeNotNullDescending(b []byte) []byte

EncodeNotNullDescending is the descending equivalent of EncodeNotNullAscending.

func EncodeNullAscending 

func EncodeNullAscending(b []byte) []byte

EncodeNullAscending encodes a NULL value. The encodes bytes are appended to the supplied buffer and the final buffer is returned. The encoded value for a NULL is guaranteed to not be a prefix for the EncodeVarint, EncodeFloat, EncodeBytes and EncodeString encodings.

func EncodeNullDescending 

func EncodeNullDescending(b []byte) []byte

EncodeNullDescending is the descending equivalent of EncodeNullAscending.

func EncodeNullValue 

func EncodeNullValue(appendTo []byte, colID uint32) []byte

EncodeNullValue encodes a null value, appends it to the supplied buffer, and returns the final buffer.

func EncodeStringAscending 

func EncodeStringAscending(b []byte, s string) []byte

EncodeStringAscending encodes the string value using an escape-based encoding. See EncodeBytes for details. The encoded bytes are append to the supplied buffer and the resulting buffer is returned.

func EncodeStringDescending 

func EncodeStringDescending(b []byte, s string) []byte

EncodeStringDescending is the descending version of EncodeStringAscending.

func EncodeTimeAscending 

func EncodeTimeAscending(b []byte, t time.Time) []byte

EncodeTimeAscending encodes a time value, appends it to the supplied buffer, and returns the final buffer. The encoding is guaranteed to be ordered Such that if t1.Before(t2) then after EncodeTime(b1, t1), and EncodeTime(b2, t1), Compare(b1, b2) < 0. The time zone offset not included in the encoding.

func EncodeTimeDescending 

func EncodeTimeDescending(b []byte, t time.Time) []byte

EncodeTimeDescending is the descending version of EncodeTimeAscending.

func EncodeTimeValue 

func EncodeTimeValue(appendTo []byte, colID uint32, t time.Time) []byte

EncodeTimeValue encodes a time.Time value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUint32Ascending 

func EncodeUint32Ascending(b []byte, v uint32) []byte

EncodeUint32Ascending encodes the uint32 value using a big-endian 8 byte representation. The bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUint32Descending 

func EncodeUint32Descending(b []byte, v uint32) []byte

EncodeUint32Descending encodes the uint32 value so that it sorts in reverse order, from largest to smallest.

func EncodeUint64Ascending 

func EncodeUint64Ascending(b []byte, v uint64) []byte

EncodeUint64Ascending encodes the uint64 value using a big-endian 8 byte representation. The bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUint64Descending 

func EncodeUint64Descending(b []byte, v uint64) []byte

EncodeUint64Descending encodes the uint64 value so that it sorts in reverse order, from largest to smallest.

func EncodeUvarintAscending 

func EncodeUvarintAscending(b []byte, v uint64) []byte

EncodeUvarintAscending encodes the uint64 value using a variable length (length-prefixed) representation. The length is encoded as a single byte indicating the number of encoded bytes (-8) to follow. See EncodeVarintAscending for rationale. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUvarintDescending 

func EncodeUvarintDescending(b []byte, v uint64) []byte

EncodeUvarintDescending encodes the uint64 value so that it sorts in reverse order, from largest to smallest.

func EncodeVarintAscending 

func EncodeVarintAscending(b []byte, v int64) []byte

EncodeVarintAscending encodes the int64 value using a variable length (length-prefixed) representation. The length is encoded as a single byte. If the value to be encoded is negative the length is encoded as 8-numBytes. If the value is positive it is encoded as 8+numBytes. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeVarintDescending 

func EncodeVarintDescending(b []byte, v int64) []byte

EncodeVarintDescending encodes the int64 value so that it sorts in reverse order, from largest to smallest.

func GetMultiVarintLen 

func GetMultiVarintLen(b []byte, num int) (int, error)

GetMultiVarintLen find the length of <num> encoded varints that follow a 1-byte tag.

func PeekLength 

func PeekLength(b []byte) (int, error)

PeekLength returns the length of the encoded value at the start of b. Note: if this function succeeds, it's not a guarantee that decoding the value will succeed.

func PeekLengthNonsortingUvarint 

func PeekLengthNonsortingUvarint(buf []byte) int

PeekLengthNonsortingUvarint returns the length of the value that starts at the beginning of buf and was encoded by EncodeNonsortingUvarint.

func PeekValueLength 

func PeekValueLength(b []byte) (typeOffset int, length int, err error)

PeekValueLength returns the length of the encoded value at the start of b. Note: If this function succeeds, it's not a guarantee that decoding the value will succeed.

`b` can point either at beginning of the "full tag" with the column id, or it can point to the beginning of the type part of the tag, as indicated by the `typeOffset` returned by this or DecodeValueTag.

The length returned is the full length of the encoded value, including the entire tag.

func PrettyPrintValue 

func PrettyPrintValue(b []byte, sep string) string

PrettyPrintValue returns the string representation of all contiguous decodable values in the provided byte slice, separated by a provided separator.

func UpperBoundNonsortingDecimalSize 

func UpperBoundNonsortingDecimalSize(d *inf.Dec) int

UpperBoundNonsortingDecimalSize returns the upper bound number of bytes that the decimal will need for the non-sorting encoding.

Types

type Direction 

type Direction int

Direction for ordering results. 

const (
	Ascending Direction
	Descending
)

Direction values.

func (Direction) Reverse 

func (d Direction) Reverse() Direction

Reverse returns the opposite direction.

type Type 

type Type int

Type represents the type of a value encoded by Encode{Null,NotNull,Varint,Uvarint,Float,Bytes}. 

const (
	Unknown Type = iota
	Null
	NotNull
	Int
	Float
	Decimal
	Bytes
	BytesDesc // Bytes encoded descendingly
	Time
	Duration
	True
	False

	SentinelType Type = 15 // Used in the Value encoding.
)

Type values. TODO(dan): Make this into a proto enum.

func DecodeValueTag 

func DecodeValueTag(b []byte) (typeOffset int, dataOffset int, colID uint32, typ Type, err error)

DecodeValueTag decodes a value encoded by encodeValueTag, used as a prefix in each of the other EncodeFooValue methods.

The tag is structured such that the encoded column id can be dropped from the front by removing the first `typeOffset` bytes. DecodeValueTag, PeekValueLength and each of the DecodeFooValue methods will still work as expected with `b[typeOffset:]`. (Except, obviously, the column id is no longer encoded so if this suffix is passed back to DecodeValueTag, the returned colID should be discarded.)

Concretely: 

b := ...
typeOffset, _, colID, typ, err := DecodeValueTag(b)
_, _, _, typ, err := DecodeValueTag(b[typeOffset:])

will return the same typ and err and 

DecodeFooValue(b)
DecodeFooValue(b[typeOffset:])

will return the same thing. PeekValueLength works as expected with either of `b` or `b[typeOffset:]`.

func PeekType 

func PeekType(b []byte) Type

PeekType peeks at the type of the value encoded at the start of b.

func (Type) String 

func (i Type) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.