encoding

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 28, 2023 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// IntMin is chosen such that the range of int tags does not overlap the
	// ascii character set that is frequently used in testing.
	IntMin = 0x80 // 128

	// IntMax is the maximum int tag value.
	IntMax = 0xfd // 253

)
View Source 
const MaxVarintLen = 9

MaxVarintLen is the maximum length of a value encoded using any of: - EncodeVarintAscending - EncodeVarintDescending - EncodeUvarintAscending - EncodeUvarintDescending

Variables

This section is empty.

Functions

func AddJSONPathTerminator 

func AddJSONPathTerminator(b []byte) []byte

AddJSONPathTerminator adds a json path terminator to a byte array.

func DecodeDecimalAscending 

func DecodeDecimalAscending(buf []byte, tmp []byte) ([]byte, apd.Decimal, error)

DecodeDecimalAscending returns the remaining byte slice after decoding and the decoded decimal from buf.

func DecodeDecimalDescending 

func DecodeDecimalDescending(buf []byte, tmp []byte) ([]byte, apd.Decimal, error)

DecodeDecimalDescending decodes decimals encoded with EncodeDecimalDescending.

func DecodeFloatAscending 

func DecodeFloatAscending(buf []byte) ([]byte, float64, error)

DecodeFloatAscending returns the remaining byte slice after decoding and the decoded float64 from buf.

func DecodeFloatDescending 

func DecodeFloatDescending(buf []byte) ([]byte, float64, error)

DecodeFloatDescending decodes floats encoded with EncodeFloatDescending.

func DecodeIntoNonsortingDecimal 

func DecodeIntoNonsortingDecimal(dec *apd.Decimal, buf []byte, tmp []byte) error

DecodeIntoNonsortingDecimal is like DecodeNonsortingDecimal, but it operates on the passed-in *apd.Decimal instead of producing a new one.

func DecodeNonsortingDecimal 

func DecodeNonsortingDecimal(buf []byte, tmp []byte) (apd.Decimal, error)

DecodeNonsortingDecimal returns the decoded decimal from buf encoded with EncodeNonsortingDecimal. buf is assumed to contain only the encoded decimal, as the function does not know from the encoding itself what the length of the encoded value is.

func DecodeNonsortingStdlibUvarint 

func DecodeNonsortingStdlibUvarint(
	buf []byte,
) (remaining []byte, length int, value uint64, err error)

DecodeNonsortingStdlibUvarint decodes a value encoded with binary.PutUvarint. It returns the length of the encoded varint and value.

func DecodeNonsortingStdlibVarint 

func DecodeNonsortingStdlibVarint(b []byte) (remaining []byte, length int, value int64, err error)

DecodeNonsortingStdlibVarint decodes a value encoded by EncodeNonsortingVarint. It returns the length of the encoded varint and value.

func DecodeNonsortingUvarint 

func DecodeNonsortingUvarint(buf []byte) (remaining []byte, length int, value uint64, err error)

DecodeNonsortingUvarint decodes a value encoded by EncodeNonsortingUvarint. It returns the length of the encoded varint and value.

func DecodeUint32Ascending 

func DecodeUint32Ascending(b []byte) ([]byte, uint32, error)

DecodeUint32Ascending decodes a uint32 from the input buffer, treating the input as a big-endian 4 byte uint32 representation. The remainder of the input buffer and the decoded uint32 are returned.

func DecodeUint64Ascending 

func DecodeUint64Ascending(b []byte) ([]byte, uint64, error)

DecodeUint64Ascending decodes a uint64 from the input buffer, treating the input as a big-endian 8 byte uint64 representation. The remainder of the input buffer and the decoded uint64 are returned.

func DecodeUntaggedDecimalValue 

func DecodeUntaggedDecimalValue(b []byte) (remaining []byte, d apd.Decimal, err error)

DecodeUntaggedDecimalValue decodes a value encoded by EncodeUntaggedDecimalValue.

func DecodeUvarintAscending 

func DecodeUvarintAscending(b []byte) ([]byte, uint64, error)

DecodeUvarintAscending decodes a varint encoded uint64 from the input buffer. The remainder of the input buffer and the decoded uint64 are returned.

func DecodeUvarintDescending 

func DecodeUvarintDescending(b []byte) ([]byte, uint64, error)

DecodeUvarintDescending decodes a uint64 value which was encoded using EncodeUvarintDescending.

func EncLenUvarintAscending 

func EncLenUvarintAscending(v uint64) int

EncLenUvarintAscending returns the encoding length for EncodeUvarintAscending without actually encoding.

func EncLenUvarintDescending 

func EncLenUvarintDescending(v uint64) int

EncLenUvarintDescending returns the encoding length for EncodeUvarintDescending without actually encoding.

func EncodeArrayAscending 

func EncodeArrayAscending(b []byte) []byte

EncodeArrayAscending encodes a value used to signify membership of an array for JSON objects.

func EncodeDecimalAscending 

func EncodeDecimalAscending(appendTo []byte, d *apd.Decimal) []byte

EncodeDecimalAscending returns the resulting byte slice with the encoded decimal appended to the given buffer.

Values are classified as large, medium, or small according to the value of E. If E is 11 or more, the value is large. For E between 0 and 10, the value is medium. For E less than zero, the value is small.

Large positive values are encoded as a single byte 0x34 followed by E as a varint and then M. Medium positive values are a single byte of 0x29+E followed by M. Small positive values are encoded as a single byte 0x28 followed by a descending varint encoding for -E followed by M.

Small negative values are encoded as a single byte 0x26 followed by -E as a varint and then the ones-complement of M. Medium negative values are encoded as a byte 0x25-E followed by the ones-complement of M. Large negative values consist of the single byte 0x1a followed by a descending varint encoding of E followed by the ones-complement of M.

func EncodeDecimalDescending 

func EncodeDecimalDescending(appendTo []byte, d *apd.Decimal) []byte

EncodeDecimalDescending is the descending version of EncodeDecimalAscending.

func EncodeFalseAscending 

func EncodeFalseAscending(b []byte) []byte

EncodeFalseAscending encodes the boolean value false for use with JSON inverted indexes.

func EncodeFloatAscending 

func EncodeFloatAscending(b []byte, f float64) []byte

EncodeFloatAscending returns the resulting byte slice with the encoded float64 appended to b. The encoded format for a float64 value f is, for positive f, the encoding of the 64 bits (in IEEE 754 format) re-interpreted as an int64 and encoded using EncodeUint64Ascending. For negative f, we keep the sign bit and invert all other bits, encoding this value using EncodeUint64Descending. This approach was inspired by in github.com/google/orderedcode/orderedcode.go.

One of five single-byte prefix tags are appended to the front of the encoding. These tags enforce logical ordering of keys for both ascending and descending encoding directions. The tags split the encoded floats into five categories: - NaN for an ascending encoding direction - Negative valued floats - Zero (positive and negative) - Positive valued floats - NaN for a descending encoding direction This ordering ensures that NaNs are always sorted first in either encoding direction, and that after them a logical ordering is followed.

func EncodeFloatDescending 

func EncodeFloatDescending(b []byte, f float64) []byte

EncodeFloatDescending is the descending version of EncodeFloatAscending.

func EncodeJSONAscending 

func EncodeJSONAscending(b []byte) []byte

EncodeJSONAscending encodes a JSON Type. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeJSONEmptyArray 

func EncodeJSONEmptyArray(b []byte) []byte

EncodeJSONEmptyArray returns a byte array b with a byte to signify an empty JSON array.

func EncodeJSONEmptyObject 

func EncodeJSONEmptyObject(b []byte) []byte

EncodeJSONEmptyObject returns a byte array b with a byte to signify an empty JSON object.

func EncodeJSONKeyStringAscending 

func EncodeJSONKeyStringAscending(b []byte, s string, end bool) []byte

EncodeJSONKeyStringAscending encodes the JSON key string value with a JSON specific escaped terminator. This allows us to encode keys in the same number of bytes as a string, while at the same time giving us a sentinel to identify JSON keys. The end parameter is used to determine if this is the last key in a a JSON path. If it is we don't add a separator after it.

func EncodeNonsortingDecimal 

func EncodeNonsortingDecimal(b []byte, d *apd.Decimal) []byte

EncodeNonsortingDecimal returns the resulting byte slice with the encoded decimal appended to b. The encoding is limited compared to standard encodings in this package in that

  • It will not sort lexicographically
  • It does not encode its length or terminate itself, so decoding functions must be provided the exact encoded bytes

The encoding assumes that any number can be written as ±0.xyz... * 10^exp, where xyz is a digit string, x != 0, and the last decimal in xyz is also not 0.

The encoding uses its first byte to split decimals into 7 distinct ordered groups (no NaN or Infinity support yet). The groups can be seen in encoding.go's const definition. Following this, the absolute value of the exponent of the decimal (as defined above) is encoded as an unsigned varint. Second, the absolute value of the digit string is added as a big-endian byte slice.

All together, the encoding looks like: 

<marker><uvarint exponent><big-endian encoded big.Int>.

The markers are shared with the sorting decimal encoding as follows: 

decimalNaN              -> decimalNaN
decimalNegativeInfinity -> decimalNegativeInfinity
decimalNegLarge         -> decimalNegValPosExp
decimalNegMedium        -> decimalNegValZeroExp
decimalNegSmall         -> decimalNegValNegExp
decimalZero             -> decimalZero
decimalPosSmall         -> decimalPosValNegExp
decimalPosMedium        -> decimalPosValZeroExp
decimalPosLarge         -> decimalPosValPosExp
decimalInfinity         -> decimalInfinity
decimalNaNDesc          -> decimalNaNDesc

func EncodeNullAscending 

func EncodeNullAscending(b []byte) []byte

EncodeNullAscending encodes a NULL value. The encodes bytes are appended to the supplied buffer and the final buffer is returned. The encoded value for a NULL is guaranteed to not be a prefix for the EncodeVarint, EncodeFloat, EncodeBytes and EncodeString encodings.

func EncodeStringAscending 

func EncodeStringAscending(b []byte, s string) []byte

EncodeStringAscending encodes the string value using an escape-based encoding. See EncodeBytes for details. The encoded bytes are append to the supplied buffer and the resulting buffer is returned.

func EncodeTrueAscending 

func EncodeTrueAscending(b []byte) []byte

EncodeTrueAscending encodes the boolean value true for use with JSON inverted indexes.

func EncodeUint32Ascending 

func EncodeUint32Ascending(b []byte, v uint32) []byte

EncodeUint32Ascending encodes the uint32 value using a big-endian 4 byte representation. The bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUint64Ascending 

func EncodeUint64Ascending(b []byte, v uint64) []byte

EncodeUint64Ascending encodes the uint64 value using a big-endian 8 byte representation. The bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUntaggedDecimalValue 

func EncodeUntaggedDecimalValue(appendTo []byte, d *apd.Decimal) []byte

EncodeUntaggedDecimalValue encodes an apd.Decimal value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUvarintAscending 

func EncodeUvarintAscending(b []byte, v uint64) []byte

EncodeUvarintAscending encodes the uint64 value using a variable length (length-prefixed) representation. The length is encoded as a single byte indicating the number of encoded bytes (-8) to follow. See EncodeVarintAscending for rationale. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUvarintDescending 

func EncodeUvarintDescending(b []byte, v uint64) []byte

EncodeUvarintDescending encodes the uint64 value so that it sorts in reverse order, from largest to smallest.

func GetMultiVarintLen 

func GetMultiVarintLen(b []byte, num int) (int, error)

GetMultiVarintLen find the length of <num> encoded varints that follow a 1-byte tag.

func IsArrayKeyDone 

func IsArrayKeyDone(buf []byte, dir Direction) bool

IsArrayKeyDone returns if the first byte in the input is the array terminator for the input direction.

func PeekLength 

func PeekLength(b []byte) (int, error)

PeekLength returns the length of the encoded value at the start of b. Note: if this function succeeds, it's not a guarantee that decoding the value will succeed. PeekLength is meant to be used on key encoded data only.

func PutUint32Ascending 

func PutUint32Ascending(b []byte, v uint32, idx int) []byte

PutUint32Ascending encodes the uint32 value using a big-endian 4 byte representation at the specified index, lengthening the input slice if necessary.

func UnsafeConvertStringToBytes 

func UnsafeConvertStringToBytes(s string) []byte

UnsafeConvertStringToBytes converts a string to a byte array to be used with string encoding functions. Note that the output byte array should not be modified if the input string is expected to be used again - doing so could violate Go semantics.

func UpperBoundNonsortingDecimalSize 

func UpperBoundNonsortingDecimalSize(d *apd.Decimal) int

UpperBoundNonsortingDecimalSize returns the upper bound number of bytes that the decimal will need for the non-sorting encoding.

func WordLen 

func WordLen(nat []big.Word) int

WordLen returns the size in bytes of the given array of Words.

Types

type Direction 

type Direction int

Direction for ordering results. 

const (
	Ascending Direction
	Descending
)

Direction values.

type Type 

type Type int

Type represents the type of a value encoded by Encode{Null,NotNull,Varint,Uvarint,Float,Bytes}. 

const (
	Unknown   Type = 0
	Null      Type = 1
	NotNull   Type = 2
	Int       Type = 3
	Float     Type = 4
	Decimal   Type = 5
	Bytes     Type = 6
	BytesDesc Type = 7 // Bytes encoded descendingly
	Time      Type = 8
	Duration  Type = 9
	True      Type = 10
	False     Type = 11
	UUID      Type = 12
	Array     Type = 13
	IPAddr    Type = 14
	// SentinelType is used for bit manipulation to check if the encoded type
	// value requires more than 4 bits, and thus will be encoded in two bytes. It
	// is not used as a type value, and thus intentionally overlaps with the
	// subsequent type value. The 'Type' annotation is intentionally omitted here.
	SentinelType      = 15
	JSON         Type = 15
	Tuple        Type = 16
	BitArray     Type = 17
	BitArrayDesc Type = 18 // BitArray encoded descendingly
	TimeTZ       Type = 19
	Geo          Type = 20
	GeoDesc      Type = 21
	ArrayKeyAsc  Type = 22 // Array key encoding
	ArrayKeyDesc Type = 23 // Array key encoded descendingly
	Box2D        Type = 24
)

Type values. TODO(dan, arjun): Make this into a proto enum. The 'Type' annotations are necessary for producing stringer-generated values.

func DecodeValueTag 

func DecodeValueTag(b []byte) (typeOffset int, dataOffset int, colID uint32, typ Type, err error)

DecodeValueTag decodes a value encoded by EncodeValueTag, used as a prefix in each of the other EncodeFooValue methods.

The tag is structured such that the encoded column id can be dropped from the front by removing the first `typeOffset` bytes. DecodeValueTag, PeekValueLength and each of the DecodeFooValue methods will still work as expected with `b[typeOffset:]`. (Except, obviously, the column id is no longer encoded so if this suffix is passed back to DecodeValueTag, the returned colID should be discarded.)

Concretely: 

b := ...
typeOffset, _, colID, typ, err := DecodeValueTag(b)
_, _, _, typ, err := DecodeValueTag(b[typeOffset:])

will return the same typ and err and 

DecodeFooValue(b)
DecodeFooValue(b[typeOffset:])

will return the same thing. PeekValueLength works as expected with either of `b` or `b[typeOffset:]`.

func PeekType 

func PeekType(b []byte) Type

PeekType peeks at the type of the value encoded at the start of b.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.