Documentation
¶
Overview ¶
Package coders is a pair of convenience handles for encoding and decoding values to [][byte].
Index ¶
- func EncodeWindowedValueHeader[W window](e *Encoder, eventTime time.Time, windows []W, pane PaneInfo)
- type Codable
- type Coder
- type Decoder
- func (d *Decoder) Bool() bool
- func (d *Decoder) Byte() byte
- func (d *Decoder) Bytes() []byte
- func (d *Decoder) Complex128() complex128
- func (d *Decoder) Complex64() complex64
- func (d *Decoder) DecodeBinaryUnmarshaler(value encoding.BinaryUnmarshaler)
- func (d *Decoder) DecodeProto(value proto.Message)
- func (d *Decoder) Double() float64
- func (d *Decoder) Empty() bool
- func (d *Decoder) Float() float32
- func (d *Decoder) GlobalWindow()
- func (d *Decoder) Int() int
- func (d *Decoder) Int16() int16
- func (d *Decoder) Int32() int32
- func (d *Decoder) Int64() int64
- func (d *Decoder) Int8() int8
- func (d *Decoder) Pane() PaneInfo
- func (d *Decoder) Read(n int) []byte
- func (d *Decoder) Rune() rune
- func (d *Decoder) StringUtf8() string
- func (d *Decoder) Timestamp() time.Time
- func (d *Decoder) Uint() uint
- func (d *Decoder) Uint16() uint16
- func (d *Decoder) Uint32() uint32
- func (d *Decoder) Uint64() uint64
- func (d *Decoder) Uint8() uint8
- func (d *Decoder) Varint() uint64
- type Encoder
- func (e *Encoder) Bool(arg bool)
- func (e *Encoder) Byte(arg byte)
- func (e *Encoder) Bytes(arg []byte)
- func (e *Encoder) Complex128(arg complex128)
- func (e *Encoder) Complex64(arg complex64)
- func (e *Encoder) Data() []byte
- func (e *Encoder) Double(arg float64)
- func (e *Encoder) Float(arg float32)
- func (e *Encoder) GlobalWindow()
- func (e *Encoder) Grow(bytesNeeded int) []byte
- func (e *Encoder) Int(arg int)
- func (e *Encoder) Int16(arg int16)
- func (e *Encoder) Int32(arg int32)
- func (e *Encoder) Int64(arg int64)
- func (e *Encoder) Int8(arg int8)
- func (e *Encoder) IntervalWindow(end time.Time, dur time.Duration)
- func (e *Encoder) Nullable(isNil bool)
- func (e *Encoder) Pane(pane PaneInfo)
- func (e *Encoder) Reset(n int)
- func (e *Encoder) Rune(arg rune)
- func (e *Encoder) StringUtf8(arg string)
- func (e *Encoder) Timestamp(ts time.Time)
- func (e *Encoder) Uint(arg uint)
- func (e *Encoder) Uint16(arg uint16)
- func (e *Encoder) Uint32(arg uint32)
- func (e *Encoder) Uint64(arg uint64)
- func (e *Encoder) Uint8(arg uint8)
- func (e *Encoder) Varint(i uint64)
- type GWC
- type PaneInfo
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EncodeWindowedValueHeader ¶
func EncodeWindowedValueHeader[W window](e *Encoder, eventTime time.Time, windows []W, pane PaneInfo)
EncodeWindowedValueHeader encodes the WindowedValue but not the value
Encodes an element, the windows it is in, the timestamp of the element,
and the pane of the element. The encoding is represented as: timestamp windows pane element
timestamp - A big endian 8 byte integer representing millis-since-epoch.
The encoded representation is shifted so that the byte representation
of negative values are lexicographically ordered before the byte
representation of positive values. This is typically done by
subtracting -9223372036854775808 from the value and encoding it as a
signed big endian integer. Example values:
-9223372036854775808: 00 00 00 00 00 00 00 00
-255: 7F FF FF FF FF FF FF 01
-1: 7F FF FF FF FF FF FF FF
0: 80 00 00 00 00 00 00 00
1: 80 00 00 00 00 00 00 01
256: 80 00 00 00 00 00 01 00
9223372036854775807: FF FF FF FF FF FF FF FF
windows - The windows are encoded using the beam:coder:iterable:v1
format, where the windows are encoded using the supplied window
coder.
pane - The first byte of the pane info determines which type of
encoding is used, as well as the is_first, is_last, and timing
fields. If this byte is bits [0 1 2 3 4 5 6 7], then:
* bits [0 1 2 3] determine the encoding as follows:
0000 - The entire pane info is encoded as a single byte.
The is_first, is_last, and timing fields are encoded
as below, and the index and non-speculative index are
both zero (and hence are not encoded here).
0001 - The pane info is encoded as this byte plus a single
VarInt encoed integer representing the pane index. The
non-speculative index can be derived as follows:
-1 if the pane is early, otherwise equal to index.
0010 - The pane info is encoded as this byte plus two VarInt
encoded integers representing the pane index and
non-speculative index respectively.
* bits [4 5] encode the timing as follows:
00 - early
01 - on time
10 - late
11 - unknown
* bit 6 is 1 if this is the first pane, 0 otherwise.
* bit 7 is 1 if this is the last pane, 0 otherwise.
element - The element incoded using the supplied element coder.
Components: The element coder and the window coder, in that order. WINDOWED_VALUE = 8 [(beam_urn) = "beam:coder:windowed_value:v1"];
Types ¶
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder deserializes data from a byte slice data in the expected results.
func NewDecoder ¶
NewDecoder instantiates a new Decoder for a given byte slice.
func (*Decoder) Complex128 ¶
func (d *Decoder) Complex128() complex128
Complex128 decodes a value of type complex128.
func (*Decoder) DecodeBinaryUnmarshaler ¶
func (d *Decoder) DecodeBinaryUnmarshaler(value encoding.BinaryUnmarshaler)
DecodeBinaryUnmarshaler deserializes the value from a byte slice using UnmarshalBinary.
func (*Decoder) DecodeProto ¶
DecodeProto deserializes the value from a byte slice using proto serialization.
func (*Decoder) GlobalWindow ¶
func (d *Decoder) GlobalWindow()
GlobalWindow encodes as no bytes, making this a no-op.
This matches with "beam:coder:global_window:v1" of the beam_runner_api.proto coders.
func (*Decoder) Read ¶
Read reads and returns n bytes from the decoder and advances the decode past the read bytes.
func (*Decoder) StringUtf8 ¶
String decodes a value of type string.
type Encoder ¶
type Encoder struct {
// contains filtered or unexported fields
}
Encoder encapsulates building a []byte from some other type, based on the Apache Beam standard Coders.
When the value is fully encode, retrive the completed []byte with *Encoder.Data.
func NewEncoder ¶
func NewEncoder() *Encoder
func (*Encoder) Bool ¶
Bool encodes an arg of type bool. Serialize boolean values as an uint8 that encodes either 0 (false) or 1 (true).
This matches with "beam:coder:bool:v1" of the beam_runner_api.proto coders.
func (*Encoder) Bytes ¶
Bytes encodes an arg of type []byte. For a byte slice, we encode its length as a varint, followed by the serialized content. Nil slices are encoded identically to 0 length slices.
This matches with "beam:coder:bytes:v1" of the beam_runner_api.proto coders.
func (*Encoder) Complex128 ¶
func (e *Encoder) Complex128(arg complex128)
Complex128 encodes an arg of type complex128.
func (*Encoder) Complex64 ¶
Complex64 encodes an arg of type complex64. We encode the real and the imaginary parts one after the other.
func (*Encoder) Double ¶
Double encodes the floating point value as a big-endian 64-bit integer according to the IEEE 754 double format bit layout. Components: None This matches with "beam:coder:double:v1" of the beam_runner_api.proto coders.
func (*Encoder) GlobalWindow ¶
func (e *Encoder) GlobalWindow()
GlobalWindow encodes as no bytes, making this a no-op.
This matches with "beam:coder:global_window:v1" of the beam_runner_api.proto coders.
func (*Encoder) Grow ¶
Grow increases the size of the encoder's data if needed. Only appends a new slice if there is not enough capacity to satisfy bytesNeeded. Returns the slice fragment that contains bytesNeeded.
func (*Encoder) Int ¶
Int encodes an arg of type int. Int can have 32 bits or 64 bits based on the machine type. To simplify our reasoning, we encode the highest possible value.
func (*Encoder) IntervalWindow ¶
IntervalWindow encodes a single interval window, which is the end time as a beam timestamp, and a varint of the duration in milliseconds.
This matches with "beam:coder:interval_window:v1" of the beam_runner_api.proto coders.
func (*Encoder) Nullable ¶
Nullable handles the nil bit of a value. Wraps a coder of a potentially null value A Nullable Type is encoded by:
- A one byte null indicator, 0x00 for null values, or 0x01 for present values.
- For present values the null indicator is followed by the value encoded with it's corresponding coder.
Components: single coder for the value NULLABLE = 17 [(beam_urn) = "beam:coder:nullable:v1"];
func (*Encoder) Reset ¶
Reset resets the Encoder to use a buffer with a capacity of at least the provided size. All encoded data is lost.
Reset may re-use the existing byte buffer.
func (*Encoder) StringUtf8 ¶
String encodes an arg of type string. For a string, we encode its length as a varint, followed by the serialized content.
This matches with "beam:coder:string_utf8:v1" of the beam_runner_api.proto coders.
func (*Encoder) Timestamp ¶
Timestamp encodes event times in the following fashion.
timestamp - A big endian 8 byte integer representing millis-since-epoch.
The encoded representation is shifted so that the byte representation
of negative values are lexicographically ordered before the byte
representation of positive values. This is typically done by
subtracting -9223372036854775808 from the value and encoding it as a
signed big endian integer. Example values:
-9223372036854775808: 00 00 00 00 00 00 00 00
-255: 7F FF FF FF FF FF FF 01
-1: 7F FF FF FF FF FF FF FF
0: 80 00 00 00 00 00 00 00
1: 80 00 00 00 00 00 00 01
256: 80 00 00 00 00 00 01 00
9223372036854775807: FF FF FF FF FF FF FF FF
Source Files