decoder

Documentation

Overview

Package implements Msgpack decoder. Fluent Bit Go already has a Msgpack decoder; however, it will decode strings as []int8. This has two undesirable consequences.

1. Printing values with %v may output non-human readable arrays.

2. Strings in []int8 format marshalled to JSON will output non-human readable base64 encoded strings.

To solve these issues, all other plugins such as the aws firehose plugin, have recursive functions which comb through decoded Msgpack structures and convert bytes to strings (effectively another decoder). Creating a new decoder to output strings instead of bytes is cleaner, removes complex recursive functions, and likely more performant.

Index

• func GetRecord(decoder *codec.Decoder) (interface{}, []byte, error)
• func New(data unsafe.Pointer, length int) *codec.Decoder
• type FlbTime
  • func (f FlbTime) ConvertExt(v interface{}) interface{}
  • func (f FlbTime) ReadExt(i interface{}, b []byte)
  • func (f FlbTime) UpdateExt(dest interface{}, v interface{})
  • func (f FlbTime) WriteExt(interface{}) []byte

Functions

func GetRecord

func GetRecord(decoder *codec.Decoder) (interface{}, []byte, error)

Retrieves data and timestamp from Msgpack object.

Parameters:

• decoder: Msgpack decoder

Returns:

• timestamp: Timestamp retrieved from Fluent Bit
• record: JSON record from Fluent Bit with variable amount of keys
• err: decode error, error retrieving timestamp, error marshalling record

func New

func New(data unsafe.Pointer, length int) *codec.Decoder

Initializes a Msgpack decoder which automatically converts bytes to strings. Decoder has an extension setup for a custom Fluent Bit timestamp format. During timestamp encoding, Fluent Bit will set the Msgpack extension type to "0". This decoder can recognize the extension type, and will then decode the custom Fluent Bit timestamp using a specific function [ReadExt].

Parameters:

• data: Msgpack data
• length: Byte length

Returns:

• decoder: Msgpack decoder

Types

type FlbTime

type FlbTime struct {
	time.Time
}

Fluent-bit can encode timestamps in Msgpack [fixext 8] format. Format stores an integer and a byte array whose length is 8 bytes. The integer is the type, and the 4 MSBs are the seconds (big-endian uint32) and 4 LSBs are nanoseconds. [fixext 8]: https://github.com/msgpack/msgpack/blob/master/spec.md#ext-format-family

func (FlbTime) ConvertExt

func (f FlbTime) ConvertExt(v interface{}) interface{}

Function required by codec but not being used by decoder.

func (FlbTime) ReadExt

func (f FlbTime) ReadExt(i interface{}, b []byte)

Updates a value from a []byte.

Parameters:

• i: Pointer to the registered extension type
• b: Msgpack data in fixext 8 format

func (FlbTime) UpdateExt

func (f FlbTime) UpdateExt(dest interface{}, v interface{})

Function required by codec but not being used by decoder.

func (FlbTime) WriteExt

func (f FlbTime) WriteExt(interface{}) []byte

Function required by codec but not being used by decoder.