txfbuiltin

Documentation

Index

• Variables
• func ExtractFieldKey(config transform.Config) (transform.Transform, error)
• func ExtractFieldPayload(config transform.Config) (transform.Transform, error)
• func FilterFieldKey(config transform.Config) (transform.Transform, error)
• func FilterFieldPayload(config transform.Config) (transform.Transform, error)
• func HoistFieldKey(config transform.Config) (transform.Transform, error)
• func HoistFieldPayload(config transform.Config) (transform.Transform, error)
• func InsertFieldKey(config transform.Config) (transform.Transform, error)
• func InsertFieldPayload(config transform.Config) (transform.Transform, error)
• func MaskFieldKey(config transform.Config) (transform.Transform, error)
• func MaskFieldPayload(config transform.Config) (transform.Transform, error)
• func ReplaceFieldKey(config transform.Config) (transform.Transform, error)
• func ReplaceFieldPayload(config transform.Config) (transform.Transform, error)
• func TimestampConvertorKey(config transform.Config) (transform.Transform, error)
• func TimestampConvertorPayload(config transform.Config) (transform.Transform, error)
• func ValueToKey(config transform.Config) (transform.Transform, error)

Variables

var (
	// ErrDropRecord is returned when a Record is not being forwarded.
	ErrDropRecord = cerrors.New("ErrDropRecord")
)

Functions

func ExtractFieldKey

func ExtractFieldKey(config transform.Config) (transform.Transform, error)

ExtractFieldKey builds the following transform:

• If the key is raw and has a schema attached, extract the field and use it to replace the entire key.
• If the key is raw and has no schema, return an error (not supported).
• If the key is structured, extract the field and use it to replace the entire key.

func ExtractFieldPayload

func ExtractFieldPayload(config transform.Config) (transform.Transform, error)

ExtractFieldPayload builds the following transformation:

• If the payload is raw and has a schema attached, extract the field and use it to replace the entire payload.
• If the payload is raw and has no schema, return an error (not supported).
• If the payload is structured, extract the field and use it to replace the entire payload.

func FilterFieldKey

func FilterFieldKey(config transform.Config) (transform.Transform, error)

func FilterFieldPayload

func FilterFieldPayload(config transform.Config) (transform.Transform, error)

func HoistFieldKey

func HoistFieldKey(config transform.Config) (transform.Transform, error)

HoistFieldKey builds the following transform:

• If the key is raw and has a schema attached, wrap it using the specified field name in a struct.
• If the key is raw and has no schema, transform it into structured data by creating a map with the hoisted field and raw data as the value.
• If the key is structured, wrap it using the specified field name in a map.

func HoistFieldPayload

func HoistFieldPayload(config transform.Config) (transform.Transform, error)

HoistFieldPayload builds the following transformation:

• If the payload is raw and has a schema attached, wrap it using the specified field name in a struct.
• If the payload is raw and has no schema, transform it into structured data by creating a map with the hoisted field and raw data as the value.
• If the payload is structured, wrap it using the specified field name in a map.

func InsertFieldKey

func InsertFieldKey(config transform.Config) (transform.Transform, error)

InsertFieldKey builds the following transform:

• If the key is raw and has a schema attached, insert the field(s) in the key data.
• If the key is raw and has no schema, return an error (not supported).
• If the key is structured, set the field(s) in the key data.

func InsertFieldPayload

func InsertFieldPayload(config transform.Config) (transform.Transform, error)

InsertFieldPayload builds the following transformation:

• If the payload is raw and has a schema attached, insert the field(s) in the payload data.
• If the payload is raw and has no schema, return an error (not supported).
• If the payload is structured, set the field(s) in the payload data.

func MaskFieldKey

func MaskFieldKey(config transform.Config) (transform.Transform, error)

MaskFieldKey builds the following transform:

• If the key is raw and has a schema attached, replace the field with the zero value of the fields type.
• If the key is raw and has no schema, return an error (not supported).
• If the key is structured, replace the field with the zero value of the fields type.

func MaskFieldPayload

func MaskFieldPayload(config transform.Config) (transform.Transform, error)

MaskFieldPayload builds the following transformation:

• If the payload is raw and has a schema attached, replace the field with the zero value of the fields type.
• If the payload is raw and has no schema, return an error (not supported).
• If the payload is structured, replace the field with the zero value of the fields type.

func ReplaceFieldKey

func ReplaceFieldKey(config transform.Config) (transform.Transform, error)

ReplaceFieldKey builds a transform which replaces a field in the key in raw data with a schema or in structured data. Raw data without a schema is not supported. The transform can be controlled by 3 variables:

• "exclude" - is a comma separated list of fields that should be excluded from the transformed record ("exclude" takes precedence over "include").
• "include" - is a comma separated list of fields that should be included in the transformed record.
• "rename" - is a comma separated list of pairs separated by colons, that controls the mapping of old field names to new field names.

If "include" is not configured or is empty then all fields in the record will be included by default (except if they are configured in "exclude"). If "include" is not empty, then all fields are excluded by default and only fields in "include" will be added to the transformed record.

func ReplaceFieldPayload

func ReplaceFieldPayload(config transform.Config) (transform.Transform, error)

ReplaceFieldPayload builds a transform which replaces a field in the payload in raw data with a schema or in structured data. Raw data without a schema is not supported. The transform can be controlled by 3 variables:

• "exclude" - is a comma separated list of fields that should be excluded from the transformed record ("exclude" takes precedence over "include").
• "include" - is a comma separated list of fields that should be included in the transformed record.
• "rename" - is a comma separated list of pairs separated by colons, that controls the mapping of old field names to new field names.

If "include" is not configured or is empty then all fields in the record will be included by default (except if they are configured in "exclude"). If "include" is not empty, then all fields are excluded by default and only fields in "include" will be added to the transformed record.

func TimestampConvertorKey

func TimestampConvertorKey(config transform.Config) (transform.Transform, error)

TimestampConvertorKey todo

func TimestampConvertorPayload

func TimestampConvertorPayload(config transform.Config) (transform.Transform, error)

TimestampConvertorPayload todo

func ValueToKey

func ValueToKey(config transform.Config) (transform.Transform, error)

ValueToKey builds a transform that replaces the record key with a new key formed from a subset of fields in the record value.

• If the payload is raw and has a schema attached, the created key will also have a schema with a subset of fields.
• If the payload is structured, the created key will also be structured with a subset of fields.
• If the payload is raw and has no schema, return an error.