encodingextension

README

Encoding extension

Status
Stability	development
Distributions	[]
Issues
Code Owners	@MovieStoreGuy

The encoding extension allows for other components to reference ingress/egress data formats to allow for simplified operability.

🚧 Under active development 🚧

Component Milestones

To help track what work needs to be done with this component, these are the currently active goals being worked towards.

Development

• CodecFactory: Allows other components to reference code by name and version
• Codec: Defining the expected interface for a codec
  • Length Delimited Codec: Used fixed length payloads to represent a collection of data that can be merged together
• Add existing formats to the codec's libraries.
• Add support to the following components
  • file receiver
  • file exporter
  • kafka receiver
  • kafka exporter
  • kinesis exporter
• Add codecs for open source formats, ie: otlp, zipkin, jaeger

Component support will be behind a feature gate to avoid change in behavior

Alpha

• Adopt vendor formats as codecs
• Transition feature gate to default to using encoding extension
• Add dynamic mapping support for avro, parquet, and gotemplate
  • Not fixed on these but I am aware that users are interested in these

Beta

• Add support for remote schema management
  • Using AWS Glue, kafka schema registry
• Performance break down of difference codecs (static, dynamic, remote)
  • This is more to help provide information to users to expected performance impact.

Stable

• All features complete
• Absent of bugs that impact core behaviours
• Dependencies can also be marked as stable

Example configuration

extensions:
  encoding:

receivers:
  kafka:
    encoding:
      name:   zipkin/v1
      format: proto
    # ... other configuration values