marshal

Documentation

Overview

Package marshal implements the marshaling side of network encoding

Index

• Constants
• func FormatCORETelemetry(builder *model.ConnectionsBuilder, telByAsset map[string]int32)
• func FormatCompilationTelemetry(builder *model.ConnectionsBuilder, ...)
• func FormatConnection(builder *model.ConnectionBuilder, conn network.ConnectionStats, ...)
• func FormatConnectionTelemetry(builder *model.ConnectionsBuilder, tel map[network.ConnTelemetryType]int64)
• func FormatProtocolStack(originalStack protocols.Stack, staticTags uint64) *model.ProtocolStack
• func GetUSMPayloadTelemetry() map[string]int64
• type ConnectionsModeler
  • func NewConnectionsModeler(conns *network.Connections) *ConnectionsModeler
  • func (c *ConnectionsModeler) Close()
• type Marshaler
  • func GetMarshaler(accept string) Marshaler
• type RouteIdx
• type USMConnectionData
  • func USMLookup[K comparable, V any](c network.ConnectionStats, ...) *USMConnectionData[K, V]
  • func (gd *USMConnectionData[K, V]) IsPIDCollision(c network.ConnectionStats) bool
• type USMConnectionIndex
  • func GroupByConnection[K comparable, V any](protocol string, data map[K]V, keyGen func(K) types.ConnectionKey) *USMConnectionIndex[K, V]
  • func (bc *USMConnectionIndex[K, V]) Close()
  • func (bc *USMConnectionIndex[K, V]) Find(c network.ConnectionStats) *USMConnectionData[K, V]
• type USMKeyValue

Constants

const ContentTypeJSON = "application/json"

ContentTypeJSON holds the HTML content-type of a JSON payload

const ContentTypeProtobuf = "application/protobuf"

ContentTypeProtobuf holds the HTML content-type of a Protobuf payload

Functions

func FormatCORETelemetry

func FormatCORETelemetry(builder *model.ConnectionsBuilder, telByAsset map[string]int32)

FormatCORETelemetry writes the CORETelemetryByAsset map into a connections payload

func FormatCompilationTelemetry

func FormatCompilationTelemetry(builder *model.ConnectionsBuilder, telByAsset map[string]network.RuntimeCompilationTelemetry)

FormatCompilationTelemetry converts telemetry from its internal representation to a protobuf message

func FormatConnection

func FormatConnection(builder *model.ConnectionBuilder, conn network.ConnectionStats, routes map[string]RouteIdx,
	httpEncoder *httpEncoder, http2Encoder *http2Encoder, kafkaEncoder *kafkaEncoder, postgresEncoder *postgresEncoder,
	redisEncoder *redisEncoder, dnsFormatter *dnsFormatter, ipc ipCache, tagsSet *network.TagsSet)

FormatConnection converts a ConnectionStats into an model.Connection

func FormatConnectionTelemetry

func FormatConnectionTelemetry(builder *model.ConnectionsBuilder, tel map[network.ConnTelemetryType]int64)

FormatConnectionTelemetry converts telemetry from its internal representation to a protobuf message

func FormatProtocolStack

func FormatProtocolStack(originalStack protocols.Stack, staticTags uint64) *model.ProtocolStack

FormatProtocolStack generates a protobuf representation of protocol stack from the internal one (`protocols.Stack`) i.e: if the input is protocols.Stack{Application: protocols.HTTP2} the output should be:

&model.ProtocolStack{
		Stack: []model.ProtocolType{
			model.ProtocolType_protocolHTTP2,
		},
	}

Additionally, if the staticTags contains TLS tags, the TLS protocol is added to the protocol stack, giving an output like this:

&model.ProtocolStack{
		Stack: []model.ProtocolType{
			model.ProtocolType_protocolTLS,
			model.ProtocolType_protocolHTTP2,
		},
	}

func GetUSMPayloadTelemetry

func GetUSMPayloadTelemetry() map[string]int64

GetUSMPayloadTelemetry returns a map with all metrics that are meant to be sent as payload telemetry.

In order to emit a new payload metric, make sure to use the option `telemetry.OptPayloadTelemetry`. Example: myMetric := telemetry.NewMetric("metric_name", telemetry.OptPayloadTelemetry)

Finally, make sure to add an entry to pkg/network/event_common.go for the sake of documentation.

Types

type ConnectionsModeler

type ConnectionsModeler struct {
	// contains filtered or unexported fields
}

ConnectionsModeler contains all the necessary structs for modeling a connection.

func NewConnectionsModeler

func NewConnectionsModeler(conns *network.Connections) *ConnectionsModeler

NewConnectionsModeler initializes the connection modeler with encoders, dns formatter for the existing connections. The ConnectionsModeler holds the traffic encoders grouped by USM logic. It also includes formatted connection telemetry related to all batches, not specific batches. Furthermore, it stores the current agent configuration which applies to all instances related to the entire set of connections, rather than just individual batches.

func (*ConnectionsModeler) Close

func (c *ConnectionsModeler) Close()

Close cleans all encoders resources.

type Marshaler

type Marshaler interface {
	Marshal(conns *network.Connections, writer io.Writer, connsModeler *ConnectionsModeler) error
	ContentType() string
}

Marshaler is an interface implemented by all Connections serializers

func GetMarshaler

func GetMarshaler(accept string) Marshaler

GetMarshaler returns the appropriate Marshaler based on the given accept header

type RouteIdx

type RouteIdx struct {
	Idx   int32
	Route model.Route
}

RouteIdx stores the route and the index into the route collection for a route

type USMConnectionData

type USMConnectionData[K comparable, V any] struct {
	Data []USMKeyValue[K, V]
	// contains filtered or unexported fields
}

USMConnectionData aggregates all USM data associated to a specific connection

func USMLookup

func USMLookup[K comparable, V any](c network.ConnectionStats, data map[types.ConnectionKey]*USMConnectionData[K, V]) *USMConnectionData[K, V]

USMLookup determines the strategy for associating a given connection to USM In the context of Linux we may perform up to 4 lookups as described below

func (*USMConnectionData[K, V]) IsPIDCollision

func (gd *USMConnectionData[K, V]) IsPIDCollision(c network.ConnectionStats) bool

IsPIDCollision can be called on each lookup result returned by `USMConnectionIndex.Find`. This is intended to avoid over-reporting USM stats in the context of PID "collisions". For example, let's say you have the following two connections:

Connection 1: srcA, dstB, pid X Connection 2: srcA, dstB, pid Y

And some USM data that is associated to: srcA, dstB (note that data from socket filter programs doesn't include PIDs)

The purpose of this check is to avoid letting `Connection 1` and `Connection 2` be associated to the same USM aggregation object.

So whichever connection "claims" the aggregation first will return `false` for `IsPIDCollision`, and any other following connection calling this method will get a `true` return value back.

Notice that this PID collision scenario is typical in the context pre-forked webservers such as NGINX, where multiple worker processes will share the same listen socket.

type USMConnectionIndex

type USMConnectionIndex[K comparable, V any] struct {
	// contains filtered or unexported fields
}

USMConnectionIndex provides a generic container for USM data pre-aggregated by connection

func GroupByConnection

func GroupByConnection[K comparable, V any](protocol string, data map[K]V, keyGen func(K) types.ConnectionKey) *USMConnectionIndex[K, V]

GroupByConnection generates a `USMConnectionIndex` from a generic `map[K]V` data structure. In addition to the `data` argument the caller must provide a `keyGen` function that essentially translates `K` to a `types.ConnectionKey` and a `protocol` name.

func (*USMConnectionIndex[K, V]) Close

func (bc *USMConnectionIndex[K, V]) Close()

Close `USMConnectionIndex` and report orphan aggregations

func (*USMConnectionIndex[K, V]) Find

func (bc *USMConnectionIndex[K, V]) Find(c network.ConnectionStats) *USMConnectionData[K, V]

Find returns a `USMConnectionData` object associated to given `network.ConnectionStats` The returned object will include all USM aggregation associated to this connection

type USMKeyValue

type USMKeyValue[K comparable, V any] struct {
	Key   K
	Value V
}

USMKeyValue is a generic container for USM data