netstats

README

Network Statistics package

Overview

This package provides general support for counting the number of bytes read and written by an exporter or receiver component. This package specifically supports monitoring the compression rate achieved by OpenTelemetry Protocol with Apache Arrow, but it can be easily adopted for any gRPC-based component, for both unary and streaming RPCs.

Usage

To create a network reporter, pass the exporter or receiver settings to netstats.NewExporterNetworkReporter or netstats.NewExporterNetworkReporter, then register with gRPC:

	dialOpts = append(dialOpts, grpc.WithStatsHandler(netReporter.Handler()))

Because OTel-Arrow supports the use of compressed payloads, configured through Arrow IPC, it is necessary for the exporter and receiver components to manually account for uncompressed payload sizes.

The SizesStruct supports recording either one or both of the compressed and uncompressed sizes. To report only uncompressed size in the exporter case, for example:

	var sized netstats.SizesStruct
	sized.Method = s.method
	sized.Length = int64(uncompressedSize)
	netReporter.CountSend(ctx, sized)

Likewise, the receiver uses CountRecv with sized.Length set to report its uncompressed size after OTel-Arrow decompression.

Documentation

Index

• Constants
• func GRPCStreamMethodName(desc grpc.ServiceDesc, stream grpc.StreamDesc) string
• func GRPCUnaryMethodName(desc grpc.ServiceDesc, method grpc.MethodDesc) string
• type Interface
• type NetworkReporter
  • func NewExporterNetworkReporter(settings exporter.Settings) (*NetworkReporter, error)
  • func NewReceiverNetworkReporter(settings receiver.Settings) (*NetworkReporter, error)
  • func (rep *NetworkReporter) CountReceive(ctx context.Context, ss SizesStruct)
  • func (rep *NetworkReporter) CountSend(ctx context.Context, ss SizesStruct)
  • func (rep *NetworkReporter) Handler() stats.Handler
• type Noop
  • func (Noop) CountReceive(context.Context, SizesStruct)
  • func (Noop) CountSend(context.Context, SizesStruct)
• type SizesStruct

Constants

const (
	// ExporterKey is an attribute name that identifies an
	// exporter component that produces internal metrics, logs,
	// and traces.
	ExporterKey = "exporter"

	// ReceiverKey is an attribute name that identifies an
	// receiver component that produces internal metrics, logs,
	// and traces.
	ReceiverKey = "receiver"

	// SentBytes is used to track bytes sent by exporters and receivers.
	SentBytes = "sent"

	// SentWireBytes is used to track bytes sent on the wire
	// (includes compression) by exporters and receivers.
	SentWireBytes = "sent_wire"

	// RecvBytes is used to track bytes received by exporters and receivers.
	RecvBytes = "recv"

	// RecvWireBytes is used to track bytes received on the wire
	// (includes compression) by exporters and receivers.
	RecvWireBytes = "recv_wire"

	// CompSize is used for compressed size histogram metrics.
	CompSize = "compressed_size"
)

Functions

func GRPCStreamMethodName

func GRPCStreamMethodName(desc grpc.ServiceDesc, stream grpc.StreamDesc) string

GRPCStreamMethodName applies the logic gRPC uses but does not expose to construct method names. This allows direct calling of the netstats interface from outside a gRPC stats handler.

func GRPCUnaryMethodName

func GRPCUnaryMethodName(desc grpc.ServiceDesc, method grpc.MethodDesc) string

GRPCUnaryMethodName applies the logic gRPC uses but does not expose to construct method names. This allows direct calling of the netstats interface from outside a gRPC stats handler.

Types

type Interface

type Interface interface {
	// CountSend reports outbound bytes.
	CountSend(ctx context.Context, ss SizesStruct)

	// CountSend reports inbound bytes.
	CountReceive(ctx context.Context, ss SizesStruct)
}

Interface describes a *NetworkReporter or a Noop.

type NetworkReporter

type NetworkReporter struct {
	// contains filtered or unexported fields
}

NetworkReporter is a helper to add network-level observability to an exporter or receiver.

func NewExporterNetworkReporter

func NewExporterNetworkReporter(settings exporter.Settings) (*NetworkReporter, error)

NewExporterNetworkReporter creates a new NetworkReporter configured for an exporter.

func NewReceiverNetworkReporter

func NewReceiverNetworkReporter(settings receiver.Settings) (*NetworkReporter, error)

NewReceiverNetworkReporter creates a new NetworkReporter configured for an exporter.

func (*NetworkReporter) CountReceive

func (rep *NetworkReporter) CountReceive(ctx context.Context, ss SizesStruct)

CountReceive is used to report a message received by the component. For exporters, SizesStruct indicates the size of a response. For receivers, SizesStruct indicates the size of a request.

func (*NetworkReporter) CountSend

func (rep *NetworkReporter) CountSend(ctx context.Context, ss SizesStruct)

CountSend is used to report a message sent by the component. For exporters, SizesStruct indicates the size of a request. For receivers, SizesStruct indicates the size of a response.

func (*NetworkReporter) Handler

func (rep *NetworkReporter) Handler() stats.Handler

type Noop

type Noop struct{}

Noop is a no-op implementation of Interface.

func (Noop) CountReceive

func (Noop) CountReceive(context.Context, SizesStruct)

func (Noop) CountSend

func (Noop) CountSend(context.Context, SizesStruct)

type SizesStruct

type SizesStruct struct {
	// Method refers to the gRPC method name
	Method string
	// Length is the uncompressed size
	Length int64
	// WireLength is compressed size
	WireLength int64
}

SizesStruct is used to pass uncompressed on-wire message lengths to the CountSend() and CountReceive() methods.