activity

package
v0.51.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 9, 2024 License: Apache-2.0 Imports: 15 Imported by: 0

README

Events telemetry

Types of telemetry in Rill

In Rill, we have two kinds of telemetry:

  • OpenTelemetry: Used for emitting and collecting traditional logging, tracing, and metrics in a hosted setting. Encapsulated in the runtime/pkg/observability package.
  • Events telemetry: Used for event telemetry, such as user behavior, browser errors, system metrics, billing events, etc. Encapsulated in runtime/pkg/activity (this package).

Why do we have a separate implementation for events telemetry? Two reasons:

  1. The event telemetry is collected from all combinations of frontend/CLI/backend services and local/cloud-hosted environments. At the time of writing, OpenTelemetry is not well-suited for capturing data across such disparate environments.
  2. At the time of writing, OpenTelemetry does not have mature abstractions for capturing event data.

Sources of events telemetry and how it propagates

Event telemetry propagate in various ways depending on the source service and environment:

  1. The cloud admin server sends events directly to Kafka.
  2. The cloud runtime sends events directly to Kafka.
  3. The cloud admin UI sends events to the cloud admin server, which proxies them to Kafka.
  4. The local CLI sends events to Rill's intake API.
  5. The local UI sends events to the local server hosted by the CLI, which proxies them to Rill's intake API.
  6. The local runtime sends events to Rill's intake API (using the same client as the CLI).

Required event format

We do not enforce strict schemas for different event types, but we do require all events to have the following format:

{
    "event_id": "<generated unique ID for deduplication>",
    "event_time": "<timestamp in ISO8601 format>",
    "event_type": "<high-level event type, such as 'behavioral', 'metric', ...>",
    "event_name": "<unique identifer of the event within the event type, such as 'login_start' or 'duckdb_estimated_size'>"
    // Other event-specific attributes
}

Events that do not contain these fields cannot be generated or proxied through the client.

Common event types, names and attributes

The event.go file in this package contains constants for common event types/names and common attributes.

Documentation

Index

Constants

View Source
const (
	EventTypeLog        = "log"
	EventTypeMetric     = "metric"
	EventTypeBehavioral = "behavioral"
)

Constants for common event types.

View Source
const (
	AttrKeyServiceName    = "service_name"
	AttrKeyServiceVersion = "service_version"
	AttrKeyServiceCommit  = "service_commit"
	AttrKeyIsDev          = "is_dev"
	AttrKeyInstallID      = "install_id"
	AttrKeyUserID         = "user_id"
)

Constants for common event attribute keys.

View Source
const (
	BehavioralEventInstallSuccess         = "install-success"
	BehavioralEventAppStart               = "app-start"
	BehavioralEventLoginStart             = "login-start"
	BehavioralEventLoginSuccess           = "login-success"
	BehavioralEventDeployStart            = "deploy-start"
	BehavioralEventDeploySuccess          = "deploy-success"
	BehavioralEventGithubConnectedStart   = "ghconnected-start"
	BehavioralEventGithubConnectedSuccess = "ghconnected-success"
	BehavioralEventDataAccessStart        = "dataaccess-start"
	BehavioralEventDataAccessSuccess      = "dataaccess-success"
)

Constants for event names of type EventTypeBehavioral. Note: This list is not exhaustive. Proxied events may contain other names.

Variables

This section is empty.

Functions

func SetAttributes added in v0.42.0

func SetAttributes(ctx context.Context, attrs ...attribute.KeyValue) context.Context

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client constructs telemetry events and sends them to a sink.

func NewClient added in v0.42.0

func NewClient(sink Sink, logger *zap.Logger) *Client

NewClient creates a base telemetry client that sends events to the provided sink. The client will close the sink when Close is called.

func NewNoopClient

func NewNoopClient() *Client

NewNoopClient creates a client that discards all events.

func (*Client) Close

func (c *Client) Close(ctx context.Context) error

Close the client. Also closes the sink passed to NewClient.

func (*Client) Record added in v0.42.0

func (c *Client) Record(ctx context.Context, typ, name string, extraAttrs ...attribute.KeyValue)

Record sends a generic telemetry event with the provided event type and name.

func (*Client) RecordBehavioralLegacy added in v0.42.0

func (c *Client) RecordBehavioralLegacy(name string, extraAttrs ...attribute.KeyValue)

EmitBehavioral sends a telemetry event of type "behavioral" with the provided name and attributes. The event additionally has all the attributes associated with out legacy behavioral events. It will panic if all of WithServiceName, WithServiceVersion, WithInstallID and WithUserID have not been called on the client.

func (*Client) RecordMetric added in v0.42.0

func (c *Client) RecordMetric(ctx context.Context, name string, value float64, attrs ...attribute.KeyValue)

RecordMetric sends a telemetry event of type "metric" with the provided name and value.

func (*Client) RecordRaw added in v0.42.0

func (c *Client) RecordRaw(data map[string]any) error

RecordRaw proxies a raw event represented as a map to the client's sink. It does not enrich the provided event with any of the client's contextual attributes. It returns an error if the event does not contain the required fields (see the Event type for required fields).

func (*Client) With added in v0.32.0

func (c *Client) With(attrs ...attribute.KeyValue) *Client

With returns a copy of the client that will set the provided attributes on all events.

func (*Client) WithInstallID added in v0.42.0

func (c *Client) WithInstallID(installID string) *Client

WithInstallID returns a copy of the client with an attribute set for AttrKeyInstallID.

func (*Client) WithIsDev added in v0.42.0

func (c *Client) WithIsDev() *Client

WithIsDev returns a copy of the client with an attribute set for AttrKeyIsDev.

func (*Client) WithServiceName added in v0.42.0

func (c *Client) WithServiceName(serviceName string) *Client

WithServiceName returns a copy of the client with an attribute set for AttrKeyServiceName.

func (*Client) WithServiceVersion added in v0.42.0

func (c *Client) WithServiceVersion(number, commit string) *Client

WithServiceVersion returns a copy of the client with attributes set for AttrKeyServiceVersion and AttrKeyServiceCommit.

func (*Client) WithUserID added in v0.42.0

func (c *Client) WithUserID(userID string) *Client

WithUserID returns a copy of the client with an attribute set for AttrKeyUserID.

type Event

type Event struct {
	EventID   string
	EventTime time.Time
	EventType string
	EventName string
	Data      map[string]any
}

Event is a telemetry event. It consists of a few required fields that are common to all events and a payload of type-specific data. All the common fields are prefixed with "event_" to avoid conflicts with the payload data.

func (Event) MarshalJSON added in v0.42.0

func (e Event) MarshalJSON() ([]byte, error)

type IntakeSinkOptions added in v0.42.0

type IntakeSinkOptions struct {
	IntakeURL      string
	IntakeUser     string
	IntakePassword string
	BufferSize     int
	SinkInterval   time.Duration
}

IntakeSinkOptions provides options for NewIntakeSink.

type Sink

type Sink interface {
	Emit(event Event) error
	Close()
}

Sink is a destination for sending telemetry events.

func NewFilterSink added in v0.42.0

func NewFilterSink(sink Sink, fn func(Event) bool) Sink

NewFilterSink returns a sink that filters events based on the provided filter function. Only events for which the filter function returns true will be emitted to the wrapped sink.

func NewIntakeSink added in v0.42.0

func NewIntakeSink(logger *zap.Logger, opts IntakeSinkOptions) Sink

NewIntakeSink creates a new sink that sends events to the Rill intake API.

func NewKafkaSink

func NewKafkaSink(brokers, topic string, logger *zap.Logger) (Sink, error)

NewKafkaSink returns a sink that sends events to a Kafka topic.

func NewLoggerSink added in v0.42.0

func NewLoggerSink(logger *zap.Logger, level zapcore.Level) Sink

NewLoggerSink returns a sink that logs events to the given logger.

func NewNoopSink

func NewNoopSink() Sink

NewNoopSink returns a sink that drops all events.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL