README
¶
Events telemetry
Types of telemetry in Rill
In Rill, we have two kinds of telemetry:
- OpenTelemetry: Used for emitting and collecting traditional logging, tracing, and metrics in a hosted setting. Encapsulated in the
runtime/pkg/observability
package. - Events telemetry: Used for event telemetry, such as user behavior, browser errors, system metrics, billing events, etc. Encapsulated in
runtime/pkg/activity
(this package).
Why do we have a separate implementation for events telemetry? Two reasons:
- The event telemetry is collected from all combinations of frontend/CLI/backend services and local/cloud-hosted environments. At the time of writing, OpenTelemetry is not well-suited for capturing data across such disparate environments.
- At the time of writing, OpenTelemetry does not have mature abstractions for capturing event data.
Sources of events telemetry and how it propagates
Event telemetry propagate in various ways depending on the source service and environment:
- The cloud admin server sends events directly to Kafka.
- The cloud runtime sends events directly to Kafka.
- The cloud admin UI sends events to the cloud admin server, which proxies them to Kafka.
- The local CLI sends events to Rill's intake API.
- The local UI sends events to the local server hosted by the CLI, which proxies them to Rill's intake API.
- The local runtime sends events to Rill's intake API (using the same client as the CLI).
Required event format
We do not enforce strict schemas for different event types, but we do require all events to have the following format:
{
"event_id": "<generated unique ID for deduplication>",
"event_time": "<timestamp in ISO8601 format>",
"event_type": "<high-level event type, such as 'behavioral', 'metric', ...>",
"event_name": "<unique identifer of the event within the event type, such as 'login_start' or 'duckdb_estimated_size'>"
// Other event-specific attributes
}
Events that do not contain these fields cannot be generated or proxied through the client.
Common event types, names and attributes
The event.go
file in this package contains constants for common event types/names and common attributes.
Documentation
¶
Index ¶
- Constants
- func SetAttributes(ctx context.Context, attrs ...attribute.KeyValue) context.Context
- type Client
- func (c *Client) Close(ctx context.Context) error
- func (c *Client) Record(ctx context.Context, typ, name string, extraAttrs ...attribute.KeyValue)
- func (c *Client) RecordBehavioralLegacy(name string, extraAttrs ...attribute.KeyValue)
- func (c *Client) RecordMetric(ctx context.Context, name string, value float64, attrs ...attribute.KeyValue)
- func (c *Client) RecordRaw(data map[string]any) error
- func (c *Client) With(attrs ...attribute.KeyValue) *Client
- func (c *Client) WithInstallID(installID string) *Client
- func (c *Client) WithIsDev() *Client
- func (c *Client) WithServiceName(serviceName string) *Client
- func (c *Client) WithServiceVersion(number, commit string) *Client
- func (c *Client) WithUserID(userID string) *Client
- type Event
- type IntakeSinkOptions
- type Sink
Constants ¶
const ( EventTypeLog = "log" EventTypeMetric = "metric" EventTypeBehavioral = "behavioral" )
Constants for common event types.
const ( AttrKeyServiceName = "service_name" AttrKeyServiceVersion = "service_version" AttrKeyServiceCommit = "service_commit" AttrKeyIsDev = "is_dev" AttrKeyInstallID = "install_id" AttrKeyUserID = "user_id" )
Constants for common event attribute keys.
const ( BehavioralEventInstallSuccess = "install-success" BehavioralEventAppStart = "app-start" BehavioralEventLoginStart = "login-start" BehavioralEventLoginSuccess = "login-success" BehavioralEventDeployStart = "deploy-start" BehavioralEventDeploySuccess = "deploy-success" BehavioralEventGithubConnectedStart = "ghconnected-start" BehavioralEventGithubConnectedSuccess = "ghconnected-success" BehavioralEventDataAccessStart = "dataaccess-start" BehavioralEventDataAccessSuccess = "dataaccess-success" )
Constants for event names of type EventTypeBehavioral. Note: This list is not exhaustive. Proxied events may contain other names.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client constructs telemetry events and sends them to a sink.
func NewClient ¶ added in v0.42.0
NewClient creates a base telemetry client that sends events to the provided sink. The client will close the sink when Close is called.
func NewNoopClient ¶
func NewNoopClient() *Client
NewNoopClient creates a client that discards all events.
func (*Client) Record ¶ added in v0.42.0
Record sends a generic telemetry event with the provided event type and name.
func (*Client) RecordBehavioralLegacy ¶ added in v0.42.0
EmitBehavioral sends a telemetry event of type "behavioral" with the provided name and attributes. The event additionally has all the attributes associated with out legacy behavioral events. It will panic if all of WithServiceName, WithServiceVersion, WithInstallID and WithUserID have not been called on the client.
func (*Client) RecordMetric ¶ added in v0.42.0
func (c *Client) RecordMetric(ctx context.Context, name string, value float64, attrs ...attribute.KeyValue)
RecordMetric sends a telemetry event of type "metric" with the provided name and value.
func (*Client) RecordRaw ¶ added in v0.42.0
RecordRaw proxies a raw event represented as a map to the client's sink. It does not enrich the provided event with any of the client's contextual attributes. It returns an error if the event does not contain the required fields (see the Event type for required fields).
func (*Client) With ¶ added in v0.32.0
With returns a copy of the client that will set the provided attributes on all events.
func (*Client) WithInstallID ¶ added in v0.42.0
WithInstallID returns a copy of the client with an attribute set for AttrKeyInstallID.
func (*Client) WithIsDev ¶ added in v0.42.0
WithIsDev returns a copy of the client with an attribute set for AttrKeyIsDev.
func (*Client) WithServiceName ¶ added in v0.42.0
WithServiceName returns a copy of the client with an attribute set for AttrKeyServiceName.
func (*Client) WithServiceVersion ¶ added in v0.42.0
WithServiceVersion returns a copy of the client with attributes set for AttrKeyServiceVersion and AttrKeyServiceCommit.
func (*Client) WithUserID ¶ added in v0.42.0
WithUserID returns a copy of the client with an attribute set for AttrKeyUserID.
type Event ¶
type Event struct { EventID string EventTime time.Time EventType string EventName string Data map[string]any }
Event is a telemetry event. It consists of a few required fields that are common to all events and a payload of type-specific data. All the common fields are prefixed with "event_" to avoid conflicts with the payload data.
func (Event) MarshalJSON ¶ added in v0.42.0
type IntakeSinkOptions ¶ added in v0.42.0
type IntakeSinkOptions struct { IntakeURL string IntakeUser string IntakePassword string BufferSize int SinkInterval time.Duration }
IntakeSinkOptions provides options for NewIntakeSink.
type Sink ¶
Sink is a destination for sending telemetry events.
func NewFilterSink ¶ added in v0.42.0
NewFilterSink returns a sink that filters events based on the provided filter function. Only events for which the filter function returns true will be emitted to the wrapped sink.
func NewIntakeSink ¶ added in v0.42.0
func NewIntakeSink(logger *zap.Logger, opts IntakeSinkOptions) Sink
NewIntakeSink creates a new sink that sends events to the Rill intake API.
func NewKafkaSink ¶
NewKafkaSink returns a sink that sends events to a Kafka topic.
func NewLoggerSink ¶ added in v0.42.0
NewLoggerSink returns a sink that logs events to the given logger.