distributor

README

distributor

This module is responsible for accepting incoming client requests. This includes:

• Snapshots: Requests to ingest new snapshots from clients
• Polls: Requests for client tracepoint configs

Overview

This module is deployed using the target 'distributor'. Once started it will create receivers based on the config, which will accept client requests. These requests are then processed into a common format deep-proto, and processed by a common logic.

Snapshots

When the distributor receives a snapshot, it will forward the data to an ingester instance. This will be done by using a Ring client to select the correct ingester.

Polls

When the distributor receives a poll, it will forward the request to a tracepoint instance. These instances will then load the config for the Org Id and client and send the response back to the client.

Metrics

There are several metrics generated by the distributor that track requests and performance.

See distributor dashboards

Documentation

Index

• type Config
  • func (cfg *Config) RegisterFlagsAndApplyDefaults(prefix string, f *flag.FlagSet)
• type Distributor
  • func New(cfg Config, tpClient *client.TPClient, clientCfg ingester_client.Config, ...) (*Distributor, error)
  • func (*Distributor) Check(_ context.Context, _ *grpc_health_v1.HealthCheckRequest) (*grpc_health_v1.HealthCheckResponse, error)
  • func (d *Distributor) PushPoll(ctx context.Context, pollRequest *pb.PollRequest) (*pb.PollResponse, error)
  • func (d *Distributor) PushSnapshot(ctx context.Context, in *tp.Snapshot) (*tp.SnapshotResponse, error)
• type LogReceivedSnapshotsConfig
• type ReadLifecycler
• type RingConfig
  • func (cfg *RingConfig) RegisterFlags(f *flag.FlagSet)
  • func (cfg *RingConfig) ToLifecyclerConfig() ring.LifecyclerConfig

Types

type Config

type Config struct {
	// Distributors ring
	DistributorRing      RingConfig                 `yaml:"ring,omitempty"`
	Receivers            map[string]interface{}     `yaml:"receivers"`
	OverrideRingKey      string                     `yaml:"override_ring_key"`
	LogReceivedSnapshots LogReceivedSnapshotsConfig `yaml:"log_received_snapshots"`

	Forwarders forwarder.ConfigList `yaml:"forwarders"`

	// disables write extension with inactive ingesters. Use this along with ingester.lifecycler.unregister_on_shutdown = true
	//  note that setting these two config values reduces tolerance to failures on rollout b/c there is always one guaranteed to be failing replica
	ExtendWrites bool `yaml:"extend_writes"`
	// contains filtered or unexported fields
}

Config for a Distributor.

func (*Config) RegisterFlagsAndApplyDefaults

func (cfg *Config) RegisterFlagsAndApplyDefaults(prefix string, f *flag.FlagSet)

RegisterFlagsAndApplyDefaults registers flags and applies defaults

type Distributor

type Distributor struct {
	services.Service

	DistributorRing *ring.Ring

	SnapshotReceiver tp.SnapshotServiceServer
	// contains filtered or unexported fields
}

Distributor coordinates replicates and distribution of log streams.

func New

func New(cfg Config, tpClient *client.TPClient, clientCfg ingester_client.Config, middleware snapshotreceiver.Middleware, ingestersRing ring.ReadRing, generatorClientCfg generator_client.Config, generatorsRing ring.ReadRing, o *overrides.Overrides, logger log.Logger, reg prometheus.Registerer) (*Distributor, error)

New crates a new distributor services

func (*Distributor) Check

func (*Distributor) Check(_ context.Context, _ *grpc_health_v1.HealthCheckRequest) (*grpc_health_v1.HealthCheckResponse, error)

Check implements the grpc healthcheck

func (*Distributor) PushPoll

func (d *Distributor) PushPoll(ctx context.Context, pollRequest *pb.PollRequest) (*pb.PollResponse, error)

func (*Distributor) PushSnapshot

func (d *Distributor) PushSnapshot(ctx context.Context, in *tp.Snapshot) (*tp.SnapshotResponse, error)

type LogReceivedSnapshotsConfig

type LogReceivedSnapshotsConfig struct {
	Enabled              bool `yaml:"enabled"`
	IncludeAllAttributes bool `yaml:"include_all_attributes"`
}

type ReadLifecycler

type ReadLifecycler interface {
	HealthyInstancesCount() int
}

ReadLifecycler represents the read interface to the lifecycler.

type RingConfig

type RingConfig struct {
	KVStore          kv.Config     `yaml:"kvstore"`
	HeartbeatPeriod  time.Duration `yaml:"heartbeat_period"`
	HeartbeatTimeout time.Duration `yaml:"heartbeat_timeout"`

	// Instance details
	InstanceID             string   `yaml:"instance_id" doc:"hidden"`
	InstanceInterfaceNames []string `yaml:"instance_interface_names"`
	InstancePort           int      `yaml:"instance_port" doc:"hidden"`
	InstanceAddr           string   `yaml:"instance_addr" doc:"hidden"`

	// Injected internally
	ListenPort int `yaml:"-"`
}

RingConfig masks the ring lifecycler config which contains many options not really required by the distributors ring. This config is used to strip down the config to the minimum, and avoid confusion to the user.

func (*RingConfig) RegisterFlags

func (cfg *RingConfig) RegisterFlags(f *flag.FlagSet)

RegisterFlags adds the flags required to config this to the given FlagSet

func (*RingConfig) ToLifecyclerConfig

func (cfg *RingConfig) ToLifecyclerConfig() ring.LifecyclerConfig

ToLifecyclerConfig returns a LifecyclerConfig based on the distributor ring config.