ingester

package
v1.18.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 3, 2024 License: Apache-2.0 Imports: 57 Imported by: 1

Documentation

Index

Constants

View Source 
const (
	// RingKey is the key under which we store the ingesters ring in the KVStore.
	RingKey = "ring"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type ActiveSeries added in v1.5.0 

type ActiveSeries struct {
	// contains filtered or unexported fields
}

ActiveSeries is keeping track of recently active series for a single tenant.

func NewActiveSeries added in v1.5.0 

func NewActiveSeries() *ActiveSeries

func (*ActiveSeries) Active added in v1.5.0 

func (c *ActiveSeries) Active() int

func (*ActiveSeries) Purge added in v1.5.0 

func (c *ActiveSeries) Purge(keepUntil time.Time)

Purge removes expired entries from the cache. This function should be called periodically to avoid memory leaks.

func (*ActiveSeries) UpdateSeries added in v1.5.0 

func (c *ActiveSeries) UpdateSeries(series labels.Labels, hash uint64, now time.Time, labelsCopy func(labels.Labels) labels.Labels)

Updates series timestamp to 'now'. Function is called to make a copy of labels if entry doesn't exist yet.

type Config 

type Config struct {
	LifecyclerConfig ring.LifecyclerConfig `yaml:"lifecycler"`

	// Config for metadata purging.
	MetadataRetainPeriod time.Duration `yaml:"metadata_retain_period"`

	RateUpdatePeriod            time.Duration `yaml:"rate_update_period"`
	UserTSDBConfigsUpdatePeriod time.Duration `yaml:"user_tsdb_configs_update_period"`

	ActiveSeriesMetricsEnabled      bool          `yaml:"active_series_metrics_enabled"`
	ActiveSeriesMetricsUpdatePeriod time.Duration `yaml:"active_series_metrics_update_period"`
	ActiveSeriesMetricsIdleTimeout  time.Duration `yaml:"active_series_metrics_idle_timeout"`

	// Use blocks storage.
	BlocksStorageConfig cortex_tsdb.BlocksStorageConfig `yaml:"-"`

	// UploadCompactedBlocksEnabled enables uploading compacted blocks.
	UploadCompactedBlocksEnabled bool `yaml:"upload_compacted_blocks_enabled"`

	// Injected at runtime and read from the distributor config, required
	// to accurately apply global limits.
	DistributorShardingStrategy string `yaml:"-"`
	DistributorShardByAllLabels bool   `yaml:"-"`

	// Injected at runtime and read from querier config.
	QueryIngestersWithin time.Duration `yaml:"-"`

	DefaultLimits    InstanceLimits         `yaml:"instance_limits"`
	InstanceLimitsFn func() *InstanceLimits `yaml:"-"`

	IgnoreSeriesLimitForMetricNames string `yaml:"ignore_series_limit_for_metric_names"`

	// For admin contact details
	AdminLimitMessage string `yaml:"admin_limit_message"`

	LabelsStringInterningEnabled bool `yaml:"labels_string_interning_enabled"`
	// contains filtered or unexported fields
}

Config for an Ingester.

func (*Config) RegisterFlags 

func (cfg *Config) RegisterFlags(f *flag.FlagSet)

RegisterFlags adds the flags required to config this to the given FlagSet

func (*Config) Validate added in v1.17.0 

func (cfg *Config) Validate() error

type Ingester 

type Ingester struct {
	*services.BasicService

	// Prometheus block storage
	TSDBState TSDBState
	// contains filtered or unexported fields
}

Ingester deals with "in flight" chunks. Based on Prometheus 1.x MemorySeriesStorage.

func New 

func New(cfg Config, limits *validation.Overrides, registerer prometheus.Registerer, logger log.Logger) (*Ingester, error)

New returns a new Ingester that uses Cortex block storage instead of chunks storage.

func NewForFlusher added in v1.0.0 

func NewForFlusher(cfg Config, limits *validation.Overrides, registerer prometheus.Registerer, logger log.Logger) (*Ingester, error)

NewForFlusher constructs a new Ingester to be used by flusher target. Compared to the 'New' method:

  • Always replays the WAL.
  • Does not start the lifecycler.

this is a special version of ingester used by Flusher. This ingester is not ingesting anything, its only purpose is to react on Flush method and flush all opened TSDBs when called.

func (*Ingester) AllUserStats 

func (i *Ingester) AllUserStats(_ context.Context, _ *client.UserStatsRequest) (*client.UsersStatsResponse, error)

AllUserStats returns ingestion statistics for all users known to this ingester.

func (*Ingester) CheckReady added in v0.7.0 

func (i *Ingester) CheckReady(ctx context.Context) error

CheckReady is the readiness handler used to indicate to k8s when the ingesters are ready for the addition or removal of another ingester.

func (*Ingester) Flush 

func (i *Ingester) Flush()

Flush triggers a flush of all the chunks and closes the flush queues. Called from the Lifecycler as part of the ingester shutdown.

func (*Ingester) FlushHandler 

func (i *Ingester) FlushHandler(w http.ResponseWriter, r *http.Request)

FlushHandler triggers a flush of all in memory chunks. Mainly used for local testing.

func (*Ingester) LabelNames 

func (i *Ingester) LabelNames(ctx context.Context, req *client.LabelNamesRequest) (*client.LabelNamesResponse, error)

LabelNames return all the label names.

func (*Ingester) LabelNamesStream added in v1.13.0 

func (i *Ingester) LabelNamesStream(req *client.LabelNamesRequest, stream client.Ingester_LabelNamesStreamServer) error

LabelNamesStream return all the label names.

func (*Ingester) LabelValues 

func (i *Ingester) LabelValues(ctx context.Context, req *client.LabelValuesRequest) (*client.LabelValuesResponse, error)

LabelValues returns all label values that are associated with a given label name.

func (*Ingester) LabelValuesStream added in v1.13.0 

func (i *Ingester) LabelValuesStream(req *client.LabelValuesRequest, stream client.Ingester_LabelValuesStreamServer) error

LabelValuesStream returns all label values that are associated with a given label name.

func (*Ingester) MetricsForLabelMatchers 

func (i *Ingester) MetricsForLabelMatchers(ctx context.Context, req *client.MetricsForLabelMatchersRequest) (*client.MetricsForLabelMatchersResponse, error)

MetricsForLabelMatchers returns all the metrics which match a set of matchers.

func (*Ingester) MetricsForLabelMatchersStream added in v1.13.0 

func (i *Ingester) MetricsForLabelMatchersStream(req *client.MetricsForLabelMatchersRequest, stream client.Ingester_MetricsForLabelMatchersStreamServer) error

func (*Ingester) MetricsMetadata added in v1.1.0 

func (i *Ingester) MetricsMetadata(ctx context.Context, _ *client.MetricsMetadataRequest) (*client.MetricsMetadataResponse, error)

MetricsMetadata returns all the metric metadata of a user.

func (*Ingester) Push 

func (i *Ingester) Push(ctx context.Context, req *cortexpb.WriteRequest) (*cortexpb.WriteResponse, error)

Push adds metrics to a block

func (*Ingester) QueryExemplars added in v1.10.0 

func (i *Ingester) QueryExemplars(ctx context.Context, req *client.ExemplarQueryRequest) (*client.ExemplarQueryResponse, error)

QueryExemplars implements service.IngesterServer

func (*Ingester) QueryStream 

func (i *Ingester) QueryStream(req *client.QueryRequest, stream client.Ingester_QueryStreamServer) error

QueryStream implements service.IngesterServer Streams metrics from a TSDB. This implements the client.IngesterServer interface

func (*Ingester) RenewTokenHandler added in v1.18.0 

func (i *Ingester) RenewTokenHandler(w http.ResponseWriter, r *http.Request)

func (*Ingester) ShutdownHandler added in v0.4.0 

func (i *Ingester) ShutdownHandler(w http.ResponseWriter, _ *http.Request)

ShutdownHandler triggers the following set of operations in order:

  • Change the state of ring to stop accepting writes.
  • Flush all the chunks.

func (*Ingester) UserStats 

func (i *Ingester) UserStats(ctx context.Context, req *client.UserStatsRequest) (*client.UserStatsResponse, error)

UserStats returns ingestion statistics for the current user.

type InstanceLimits added in v1.9.0 

type InstanceLimits struct {
	MaxIngestionRate         float64 `yaml:"max_ingestion_rate"`
	MaxInMemoryTenants       int64   `yaml:"max_tenants"`
	MaxInMemorySeries        int64   `yaml:"max_series"`
	MaxInflightPushRequests  int64   `yaml:"max_inflight_push_requests"`
	MaxInflightQueryRequests int64   `yaml:"max_inflight_query_requests"`
}

InstanceLimits describes limits used by ingester. Reaching any of these will result in Push method to return (internal) error.

func (*InstanceLimits) UnmarshalYAML added in v1.9.0 

func (l *InstanceLimits) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements the yaml.Unmarshaler interface. If give

type Limiter added in v1.1.0 

type Limiter struct {
	AdminLimitMessage string
	// contains filtered or unexported fields
}

Limiter implements primitives to get the maximum number of series an ingester can handle for a specific tenant

func NewLimiter added in v1.1.0 

func NewLimiter(
	limits *validation.Overrides,
	ring RingCount,
	shardingStrategy string,
	shardByAllLabels bool,
	replicationFactor int,
	zoneAwarenessEnabled bool,
	AdminLimitMessage string,
) *Limiter

NewLimiter makes a new in-memory series limiter

func (*Limiter) AssertMaxMetadataPerMetric added in v1.1.0 

func (l *Limiter) AssertMaxMetadataPerMetric(userID string, metadata int) error

AssertMaxMetadataPerMetric limit has not been reached compared to the current number of metadata per metric in input and returns an error if so.

func (*Limiter) AssertMaxMetricsWithMetadataPerUser added in v1.1.0 

func (l *Limiter) AssertMaxMetricsWithMetadataPerUser(userID string, metrics int) error

AssertMaxMetricsWithMetadataPerUser limit has not been reached compared to the current number of metrics with metadata in input and returns an error if so.

func (*Limiter) AssertMaxSeriesPerLabelSet added in v1.18.0 

func (l *Limiter) AssertMaxSeriesPerLabelSet(userID string, metric labels.Labels, f func(validation.LimitsPerLabelSet) (int, error)) error

AssertMaxSeriesPerLabelSet limit has not been reached compared to the current number of metrics with metadata in input and returns an error if so.

func (*Limiter) AssertMaxSeriesPerMetric added in v1.1.0 

func (l *Limiter) AssertMaxSeriesPerMetric(userID string, series int) error

AssertMaxSeriesPerMetric limit has not been reached compared to the current number of series in input and returns an error if so.

func (*Limiter) AssertMaxSeriesPerUser added in v1.1.0 

func (l *Limiter) AssertMaxSeriesPerUser(userID string, series int) error

AssertMaxSeriesPerUser limit has not been reached compared to the current number of series in input and returns an error if so.

func (*Limiter) FormatError added in v1.9.0 

func (l *Limiter) FormatError(userID string, err error) error

FormatError returns the input error enriched with the actual limits for the given user. It acts as pass-through if the input error is unknown.

type RingCount added in v0.4.0 

type RingCount interface {
	HealthyInstancesCount() int
	ZonesCount() int
}

RingCount is the interface exposed by a ring implementation which allows to count members

type Shipper added in v0.7.0 

type Shipper interface {
	Sync(ctx context.Context) (uploaded int, err error)
}

Shipper interface is used to have an easy way to mock it in tests.

type TSDBState added in v0.4.0 

type TSDBState struct {
	// contains filtered or unexported fields
}

TSDBState holds data structures used by the TSDB storage engine

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.