storage

package
v1.102.1-cluster Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 1, 2024 License: Apache-2.0 Imports: 46 Imported by: 10

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// MetricsWithDroppedLabels is the number of metrics with at least a single dropped label
	MetricsWithDroppedLabels atomic.Uint64

	// TooLongLabelNames is the number of too long label names
	TooLongLabelNames atomic.Uint64

	// TooLongLabelValues is the number of too long label values
	TooLongLabelValues atomic.Uint64
)
View Source 
var ErrDeadlineExceeded = fmt.Errorf("deadline exceeded")

ErrDeadlineExceeded is returned when the request times out.

Functions

func DeduplicateSamples added in v1.33.0 

func DeduplicateSamples(srcTimestamps []int64, srcValues []float64, dedupInterval int64) ([]int64, []float64)

DeduplicateSamples removes samples from src* if they are closer to each other than dedupInterval in milliseconds. DeduplicateSamples treats StaleNaN (Prometheus stale markers) as values and doesn't skip them on purpose - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5587

func GetDedupInterval added in v1.71.0 

func GetDedupInterval() int64

GetDedupInterval returns the dedup interval in milliseconds, which has been set via SetDedupInterval.

func MarshalBlock 

func MarshalBlock(dst []byte, b *Block) []byte

MarshalBlock marshals b to dst.

b.MarshalData must be called on b before calling MarshalBlock.

func MarshalMetricLabelRaw 

func MarshalMetricLabelRaw(dst []byte, label *prompb.Label) []byte

MarshalMetricLabelRaw marshals label to dst.

func MarshalMetricNameRaw 

func MarshalMetricNameRaw(dst []byte, accountID, projectID uint32, labels []prompb.Label) []byte

MarshalMetricNameRaw marshals labels to dst and returns the result.

The result must be unmarshaled with MetricName.UnmarshalRaw

func MarshalMetricRow 

func MarshalMetricRow(dst []byte, metricNameRaw []byte, timestamp int64, value float64) []byte

MarshalMetricRow marshals MetricRow data to dst and returns the result.

func PutMetricName 

func PutMetricName(mn *MetricName)

PutMetricName returns mn to the pool.

func RegexpCacheMaxSizeBytes added in v1.76.0 

func RegexpCacheMaxSizeBytes() int

RegexpCacheMaxSizeBytes returns the maximum size in bytes for the cached regexps for tag filters.

func RegexpCacheMisses 

func RegexpCacheMisses() uint64

RegexpCacheMisses returns the number of cache misses for regexp cache for tag filters.

func RegexpCacheRequests 

func RegexpCacheRequests() uint64

RegexpCacheRequests returns the number of requests to regexp cache for tag filters.

func RegexpCacheSize 

func RegexpCacheSize() int

RegexpCacheSize returns the number of cached regexps for tag filters.

func RegexpCacheSizeBytes added in v1.76.0 

func RegexpCacheSizeBytes() int

RegexpCacheSizeBytes returns an approximate size in bytes for the cached regexps for tag filters.

func RegexpPrefixesCacheMaxSizeBytes added in v1.76.0 

func RegexpPrefixesCacheMaxSizeBytes() int

RegexpPrefixesCacheMaxSizeBytes returns the maximum size in bytes for cached regexp prefixes for tag filters in bytes.

func RegexpPrefixesCacheMisses added in v1.76.0 

func RegexpPrefixesCacheMisses() uint64

RegexpPrefixesCacheMisses returns the number of cache misses for regexp prefixes cache.

func RegexpPrefixesCacheRequests added in v1.76.0 

func RegexpPrefixesCacheRequests() uint64

RegexpPrefixesCacheRequests returns the number of requests to regexp prefixes cache.

func RegexpPrefixesCacheSize added in v1.76.0 

func RegexpPrefixesCacheSize() int

RegexpPrefixesCacheSize returns the number of cached regexp prefixes for tag filters.

func RegexpPrefixesCacheSizeBytes added in v1.76.0 

func RegexpPrefixesCacheSizeBytes() int

RegexpPrefixesCacheSizeBytes returns an approximate size in bytes for cached regexp prefixes for tag filters.

func SetDataFlushInterval added in v1.85.0 

func SetDataFlushInterval(d time.Duration)

SetDataFlushInterval sets the interval for guaranteed flush of recently ingested data from memory to disk.

The data can be flushed from memory to disk more frequently if it doesn't fit the memory limit.

This function must be called before initializing the storage.

func SetDedupInterval added in v1.71.0 

func SetDedupInterval(dedupInterval time.Duration)

SetDedupInterval sets the deduplication interval, which is applied to raw samples during data ingestion and querying.

De-duplication is disabled if dedupInterval is 0.

This function must be called before initializing the storage.

func SetFreeDiskSpaceLimit added in v1.67.0 

func SetFreeDiskSpaceLimit(bytes int64)

SetFreeDiskSpaceLimit sets the minimum free disk space size of current storage path

The function must be called before opening or creating any storage.

func SetLogNewSeries added in v1.56.0 

func SetLogNewSeries(ok bool)

SetLogNewSeries updates new series logging.

This function must be called before any calling any storage functions.

func SetMaxLabelValueLen added in v1.71.0 

func SetMaxLabelValueLen(n int)

SetMaxLabelValueLen sets the limit on the label value length.

This function can be called before using the storage package.

Label values with longer length are truncated.

func SetMaxLabelsPerTimeseries added in v1.27.0 

func SetMaxLabelsPerTimeseries(maxLabels int)

SetMaxLabelsPerTimeseries sets the limit on the number of labels per each time series.

This function can be called before using the storage package.

Superfluous labels are dropped.

func SetRetentionTimezoneOffset added in v1.78.0 

func SetRetentionTimezoneOffset(offset time.Duration)

SetRetentionTimezoneOffset sets the offset, which is used for calculating the time for indexdb rotation. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2574

func SetTSIDCacheSize added in v1.73.1 

func SetTSIDCacheSize(size int)

SetTSIDCacheSize overrides the default size of storage/tsid cache

func SetTagFiltersCacheSize added in v1.83.0 

func SetTagFiltersCacheSize(size int)

SetTagFiltersCacheSize overrides the default size of tagFiltersToMetricIDsCache

func TimestampToHumanReadableFormat added in v1.79.0 

func TimestampToHumanReadableFormat(timestamp int64) string

TimestampToHumanReadableFormat converts the given timestamp to human-readable format.

func UnmarshalBlock 

func UnmarshalBlock(dst *Block, src []byte) ([]byte, error)

UnmarshalBlock unmarshal Block from src to dst.

dst.UnmarshalData isn't called on the block.

Types

type Block 

type Block struct {
	// contains filtered or unexported fields
}

Block represents a block of time series values for a single TSID.

func (*Block) AppendRowsWithTimeRangeFilter added in v1.42.0 

func (b *Block) AppendRowsWithTimeRangeFilter(dstTimestamps []int64, dstValues []float64, tr TimeRange) ([]int64, []float64)

AppendRowsWithTimeRangeFilter filters samples from b according to tr and appends them to dst*.

It is expected that UnmarshalData has been already called on b.

func (*Block) CopyFrom 

func (b *Block) CopyFrom(src *Block)

CopyFrom copies src to b.

func (*Block) Init 

func (b *Block) Init(tsid *TSID, timestamps, values []int64, scale int16, precisionBits uint8)

Init initializes b with the given tsid, timestamps, values and scale.

func (*Block) MarshalData 

func (b *Block) MarshalData(timestampsBlockOffset, valuesBlockOffset uint64) ([]byte, []byte, []byte)

MarshalData marshals the block into binary representation.

func (*Block) MarshalPortable added in v1.42.0 

func (b *Block) MarshalPortable(dst []byte) []byte

MarshalPortable marshals b to dst, so it could be portably migrated to other VictoriaMetrics instance.

The marshaled value must be unmarshaled with UnmarshalPortable function.

func (*Block) Reset 

func (b *Block) Reset()

Reset resets b.

func (*Block) RowsCount 

func (b *Block) RowsCount() int

RowsCount returns the number of rows in the block.

func (*Block) UnmarshalData 

func (b *Block) UnmarshalData() error

UnmarshalData unmarshals block data.

func (*Block) UnmarshalPortable added in v1.42.0 

func (b *Block) UnmarshalPortable(src []byte) ([]byte, error)

UnmarshalPortable unmarshals block from src to b and returns the remaining tail.

It is assumed that the block has been marshaled with MarshalPortable.

type BlockRef added in v1.35.0 

type BlockRef struct {
	// contains filtered or unexported fields
}

BlockRef references a Block.

BlockRef is valid only until the corresponding Search is valid, i.e. it becomes invalid after Search.MustClose is called.

func (*BlockRef) MustReadBlock added in v1.35.0 

func (br *BlockRef) MustReadBlock(dst *Block)

MustReadBlock reads block from br to dst.

type IndexDBMetrics 

type IndexDBMetrics struct {
	TagFiltersToMetricIDsCacheSize         uint64
	TagFiltersToMetricIDsCacheSizeBytes    uint64
	TagFiltersToMetricIDsCacheSizeMaxBytes uint64
	TagFiltersToMetricIDsCacheRequests     uint64
	TagFiltersToMetricIDsCacheMisses       uint64

	DeletedMetricsCount uint64

	IndexDBRefCount uint64

	MissingTSIDsForMetricID uint64

	RecentHourMetricIDsSearchCalls uint64
	RecentHourMetricIDsSearchHits  uint64

	DateRangeSearchCalls uint64
	DateRangeSearchHits  uint64
	GlobalSearchCalls    uint64

	MissingMetricNamesForMetricID uint64

	IndexBlocksWithMetricIDsProcessed      uint64
	IndexBlocksWithMetricIDsIncorrectOrder uint64

	MinTimestampForCompositeIndex     uint64
	CompositeFilterSuccessConversions uint64
	CompositeFilterMissingConversions uint64

	mergeset.TableMetrics
}

IndexDBMetrics contains essential metrics for indexDB.

type MetricBlock 

type MetricBlock struct {
	// MetricName is metric name for the given Block.
	MetricName []byte

	// Block is a block for the given MetricName
	Block Block
}

MetricBlock is a time series block for a single metric.

func (*MetricBlock) CopyFrom 

func (mb *MetricBlock) CopyFrom(src *MetricBlock)

CopyFrom copies src to mb.

func (*MetricBlock) Marshal 

func (mb *MetricBlock) Marshal(dst []byte) []byte

Marshal marshals MetricBlock to dst

func (*MetricBlock) Unmarshal 

func (mb *MetricBlock) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals MetricBlock from src

type MetricBlockRef added in v1.35.0 

type MetricBlockRef struct {
	// The metric name
	MetricName []byte

	// The block reference. Call BlockRef.MustReadBlock in order to obtain the block.
	BlockRef *BlockRef
}

MetricBlockRef contains reference to time series block for a single metric.

type MetricName 

type MetricName struct {
	AccountID uint32
	ProjectID uint32

	MetricGroup []byte

	// Tags are optional. They must be sorted by tag Key for canonical view.
	// Use sortTags method.
	Tags []Tag
}

MetricName reperesents a metric name.

func GetMetricName 

func GetMetricName() *MetricName

GetMetricName returns a MetricName from pool.

func (*MetricName) AddTag 

func (mn *MetricName) AddTag(key, value string)

AddTag adds new tag to mn with the given key and value.

func (*MetricName) AddTagBytes 

func (mn *MetricName) AddTagBytes(key, value []byte)

AddTagBytes adds new tag to mn with the given key and value.

func (*MetricName) CopyFrom 

func (mn *MetricName) CopyFrom(src *MetricName)

CopyFrom copies src to mn.

func (*MetricName) GetTagValue 

func (mn *MetricName) GetTagValue(tagKey string) []byte

GetTagValue returns tag value for the given tagKey.

func (*MetricName) Marshal 

func (mn *MetricName) Marshal(dst []byte) []byte

Marshal appends marshaled mn to dst and returns the result.

mn.sortTags must be called before calling this function in order to sort and de-duplcate tags.

func (*MetricName) MarshalNoAccountIDProjectID 

func (mn *MetricName) MarshalNoAccountIDProjectID(dst []byte) []byte

MarshalNoAccountIDProjectID appends marshaled mn without AccountID and ProjectID to dst and returns the result.

The result must be unmarshaled with UnmarshalNoAccountIDProjectID.

It is expected that mn.Tags are already sorted and de-duplicated with mn.sortTags.

func (*MetricName) MoveFrom added in v1.86.0 

func (mn *MetricName) MoveFrom(src *MetricName)

MoveFrom moves src to mn.

The src is reset after the call.

func (*MetricName) RemoveTag 

func (mn *MetricName) RemoveTag(tagKey string)

RemoveTag removes a tag with the given tagKey

func (*MetricName) RemoveTagsIgnoring 

func (mn *MetricName) RemoveTagsIgnoring(ignoringTags []string)

RemoveTagsIgnoring removes all the tags included in ignoringTags.

func (*MetricName) RemoveTagsOn 

func (mn *MetricName) RemoveTagsOn(onTags []string)

RemoveTagsOn removes all the tags not included to onTags.

func (*MetricName) Reset 

func (mn *MetricName) Reset()

Reset resets the mn.

func (*MetricName) ResetMetricGroup 

func (mn *MetricName) ResetMetricGroup()

ResetMetricGroup resets mn.MetricGroup

func (*MetricName) SetTagBytes added in v1.92.0 

func (mn *MetricName) SetTagBytes(key, value []byte)

SetTagBytes sets tag with the given key to the given value.

func (*MetricName) SetTags added in v1.37.3 

func (mn *MetricName) SetTags(addTags []string, prefix string, skipTags []string, src *MetricName)

SetTags sets tags from src with keys matching addTags.

It adds prefix to copied label names. skipTags contains a list of tags, which must be skipped.

func (*MetricName) String 

func (mn *MetricName) String() string

String returns user-readable representation of the metric name.

func (*MetricName) Unmarshal 

func (mn *MetricName) Unmarshal(src []byte) error

Unmarshal unmarshals mn from src.

func (*MetricName) UnmarshalNoAccountIDProjectID 

func (mn *MetricName) UnmarshalNoAccountIDProjectID(src []byte) error

UnmarshalNoAccountIDProjectID unmarshals mn, which has been marshaled with MarshalNoAccountIDProjectID

func (*MetricName) UnmarshalRaw added in v1.60.0 

func (mn *MetricName) UnmarshalRaw(src []byte) error

UnmarshalRaw unmarshals mn encoded with MarshalMetricNameRaw.

func (*MetricName) UnmarshalString added in v1.79.0 

func (mn *MetricName) UnmarshalString(s string) error

UnmarshalString unmarshals mn from s

type MetricRow 

type MetricRow struct {
	// MetricNameRaw contains raw metric name, which must be decoded
	// with MetricName.UnmarshalRaw.
	MetricNameRaw []byte

	Timestamp int64
	Value     float64
}

MetricRow is a metric to insert into storage.

func UnmarshalMetricRows 

func UnmarshalMetricRows(dst []MetricRow, src []byte, maxRows int) ([]MetricRow, []byte, error)

UnmarshalMetricRows appends unmarshaled MetricRow items from src to dst and returns the result.

Up to maxRows rows are unmarshaled at once. The remaining byte slice is returned to the caller.

The returned MetricRow items refer to src, so they become invalid as soon as src changes.

func (*MetricRow) CopyFrom 

func (mr *MetricRow) CopyFrom(src *MetricRow)

CopyFrom copies src to mr.

func (*MetricRow) Marshal 

func (mr *MetricRow) Marshal(dst []byte) []byte

Marshal appends marshaled mr to dst and returns the result.

func (*MetricRow) ResetX 

func (mr *MetricRow) ResetX()

ResetX resets mr after UnmarshalX or after UnmarshalMetricRows

func (*MetricRow) String 

func (mr *MetricRow) String() string

String returns string representation of the mr.

func (*MetricRow) UnmarshalX added in v1.60.0 

func (mr *MetricRow) UnmarshalX(src []byte) ([]byte, error)

UnmarshalX unmarshals mr from src and returns the remaining tail from src.

mr refers to src, so it remains valid until src changes.

type Metrics 

type Metrics struct {
	RowsAddedTotal    uint64
	DedupsDuringMerge uint64
	SnapshotsCount    uint64

	TooSmallTimestampRows uint64
	TooBigTimestampRows   uint64

	TimeseriesRepopulated  uint64
	TimeseriesPreCreated   uint64
	NewTimeseriesCreated   uint64
	SlowRowInserts         uint64
	SlowPerDayIndexInserts uint64
	SlowMetricNameLoads    uint64

	HourlySeriesLimitRowsDropped   uint64
	HourlySeriesLimitMaxSeries     uint64
	HourlySeriesLimitCurrentSeries uint64

	DailySeriesLimitRowsDropped   uint64
	DailySeriesLimitMaxSeries     uint64
	DailySeriesLimitCurrentSeries uint64

	TimestampsBlocksMerged uint64
	TimestampsBytesSaved   uint64

	TSIDCacheSize         uint64
	TSIDCacheSizeBytes    uint64
	TSIDCacheSizeMaxBytes uint64
	TSIDCacheRequests     uint64
	TSIDCacheMisses       uint64
	TSIDCacheCollisions   uint64

	MetricIDCacheSize         uint64
	MetricIDCacheSizeBytes    uint64
	MetricIDCacheSizeMaxBytes uint64
	MetricIDCacheRequests     uint64
	MetricIDCacheMisses       uint64
	MetricIDCacheCollisions   uint64

	MetricNameCacheSize         uint64
	MetricNameCacheSizeBytes    uint64
	MetricNameCacheSizeMaxBytes uint64
	MetricNameCacheRequests     uint64
	MetricNameCacheMisses       uint64
	MetricNameCacheCollisions   uint64

	DateMetricIDCacheSize        uint64
	DateMetricIDCacheSizeBytes   uint64
	DateMetricIDCacheSyncsCount  uint64
	DateMetricIDCacheResetsCount uint64

	HourMetricIDCacheSize      uint64
	HourMetricIDCacheSizeBytes uint64

	NextDayMetricIDCacheSize      uint64
	NextDayMetricIDCacheSizeBytes uint64

	PrefetchedMetricIDsSize      uint64
	PrefetchedMetricIDsSizeBytes uint64

	NextRetentionSeconds uint64

	IndexDBMetrics IndexDBMetrics
	TableMetrics   TableMetrics
}

Metrics contains essential metrics for the Storage.

func (*Metrics) Reset 

func (m *Metrics) Reset()

Reset resets m. 

type Search struct {
	// MetricBlockRef is updated with each Search.NextMetricBlock call.
	MetricBlockRef MetricBlockRef
	// contains filtered or unexported fields
}

Search is a search for time series.

func (*Search) Error 

func (s *Search) Error() error

Error returns the last error from s.

func (*Search) Init 

func (s *Search) Init(qt *querytracer.Tracer, storage *Storage, tfss []*TagFilters, tr TimeRange, maxMetrics int, deadline uint64) int

Init initializes s from the given storage, tfss and tr.

MustClose must be called when the search is done.

Init returns the upper bound on the number of found time series.

func (*Search) MustClose 

func (s *Search) MustClose()

MustClose closes the Search.

func (*Search) NextMetricBlock 

func (s *Search) NextMetricBlock() bool

NextMetricBlock proceeds to the next MetricBlockRef.

type SearchQuery 

type SearchQuery struct {
	AccountID uint32
	ProjectID uint32

	// The time range for searching time series
	MinTimestamp int64
	MaxTimestamp int64

	// Tag filters for the search query
	TagFilterss [][]TagFilter

	// The maximum number of time series the search query can return.
	MaxMetrics int
}

SearchQuery is used for sending search queries from vmselect to vmstorage.

func NewSearchQuery added in v1.47.0 

func NewSearchQuery(accountID, projectID uint32, start, end int64, tagFilterss [][]TagFilter, maxMetrics int) *SearchQuery

NewSearchQuery creates new search query for the given args.

func (*SearchQuery) GetTimeRange added in v1.79.0 

func (sq *SearchQuery) GetTimeRange() TimeRange

GetTimeRange returns time range for the given sq.

func (*SearchQuery) Marshal 

func (sq *SearchQuery) Marshal(dst []byte) []byte

Marshal appends marshaled sq to dst and returns the result.

func (*SearchQuery) String 

func (sq *SearchQuery) String() string

String returns string representation of the search query.

func (*SearchQuery) Unmarshal 

func (sq *SearchQuery) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals sq from src and returns the tail.

type Storage 

type Storage struct {
	// contains filtered or unexported fields
}

Storage represents TSDB storage.

func MustOpenStorage added in v1.91.0 

func MustOpenStorage(path string, retention time.Duration, maxHourlySeries, maxDailySeries int) *Storage

MustOpenStorage opens storage on the given path with the given retentionMsecs.

func (*Storage) AddRows 

func (s *Storage) AddRows(mrs []MetricRow, precisionBits uint8)

AddRows adds the given mrs to s.

The caller should limit the number of concurrent AddRows calls to the number of available CPU cores in order to limit memory usage.

func (*Storage) CreateSnapshot 

func (s *Storage) CreateSnapshot() (string, error)

CreateSnapshot creates snapshot for s and returns the snapshot name.

func (*Storage) DebugFlush added in v1.47.0 

func (s *Storage) DebugFlush()

DebugFlush makes sure all the recently added data is visible to search.

Note: this function doesn't store all the in-memory data to disk - it just converts recently added items to searchable parts, which can be stored either in memory (if they are quite small) or to persistent disk.

This function is for debugging and testing purposes only, since it may slow down data ingestion when used frequently.

func (*Storage) DeleteSeries added in v1.79.0 

func (s *Storage) DeleteSeries(qt *querytracer.Tracer, tfss []*TagFilters) (int, error)

DeleteSeries deletes all the series matching the given tfss.

Returns the number of metrics deleted.

func (*Storage) DeleteSnapshot 

func (s *Storage) DeleteSnapshot(snapshotName string) error

DeleteSnapshot deletes the given snapshot.

func (*Storage) DeleteStaleSnapshots added in v1.77.0 

func (s *Storage) DeleteStaleSnapshots(maxAge time.Duration) error

DeleteStaleSnapshots deletes snapshot older than given maxAge

func (*Storage) ForceMergePartitions added in v1.41.1 

func (s *Storage) ForceMergePartitions(partitionNamePrefix string) error

ForceMergePartitions force-merges partitions in s with names starting from the given partitionNamePrefix.

Partitions are merged sequentially in order to reduce load on the system.

func (*Storage) GetSeriesCount 

func (s *Storage) GetSeriesCount(accountID, projectID uint32, deadline uint64) (uint64, error)

GetSeriesCount returns the approximate number of unique time series for the given (accountID, projectID).

It includes the deleted series too and may count the same series up to two times - in db and extDB.

func (*Storage) GetTSDBStatus added in v1.78.0 

func (s *Storage) GetTSDBStatus(qt *querytracer.Tracer, accountID, projectID uint32, tfss []*TagFilters, date uint64, focusLabel string, topN, maxMetrics int, deadline uint64) (*TSDBStatus, error)

GetTSDBStatus returns TSDB status data for /api/v1/status/tsdb

func (*Storage) IsReadOnly added in v1.67.0 

func (s *Storage) IsReadOnly() bool

IsReadOnly returns information is storage in read only mode

func (*Storage) ListSnapshots 

func (s *Storage) ListSnapshots() ([]string, error)

ListSnapshots returns sorted list of existing snapshots for s.

func (*Storage) MustClose 

func (s *Storage) MustClose()

MustClose closes the storage.

It is expected that the s is no longer used during the close.

func (*Storage) RegisterMetricNames added in v1.47.0 

func (s *Storage) RegisterMetricNames(qt *querytracer.Tracer, mrs []MetricRow)

RegisterMetricNames registers all the metric names from mrs in the indexdb, so they can be queried later.

The the MetricRow.Timestamp is used for registering the metric name at the given day according to the timestamp. Th MetricRow.Value field is ignored.

func (*Storage) RetentionMsecs 

func (s *Storage) RetentionMsecs() int64

RetentionMsecs returns retentionMsecs for s.

func (*Storage) SearchGraphitePaths added in v1.53.0 

func (s *Storage) SearchGraphitePaths(qt *querytracer.Tracer, accountID, projectID uint32, tr TimeRange, query []byte, maxPaths int, deadline uint64) ([]string, error)

SearchGraphitePaths returns all the matching paths for the given graphite query on the given tr.

func (*Storage) SearchLabelNamesWithFiltersOnTimeRange added in v1.78.0 

func (s *Storage) SearchLabelNamesWithFiltersOnTimeRange(qt *querytracer.Tracer, accountID, projectID uint32, tfss []*TagFilters, tr TimeRange,
	maxLabelNames, maxMetrics int, deadline uint64,
) ([]string, error)

SearchLabelNamesWithFiltersOnTimeRange searches for label names matching the given tfss on tr.

func (*Storage) SearchLabelValuesWithFiltersOnTimeRange added in v1.78.0 

func (s *Storage) SearchLabelValuesWithFiltersOnTimeRange(qt *querytracer.Tracer, accountID, projectID uint32, labelName string, tfss []*TagFilters,
	tr TimeRange, maxLabelValues, maxMetrics int, deadline uint64,
) ([]string, error)

SearchLabelValuesWithFiltersOnTimeRange searches for label values for the given labelName, filters and tr.

func (*Storage) SearchMetricNames added in v1.47.0 

func (s *Storage) SearchMetricNames(qt *querytracer.Tracer, tfss []*TagFilters, tr TimeRange, maxMetrics int, deadline uint64) ([]string, error)

SearchMetricNames returns marshaled metric names matching the given tfss on the given tr.

The marshaled metric names must be unmarshaled via MetricName.UnmarshalString().

func (*Storage) SearchTagValueSuffixes added in v1.41.0 

func (s *Storage) SearchTagValueSuffixes(qt *querytracer.Tracer, accountID, projectID uint32, tr TimeRange, tagKey, tagValuePrefix string,
	delimiter byte, maxTagValueSuffixes int, deadline uint64,
) ([]string, error)

SearchTagValueSuffixes returns all the tag value suffixes for the given tagKey and tagValuePrefix on the given tr.

This allows implementing https://graphite-api.readthedocs.io/en/latest/api.html#metrics-find or similar APIs.

If more than maxTagValueSuffixes suffixes is found, then only the first maxTagValueSuffixes suffixes is returned.

func (*Storage) SearchTenants 

func (s *Storage) SearchTenants(qt *querytracer.Tracer, tr TimeRange, deadline uint64) ([]string, error)

SearchTenants returns list of registered tenants on the given tr.

func (*Storage) UpdateMetrics 

func (s *Storage) UpdateMetrics(m *Metrics)

UpdateMetrics updates m with metrics from s.

type TSDBStatus added in v1.35.0 

type TSDBStatus struct {
	TotalSeries                  uint64
	TotalLabelValuePairs         uint64
	SeriesCountByMetricName      []TopHeapEntry
	SeriesCountByLabelName       []TopHeapEntry
	SeriesCountByFocusLabelValue []TopHeapEntry
	SeriesCountByLabelValuePair  []TopHeapEntry
	LabelValueCountByLabelName   []TopHeapEntry
}

TSDBStatus contains TSDB status data for /api/v1/status/tsdb.

See https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-stats

type TSID 

type TSID struct {
	// AccountID is the id of the registered account.
	AccountID uint32

	// ProjectID is the id of the project.
	//
	// The ProjectID must be unique for the given AccountID.
	ProjectID uint32

	// MetricGroupID is the id of metric group inside the given project.
	//
	// MetricGroupID must be unique for the given (AccountID, ProjectID).
	//
	// Metric group contains metrics with the identical name like
	// 'memory_usage', 'http_requests', but with different
	// labels. For instance, the following metrics belong
	// to a metric group 'memory_usage':
	//
	//   memory_usage{datacenter="foo1", job="bar1", instance="baz1:1234"}
	//   memory_usage{datacenter="foo1", job="bar1", instance="baz2:1234"}
	//   memory_usage{datacenter="foo1", job="bar2", instance="baz1:1234"}
	//   memory_usage{datacenter="foo2", job="bar1", instance="baz2:1234"}
	MetricGroupID uint64

	// JobID is the id of an individual job (aka service)
	// for the given project.
	//
	// JobID must be unique for the given (AccountID, ProjectID).
	//
	// Service may consist of multiple instances.
	// See https://prometheus.io/docs/concepts/jobs_instances/ for details.
	JobID uint32

	// InstanceID is the id of an instance (aka process)
	// for the given project.
	//
	// InstanceID must be unique for the given (AccountID, ProjectID).
	//
	// See https://prometheus.io/docs/concepts/jobs_instances/ for details.
	InstanceID uint32

	// MetricID is the unique id of the metric (time series).
	//
	// All the other TSID fields may be obtained by MetricID.
	MetricID uint64
}

TSID is unique id for a time series.

Time series blocks are sorted by TSID.

All the fields except MetricID are optional. They exist solely for better grouping of related metrics. It is OK if their meaning differ from their naming.

func (*TSID) Less 

func (t *TSID) Less(b *TSID) bool

Less return true if t < b.

func (*TSID) Marshal 

func (t *TSID) Marshal(dst []byte) []byte

Marshal appends marshaled t to dst and returns the result.

func (*TSID) Unmarshal 

func (t *TSID) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals t from src and returns the rest of src.

type TableMetrics 

type TableMetrics struct {

	// LastPartition contains metrics for the last partition.
	// These metrics are important, since the majority of data ingestion
	// and querying goes to the last partition.
	LastPartition partitionMetrics

	PartitionsRefCount uint64
	// contains filtered or unexported fields
}

TableMetrics contains essential metrics for the table.

func (*TableMetrics) TotalRowsCount added in v1.85.0 

func (pm *TableMetrics) TotalRowsCount() uint64

TotalRowsCount returns total number of rows in tm.

type Tag 

type Tag struct {
	Key   []byte
	Value []byte
}

Tag represents a (key, value) tag for metric.

func (*Tag) Equal added in v1.28.0 

func (tag *Tag) Equal(t *Tag) bool

Equal returns true if tag equals t

func (*Tag) Marshal 

func (tag *Tag) Marshal(dst []byte) []byte

Marshal appends marshaled tag to dst and returns the result.

func (*Tag) Reset added in v1.28.0 

func (tag *Tag) Reset()

Reset resets the tag.

func (*Tag) Unmarshal 

func (tag *Tag) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals tag from src and returns the remaining data from src.

type TagFilter 

type TagFilter struct {
	Key        []byte
	Value      []byte
	IsNegative bool
	IsRegexp   bool
}

TagFilter represents a single tag filter from SearchQuery.

func (*TagFilter) Marshal 

func (tf *TagFilter) Marshal(dst []byte) []byte

Marshal appends marshaled tf to dst and returns the result.

func (*TagFilter) String 

func (tf *TagFilter) String() string

String returns string representation of tf.

func (*TagFilter) Unmarshal 

func (tf *TagFilter) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals tf from src and returns the tail.

type TagFilters 

type TagFilters struct {
	// contains filtered or unexported fields
}

TagFilters represents filters used for filtering tags.

func NewTagFilters 

func NewTagFilters(accountID, projectID uint32) *TagFilters

NewTagFilters returns new TagFilters for the given accountID and projectID.

func (*TagFilters) Add 

func (tfs *TagFilters) Add(key, value []byte, isNegative, isRegexp bool) error

Add adds the given tag filter to tfs.

MetricGroup must be encoded with nil key.

func (*TagFilters) AddGraphiteQuery added in v1.53.0 

func (tfs *TagFilters) AddGraphiteQuery(query []byte, paths []string, isNegative bool)

AddGraphiteQuery adds the given Graphite query that matches the given paths to tfs.

func (*TagFilters) Reset 

func (tfs *TagFilters) Reset(accountID, projectID uint32)

Reset resets the tf for the given accountID and projectID

func (*TagFilters) String 

func (tfs *TagFilters) String() string

String returns human-readable value for tfs.

type TimeRange 

type TimeRange struct {
	MinTimestamp int64
	MaxTimestamp int64
}

TimeRange is time range.

func (*TimeRange) String 

func (tr *TimeRange) String() string

type TopHeapEntry added in v1.35.0 

type TopHeapEntry struct {
	Name  string
	Count uint64
}

TopHeapEntry represents an entry from `top heap` used in stats.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.