conf

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2020 License: AGPL-3.0 Imports: 13 Imported by: 66

Documentation

Overview

Package conf reads config data from two of carbon's config files * storage-schemas.conf (old and new retention format) see https://graphite.readthedocs.io/en/0.9.9/config-carbon.html#storage-schemas-conf * storage-aggregation.conf see http://graphite.readthedocs.io/en/latest/config-carbon.html#storage-aggregation-conf as well as our own file index-rules.conf

it also adds defaults (the same ones as graphite), so that even if nothing is matched in the user provided schemas or aggregations, a setting is *always* found uses some modified snippets from github.com/lomik/go-carbon and github.com/lomik/go-whisper

Index

Constants

View Source 
const Month_sec = 60 * 60 * 24 * 28

Variables

This section is empty.

Functions

This section is empty.

Types

type Aggregation 

type Aggregation struct {
	Name              string
	Pattern           *regexp.Regexp
	XFilesFactor      float64
	AggregationMethod []Method
}

type Aggregations 

type Aggregations struct {
	Data               []Aggregation
	DefaultAggregation Aggregation
}

Aggregations holds the aggregation definitions

func NewAggregations 

func NewAggregations() Aggregations

NewAggregations create instance of Aggregations

func ReadAggregations 

func ReadAggregations(file string) (Aggregations, error)

ReadAggregations returns the defined aggregations from a storage-aggregation.conf file and adds the default

func (Aggregations) Get 

func (a Aggregations) Get(i uint16) Aggregation

Get returns the aggregation setting corresponding to the given index

func (Aggregations) Match 

func (a Aggregations) Match(metric string) (uint16, Aggregation)

Match returns the correct aggregation setting for the given metric it can always find a valid setting, because there's a default catch all also returns the index of the setting, to efficiently reference it

type IndexRule 

type IndexRule struct {
	Name     string
	Pattern  *regexp.Regexp
	MaxStale time.Duration
}

type IndexRules 

type IndexRules struct {
	Rules   []IndexRule
	Default IndexRule
}

IndexRules holds the index rule definitions

func NewIndexRules 

func NewIndexRules() IndexRules

NewIndexRules create instance of IndexRules it has a default catchall that doesn't prune

func ReadIndexRules 

func ReadIndexRules(file string) (IndexRules, error)

ReadIndexRules returns the defined index rule from a index-rules.conf file and adds the default

func (IndexRules) Cutoffs 

func (a IndexRules) Cutoffs(now time.Time) []int64

Cutoffs returns a set of cutoffs corresponding to a given timestamp and the set of all rules

func (IndexRules) Get 

func (a IndexRules) Get(i uint16) IndexRule

Get returns the index rule setting corresponding to the given index

func (IndexRules) Match 

func (a IndexRules) Match(metric string) (uint16, IndexRule)

Match returns the correct index rule setting for the given metric it can always find a valid setting, because there's a default catch all also returns the index of the setting, to efficiently reference it

func (IndexRules) Prunable 

func (a IndexRules) Prunable() bool

Prunable returns whether there's any entries that require pruning

type Method 

type Method int
const (
	Avg Method = iota + 1
	Sum
	Lst
	Max
	Min
)

type Retention 

type Retention struct {
	SecondsPerPoint int    // interval in seconds
	NumberOfPoints  int    // ~ttl
	ChunkSpan       uint32 // duration of chunk of aggregated metric for storage, controls how many aggregated points go into 1 chunk
	NumChunks       uint32 // number of chunks to keep in memory. remember, for a query from now until 3 months ago, we will end up querying the memory server as well.
	Ready           uint32 // ready for reads for data as of this timestamp (or as of now-TTL, whichever is highest)
}

A retention level.

Retention levels describe a given archive in the database. How detailed it is and how far back it records.

func NewRetention 

func NewRetention(secondsPerPoint, numberOfPoints int) Retention

func NewRetentionMT 

func NewRetentionMT(secondsPerPoint int, ttl, chunkSpan, numChunks, ready uint32) Retention

func ParseRetentionNew 

func ParseRetentionNew(def string) (Retention, error)

func (Retention) MaxRetention 

func (r Retention) MaxRetention() int

func (Retention) String added in v0.13.1 

func (r Retention) String() string

type Retentions 

type Retentions struct {
	Orig string
	Rets []Retention
}

func BuildFromRetentions added in v0.13.1 

func BuildFromRetentions(rets ...Retention) Retentions

func MustParseRetentions added in v0.13.1 

func MustParseRetentions(defs string) Retentions

func ParseRetentions 

func ParseRetentions(defs string) (Retentions, error)

ParseRetentions parses retention definitions into a Retentions structure

func (Retentions) Sub added in v0.13.1 

func (r Retentions) Sub(pos int) Retentions

Sub returns a "subslice" of Retentions starting at the given pos.

func (Retentions) Validate 

func (r Retentions) Validate() error

Validate assures the retentions are sane. As the whisper source code says: An ArchiveList must: 1. Have at least one archive config. Example: (60, 86400) 2. No archive may be a duplicate of another. 3. Higher precision archives' precision must evenly divide all lower precision archives' precision. 4. Lower precision archives must cover larger time intervals than higher precision archives. 5. Each archive must have at least enough points to consolidate to the next archive

type Schema 

type Schema struct {
	Name               string
	Pattern            *regexp.Regexp
	Retentions         Retentions
	Priority           int64
	ReorderWindow      uint32
	ReorderAllowUpdate bool
}

Schema represents one schema setting

type SchemaSlice 

type SchemaSlice []Schema

func (SchemaSlice) Len 

func (s SchemaSlice) Len() int

func (SchemaSlice) Less 

func (s SchemaSlice) Less(i, j int) bool

func (SchemaSlice) Swap 

func (s SchemaSlice) Swap(i, j int)

type Schemas 

type Schemas struct {
	DefaultSchema Schema
	// contains filtered or unexported fields
}

Schemas contains schema settings

func NewSchemas 

func NewSchemas(schemas []Schema) Schemas

func ReadSchemas 

func ReadSchemas(file string) (Schemas, error)

ReadSchemas reads and parses a storage-schemas.conf file and returns a sorted schemas structure see https://graphite.readthedocs.io/en/0.9.9/config-carbon.html#storage-schemas-conf

func (*Schemas) BuildIndex 

func (s *Schemas) BuildIndex()

func (Schemas) Get 

func (s Schemas) Get(i uint16) Schema

Get returns the schema setting corresponding to the given index

func (Schemas) Len added in v1.0.0 

func (s Schemas) Len() int

Len returns the max number of possible schemas

func (Schemas) ListRaw added in v1.0.0 

func (s Schemas) ListRaw() ([]Schema, Schema)

ListRaw returns the raw unexpanded schemas

func (Schemas) Match 

func (s Schemas) Match(metric string, interval int) (uint16, Schema)

Match returns the correct schema setting for the given metric it can always find a valid setting, because there's a default catch all also returns the index of the setting, to efficiently reference it.

A schema is just a pattern + retention policy. A retention policy is just a list of retentions. The s.index slice contains a schema for each sublist of an original schemas retentions. So if an original schema had a retention policy of 1s:1d,1m:7d,1h:1y then 3 schemas would be added to the index with same pattern as the original but retention policies of "1s:1d,1m:7d,1h:1y", "1m:7d,1h:1y" and "1h:1y".

|---------------------------------------------------------------------| | pattern 1 | pattern 2 | pattern 3 | |---------------------------------------------------------------------| | ret0 | ret1 | ret0 | ret1 | ret2 | ret0 | ret1 | |---------------------------------------------------------------------| | schema0 | schema1 | schema2 | schema3 | schema4 | schema5 | schema6 | |---------------------------------------------------------------------|

When evaluating a match we start with the first schema in the index and compare the regex pattern.

  • If it matches we then just find the retention set with the best fit. The best fit is when the interval is >= the rawInterval (first retention) and less then the interval of the next rollup.
  • If the pattern doesnt match, then we skip ahead to the next pattern.

eg. from the above diagram we would compare the pattern for schema0 

(pattern1), if it doesnt match we will then compare the pattern of
schema2 (pattern2) and if that doesnt match we would try schema5
(pattern3).

func (Schemas) MaxChunkSpan 

func (schemas Schemas) MaxChunkSpan() uint32

MaxChunkSpan returns the largest chunkspan seen amongst all archives of all schemas

func (Schemas) MaxRawRetention added in v1.0.0 

func (schemas Schemas) MaxRawRetention() int

MaxRawRetention returns the largest maximum retention seen amongst the raw archive of all schemas

func (Schemas) TTLs 

func (schemas Schemas) TTLs() []uint32

TTLs returns a slice of all TTL's seen amongst all archives of all schemas

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.