memory

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 19, 2018 License: AGPL-3.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	EQUAL      match = iota // =
	NOT_EQUAL               // !=
	MATCH                   // =~        regular expression
	MATCH_TAG               // __tag=~   relies on special key __tag. non-standard, required for `/metrics/tags` requests with "filter"
	NOT_MATCH               // !=~
	PREFIX                  // ^=        exact prefix, not regex. non-standard, required for auto complete of tag values
	PREFIX_TAG              // __tag^=   exact prefix with tag. non-standard, required for auto complete of tag keys
)

Variables

View Source 
var (
	Enabled bool

	TagSupport      bool
	TagQueryWorkers int // number of workers to spin up when evaluation tag expressions

	IndexRules conf.IndexRules
)

Functions

func ConfigProcess 

func ConfigProcess()

func ConfigSetup 

func ConfigSetup()

Types

type IdSet 

type IdSet map[schema.MKey]struct{} // set of ids

func (IdSet) String 

func (ids IdSet) String() string

type KvByCost 

type KvByCost []kv

func (KvByCost) Len 

func (a KvByCost) Len() int

func (KvByCost) Less 

func (a KvByCost) Less(i, j int) bool

func (KvByCost) Swap 

func (a KvByCost) Swap(i, j int)

type KvReByCost 

type KvReByCost []kvRe

func (KvReByCost) Len 

func (a KvReByCost) Len() int

func (KvReByCost) Less 

func (a KvReByCost) Less(i, j int) bool

func (KvReByCost) Swap 

func (a KvReByCost) Swap(i, j int)

type MemoryIdx 

type MemoryIdx struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Implements the the "MetricIndex" interface

func New 

func New() *MemoryIdx

func (*MemoryIdx) AddOrUpdate 

func (m *MemoryIdx) AddOrUpdate(mkey schema.MKey, data *schema.MetricData, partition int32) (idx.Archive, int32, bool)

AddOrUpdate returns the corresponding Archive for the MetricData. if it is existing -> updates lastUpdate based on .Time, and partition if was new -> adds new MetricDefinition to index

func (*MemoryIdx) Delete 

func (m *MemoryIdx) Delete(orgId uint32, pattern string) ([]idx.Archive, error)

func (*MemoryIdx) DeleteTagged 

func (m *MemoryIdx) DeleteTagged(orgId uint32, paths []string) ([]idx.Archive, error)

func (*MemoryIdx) Find 

func (m *MemoryIdx) Find(orgId uint32, pattern string, from int64) ([]idx.Node, error)

func (*MemoryIdx) FindByTag 

func (m *MemoryIdx) FindByTag(orgId uint32, expressions []string, from int64) ([]idx.Node, error)

func (*MemoryIdx) FindTagValues 

func (m *MemoryIdx) FindTagValues(orgId uint32, tag, prefix string, expressions []string, from int64, limit uint) ([]string, error)

FindTagValues returns tag values matching the specified conditions tag: tag key match prefix: value prefix match expressions: tagdb expressions in the same format as graphite from: tags must have at least one metric with LastUpdate >= from limit: the maximum number of results to return

the results will always be sorted alphabetically for consistency

func (*MemoryIdx) FindTags 

func (m *MemoryIdx) FindTags(orgId uint32, prefix string, expressions []string, from int64, limit uint) ([]string, error)

FindTags returns tags matching the specified conditions prefix: prefix match expressions: tagdb expressions in the same format as graphite from: tags must have at least one metric with LastUpdate >= from limit: the maximum number of results to return

the results will always be sorted alphabetically for consistency

func (*MemoryIdx) Get 

func (m *MemoryIdx) Get(id schema.MKey) (idx.Archive, bool)

func (*MemoryIdx) GetPath 

func (m *MemoryIdx) GetPath(orgId uint32, path string) []idx.Archive

GetPath returns the node under the given org and path. this is an alternative to Find for when you have a path, not a pattern, and want to lookup in a specific org tree only.

func (*MemoryIdx) Init 

func (m *MemoryIdx) Init() error

func (*MemoryIdx) List 

func (m *MemoryIdx) List(orgId uint32) []idx.Archive

func (*MemoryIdx) Load 

func (m *MemoryIdx) Load(defs []schema.MetricDefinition) int

Used to rebuild the index from an existing set of metricDefinitions.

func (*MemoryIdx) Prune 

func (m *MemoryIdx) Prune(now time.Time) ([]idx.Archive, error)

Prune prunes series from the index if they have become stale per their index-rule

func (*MemoryIdx) Stop 

func (m *MemoryIdx) Stop()

func (*MemoryIdx) TagDetails 

func (m *MemoryIdx) TagDetails(orgId uint32, key, filter string, from int64) (map[string]uint64, error)

func (*MemoryIdx) Tags 

func (m *MemoryIdx) Tags(orgId uint32, filter string, from int64) ([]string, error)

Tags returns a list of all tag keys associated with the metrics of a given organization. The return values are filtered by the regex in the second parameter. If the third parameter is >0 then only metrics will be accounted of which the LastUpdate time is >= the given value.

func (*MemoryIdx) Update 

func (m *MemoryIdx) Update(point schema.MetricPoint, partition int32) (idx.Archive, int32, bool)

Update updates an existing archive, if found. It returns whether it was found, and - if so - the (updated) existing archive and its old partition

func (*MemoryIdx) UpdateArchive 

func (m *MemoryIdx) UpdateArchive(archive idx.Archive)

UpdateArchive updates the archive information

type Node 

type Node struct {
	Path     string
	Children []string
	Defs     []schema.MKey
}

func (*Node) HasChildren 

func (n *Node) HasChildren() bool

func (*Node) Leaf 

func (n *Node) Leaf() bool

func (*Node) String 

func (n *Node) String() string

type TagIndex 

type TagIndex map[string]TagValue // key -> list of values

type TagQuery 

type TagQuery struct {
	// contains filtered or unexported fields
}

TagQuery runs a set of pattern or string matches on tag keys and values against the index. It is executed via: Run() which returns a set of matching MetricIDs RunGetTags() which returns a list of tags of the matching metrics

func NewTagQuery 

func NewTagQuery(expressions []string, from int64) (TagQuery, error)

func (*TagQuery) Run 

func (q *TagQuery) Run(index TagIndex, byId map[schema.MKey]*idx.Archive) IdSet

Run executes the tag query on the given index and returns a list of ids

func (*TagQuery) RunGetTags 

func (q *TagQuery) RunGetTags(index TagIndex, byId map[schema.MKey]*idx.Archive) map[string]struct{}

RunGetTags executes the tag query and returns all the tags of the resulting metrics

type TagValue 

type TagValue map[string]IdSet // value -> set of ids

type TimeLimiter 

type TimeLimiter struct {
	// contains filtered or unexported fields
}

TimeLimiter limits the rate of a set of serial operations. It does this by tracking how much time has been spent (updated via Add()), and comparing this to the window size and the limit, slowing down further operations as soon as one Add() is called informing it the per-window allowed budget has been exceeded. Limitations: * the last operation is allowed to exceed the budget (but the next call will be delayed to compensate) * concurrency is not supported

For correctness, you should always follow up an Add() with a Wait()

func NewTimeLimiter 

func NewTimeLimiter(window, limit time.Duration, now time.Time) *TimeLimiter

NewTimeLimiter creates a new TimeLimiter. limit must <= window

func (*TimeLimiter) Add 

func (l *TimeLimiter) Add(d time.Duration)

Add increments the "time spent" counter by "d"

func (*TimeLimiter) Wait 

func (l *TimeLimiter) Wait()

Wait returns when we are not rate limited

  • if we passed the window, we reset everything (this is only safe for callers that behave correctly, i.e. that wait the instructed time after each add)
  • if limit is not reached, no sleep is needed
  • if limit has been exceeded, sleep until next period + extra multiple to compensate this is perhaps best explained with an example: if window is 1s and limit 100ms, but we spent 250ms, then we spent effectively 2.5 seconds worth of work. let's say we are 800ms into the 1s window, that means we should sleep 2500-800 = 1.7s in order to maximize work while honoring the imposed limit.
  • if limit has been met exactly, sleep until next period (this is a special case of the above)

type Tree 

type Tree struct {
	Items map[string]*Node // key is the full path of the node.
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.