tagquery

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2020 License: AGPL-3.0, BSD-2-Clause Imports: 13 Imported by: 35

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	MatchCacheSize int
	MetaTagSupport bool

	// the function we use to get the hash for hashing the meta records
	// it can be replaced for mocking in tests
	QueryHash func() util.StringHash32
)

Functions

func IsSeriesByTagExpression added in v1.0.0 

func IsSeriesByTagExpression(query string) bool

IsSeriesByTag returns true if the given query is a `seriesByTag` query

Types

type Expression 

type Expression interface {
	// Equals takes another expression and compares it against itself. Returns true if they are equal
	// or false otherwise
	Equals(Expression) bool

	// GetDefaultDecision defines what decision should be made if the filter has not come to a conclusive
	// decision based on a single index. When looking at more than one tag index in order of decreasing
	// priority to decide whether a metric should be part of the final result set, some operators and metric
	// combinations can come to a conclusive decision without looking at all indexes and some others can't.
	// if an expression has evaluated a metric against all indexes and has not come to a conclusive
	// decision, then the default decision gets applied.
	//
	// Example using the expression returned by `ParseExpression("anothertag!=value")`
	// metric1 has tags ["name=a.b.c", "some=value"] in the metric tag index
	// 1) expression looks at the metric tag index and it sees that metric1 does not have a tag "anothertag"
	//    with the value "value", but at this point it doesn't know if another index that will be looked
	//    at later does, so it returns the decision "none".
	// 2) expression now looks at index2 and sees again that metric1 does not have the tag and value
	//    it is looking for, so it returns "none" again.
	// 3) the expression execution sees that there are no more indexes left, so it applies the default
	//    decision for the operator != which is "pass", meaning the expression "anothertag!=value" has
	//    not filtered the metric metric1 out of the result set.
	//
	// metric2 has tags ["name=a.b.c", "anothertag=value"] according to the metric tag index and it has
	// no meta tags, we still evaluate the same expression:
	// 1) expression looks at metric tag index and see metric2 has tag "anothertag" with value "value".
	//    it directly comes to a conclusive decision that this metric needs to be filtered out of the
	//    result set and returns the filter decision "fail".
	//
	// metric3 has tags ["name=aaa", "abc=cba"] according to the metric tag index and there is a meta
	// record assigning the tag "anothertag=value" to metrics matching that query expression "abc=cba".
	// 1) expression looks at metric3 and sees it does not have the tag & value it's looking for, so
	//    it returns the filter decision "none" because it cannot know for sure whether another index
	//    will assign "anothertag=value" to metric3.
	// 2) expression looks at the meta tag index and it sees that there are meta records matching the
	//    tag "anothertag" and the value "value", so it retrieves the according filter functions of
	//    of these meta records and passes metric3's tag set into them.
	// 3) the filter function of the meta record for the query set "abc=cba" returns true, indicating
	//    that its meta tag gets applied to metric3.
	// 4) based on that the tag expression comes to the decision that metric3 should not be part of
	//    final result set, so it returns "fail".
	GetDefaultDecision() FilterDecision

	// GetKey returns tag to who's values this expression get's applied to
	// example:
	// in the expression "tag1=value" GetKey() would return "tag1"
	GetKey() string

	// GetValue returns the value part of the expression
	// example:
	// in the expression "abc!=cba" this would return "cba"
	GetValue() string

	// GetOperator returns the operator of this expression
	GetOperator() ExpressionOperator

	// GetOperatorCost returns a value which should roughly reflect the cost of this operator compared
	// to other operators. F.e. = is cheaper than =~. Keep in mind that this is only a very rough
	// estimate and will never be accurate.
	GetOperatorCost() uint32

	// OperatesOnTag returns whether this expression operators on the tag key
	// (if not, it operates on the value).
	// Expressions such has expressionHasTag, expressionMatchTag, expressionPrefixTag would return true,
	// because in order to make a decision regarding whether a metric should be part of the result set
	// they need to look at a metric's tags, as opposed to looking at the values associated with some
	// specified tag.
	// If this returns true, then tags shall be passed into ValuePasses(), other values associated with
	// the tag returned by GetKey() shall be passed into ValuePasses().
	OperatesOnTag() bool

	// RequiresNonEmptyValue returns whether this expression requires a non-empty value.
	// Every valid query must have at least one expression requiring a non-empty value.
	RequiresNonEmptyValue() bool

	// ResultIsSmallerWhenInverted returns a bool indicating whether the result set after evaluating
	// this expression will likely be bigger than half of the tested index entries or smaller.
	// This is never guaranteed to actually be correct, it is only an assumption based on which we
	// can optimize performance.
	// F.e. operators = / =~ / __tag=   would return false
	//      operators != / !=~          would return true
	ResultIsSmallerWhenInverted() bool

	// Matches takes a string which should either be a tag key or value depending on the return
	// value of OperatesOnTag(), then it returns whether the given string satisfies this expression
	Matches(string) bool

	// MatchesExactly returns a bool to indicate whether the key / value of this expression (depending
	// on OperatesOnTag()) needs to be an exact match with the key / value of the metrics it evaluates
	// F.e:
	// in the case of the expression "tag1=value1" we're only looking for metrics where the value
	// associated with tag key "tag1" is exactly "value1", so a simple string comparison is sufficient.
	// in other cases like "tag1=~val.*" or "tag^=val" this isn't the case, a simple string comparison
	// is not sufficient to decide whether a metric should be part of the result set or not.
	// since simple string comparisons are cheaper than other comparison methods, whenever possible we
	// want to use string comparison.
	MatchesExactly() bool

	// GetMetricDefinitionFilter returns a MetricDefinitionFilter
	// The MetricDefinitionFilter takes a metric definition, looks at its tags and returns a decision
	// regarding this query expression applied to its tags
	GetMetricDefinitionFilter(lookup IdTagLookup) MetricDefinitionFilter

	// StringIntoWriter takes a string writer and writes a representation of this expression into it
	StringIntoWriter(writer io.Writer)
}

Expression represents a single Graphite tag expression (see documentation for `ParseExpressions`). An expression is part of a query containing one or many expressions. It provides all the necessary methods that are required to do a tag lookup from an index keyed by tags & values, such as the type memory.TagIndex or the type memory.metaTagIndex. It is also comes with a method to generate a filter which decides whether a given MetricDefinition matches the requirements defined by this expression or not. This filter can be obtained from the method GetMetricDefinitionFilter().

func ParseExpression 

func ParseExpression(expr string) (Expression, error)

ParseExpression returns an expression that's been generated from the given string, in case of an error the error gets returned as the second value

type ExpressionOperator 

type ExpressionOperator uint16
const (
	EQUAL       ExpressionOperator = iota // =
	NOT_EQUAL                             // !=
	MATCH                                 // =~        regular expression
	MATCH_TAG                             // __tag=~   relies on special key __tag. non-standard, required for `/metrics/tags` requests with "filter"
	NOT_MATCH                             // !=~
	PREFIX                                // ^=        exact prefix, not regex. non-standard, required for auto complete of tag values
	PREFIX_TAG                            // __tag^=   exact prefix with tag. non-standard, required for auto complete of tag keys
	HAS_TAG                               // <tag>!="" specified tag must be present
	NOT_HAS_TAG                           // <tag>="" specified tag must not be present
	MATCH_ALL                             // special case of expression that matches every metric (f.e. key=.*)
	MATCH_NONE                            // special case of expression that matches no metric (f.e. key!=.*)
)

func (ExpressionOperator) StringIntoWriter added in v0.13.1 

func (o ExpressionOperator) StringIntoWriter(writer io.Writer)

type Expressions 

type Expressions []Expression

func ParseExpressions 

func ParseExpressions(expressions []string) (Expressions, error)

ParseExpressions parses a list of graphite tag expressions (as used by the `seriesByTag` function). tag expression definitions: https://graphite.readthedocs.io/en/latest/tags.html#querying seriesBytTag documentation: https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.seriesByTag Some possible tag expressions are: "status=200", "path!=/", "name=~cpu\..*" (`name` is a special tag which is automatically applied to the metric name).

func ParseSeriesByTagExpression added in v1.0.0 

func ParseSeriesByTagExpression(seriesByTagQuery string) (Expressions, error)

ParseSeriesByTagExpression takes a `seriesByTag` query which includes multiple tag query expressions example query: "seriesByTag('a=b', 'c=d', 'e!=~f.*')" it then returns a slice of strings where each string is one of the expressions, and an error which is non-nil if there was an error in the expression parsing and validation. All expressions are validated and an error is returned if one or more are invalid. This method does not validate that the passed in query is a valid `seriesByTag` query. It is assumed that `IsSeriesByTagExpression` has been run (and returned true) before this function is called.

func (Expressions) Equal added in v0.13.1 

func (e Expressions) Equal(other Expressions) bool

func (Expressions) MarshalJSON 

func (e Expressions) MarshalJSON() ([]byte, error)

MarshalJSON satisfies the json.Marshaler interface it is used by the api endpoint /metaTags to list the meta tag records

func (Expressions) Sort 

func (e Expressions) Sort()

func (Expressions) Strings 

func (e Expressions) Strings() []string

func (*Expressions) UnmarshalJSON added in v0.13.1 

func (e *Expressions) UnmarshalJSON(data []byte) error

type FilterDecision 

type FilterDecision uint8
const (
	None FilterDecision = iota // no decision has been made, because the decision might change depending on what other indexes defines
	Fail                       // it has been decided by the filter that this metric does not end up in the result set
	Pass                       // the filter has passed
)

type IdTagLookup 

type IdTagLookup func(id schema.MKey, tag, value string) bool

type InvalidExpressionError 

type InvalidExpressionError string

func (InvalidExpressionError) Error 

func (i InvalidExpressionError) Error() string

func (InvalidExpressionError) HTTPStatusCode added in v1.0.0 

func (i InvalidExpressionError) HTTPStatusCode() int

type MetaTagRecord 

type MetaTagRecord struct {
	MetaTags    Tags        `json:"metaTags"`
	Expressions Expressions `json:"expressions"`
}

func ParseMetaTagRecord 

func ParseMetaTagRecord(metaTags []string, expressions []string) (MetaTagRecord, error)

func (*MetaTagRecord) Equals 

func (m *MetaTagRecord) Equals(other *MetaTagRecord) bool

Equals takes another MetaTagRecord and compares all its properties to its own properties. It is assumed that the expressions of both meta tag records are already sorted.

func (*MetaTagRecord) GetMetricDefinitionFilter 

func (m *MetaTagRecord) GetMetricDefinitionFilter(lookup IdTagLookup) MetricDefinitionFilter

func (*MetaTagRecord) HasMetaTags 

func (m *MetaTagRecord) HasMetaTags() bool

HasMetaTags returns true if the meta tag record has one or more meta tags, otherwise it returns false

func (*MetaTagRecord) HashExpressions 

func (m *MetaTagRecord) HashExpressions() uint32

HashExpressions returns a hash of all expressions in this meta tag record It is assumed that the expressions are already sorted

func (*MetaTagRecord) HashMetaTags added in v0.13.1 

func (m *MetaTagRecord) HashMetaTags() uint32

HashMetaTags returns a hash of all meta tags in this meta tag record It is assumed that the meta tags are already sorted

func (*MetaTagRecord) HashRecord added in v0.13.1 

func (m *MetaTagRecord) HashRecord() uint64

type MetricDefinitionFilter 

type MetricDefinitionFilter func(id schema.MKey, name string, tags []string) FilterDecision

MetricDefinitionFilter takes a metric name together with its tags and returns a FilterDecision

type Query 

type Query struct {
	// clause that operates on LastUpdate field
	From int64

	// slice of expressions sorted by the estimated cost of their operators
	Expressions Expressions
	// contains filtered or unexported fields
}

Query is a set of graphite tag expressions as used by the graphite `seriesByTag` function. tag expression definitions: https://graphite.readthedocs.io/en/latest/tags.html#querying seriesByTag documentation: https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.seriesByTag

func NewQuery 

func NewQuery(expressions Expressions, from int64) (Query, error)

func NewQueryFromStrings 

func NewQueryFromStrings(expressionStrs []string, from int64) (Query, error)

NewQueryFromStrings parses a list of graphite tag expressions as used by the graphite `seriesByTag` function. tag expression definitions: https://graphite.readthedocs.io/en/latest/tags.html#querying seriesByTag documentation: https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.seriesByTag Some possible tag expressions are: "status=200", "path!=/", "name=~cpu\..*" (`name` is a special tag which is automatically applied to the metric name).

func (*Query) GetTagClause 

func (q *Query) GetTagClause() Expression

GetTagClause returns the expression which operates on tags, if one is present. This assumes that Query has been instantiated via NewQuery(), which either sets .tagClause to a valid value or returns an error. There can only be one tagClause per Query.

type Tag 

type Tag struct {
	Key   string
	Value string
}

func ParseTag 

func ParseTag(tag string) (Tag, error)

func (*Tag) DecodeMsg 

func (z *Tag) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (Tag) EncodeMsg 

func (z Tag) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (Tag) MarshalMsg 

func (z Tag) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (Tag) Msgsize 

func (z Tag) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*Tag) StringIntoWriter added in v0.13.1 

func (t *Tag) StringIntoWriter(writer io.Writer)

func (*Tag) UnmarshalMsg 

func (z *Tag) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

type Tags 

type Tags []Tag

func ParseTags 

func ParseTags(tags []string) (Tags, error)

ParseTags parses a list of key value pairs into a list of Tags. ParseTags([]string{"key=value", "name=whatever"}) -> Tags{{Key: "key", Value: "value"}, {Key: "name", Value: "whatever"}}}

func ParseTagsFromMetricName 

func ParseTagsFromMetricName(name string) (Tags, error)

func (*Tags) DecodeMsg 

func (z *Tags) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (Tags) EncodeMsg 

func (z Tags) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (Tags) Equal added in v0.13.1 

func (t Tags) Equal(other Tags) bool

func (Tags) Len 

func (t Tags) Len() int

func (Tags) Less 

func (t Tags) Less(i, j int) bool

func (Tags) MarshalJSON 

func (t Tags) MarshalJSON() ([]byte, error)

MarshalJSON satisfies the json.Marshaler interface it is used by the api endpoint /metaTags to list the meta tag records

func (Tags) MarshalMsg 

func (z Tags) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (Tags) Msgsize 

func (z Tags) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (Tags) Sort added in v0.13.1 

func (t Tags) Sort()

func (Tags) Strings 

func (t Tags) Strings() []string

func (Tags) Swap 

func (t Tags) Swap(i, j int)

func (*Tags) UnmarshalJSON added in v0.13.1 

func (t *Tags) UnmarshalJSON(data []byte) error

func (*Tags) UnmarshalMsg 

func (z *Tags) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.