Documentation ¶
Overview ¶
Package logqlengine implements LogQL evaluation engine.
Index ¶
- Variables
- func IsExplainQuery(ctx context.Context) bool
- func LineFromEntry(entry Entry) string
- func VisitNode[N Node](root Node, cb func(N) error) error
- type AndLabelMatcher
- type BinOp
- type BytesLabelFilter
- type Comparator
- type ContainsMatcher
- type Decolorize
- type Direction
- type DistinctFilter
- type DropLabels
- type DurationLabelFilter
- type Engine
- type Entry
- type EntryIterator
- type EqComparator
- type EqualIPMatcher
- type EqualsMatcher
- type EvalParams
- type ExplainQuery
- type FastRegexpMatcher
- type GtComparator
- type GteComparator
- type IPLabelFilter
- type IPLineMatcher
- type IPMatcher
- type JSONExtractor
- type KeepLabels
- type LabelFormat
- type LabelMatcher
- type LabelReplace
- type LineFilter
- type LineFormat
- type LiteralBinOp
- type LiteralQuery
- type LogQuery
- type LogfmtExtractor
- type LtComparator
- type LteComparator
- type Matcher
- type MetricNode
- type MetricParams
- type MetricQuery
- type Node
- type NodeVisitor
- type NotEqComparator
- type NotMatcher
- type NumberLabelFilter
- type Optimizer
- type Options
- type OrLabelMatcher
- type OrLineFilter
- type PatternExtractor
- type Pipeline
- type PipelineNode
- type PrefixIPMatcher
- type Processor
- type ProcessorNode
- type Querier
- type QuerierCapabilities
- type Query
- type RangeAggregation
- type RangeIPMatcher
- type RegexpExtractor
- type RegexpMatcher
- type RenameLabel
- type SampleIterator
- type SampleNode
- type SamplingNode
- type StepIterator
- type StringMatcher
- type SupportedOps
- type UnpackExtractor
- type Vector
- type VectorAggregation
Constants ¶
This section is empty.
Variables ¶
var NopProcessor = &nopProcessor{}
NopProcessor is a processor that does nothing.
Functions ¶
func IsExplainQuery ¶ added in v0.14.1
IsExplainQuery whether if this LogQL query is explained.
func LineFromEntry ¶ added in v0.10.0
LineFromEntry returns a JSON line from a log record.
Types ¶
type AndLabelMatcher ¶
AndLabelMatcher is a AND logical operation for two label matchers.
func (*AndLabelMatcher) Process ¶
func (m *AndLabelMatcher) Process(ts otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type BinOp ¶ added in v0.10.0
type BinOp struct {
Left, Right MetricNode
Expr *logql.BinOpExpr
}
BinOp is a MetricNode implementing binary operation.
func (*BinOp) EvalMetric ¶ added in v0.10.0
func (n *BinOp) EvalMetric(ctx context.Context, params MetricParams) (_ StepIterator, rerr error)
EvalMetric implements [EvalMetric].
type BytesLabelFilter ¶
type BytesLabelFilter[C Comparator[uint64]] struct { // contains filtered or unexported fields }
BytesLabelFilter is a label filter Processor.
func (*BytesLabelFilter[C]) Process ¶
func (lf *BytesLabelFilter[C]) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type Comparator ¶
Comparator is a filter that compares value.
type ContainsMatcher ¶
type ContainsMatcher struct {
Value string
}
ContainsMatcher checks if a string contains value.
func (ContainsMatcher) Match ¶
func (m ContainsMatcher) Match(s string) bool
Match implements StringMatcher.
type Decolorize ¶ added in v0.1.0
type Decolorize struct{}
Decolorize removes ANSI escape codes from line.
func (*Decolorize) Process ¶ added in v0.1.0
func (d *Decolorize) Process(_ otelstorage.Timestamp, line string, _ logqlabels.LabelSet) (string, bool)
Process implements Processor.
type Direction ¶ added in v0.10.0
type Direction string
Direction describe log ordering.
func (Direction) String ¶ added in v0.12.2
String implements fmt.Stringer.
type DistinctFilter ¶ added in v0.0.8
type DistinctFilter struct {
// contains filtered or unexported fields
}
DistinctFilter filters out records with duplicate label values.
func (*DistinctFilter) Process ¶ added in v0.0.8
func (d *DistinctFilter) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type DropLabels ¶
type DropLabels struct {
// contains filtered or unexported fields
}
DropLabels label filtering Processor.
func (*DropLabels) Process ¶
func (k *DropLabels) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type DurationLabelFilter ¶
type DurationLabelFilter[C Comparator[time.Duration]] struct { // contains filtered or unexported fields }
DurationLabelFilter is a label filter Processor.
func (*DurationLabelFilter[C]) Process ¶
func (lf *DurationLabelFilter[C]) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine is a LogQL evaluation engine.
func (*Engine) ParseOptions ¶ added in v0.13.0
func (e *Engine) ParseOptions() logql.ParseOptions
ParseOptions returns logql.ParseOptions used by engine.
type Entry ¶ added in v0.10.0
type Entry struct { Timestamp otelstorage.Timestamp Line string Set logqlabels.LabelSet }
Entry represents a log entry.
type EntryIterator ¶ added in v0.10.0
EntryIterator represents a LogQL entry log stream.
type EqComparator ¶
type EqComparator[T comparable] struct{}
EqComparator implements '==' Comparator.
func (EqComparator[T]) Compare ¶
func (EqComparator[T]) Compare(a, b T) bool
Compare implements Comparator[T].
type EqualIPMatcher ¶
EqualIPMatcher checks if an IP equal to given value.
type EqualsMatcher ¶
type EqualsMatcher struct {
Value string
}
EqualsMatcher checks if a string equals to a value.
func (EqualsMatcher) Match ¶
func (m EqualsMatcher) Match(s string) bool
Match implements StringMatcher.
type EvalParams ¶
type EvalParams struct { Start time.Time End time.Time Step time.Duration Direction Direction // forward, backward Limit int // -1 if query should be unlimited. }
EvalParams sets evaluation parameters.
func (EvalParams) IsInstant ¶
func (p EvalParams) IsInstant() bool
IsInstant whether query is instant.
type ExplainQuery ¶ added in v0.14.0
type ExplainQuery struct { Explain Query // contains filtered or unexported fields }
ExplainQuery is simple Explain expression query.
func (*ExplainQuery) Eval ¶ added in v0.14.0
func (q *ExplainQuery) Eval(ctx context.Context, params EvalParams) (lokiapi.QueryResponseData, error)
Eval implements Query.
type FastRegexpMatcher ¶ added in v0.14.0
type FastRegexpMatcher struct {
Re *labels.FastRegexMatcher
}
FastRegexpMatcher checks if a matches regular expression.
func (FastRegexpMatcher) Match ¶ added in v0.14.0
func (m FastRegexpMatcher) Match(s string) bool
Match implements StringMatcher.
type GtComparator ¶
GtComparator implements '>' Comparator.
func (GtComparator[T]) Compare ¶
func (GtComparator[T]) Compare(a, b T) bool
Compare implements Comparator[T].
type GteComparator ¶
GteComparator implements '>=' Comparator.
func (GteComparator[T]) Compare ¶
func (GteComparator[T]) Compare(a, b T) bool
Compare implements Comparator[T].
type IPLabelFilter ¶
type IPLabelFilter struct {
// contains filtered or unexported fields
}
IPLabelFilter is a label filter Processor.
func (*IPLabelFilter) Process ¶
func (lf *IPLabelFilter) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type IPLineMatcher ¶ added in v0.14.0
type IPLineMatcher struct {
// contains filtered or unexported fields
}
IPLineMatcher looks for IP address in a line and applies matcher to it.
func (IPLineMatcher) Match ¶ added in v0.14.0
func (lf IPLineMatcher) Match(line string) bool
Match implements StringMatcher.
type JSONExtractor ¶
type JSONExtractor struct {
// contains filtered or unexported fields
}
JSONExtractor is a JSON label extractor.
func (*JSONExtractor) Process ¶
func (e *JSONExtractor) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type KeepLabels ¶
type KeepLabels struct {
// contains filtered or unexported fields
}
KeepLabels label filtering Processor.
func (*KeepLabels) Process ¶
func (k *KeepLabels) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type LabelFormat ¶ added in v0.0.8
type LabelFormat struct {
// contains filtered or unexported fields
}
LabelFormat is a label formatting Processor.
func (*LabelFormat) Process ¶ added in v0.0.8
func (lf *LabelFormat) Process(ts otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type LabelMatcher ¶
type LabelMatcher struct {
// contains filtered or unexported fields
}
LabelMatcher is a label filter Processor.
func (*LabelMatcher) Process ¶
func (lf *LabelMatcher) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type LabelReplace ¶ added in v0.10.0
type LabelReplace struct { Input MetricNode Expr *logql.LabelReplaceExpr }
LabelReplace is a MetricNode implementing `label_replace` function.
func (*LabelReplace) EvalMetric ¶ added in v0.10.0
func (n *LabelReplace) EvalMetric(ctx context.Context, params MetricParams) (_ StepIterator, rerr error)
EvalMetric implements [EvalMetric].
func (*LabelReplace) Traverse ¶ added in v0.10.0
func (n *LabelReplace) Traverse(cb NodeVisitor) error
Traverse implements Node.
type LineFilter ¶
type LineFilter struct {
// contains filtered or unexported fields
}
LineFilter is a line matching Processor.
func (*LineFilter) Process ¶
func (lf *LineFilter) Process(_ otelstorage.Timestamp, line string, _ logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type LineFormat ¶ added in v0.0.8
type LineFormat struct {
// contains filtered or unexported fields
}
LineFormat is a line formatting Processor.
func (*LineFormat) Process ¶ added in v0.0.8
func (lf *LineFormat) Process(ts otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type LiteralBinOp ¶ added in v0.10.0
type LiteralBinOp struct { Input MetricNode Literal float64 IsLiteralOnLeft bool Expr *logql.BinOpExpr }
LiteralBinOp is a MetricNode implementing binary operation on literal.
func (*LiteralBinOp) EvalMetric ¶ added in v0.10.0
func (n *LiteralBinOp) EvalMetric(ctx context.Context, params MetricParams) (_ StepIterator, rerr error)
EvalMetric implements [EvalMetric].
func (*LiteralBinOp) Traverse ¶ added in v0.10.0
func (n *LiteralBinOp) Traverse(cb NodeVisitor) error
Traverse implements Node.
type LiteralQuery ¶ added in v0.10.0
type LiteralQuery struct { Value float64 // contains filtered or unexported fields }
LiteralQuery is simple literal expression query.
func (*LiteralQuery) Eval ¶ added in v0.10.0
func (q *LiteralQuery) Eval(ctx context.Context, params EvalParams) (data lokiapi.QueryResponseData, rerr error)
Eval implements Query.
type LogQuery ¶ added in v0.10.0
type LogQuery struct { Root PipelineNode LookbackDuration time.Duration // contains filtered or unexported fields }
LogQuery represents a log query.
func (*LogQuery) Eval ¶ added in v0.10.0
func (q *LogQuery) Eval(ctx context.Context, params EvalParams) (data lokiapi.QueryResponseData, _ error)
Eval implements Query.
type LogfmtExtractor ¶
type LogfmtExtractor struct {
// contains filtered or unexported fields
}
LogfmtExtractor is a Logfmt label extractor.
func (*LogfmtExtractor) Process ¶
func (e *LogfmtExtractor) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type LtComparator ¶
LtComparator implements '<' Comparator.
func (LtComparator[T]) Compare ¶
func (LtComparator[T]) Compare(a, b T) bool
Compare implements Comparator[T].
type LteComparator ¶
LteComparator implements '<=' Comparator.
func (LteComparator[T]) Compare ¶
func (LteComparator[T]) Compare(a, b T) bool
Compare implements Comparator[T].
type MetricNode ¶ added in v0.10.0
type MetricNode interface { Node EvalMetric(ctx context.Context, params MetricParams) (StepIterator, error) }
MetricNode represents a LogQL metric function node.
type MetricParams ¶ added in v0.10.0
MetricParams defines MetricNode parameters.
type MetricQuery ¶ added in v0.10.0
type MetricQuery struct { Root MetricNode // contains filtered or unexported fields }
MetricQuery represents a metric query.
func (*MetricQuery) Eval ¶ added in v0.10.0
func (q *MetricQuery) Eval(ctx context.Context, params EvalParams) (lokiapi.QueryResponseData, error)
Eval implements Query.
type Node ¶ added in v0.10.0
type Node interface { // Traverse calls given callback on child nodes. Traverse(cb NodeVisitor) error }
Node is a generic node interface.
type NodeVisitor ¶ added in v0.10.0
NodeVisitor is a callback to traverse Node.
type NotEqComparator ¶
type NotEqComparator[T comparable] struct{}
NotEqComparator implements '!=' Comparator.
func (NotEqComparator[T]) Compare ¶
func (NotEqComparator[T]) Compare(a, b T) bool
Compare implements Comparator[T].
type NotMatcher ¶
NotMatcher is a NOT logical matcher.
func (NotMatcher[T, M]) Match ¶
func (m NotMatcher[T, M]) Match(v T) bool
Match implements StringMatcher.
type NumberLabelFilter ¶
type NumberLabelFilter[C Comparator[float64]] struct { // contains filtered or unexported fields }
NumberLabelFilter is a label filter Processor.
func (*NumberLabelFilter[C]) Process ¶
func (lf *NumberLabelFilter[C]) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type Optimizer ¶ added in v0.10.0
type Optimizer interface { // Name returns optimizer name. Name() string Optimize(ctx context.Context, q Query) (Query, error) }
Optimizer defines an interface for optimizer.
func DefaultOptimizers ¶ added in v0.10.0
func DefaultOptimizers() []Optimizer
DefaultOptimizers returns slice of default [Optimizer]s.
type Options ¶
type Options struct { // LookbackDuration sets lookback duration for instant queries. // // Should be negative, otherwise default value would be used. LookbackDuration time.Duration // ParseOptions is a LogQL parser options. ParseOptions logql.ParseOptions // OTELAdapter enables 'otel adapter' whatever it is. OTELAdapter bool // Optimizers defines a list of optimiziers to use. Optimizers []Optimizer // MeterProvider provides OpenTelemetry meter for this engine. MeterProvider metric.MeterProvider // TracerProvider provides OpenTelemetry tracer for this engine. TracerProvider trace.TracerProvider }
Options sets Engine options.
type OrLabelMatcher ¶
OrLabelMatcher is a OR logical operation for two label matchers.
func (*OrLabelMatcher) Process ¶
func (m *OrLabelMatcher) Process(ts otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type OrLineFilter ¶ added in v0.14.0
type OrLineFilter struct {
// contains filtered or unexported fields
}
OrLineFilter is a line matching Processor.
func (*OrLineFilter) Process ¶ added in v0.14.0
func (lf *OrLineFilter) Process(_ otelstorage.Timestamp, line string, _ logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type PatternExtractor ¶ added in v0.1.0
type PatternExtractor struct {
// contains filtered or unexported fields
}
PatternExtractor is a Pattern label extractor.
func (*PatternExtractor) Process ¶ added in v0.1.0
func (e *PatternExtractor) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type Pipeline ¶
type Pipeline struct {
Stages []Processor
}
Pipeline is a multi-stage processor.
func (*Pipeline) Process ¶
func (p *Pipeline) Process(ts otelstorage.Timestamp, line string, attrs logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type PipelineNode ¶ added in v0.10.0
type PipelineNode interface { Node EvalPipeline(ctx context.Context, params EvalParams) (EntryIterator, error) }
PipelineNode represents a LogQL pipeline node.
type PrefixIPMatcher ¶ added in v0.0.8
PrefixIPMatcher checks if an IP has given prefix.
type Processor ¶
type Processor interface {
Process(ts otelstorage.Timestamp, line string, labels logqlabels.LabelSet) (newLine string, keep bool)
}
Processor is a log record processor.
func BuildPipeline ¶
func BuildPipeline(stages ...logql.PipelineStage) (Processor, error)
BuildPipeline builds a new Pipeline.
type ProcessorNode ¶ added in v0.10.0
type ProcessorNode struct { Input PipelineNode Prefilter Processor Selector logql.Selector Pipeline []logql.PipelineStage EnableOTELAdapter bool }
ProcessorNode implements PipelineNode.
func (*ProcessorNode) EvalPipeline ¶ added in v0.10.0
func (n *ProcessorNode) EvalPipeline(ctx context.Context, params EvalParams) (_ EntryIterator, rerr error)
EvalPipeline implements PipelineNode.
func (*ProcessorNode) Traverse ¶ added in v0.10.0
func (n *ProcessorNode) Traverse(cb NodeVisitor) error
Traverse implements Node.
type Querier ¶
type Querier interface { // Capabilities returns Querier capabilities. // // NOTE: engine would call once and then save value. // Capabilities should not change over time. Capabilities() QuerierCapabilities // Query creates new [PipelineNode]. Query(ctx context.Context, selector []logql.LabelMatcher) (PipelineNode, error) }
Querier does queries to storage.
type QuerierCapabilities ¶ added in v0.1.0
type QuerierCapabilities struct { Label SupportedOps Line SupportedOps }
QuerierCapabilities defines what operations storage can do.
type Query ¶ added in v0.10.0
type Query interface {
Eval(ctx context.Context, params EvalParams) (lokiapi.QueryResponseData, error)
}
Query is a LogQL query.
type RangeAggregation ¶ added in v0.10.0
type RangeAggregation struct { Input SampleNode Expr *logql.RangeAggregationExpr }
RangeAggregation is a MetricNode implementing range aggregation.
func (*RangeAggregation) EvalMetric ¶ added in v0.10.0
func (n *RangeAggregation) EvalMetric(ctx context.Context, params MetricParams) (_ StepIterator, rerr error)
EvalMetric implements [EvalMetric].
func (*RangeAggregation) Traverse ¶ added in v0.10.0
func (n *RangeAggregation) Traverse(cb NodeVisitor) error
Traverse implements Node.
type RangeIPMatcher ¶
RangeIPMatcher checks if an IP is in given range.
type RegexpExtractor ¶
type RegexpExtractor struct {
// contains filtered or unexported fields
}
RegexpExtractor is a Regexp label extractor.
func (*RegexpExtractor) Process ¶
func (e *RegexpExtractor) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type RegexpMatcher ¶
RegexpMatcher checks if a matches regular expression.
func (RegexpMatcher) Match ¶
func (m RegexpMatcher) Match(s string) bool
Match implements StringMatcher.
type RenameLabel ¶ added in v0.0.8
type RenameLabel struct {
// contains filtered or unexported fields
}
RenameLabel is a label renaming Processor.
func (*RenameLabel) Process ¶ added in v0.0.8
func (rl *RenameLabel) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (_ string, keep bool)
Process implements Processor.
type SampleIterator ¶ added in v0.10.0
type SampleIterator = iterators.Iterator[logqlmetric.SampledEntry]
SampleIterator represents a samples stream.
type SampleNode ¶ added in v0.10.0
type SampleNode interface { Node EvalSample(ctx context.Context, params EvalParams) (SampleIterator, error) }
SampleNode represents a log sampling node.
type SamplingNode ¶ added in v0.10.0
type SamplingNode struct { Input PipelineNode Expr *logql.RangeAggregationExpr }
SamplingNode implements entry sampling.
func (*SamplingNode) EvalSample ¶ added in v0.10.0
func (n *SamplingNode) EvalSample(ctx context.Context, params EvalParams) (SampleIterator, error)
EvalSample implements SampleNode.
func (*SamplingNode) Traverse ¶ added in v0.10.0
func (n *SamplingNode) Traverse(cb NodeVisitor) error
Traverse implements Node.
type StepIterator ¶ added in v0.10.0
type StepIterator = logqlmetric.StepIterator
StepIterator represents a metric stream.
type StringMatcher ¶
StringMatcher matches a string.
type SupportedOps ¶
type SupportedOps uint64
SupportedOps is a bitset defining ops supported by Querier.
type UnpackExtractor ¶ added in v0.0.8
type UnpackExtractor struct{}
UnpackExtractor extracts log entry fron Promtail `pack`-ed entry.
func (*UnpackExtractor) Process ¶ added in v0.0.8
func (e *UnpackExtractor) Process(_ otelstorage.Timestamp, line string, set logqlabels.LabelSet) (string, bool)
Process implements Processor.
type Vector ¶ added in v0.10.0
type Vector struct {
Expr *logql.VectorExpr
}
Vector is a MetricNode implementing vector literal.
func (*Vector) EvalMetric ¶ added in v0.10.0
func (n *Vector) EvalMetric(ctx context.Context, params MetricParams) (_ StepIterator, rerr error)
EvalMetric implements [EvalMetric].
type VectorAggregation ¶ added in v0.10.0
type VectorAggregation struct { Input MetricNode Expr *logql.VectorAggregationExpr }
VectorAggregation is a MetricNode implementing vector aggregation.
func (*VectorAggregation) EvalMetric ¶ added in v0.10.0
func (n *VectorAggregation) EvalMetric(ctx context.Context, params MetricParams) (_ StepIterator, rerr error)
EvalMetric implements [EvalMetric].
func (*VectorAggregation) Traverse ¶ added in v0.10.0
func (n *VectorAggregation) Traverse(cb NodeVisitor) error
Traverse implements Node.
Source Files ¶
- comparator.go
- decolorize.go
- distinct.go
- drop.go
- engine.go
- engine_explain_query.go
- engine_literal_query.go
- engine_log_query.go
- engine_metric_nodes.go
- engine_metric_query.go
- engine_optimizer.go
- engine_plan.go
- engine_stats.go
- ip_matcher.go
- json.go
- json_unsafe.go
- keep.go
- label_filter.go
- label_format.go
- line_filter.go
- line_format.go
- logfmt.go
- matcher.go
- otel_adapter.go
- pattern.go
- precondition.go
- processor.go
- regexp.go
- sampler.go
- storage.go
- string_matcher.go
- template.go
- unpack.go
Directories ¶
Path | Synopsis |
---|---|
Package jsonexpr provides JSON extractor expression parser.
|
Package jsonexpr provides JSON extractor expression parser. |
Package logqlabels contains LogQL label utilities.
|
Package logqlabels contains LogQL label utilities. |
Package logqlerrors defines LogQL engine errors.
|
Package logqlerrors defines LogQL engine errors. |
Package logqlmetric provides metric queries implementation.
|
Package logqlmetric provides metric queries implementation. |
Package logqlpattern contains parser for LogQL `pattern` stage pattern.
|
Package logqlpattern contains parser for LogQL `pattern` stage pattern. |