logqlengine

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 10, 2023 License: Apache-2.0 Imports: 36 Imported by: 0

Documentation

Overview

Package logqlengine implements LogQL evaluation engine.

Index

Constants

This section is empty.

Variables

View Source 
var NopProcessor = &nopProcessor{}

NopProcessor is a processor that does nothing.

Functions

func LineFromRecord added in v0.1.0 

func LineFromRecord(record logstorage.Record) string

LineFromRecord returns a JSON line from a log record.

Types

type AndLabelMatcher 

type AndLabelMatcher struct {
	Left  Processor
	Right Processor
}

AndLabelMatcher is a AND logical operation for two label matchers.

func (*AndLabelMatcher) Process 

func (m *AndLabelMatcher) Process(ts otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type BytesLabelFilter 

type BytesLabelFilter[C Comparator[uint64]] struct {
	// contains filtered or unexported fields
}

BytesLabelFilter is a label filter Processor.

func (*BytesLabelFilter[C]) Process 

func (lf *BytesLabelFilter[C]) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type Comparator 

type Comparator[T any] interface {
	Compare(a, b T) bool
}

Comparator is a filter that compares value.

type ContainsMatcher 

type ContainsMatcher struct {
	Value string
}

ContainsMatcher checks if a string contains value.

func (ContainsMatcher) Match 

func (m ContainsMatcher) Match(s string) bool

Match implements StringMatcher.

type Decolorize added in v0.1.0 

type Decolorize struct{}

Decolorize removes ANSI escape codes from line.

func (*Decolorize) Process added in v0.1.0 

func (d *Decolorize) Process(_ otelstorage.Timestamp, line string, _ LabelSet) (string, bool)

Process implements Processor.

type DistinctFilter added in v0.0.8 

type DistinctFilter struct {
	// contains filtered or unexported fields
}

DistinctFilter filters out records with duplicate label values.

func (*DistinctFilter) Process added in v0.0.8 

func (d *DistinctFilter) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type DropLabels 

type DropLabels struct {
	// contains filtered or unexported fields
}

DropLabels label filtering Processor.

func (*DropLabels) Process 

func (k *DropLabels) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type DurationLabelFilter 

type DurationLabelFilter[C Comparator[time.Duration]] struct {
	// contains filtered or unexported fields
}

DurationLabelFilter is a label filter Processor.

func (*DurationLabelFilter[C]) Process 

func (lf *DurationLabelFilter[C]) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine is a LogQL evaluation engine.

func NewEngine 

func NewEngine(querier Querier, opts Options) *Engine

NewEngine creates new Engine.

func (*Engine) Eval 

func (e *Engine) Eval(ctx context.Context, query string, params EvalParams) (data lokiapi.QueryResponseData, rerr error)

Eval parses and evaluates query.

type EqComparator 

type EqComparator[T comparable] struct{}

EqComparator implements '==' Comparator.

func (EqComparator[T]) Compare 

func (EqComparator[T]) Compare(a, b T) bool

Compare implements Comparator[T].

type EqualIPMatcher 

type EqualIPMatcher struct {
	Value netip.Addr
}

EqualIPMatcher checks if an IP equal to given value.

func (EqualIPMatcher) Match 

func (m EqualIPMatcher) Match(ip netip.Addr) bool

Match implements IPMatcher.

type EqualsMatcher 

type EqualsMatcher struct {
	Value string
}

EqualsMatcher checks if a string equals to a value.

func (EqualsMatcher) Match 

func (m EqualsMatcher) Match(s string) bool

Match implements StringMatcher.

type EvalParams 

type EvalParams struct {
	Start     otelstorage.Timestamp
	End       otelstorage.Timestamp
	Step      time.Duration
	Direction string // forward, backward
	Limit     int
}

EvalParams sets evaluation parameters.

func (EvalParams) IsInstant 

func (p EvalParams) IsInstant() bool

IsInstant whether query is instant.

type GtComparator 

type GtComparator[T cmp.Ordered] struct{}

GtComparator implements '>' Comparator.

func (GtComparator[T]) Compare 

func (GtComparator[T]) Compare(a, b T) bool

Compare implements Comparator[T].

type GteComparator 

type GteComparator[T cmp.Ordered] struct{}

GteComparator implements '>=' Comparator.

func (GteComparator[T]) Compare 

func (GteComparator[T]) Compare(a, b T) bool

Compare implements Comparator[T].

type IPLabelFilter 

type IPLabelFilter struct {
	// contains filtered or unexported fields
}

IPLabelFilter is a label filter Processor.

func (*IPLabelFilter) Process 

func (lf *IPLabelFilter) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type IPLineFilter added in v0.0.8 

type IPLineFilter struct {
	// contains filtered or unexported fields
}

IPLineFilter looks for IP address in a line and applies matcher to it.

func (*IPLineFilter) Process added in v0.0.8 

func (lf *IPLineFilter) Process(_ otelstorage.Timestamp, line string, _ LabelSet) (_ string, keep bool)

Process implements Processor.

type IPMatcher 

type IPMatcher interface {
	Matcher[netip.Addr]
}

IPMatcher matches an IP.

type JSONExtractor 

type JSONExtractor struct {
	// contains filtered or unexported fields
}

JSONExtractor is a JSON label extractor.

func (*JSONExtractor) Process 

func (e *JSONExtractor) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type KeepLabels 

type KeepLabels struct {
	// contains filtered or unexported fields
}

KeepLabels label filtering Processor.

func (*KeepLabels) Process 

func (k *KeepLabels) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type LabelFormat added in v0.0.8 

type LabelFormat struct {
	// contains filtered or unexported fields
}

LabelFormat is a label formatting Processor.

func (*LabelFormat) Process added in v0.0.8 

func (lf *LabelFormat) Process(ts otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type LabelMatcher 

type LabelMatcher struct {
	// contains filtered or unexported fields
}

LabelMatcher is a label filter Processor.

func (*LabelMatcher) Process 

func (lf *LabelMatcher) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type LabelSet 

type LabelSet struct {
	// contains filtered or unexported fields
}

LabelSet is a log record's label set.

func (*LabelSet) AsLokiAPI 

func (l *LabelSet) AsLokiAPI() lokiapi.LabelSet

AsLokiAPI returns lokiapi.LabelSet

func (*LabelSet) AsMap added in v0.0.8 

func (l *LabelSet) AsMap() map[string]string

AsMap returns labels as strings map.

func (*LabelSet) Delete 

func (l *LabelSet) Delete(s logql.Label)

Delete deletes label.

func (*LabelSet) Get 

func (l *LabelSet) Get(name logql.Label) (v pcommon.Value, ok bool)

Get returns attr value.

func (*LabelSet) GetError 

func (l *LabelSet) GetError() (string, bool)

GetError returns error label.

func (*LabelSet) GetFloat added in v0.0.8 

func (l *LabelSet) GetFloat(name logql.Label) (_ float64, ok bool, err error)

GetFloat returns number attr value.

func (*LabelSet) GetString 

func (l *LabelSet) GetString(name logql.Label) (string, bool)

GetString returns stringified attr value.

func (*LabelSet) Range 

func (l *LabelSet) Range(cb func(logql.Label, pcommon.Value))

Range iterates over label set.

func (*LabelSet) Set added in v0.0.8 

func (l *LabelSet) Set(s logql.Label, val pcommon.Value)

Set sets label.

func (*LabelSet) SetAttrs 

func (l *LabelSet) SetAttrs(attrMaps ...otelstorage.Attrs)

SetAttrs sets labels from attrs.

func (*LabelSet) SetError 

func (l *LabelSet) SetError(typ string, err error)

SetError sets special error label.

func (*LabelSet) SetFromRecord 

func (l *LabelSet) SetFromRecord(record logstorage.Record)

SetFromRecord sets labels from given log record.

func (*LabelSet) String 

func (l *LabelSet) String() string

String returns text representation of labels.

type LineFilter 

type LineFilter struct {
	// contains filtered or unexported fields
}

LineFilter is a line matching Processor.

func (*LineFilter) Process 

func (lf *LineFilter) Process(_ otelstorage.Timestamp, line string, _ LabelSet) (_ string, keep bool)

Process implements Processor.

type LineFormat added in v0.0.8 

type LineFormat struct {
	// contains filtered or unexported fields
}

LineFormat is a line formatting Processor.

func (*LineFormat) Process added in v0.0.8 

func (lf *LineFormat) Process(ts otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type LogfmtExtractor 

type LogfmtExtractor struct {
	// contains filtered or unexported fields
}

LogfmtExtractor is a Logfmt label extractor.

func (*LogfmtExtractor) Process 

func (e *LogfmtExtractor) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type LtComparator 

type LtComparator[T cmp.Ordered] struct{}

LtComparator implements '<' Comparator.

func (LtComparator[T]) Compare 

func (LtComparator[T]) Compare(a, b T) bool

Compare implements Comparator[T].

type LteComparator 

type LteComparator[T cmp.Ordered] struct{}

LteComparator implements '<=' Comparator.

func (LteComparator[T]) Compare 

func (LteComparator[T]) Compare(a, b T) bool

Compare implements Comparator[T].

type Matcher 

type Matcher[T any] interface {
	Match(T) bool
}

Matcher is a generic matcher.

type NotEqComparator 

type NotEqComparator[T comparable] struct{}

NotEqComparator implements '!=' Comparator.

func (NotEqComparator[T]) Compare 

func (NotEqComparator[T]) Compare(a, b T) bool

Compare implements Comparator[T].

type NotMatcher 

type NotMatcher[T any, M Matcher[T]] struct {
	Next M
}

NotMatcher is a NOT logical matcher.

func (NotMatcher[T, M]) Match 

func (m NotMatcher[T, M]) Match(v T) bool

Match implements StringMatcher.

type NumberLabelFilter 

type NumberLabelFilter[C Comparator[float64]] struct {
	// contains filtered or unexported fields
}

NumberLabelFilter is a label filter Processor.

func (*NumberLabelFilter[C]) Process 

func (lf *NumberLabelFilter[C]) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type Options 

type Options struct {
	// LookbackDuration sets lookback duration for instant queries.
	//
	// Should be negative, otherwise default value would be used.
	LookbackDuration time.Duration

	// ParseOptions is a LogQL parser options.
	ParseOptions logql.ParseOptions

	// TracerProvider provides OpenTelemetry tracer for this engine.
	TracerProvider trace.TracerProvider
}

Options sets Engine options.

type OrLabelMatcher 

type OrLabelMatcher struct {
	Left  Processor
	Right Processor
}

OrLabelMatcher is a OR logical operation for two label matchers.

func (*OrLabelMatcher) Process 

func (m *OrLabelMatcher) Process(ts otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type PatternExtractor added in v0.1.0 

type PatternExtractor struct {
	// contains filtered or unexported fields
}

PatternExtractor is a Pattern label extractor.

func (*PatternExtractor) Process added in v0.1.0 

func (e *PatternExtractor) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type Pipeline 

type Pipeline struct {
	Stages []Processor
}

Pipeline is a multi-stage processor.

func (*Pipeline) Process 

func (p *Pipeline) Process(ts otelstorage.Timestamp, line string, attrs LabelSet) (_ string, keep bool)

Process implements Processor.

type PrefixIPMatcher added in v0.0.8 

type PrefixIPMatcher struct {
	Prefix netip.Prefix
}

PrefixIPMatcher checks if an IP has given prefix.

func (PrefixIPMatcher) Match added in v0.0.8 

func (m PrefixIPMatcher) Match(ip netip.Addr) bool

Match implements IPMatcher.

type Processor 

type Processor interface {
	Process(ts otelstorage.Timestamp, line string, labels LabelSet) (newLine string, keep bool)
}

Processor is a log record processor.

func BuildPipeline 

func BuildPipeline(stages ...logql.PipelineStage) (Processor, error)

BuildPipeline builds a new Pipeline.

type Querier 

type Querier interface {
	// Capabilities returns Querier capabilities.
	//
	// NOTE: engine would call once and then save value.
	// 	Capabilities should not change over time.
	Capabilities() QuerierCapabilities
	// SelectLogs selects log records from storage.
	SelectLogs(ctx context.Context, start, end otelstorage.Timestamp, params SelectLogsParams) (iterators.Iterator[logstorage.Record], error)
}

Querier does queries to storage.

type QuerierCapabilities added in v0.1.0 

type QuerierCapabilities struct {
	Label SupportedOps
	Line  SupportedOps
}

QuerierCapabilities defines what operations storage can do.

type RangeIPMatcher 

type RangeIPMatcher struct {
	// FIXME(tdakkota): probably, it is better to just use two addrs
	// 	and compare them.
	Range netipx.IPRange
}

RangeIPMatcher checks if an IP is in given range.

func (RangeIPMatcher) Match 

func (m RangeIPMatcher) Match(ip netip.Addr) bool

Match implements IPMatcher.

type RegexpExtractor 

type RegexpExtractor struct {
	// contains filtered or unexported fields
}

RegexpExtractor is a Regexp label extractor.

func (*RegexpExtractor) Process 

func (e *RegexpExtractor) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type RegexpMatcher 

type RegexpMatcher struct {
	Re *regexp.Regexp
}

RegexpMatcher checks if a matches regular expression.

func (RegexpMatcher) Match 

func (m RegexpMatcher) Match(s string) bool

Match implements StringMatcher.

type RenameLabel added in v0.0.8 

type RenameLabel struct {
	// contains filtered or unexported fields
}

RenameLabel is a label renaming Processor.

func (*RenameLabel) Process added in v0.0.8 

func (rl *RenameLabel) Process(_ otelstorage.Timestamp, line string, set LabelSet) (_ string, keep bool)

Process implements Processor.

type SelectLogsParams 

type SelectLogsParams struct {
	Labels []logql.LabelMatcher
	Line   []logql.LineFilter
}

SelectLogsParams is a storage query params.

type StringMatcher 

type StringMatcher interface {
	Matcher[string]
}

StringMatcher matches a string.

type SupportedOps 

type SupportedOps uint64

SupportedOps is a bitset defining ops supported by Querier.

func (*SupportedOps) Add 

func (caps *SupportedOps) Add(ops ...logql.BinOp)

Add sets capability.

func (SupportedOps) Supports 

func (caps SupportedOps) Supports(op logql.BinOp) bool

Supports checks if storage supports given ops.

type UnpackExtractor added in v0.0.8 

type UnpackExtractor struct{}

UnpackExtractor extracts log entry fron Promtail `pack`-ed entry.

func (*UnpackExtractor) Process added in v0.0.8 

func (e *UnpackExtractor) Process(_ otelstorage.Timestamp, line string, set LabelSet) (string, bool)

Process implements Processor.

type UnsupportedError 

type UnsupportedError struct {
	Msg string
}

UnsupportedError is an error that reports unsupported expressions.

func (*UnsupportedError) Error 

func (e *UnsupportedError) Error() string

Error implements error.

Source Files

View all Source files

Directories

Path Synopsis
Package jsonexpr provides JSON extractor expression parser.
 Package jsonexpr provides JSON extractor expression parser.
Package logqlmetric provides metric queries implementation.
 Package logqlmetric provides metric queries implementation.
Package logqlpattern contains parser for LogQL `pattern` stage pattern.
 Package logqlpattern contains parser for LogQL `pattern` stage pattern.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.