textparse

package
v0.300.0-rc.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2024 License: Apache-2.0 Imports: 22 Imported by: 32

README

Making changes to textparse lexers

In the rare case that you need to update the textparse lexers, edit promlex.l or openmetricslex.l and then run the following command: golex -o=promlex.l.go promlex.l

Note that you need golex installed: go get -u modernc.org/golex

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Entry 

type Entry int

Entry represents the type of a parsed entry. 

const (
	EntryInvalid   Entry = -1
	EntryType      Entry = 0
	EntryHelp      Entry = 1
	EntrySeries    Entry = 2 // EntrySeries marks a series with a simple float64 as value.
	EntryComment   Entry = 3
	EntryUnit      Entry = 4
	EntryHistogram Entry = 5 // EntryHistogram marks a series with a native histogram as a value.
)

type NHCBParser 

type NHCBParser struct {
	// contains filtered or unexported fields
}

The NHCBParser wraps a Parser and converts classic histograms to native histograms with custom buckets.

Since Parser interface is line based, this parser needs to keep track of the last classic histogram series it saw to collate them into a single native histogram.

Note:

  • Only series that have the histogram metadata type are considered for conversion.
  • The classic series are also returned if keepClassicHistograms is true.

func (*NHCBParser) Comment 

func (p *NHCBParser) Comment() []byte

func (*NHCBParser) CreatedTimestamp 

func (p *NHCBParser) CreatedTimestamp() *int64

func (*NHCBParser) Exemplar 

func (p *NHCBParser) Exemplar(ex *exemplar.Exemplar) bool

func (*NHCBParser) Help 

func (p *NHCBParser) Help() ([]byte, []byte)

func (*NHCBParser) Histogram 

func (p *NHCBParser) Histogram() ([]byte, *int64, *histogram.Histogram, *histogram.FloatHistogram)

func (*NHCBParser) Metric 

func (p *NHCBParser) Metric(l *labels.Labels) string

func (*NHCBParser) Next 

func (p *NHCBParser) Next() (Entry, error)

func (*NHCBParser) Series 

func (p *NHCBParser) Series() ([]byte, *int64, float64)

func (*NHCBParser) Type 

func (p *NHCBParser) Type() ([]byte, model.MetricType)

func (*NHCBParser) Unit 

func (p *NHCBParser) Unit() ([]byte, []byte)

type OpenMetricsOption added in v0.55.0 

type OpenMetricsOption func(*openMetricsParserOptions)

func WithOMParserCTSeriesSkipped added in v0.55.0 

func WithOMParserCTSeriesSkipped() OpenMetricsOption

WithOMParserCTSeriesSkipped turns off exposing _created lines as series, which makes those only used for parsing created timestamp for `CreatedTimestamp` method purposes.

It's recommended to use this option to avoid using _created lines for other purposes than created timestamp, but leave false by default for the best-effort compatibility.

type OpenMetricsParser 

type OpenMetricsParser struct {
	// contains filtered or unexported fields
}

OpenMetricsParser parses samples from a byte slice of samples in the official OpenMetrics text exposition format. This is based on the working draft https://docs.google.com/document/u/1/d/1KwV0mAXwwbvvifBvDKH_LU1YjyXE_wxCkHNoCGq1GX0/edit

func (*OpenMetricsParser) Comment 

func (p *OpenMetricsParser) Comment() []byte

Comment returns the text of the current comment. Must only be called after Next returned a comment entry. The returned byte slice becomes invalid after the next call to Next.

func (*OpenMetricsParser) CreatedTimestamp added in v0.49.0 

func (p *OpenMetricsParser) CreatedTimestamp() *int64

CreatedTimestamp returns the created timestamp for a current Metric if exists or nil. NOTE(Maniktherana): Might use additional CPU/mem resources due to deep copy of parser required for peeking given 1.0 OM specification on _created series.

func (*OpenMetricsParser) Exemplar 

func (p *OpenMetricsParser) Exemplar(e *exemplar.Exemplar) bool

Exemplar writes the exemplar of the current sample into the passed exemplar. It returns whether an exemplar exists. As OpenMetrics only ever has one exemplar per sample, every call after the first (for the same sample) will always return false.

func (*OpenMetricsParser) Help 

func (p *OpenMetricsParser) Help() ([]byte, []byte)

Help returns the metric name and help text in the current entry. Must only be called after Next returned a help entry. The returned byte slices become invalid after the next call to Next.

func (*OpenMetricsParser) Histogram added in v0.40.0 

func (p *OpenMetricsParser) Histogram() ([]byte, *int64, *histogram.Histogram, *histogram.FloatHistogram)

Histogram returns (nil, nil, nil, nil) for now because OpenMetrics does not support sparse histograms yet.

func (*OpenMetricsParser) Metric 

func (p *OpenMetricsParser) Metric(l *labels.Labels) string

Metric writes the labels of the current sample into the passed labels. It returns the string from which the metric was parsed.

func (*OpenMetricsParser) Next 

func (p *OpenMetricsParser) Next() (Entry, error)

Next advances the parser to the next sample. It returns (EntryInvalid, io.EOF) if no samples were read.

func (*OpenMetricsParser) Series 

func (p *OpenMetricsParser) Series() ([]byte, *int64, float64)

Series returns the bytes of the series, the timestamp if set, and the value of the current sample.

func (*OpenMetricsParser) Type 

func (p *OpenMetricsParser) Type() ([]byte, model.MetricType)

Type returns the metric name and type in the current entry. Must only be called after Next returned a type entry. The returned byte slices become invalid after the next call to Next.

func (*OpenMetricsParser) Unit 

func (p *OpenMetricsParser) Unit() ([]byte, []byte)

Unit returns the metric name and unit in the current entry. Must only be called after Next returned a unit entry. The returned byte slices become invalid after the next call to Next.

type Parser 

type Parser interface {
	// Series returns the bytes of a series with a simple float64 as a
	// value, the timestamp if set, and the value of the current sample.
	Series() ([]byte, *int64, float64)

	// Histogram returns the bytes of a series with a sparse histogram as a
	// value, the timestamp if set, and the histogram in the current sample.
	// Depending on the parsed input, the function returns an (integer) Histogram
	// or a FloatHistogram, with the respective other return value being nil.
	Histogram() ([]byte, *int64, *histogram.Histogram, *histogram.FloatHistogram)

	// Help returns the metric name and help text in the current entry.
	// Must only be called after Next returned a help entry.
	// The returned byte slices become invalid after the next call to Next.
	Help() ([]byte, []byte)

	// Type returns the metric name and type in the current entry.
	// Must only be called after Next returned a type entry.
	// The returned byte slices become invalid after the next call to Next.
	Type() ([]byte, model.MetricType)

	// Unit returns the metric name and unit in the current entry.
	// Must only be called after Next returned a unit entry.
	// The returned byte slices become invalid after the next call to Next.
	Unit() ([]byte, []byte)

	// Comment returns the text of the current comment.
	// Must only be called after Next returned a comment entry.
	// The returned byte slice becomes invalid after the next call to Next.
	Comment() []byte

	// Metric writes the labels of the current sample into the passed labels.
	// It returns the string from which the metric was parsed.
	// The values of the "le" labels of classic histograms and "quantile" labels
	// of summaries should follow the OpenMetrics formatting rules.
	Metric(l *labels.Labels) string

	// Exemplar writes the exemplar of the current sample into the passed
	// exemplar. It can be called repeatedly to retrieve multiple exemplars
	// for the same sample. It returns false once all exemplars are
	// retrieved (including the case where no exemplars exist at all).
	Exemplar(l *exemplar.Exemplar) bool

	// CreatedTimestamp returns the created timestamp (in milliseconds) for the
	// current sample. It returns nil if it is unknown e.g. if it wasn't set,
	// if the scrape protocol or metric type does not support created timestamps.
	// Assume the CreatedTimestamp returned pointer is only valid until
	// the Next iteration.
	CreatedTimestamp() *int64

	// Next advances the parser to the next sample.
	// It returns (EntryInvalid, io.EOF) if no samples were read.
	Next() (Entry, error)
}

Parser parses samples from a byte slice of samples in different exposition formats.

func New 

func New(b []byte, contentType, fallbackType string, parseClassicHistograms, skipOMCTSeries bool, st *labels.SymbolTable) (Parser, error)

New returns a new parser of the byte slice.

This function no longer guarantees to return a valid parser.

It only returns a valid parser if the supplied contentType and fallbackType allow. An error may also be returned if fallbackType had to be used or there was some other error parsing the supplied Content-Type. If the returned parser is nil then the scrape must fail.

func NewNHCBParser 

func NewNHCBParser(p Parser, st *labels.SymbolTable, keepClassicHistograms bool) Parser

func NewOpenMetricsParser 

func NewOpenMetricsParser(b []byte, st *labels.SymbolTable, opts ...OpenMetricsOption) Parser

NewOpenMetricsParser returns a new parser for the byte slice with option to skip CT series parsing.

func NewPromParser 

func NewPromParser(b []byte, st *labels.SymbolTable) Parser

NewPromParser returns a new parser of the byte slice.

func NewProtobufParser added in v0.40.0 

func NewProtobufParser(b []byte, parseClassicHistograms bool, st *labels.SymbolTable) Parser

NewProtobufParser returns a parser for the payload in the byte slice.

type PromParser 

type PromParser struct {
	// contains filtered or unexported fields
}

PromParser parses samples from a byte slice of samples in the official Prometheus text exposition format.

func (*PromParser) Comment 

func (p *PromParser) Comment() []byte

Comment returns the text of the current comment. Must only be called after Next returned a comment entry. The returned byte slice becomes invalid after the next call to Next.

func (*PromParser) CreatedTimestamp added in v0.49.0 

func (p *PromParser) CreatedTimestamp() *int64

CreatedTimestamp returns nil as it's not implemented yet. TODO(bwplotka): https://github.com/prometheus/prometheus/issues/12980

func (*PromParser) Exemplar 

func (p *PromParser) Exemplar(*exemplar.Exemplar) bool

Exemplar implements the Parser interface. However, since the classic Prometheus text format does not support exemplars, this implementation simply returns false and does nothing else.

func (*PromParser) Help 

func (p *PromParser) Help() ([]byte, []byte)

Help returns the metric name and help text in the current entry. Must only be called after Next returned a help entry. The returned byte slices become invalid after the next call to Next.

func (*PromParser) Histogram added in v0.40.0 

func (p *PromParser) Histogram() ([]byte, *int64, *histogram.Histogram, *histogram.FloatHistogram)

Histogram returns (nil, nil, nil, nil) for now because the Prometheus text format does not support sparse histograms yet.

func (*PromParser) Metric 

func (p *PromParser) Metric(l *labels.Labels) string

Metric writes the labels of the current sample into the passed labels. It returns the string from which the metric was parsed.

func (*PromParser) Next 

func (p *PromParser) Next() (Entry, error)

Next advances the parser to the next sample. It returns (EntryInvalid, io.EOF) if no samples were read.

func (*PromParser) Series 

func (p *PromParser) Series() ([]byte, *int64, float64)

Series returns the bytes of the series, the timestamp if set, and the value of the current sample.

func (*PromParser) Type 

func (p *PromParser) Type() ([]byte, model.MetricType)

Type returns the metric name and type in the current entry. Must only be called after Next returned a type entry. The returned byte slices become invalid after the next call to Next.

func (*PromParser) Unit 

func (p *PromParser) Unit() ([]byte, []byte)

Unit returns the metric name and unit in the current entry. Must only be called after Next returned a unit entry. The returned byte slices become invalid after the next call to Next.

type ProtobufParser added in v0.40.0 

type ProtobufParser struct {
	// contains filtered or unexported fields
}

ProtobufParser is a very inefficient way of unmarshaling the old Prometheus protobuf format and then present it as it if were parsed by a Prometheus-2-style text parser. This is only done so that we can easily plug in the protobuf format into Prometheus 2. For future use (with the final format that will be used for native histograms), we have to revisit the parsing. A lot of the efficiency tricks of the Prometheus-2-style parsing could be used in a similar fashion (byte-slice pointers into the raw payload), which requires some hand-coded protobuf handling. But the current parsers all expect the full series name (metric name plus label pairs) as one string, which is not how things are represented in the protobuf format. If the re-arrangement work is actually causing problems (which has to be seen), that expectation needs to be changed.

func (*ProtobufParser) Comment added in v0.40.0 

func (p *ProtobufParser) Comment() []byte

Comment always returns nil because comments aren't supported by the protobuf format.

func (*ProtobufParser) CreatedTimestamp added in v0.49.0 

func (p *ProtobufParser) CreatedTimestamp() *int64

CreatedTimestamp returns CT or nil if CT is not present or invalid (as timestamp e.g. negative value) on counters, summaries or histograms.

func (*ProtobufParser) Exemplar added in v0.40.0 

func (p *ProtobufParser) Exemplar(ex *exemplar.Exemplar) bool

Exemplar writes the exemplar of the current sample into the passed exemplar. It returns if an exemplar exists or not. In case of a native histogram, the exemplars in the native histogram will be returned. If this field is empty, the classic bucket section is still used for exemplars. To ingest all exemplars, call the Exemplar method repeatedly until it returns false.

func (*ProtobufParser) Help added in v0.40.0 

func (p *ProtobufParser) Help() ([]byte, []byte)

Help returns the metric name and help text in the current entry. Must only be called after Next returned a help entry. The returned byte slices become invalid after the next call to Next.

func (*ProtobufParser) Histogram added in v0.40.0 

func (p *ProtobufParser) Histogram() ([]byte, *int64, *histogram.Histogram, *histogram.FloatHistogram)

Histogram returns the bytes of a series with a native histogram as a value, the timestamp if set, and the native histogram in the current sample.

The Compact method is called before returning the Histogram (or FloatHistogram).

If the SampleCountFloat or the ZeroCountFloat in the proto message is > 0, the histogram is parsed and returned as a FloatHistogram and nil is returned as the (integer) Histogram return value. Otherwise, it is parsed and returned as an (integer) Histogram and nil is returned as the FloatHistogram return value.

func (*ProtobufParser) Metric added in v0.40.0 

func (p *ProtobufParser) Metric(l *labels.Labels) string

Metric writes the labels of the current sample into the passed labels. It returns the string from which the metric was parsed.

func (*ProtobufParser) Next added in v0.40.0 

func (p *ProtobufParser) Next() (Entry, error)

Next advances the parser to the next "sample" (emulating the behavior of a text format parser). It returns (EntryInvalid, io.EOF) if no samples were read.

func (*ProtobufParser) Series added in v0.40.0 

func (p *ProtobufParser) Series() ([]byte, *int64, float64)

Series returns the bytes of a series with a simple float64 as a value, the timestamp if set, and the value of the current sample.

func (*ProtobufParser) Type added in v0.40.0 

func (p *ProtobufParser) Type() ([]byte, model.MetricType)

Type returns the metric name and type in the current entry. Must only be called after Next returned a type entry. The returned byte slices become invalid after the next call to Next.

func (*ProtobufParser) Unit added in v0.40.0 

func (p *ProtobufParser) Unit() ([]byte, []byte)

Unit returns the metric unit in the current entry. Must only be called after Next returned a unit entry. The returned byte slices become invalid after the next call to Next.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.