expfmt

package
v0.61.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 4, 2024 License: Apache-2.0 Imports: 19 Imported by: 3,412

Documentation

Overview

Package expfmt contains tools for reading and writing Prometheus metrics.

Index

Constants

View Source 
const (
	TextVersion   = "0.0.4"
	ProtoType     = `application/vnd.google.protobuf`
	ProtoProtocol = `io.prometheus.client.MetricFamily`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeProtoCompact) instead.
	ProtoFmt                 = ProtoType + "; proto=" + ProtoProtocol + ";"
	OpenMetricsType          = `application/openmetrics-text`
	OpenMetricsVersion_0_0_1 = "0.0.1"
	OpenMetricsVersion_1_0_0 = "1.0.0"

	// The Content-Type values for the different wire protocols. Do not do direct
	// comparisons to these constants, instead use the comparison functions.
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeUnknown) instead.
	FmtUnknown Format = `<unknown>`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeTextPlain) instead.
	FmtText Format = `text/plain; version=` + TextVersion + `; charset=utf-8`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeProtoDelim) instead.
	FmtProtoDelim Format = ProtoFmt + ` encoding=delimited`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeProtoText) instead.
	FmtProtoText Format = ProtoFmt + ` encoding=text`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeProtoCompact) instead.
	FmtProtoCompact Format = ProtoFmt + ` encoding=compact-text`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeOpenMetrics) instead.
	FmtOpenMetrics_1_0_0 Format = OpenMetricsType + `; version=` + OpenMetricsVersion_1_0_0 + `; charset=utf-8`
	// Deprecated: Use expfmt.NewFormat(expfmt.TypeOpenMetrics) instead.
	FmtOpenMetrics_0_0_1 Format = OpenMetricsType + `; version=` + OpenMetricsVersion_0_0_1 + `; charset=utf-8`
)

Constants to assemble the Content-Type values for the different wire protocols. The Content-Type strings here are all for the legacy exposition formats, where valid characters for metric names and label names are limited. Support for arbitrary UTF-8 characters in those names is already partially implemented in this module (see model.ValidationScheme), but to actually use it on the wire, new content-type strings will have to be agreed upon and added here.

Variables

This section is empty.

Functions

func ExtractSamples 

func ExtractSamples(o *DecodeOptions, fams ...*dto.MetricFamily) (model.Vector, error)

ExtractSamples builds a slice of samples from the provided metric families. If an error occurs during sample extraction, it continues to extract from the remaining metric families. The returned error is the last error that has occurred.

func FinalizeOpenMetrics added in v0.9.0 

func FinalizeOpenMetrics(w io.Writer) (written int, err error)

FinalizeOpenMetrics writes the final `# EOF\n` line required by OpenMetrics.

func MetricFamilyToOpenMetrics added in v0.9.0 

func MetricFamilyToOpenMetrics(out io.Writer, in *dto.MetricFamily, options ...EncoderOption) (written int, err error)

MetricFamilyToOpenMetrics converts a MetricFamily proto message into the OpenMetrics text format and writes the resulting lines to 'out'. It returns the number of bytes written and any error encountered. The output will have the same order as the input, no further sorting is performed. Furthermore, this function assumes the input is already sanitized and does not perform any sanity checks. If the input contains duplicate metrics or invalid metric or label names, the conversion will result in invalid text format output.

If metric names conform to the legacy validation pattern, they will be placed outside the brackets in the traditional way, like `foo{}`. If the metric name fails the legacy validation check, it will be placed quoted inside the brackets: `{"foo"}`. As stated above, the input is assumed to be santized and no error will be thrown in this case.

Similar to metric names, if label names conform to the legacy validation pattern, they will be unquoted as normal, like `foo{bar="baz"}`. If the label name fails the legacy validation check, it will be quoted: `foo{"bar"="baz"}`. As stated above, the input is assumed to be santized and no error will be thrown in this case.

This function fulfills the type 'expfmt.encoder'.

Note that OpenMetrics requires a final `# EOF` line. Since this function acts on individual metric families, it is the responsibility of the caller to append this line to 'out' once all metric families have been written. Conveniently, this can be done by calling FinalizeOpenMetrics.

The output should be fully OpenMetrics compliant. However, there are a few missing features and peculiarities to avoid complications when switching from Prometheus to OpenMetrics or vice versa:

  • Counters are expected to have the `_total` suffix in their metric name. In the output, the suffix will be truncated from the `# TYPE`, `# HELP` and `# UNIT` lines. A counter with a missing `_total` suffix is not an error. However, its type will be set to `unknown` in that case to avoid invalid OpenMetrics output.

  • According to the OM specs, the `# UNIT` line is optional, but if populated, the unit has to be present in the metric name as its suffix: (see https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#unit). However, in order to accommodate any potential scenario where such a change in the metric name is not desirable, the users are here given the choice of either explicitly opt in, in case they wish for the unit to be included in the output AND in the metric name as a suffix (see the description of the WithUnit function above), or not to opt in, in case they don't want for any of that to happen.

  • No support for the following (optional) features: info type, stateset type, gaugehistogram type.

  • The size of exemplar labels is not checked (i.e. it's possible to create exemplars that are larger than allowed by the OpenMetrics specification).

  • The value of Counters is not checked. (OpenMetrics doesn't allow counters with a `NaN` value.)

func MetricFamilyToText 

func MetricFamilyToText(out io.Writer, in *dto.MetricFamily) (written int, err error)

MetricFamilyToText converts a MetricFamily proto message into text format and writes the resulting lines to 'out'. It returns the number of bytes written and any error encountered. The output will have the same order as the input, no further sorting is performed. Furthermore, this function assumes the input is already sanitized and does not perform any sanity checks. If the input contains duplicate metrics or invalid metric or label names, the conversion will result in invalid text format output.

If metric names conform to the legacy validation pattern, they will be placed outside the brackets in the traditional way, like `foo{}`. If the metric name fails the legacy validation check, it will be placed quoted inside the brackets: `{"foo"}`. As stated above, the input is assumed to be santized and no error will be thrown in this case.

Similar to metric names, if label names conform to the legacy validation pattern, they will be unquoted as normal, like `foo{bar="baz"}`. If the label name fails the legacy validation check, it will be quoted: `foo{"bar"="baz"}`. As stated above, the input is assumed to be santized and no error will be thrown in this case.

This method fulfills the type 'prometheus.encoder'.

Types

type Closer added in v0.9.0 

type Closer interface {
	Close() error
}

Closer is implemented by Encoders that need to be closed to finalize encoding. (For example, OpenMetrics needs a final `# EOF` line.)

Note that all Encoder implementations returned from this package implement Closer, too, even if the Close call is a no-op. This happens in preparation for adding a Close method to the Encoder interface directly in a (mildly breaking) release in the future.

type DecodeOptions 

type DecodeOptions struct {
	// Timestamp is added to each value from the stream that has no explicit timestamp set.
	Timestamp model.Time
}

DecodeOptions contains options used by the Decoder and in sample extraction.

type Decoder 

type Decoder interface {
	Decode(*dto.MetricFamily) error
}

Decoder types decode an input stream into metric families.

func NewDecoder 

func NewDecoder(r io.Reader, format Format) Decoder

NewDecoder returns a new decoder based on the given input format. If the input format does not imply otherwise, a text format decoder is returned.

type Encoder 

type Encoder interface {
	Encode(*dto.MetricFamily) error
}

Encoder types encode metric families into an underlying wire protocol.

func NewEncoder 

func NewEncoder(w io.Writer, format Format, options ...EncoderOption) Encoder

NewEncoder returns a new encoder based on content type negotiation. All Encoder implementations returned by NewEncoder also implement Closer, and callers should always call the Close method. It is currently only required for FmtOpenMetrics, but a future (breaking) release will add the Close method to the Encoder interface directly. The current version of the Encoder interface is kept for backwards compatibility. In cases where the Format does not allow for UTF-8 names, the global NameEscapingScheme will be applied.

NewEncoder can be called with additional options to customize the OpenMetrics text output. For example: NewEncoder(w, FmtOpenMetrics_1_0_0, WithCreatedLines())

Extra options are ignored for all other formats.

type EncoderOption added in v0.49.0 

type EncoderOption func(*encoderOption)

func WithCreatedLines added in v0.49.0 

func WithCreatedLines() EncoderOption

WithCreatedLines is an EncoderOption that configures the OpenMetrics encoder to include _created lines (See https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#counter-1). Created timestamps can improve the accuracy of series reset detection, but come with a bandwidth cost.

At the time of writing, created timestamp ingestion is still experimental in Prometheus and need to be enabled with the feature-flag `--feature-flag=created-timestamp-zero-ingestion`, and breaking changes are still possible. Therefore, it is recommended to use this feature with caution.

func WithUnit added in v0.50.0 

func WithUnit() EncoderOption

WithUnit is an EncoderOption enabling a set unit to be written to the output and to be added to the metric name, if it's not there already, as a suffix. Without opting in this way, the unit will not be added to the metric name and, on top of that, the unit will not be passed onto the output, even if it were declared in the *dto.MetricFamily struct, i.e. even if in.Unit !=nil.

type Format 

type Format string

Format specifies the HTTP content type of the different wire protocols.

func Negotiate 

func Negotiate(h http.Header) Format

Negotiate returns the Content-Type based on the given Accept header. If no appropriate accepted type is found, FmtText is returned (which is the Prometheus text format). This function will never negotiate FmtOpenMetrics, as the support is still experimental. To include the option to negotiate FmtOpenMetrics, use NegotiateOpenMetrics.

func NegotiateIncludingOpenMetrics added in v0.9.0 

func NegotiateIncludingOpenMetrics(h http.Header) Format

NegotiateIncludingOpenMetrics works like Negotiate but includes FmtOpenMetrics as an option for the result. Note that this function is temporary and will disappear once FmtOpenMetrics is fully supported and as such may be negotiated by the normal Negotiate function.

func NewFormat added in v0.48.0 

func NewFormat(t FormatType) Format

NewFormat generates a new Format from the type provided. Mostly used for tests, most Formats should be generated as part of content negotiation in encode.go. If a type has more than one version, the latest version will be returned.

func NewOpenMetricsFormat added in v0.51.0 

func NewOpenMetricsFormat(version string) (Format, error)

NewOpenMetricsFormat generates a new OpenMetrics format matching the specified version number.

func ResponseFormat 

func ResponseFormat(h http.Header) Format

ResponseFormat extracts the correct format from a HTTP response header. If no matching format can be found FormatUnknown is returned.

func (Format) FormatType added in v0.47.0 

func (f Format) FormatType() FormatType

FormatType deduces an overall FormatType for the given format.

func (Format) ToEscapingScheme added in v0.47.0 

func (format Format) ToEscapingScheme() model.EscapingScheme

ToEscapingScheme returns an EscapingScheme depending on the Format. Iff the Format contains a escaping=allow-utf-8 term, it will select NoEscaping. If a valid "escaping" term exists, that will be used. Otherwise, the global default will be returned.

func (Format) WithEscapingScheme added in v0.59.0 

func (f Format) WithEscapingScheme(s model.EscapingScheme) Format

WithEscapingScheme returns a copy of Format with the specified escaping scheme appended to the end. If an escaping scheme already exists it is removed.

type FormatType added in v0.47.0 

type FormatType int

FormatType is a Go enum representing the overall category for the given Format. As the number of Format permutations increases, doing basic string comparisons are not feasible, so this enum captures the most useful high-level attribute of the Format string. 

const (
	TypeUnknown FormatType = iota
	TypeProtoCompact
	TypeProtoDelim
	TypeProtoText
	TypeTextPlain
	TypeOpenMetrics
)

type ParseError 

type ParseError struct {
	Line int
	Msg  string
}

ParseError signals errors while parsing the simple and flat text-based exchange format.

func (ParseError) Error 

func (e ParseError) Error() string

Error implements the error interface.

type SampleDecoder 

type SampleDecoder struct {
	Dec  Decoder
	Opts *DecodeOptions
	// contains filtered or unexported fields
}

SampleDecoder wraps a Decoder to extract samples from the metric families decoded by the wrapped Decoder.

func (*SampleDecoder) Decode 

func (sd *SampleDecoder) Decode(s *model.Vector) error

Decode calls the Decode method of the wrapped Decoder and then extracts the samples from the decoded MetricFamily into the provided model.Vector.

type TextParser 

type TextParser struct {
	// contains filtered or unexported fields
}

TextParser is used to parse the simple and flat text-based exchange format. Its zero value is ready to use.

func (*TextParser) TextToMetricFamilies 

func (p *TextParser) TextToMetricFamilies(in io.Reader) (map[string]*dto.MetricFamily, error)

TextToMetricFamilies reads 'in' as the simple and flat text-based exchange format and creates MetricFamily proto messages. It returns the MetricFamily proto messages in a map where the metric names are the keys, along with any error encountered.

If the input contains duplicate metrics (i.e. lines with the same metric name and exactly the same label set), the resulting MetricFamily will contain duplicate Metric proto messages. Similar is true for duplicate label names. Checks for duplicates have to be performed separately, if required. Also note that neither the metrics within each MetricFamily are sorted nor the label pairs within each Metric. Sorting is not required for the most frequent use of this method, which is sample ingestion in the Prometheus server. However, for presentation purposes, you might want to sort the metrics, and in some cases, you must sort the labels, e.g. for consumption by the metric family injection hook of the Prometheus registry.

Summaries and histograms are rather special beasts. You would probably not use them in the simple text format anyway. This method can deal with summaries and histograms if they are presented in exactly the way the text.Create function creates them.

This method must not be called concurrently. If you want to parse different input concurrently, instantiate a separate Parser for each goroutine.

Notes

Bugs

  • Update other names to "quantile".

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.