lineprotocol

package
v2.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 24, 2021 License: MIT Imports: 11 Imported by: 34

Documentation

Overview

Package lineprotocol implements a codec for the InfluxDB line-protocol syntax. See the documentation of the Encoder and Decoder types for details.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrValueOutOfRange = errors.New("line-protocol value out of range")

ErrValueOutOfRange signals that a value is out of the acceptable numeric range.

Functions

This section is empty.

Types

type DecodeError 

type DecodeError struct {
	// Line holds the one-based index of the line where the error occurred.
	Line int64
	// Column holds the one-based index of the column (in bytes) where the error occurred.
	Column int
	// Err holds the underlying error.
	Err error
}

DecodeError represents an error when decoding a line-protocol entry.

func (*DecodeError) Error 

func (e *DecodeError) Error() string

Error implements the error interface.

func (*DecodeError) Unwrap 

func (e *DecodeError) Unwrap() error

Unwrap implements error unwrapping so that the underlying error can be retrieved.

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder implements low level parsing of a set of line-protocol entries.

Decoder methods must be called in the same order that their respective sections appear in a line-protocol entry. See the documentation on the Decoder.Next method for details.

Example 
package main

import (
	"fmt"
	"time"

	"github.com/influxdata/line-protocol/v2/lineprotocol"
)

func main() {
	data := []byte(`
foo,tag1=val1,tag2=val2 x=1,y="hello" 1625823259000000
bar enabled=true
`)
	dec := lineprotocol.NewDecoderWithBytes(data)
	for dec.Next() {
		fmt.Printf("\nstart entry\n")
		m, err := dec.Measurement()
		if err != nil {
			panic(err)
		}
		fmt.Printf("measurement %s\n", m)
		for {
			key, val, err := dec.NextTag()
			if err != nil {
				panic(err)
			}
			if key == nil {
				break
			}
			fmt.Printf("tag %s=%s\n", key, val)
		}
		for {
			key, val, err := dec.NextField()
			if err != nil {
				panic(err)
			}
			if key == nil {
				break
			}
			fmt.Printf("field %s=%v\n", key, val)
		}
		t, err := dec.Time(lineprotocol.Microsecond, time.Time{})
		if err != nil {
			panic(err)
		}
		if t.IsZero() {
			fmt.Printf("no timestamp\n")
		} else {
			fmt.Printf("timestamp %s\n", t.UTC().Format(time.RFC3339Nano))
		}
	}
	// Note: because we're decoding from a slice of bytes, dec.Error can't return
	// an error. If we were decoding from an io.Reader, we'd need to check dec.Error
	// here.

}

Output:


start entry
measurement foo
tag tag1=val1
tag tag2=val2
field x=1
field y="hello"
timestamp 2021-07-09T09:34:19Z

start entry
measurement bar
field enabled=true
no timestamp

func NewDecoder 

func NewDecoder(r io.Reader) *Decoder

NewDecoder returns a decoder that reads from the given reader.

func NewDecoderWithBytes 

func NewDecoderWithBytes(buf []byte) *Decoder

NewDecoder returns a decoder that splits the line-protocol text inside buf.

func (*Decoder) Err 

func (d *Decoder) Err() error

Err returns any I/O error encountered when reading entries. If d was created with NewDecoderWithBytes, Err will always return nil.

func (*Decoder) Measurement 

func (d *Decoder) Measurement() ([]byte, error)

Measurement returns the measurement name. It returns nil unless called before NextTag, NextField or Time.

func (*Decoder) Next 

func (d *Decoder) Next() bool

Next advances to the next entry, and reports whether there is an entry available. Syntax errors on individual lines do not cause this to return false (the decoder attempts to recover from badly formatted lines), but I/O errors do. Call d.Err to discover if there was any I/O error. Syntax errors are returned as *DecoderError errors from Decoder methods.

After calling Next, the various components of a line can be retrieved by calling Measurement, NextTag, NextField and Time in that order (the same order that the components are held in the entry).

IMPORTANT NOTE: the byte slices returned by the Decoder methods are only valid until the next call to any other Decode method.

Decoder will skip earlier components if a later method is called, but it doesn't retain the entire entry, so it cannot go backwards.

For example, to retrieve only the timestamp of all lines, this suffices: 

for d.Next() {
	timestamp, err := d.TimeBytes()
}

func (*Decoder) NextField 

func (d *Decoder) NextField() (key []byte, val Value, err error)

NextField is a wrapper around NextFieldBytes that parses the field value. Note: the returned value is only valid until the next call method call on Decoder because when it's a string, it refers to an internal buffer.

If the value cannot be parsed because it's out of range (as opposed to being syntactically invalid), the errors.Is(err, ErrValueOutOfRange) will return true.

func (*Decoder) NextFieldBytes 

func (d *Decoder) NextFieldBytes() (key []byte, kind ValueKind, value []byte, err error)

NextFieldBytes returns the next field in the entry. If there are no more fields, it returns all zero values. Note that this must be called before Time because fields precede the timestamp in the line-protocol entry.

The returned value slice may not be valid: to check its validity, use NewValueFromBytes(kind, value), or use NextField.

func (*Decoder) NextTag 

func (d *Decoder) NextTag() (key, value []byte, err error)

NextTag returns the next tag in the entry. If there are no more tags, it returns nil, nil, nil. Note that this must be called before NextField because tags precede fields in the line-protocol entry.

func (*Decoder) Time 

func (d *Decoder) Time(prec Precision, defaultTime time.Time) (time.Time, error)

Time is a wrapper around TimeBytes that returns the timestamp assuming the given precision.

func (*Decoder) TimeBytes 

func (d *Decoder) TimeBytes() ([]byte, error)

TimeBytes returns the timestamp of the entry as a byte slice. If there is no timestamp, it returns nil, nil.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder encapsulates the encoding part of the line protocol.

The zero value of an Encoder is ready to use.

It is associated with a []byte buffer which is appended to each time a method is called.

Methods must be called in the same order that their respective data appears in the line-protocol point (Encoder doesn't reorder anything). That is, for a given entry, methods must be called in the following order: 

StartLine
AddTag (zero or more times)
AddField (one or more times)
EndLine (optional)

When an error is encountered encoding a point, the Err method returns it, and the erroneous point is omitted from the result.

Example 
package main

import (
	"fmt"
	"time"

	"github.com/influxdata/line-protocol/v2/lineprotocol"
)

func main() {
	var enc lineprotocol.Encoder
	enc.SetPrecision(lineprotocol.Microsecond)
	enc.StartLine("foo")
	enc.AddTag("tag1", "val1")
	enc.AddTag("tag2", "val2")
	enc.AddField("x", lineprotocol.MustNewValue(1.0))
	enc.AddField("y", lineprotocol.MustNewValue("hello"))
	enc.EndLine(time.Unix(0, 1625823259000000000))
	enc.StartLine("bar")
	enc.AddField("enabled", lineprotocol.BoolValue(true))
	enc.EndLine(time.Time{})
	if err := enc.Err(); err != nil {
		panic(fmt.Errorf("encoding error: %v", err))
	}
	fmt.Printf("%s", enc.Bytes())
}

Output:

foo,tag1=val1,tag2=val2 x=1,y="hello" 1625823259000000
bar enabled=true

func (*Encoder) AddField 

func (e *Encoder) AddField(key string, value Value)

AddField adds a field to the line. AddField must be called after AddTag or AddMeasurement. At least one field must be added to each line.

func (*Encoder) AddFieldRaw 

func (e *Encoder) AddFieldRaw(key []byte, value Value)

AddFieldRaw is like AddField except that the key is represented as a byte slice instead of a string, which can save allocations. TODO would it be better for this to be: 

AddFieldRaw(key []byte, kind ValueKind, data []byte) error

so that we could respect lax and be more efficient when reading directly from a Decoder?

func (*Encoder) AddTag 

func (e *Encoder) AddTag(key, value string)

AddTag adds a tag to the line. Tag keys must be added in lexical order and AddTag must be called after StartLine and before AddField.

Neither the key or the value may contain non-printable ASCII characters (0x00 to 0x1f and 0x7f) or invalid UTF-8 or a trailing backslash character.

func (*Encoder) AddTagRaw 

func (e *Encoder) AddTagRaw(key, value []byte)

AddTagRaw is like AddTag except that it accepts byte slices instead of strings, which can save allocations. Note that AddRawTag _will_ escape metacharacters such as "=" and "," when they're present.

func (*Encoder) Bytes 

func (e *Encoder) Bytes() []byte

Bytes returns the current line buffer.

func (*Encoder) ClearErr 

func (e *Encoder) ClearErr()

ClearErr clears any current encoding error.

func (*Encoder) EndLine 

func (e *Encoder) EndLine(t time.Time)

EndLine adds the timestamp and newline at the end of the line. If t is zero, no timestamp will written and this method will do nothing. If the time is outside the maximum representable time range, an ErrRange error will be returned.

func (*Encoder) Err 

func (e *Encoder) Err() error

Err returns the first encoding error that's been encountered so far, if any. TODO define a type so that we can get access to the line where it happened.

func (*Encoder) Reset 

func (e *Encoder) Reset()

Reset resets the line, clears any error, and starts writing at the start of the line buffer slice.

func (*Encoder) SetBuffer 

func (e *Encoder) SetBuffer(buf []byte)

SetBuffer sets the buffer used for the line, clears any current error and resets the line.

Encoded data will be appended to buf.

func (*Encoder) SetLax 

func (e *Encoder) SetLax(lax bool)

SetLax sets whether the Encoder methods check fully for validity or not. When Lax is true:

- measurement names, tag and field keys aren't checked for invalid characters - field values passed to AddRawField are not bounds or syntax checked - tag keys are not checked to be in alphabetical order.

This can be used to increase performance in places where values are already known to be valid.

func (*Encoder) SetPrecision 

func (e *Encoder) SetPrecision(p Precision)

SetPrecision sets the precision used to encode the time stamps in the encoded messages. The default precision is Nanosecond. Timestamps are truncated to this precision.

func (*Encoder) StartLine 

func (e *Encoder) StartLine(measurement string)

StartLine starts writing a line with the given measurement name. If this is called when it's not possible to start a new entry, or the measurement cannot be encoded, it will return an error.

Starting a new entry is always allowed when there's been an error encoding the previous entry.

func (*Encoder) StartLineRaw 

func (e *Encoder) StartLineRaw(name []byte)

StartLineRaw is the same as Start except that it accepts a byte slice instead of a string, which can save allocations.

type Precision 

type Precision byte

Precision specifies the scale at which a line-protocol timestamp is encoded. 

const (
	Nanosecond Precision = iota
	Microsecond
	Millisecond
	Second
)

func (Precision) Duration 

func (p Precision) Duration() time.Duration

Duration returns the time duration for the given precision. For example, Second.Duration() is time.Second.

func (Precision) String 

func (p Precision) String() string

String returns p as a string (ns, µs, ms or s).

type Value 

type Value struct {
	// contains filtered or unexported fields
}

Value holds one of the possible line-protocol field values.

func BoolValue 

func BoolValue(x bool) Value

BoolValue returns a Value containing the value of x.

func FloatValue 

func FloatValue(x float64) (Value, bool)

FloatValue returns a Value containing the value of x.

FloatValue will fail and return false if x is non-finite.

func IntValue 

func IntValue(x int64) Value

IntValue returns a Value containing the value of x.

func MustNewValue 

func MustNewValue(x interface{}) Value

MustNewValue is like NewValue except that it panics on failure.

func NewValue 

func NewValue(x interface{}) (Value, bool)

NewValue returns a Value containing the value of x, which must be of type int64 (Int), uint64 (Uint), float64 (Float), bool (Bool), string (String) or []byte (String).

Unlike NewValueFromBytes, NewValue will make a copy of the byte slice if x is []byte - use NewValueFromBytes if you require zero-copy semantics.

NewValue will fail and return false if x isn't a recognized type or if it's a non-finite float64, or if a string or byte slice contains invalid utf-8.

func NewValueFromBytes 

func NewValueFromBytes(kind ValueKind, data []byte) (Value, error)

NewValueFromBytes creates a value of the given kind with the given data, as returned from Decoder.NextFieldBytes.

If the value is out of range, errors.Is(err, ErrValueOutOfRange) will return true.

The data for Int and Uint kinds should not include the type suffixes present in the line-protocol field values. For example, the data for the zero Int should be "0" not "0i".

The data for String should not include the surrounding quotes, should be unescaped already and should not contain invalid utf-8. The returned value will contain a reference to data - it does not make a copy.

func StringValue 

func StringValue(x string) (Value, bool)

StringValue returns a Value containing the value of x.

StringValue will fail and return false if x contains invalid utf-8.

func StringValueFromBytes 

func StringValueFromBytes(x []byte) (Value, bool)

StringValueFromBytes returns a Value containing the value of x.

StringValueFromBytes will fail and return false if x contains invalid utf-8.

Unlike NewValueFromBytes, StringValueFromBytes will make a copy of the byte slice - use NewValueFromBytes if you require zero-copy semantics.

func UintValue 

func UintValue(x uint64) Value

UintValue returns a Value containing the value of x.

func (Value) AppendBytes 

func (v Value) AppendBytes(dst []byte) []byte

AppendTo appends the encoded value of v to buf.

func (Value) BoolV 

func (v Value) BoolV() bool

BoolV returns the value as a bool. It panics if v.Kind is not Bool.

func (Value) BytesV 

func (v Value) BytesV() []byte

BytesV returns the value as a []byte. It panics if v.Kind is not String. Note that this may return a direct reference to the byte slice within the value - modifying the returned byte slice may mutate the contents of the Value.

func (Value) Equal 

func (v1 Value) Equal(v2 Value) bool

Equal reports whether v1 is equal to v2.

func (Value) FloatV 

func (v Value) FloatV() float64

FloatV returns the value as a float64. It panics if v.Kind is not Float.

func (Value) IntV 

func (v Value) IntV() int64

IntV returns the value as an int64. It panics if v.Kind is not Int.

func (Value) Interface 

func (v Value) Interface() interface{}

Interface returns the value as an interface. The returned value will have a different dynamic type depending on the value kind; one of int64 (Int), uint64 (Uint), float64 (Float), string (String), bool (Bool).

func (Value) Kind 

func (v Value) Kind() ValueKind

func (Value) String 

func (v Value) String() string

String returns the value as it would be encoded in a line-protocol entry.

func (Value) StringV 

func (v Value) StringV() string

StringV returns the value as a string. It panics if v.Kind is not String.

func (Value) UintV 

func (v Value) UintV() uint64

UintV returns the value as a uint64. It panics if v.Kind is not Uint.

type ValueKind 

type ValueKind uint8

ValueKind represents the type of a field value. 

const (
	Unknown ValueKind = iota
	String
	Int
	Uint
	Float
	Bool
)

func (ValueKind) MarshalText 

func (k ValueKind) MarshalText() ([]byte, error)

MarshalText implements encoding.TextMarshaler for ValueKind. It returns an error if k is Unknown.

func (ValueKind) String 

func (k ValueKind) String() string

String returns k as a string. It panics if k isn't one of the enumerated ValueKind constants. The string form is the lower-case form of the constant.

func (*ValueKind) UnmarshalText 

func (k *ValueKind) UnmarshalText(data []byte) error

UnmarshalText implements encoding.TextUnmarshaler for ValueKind.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.