query

Documentation

Overview

Package query provides the datatypes and functions for construction queries using the Axiom Processing Language (APL) and working with their results.

Usage:

import "github.com/axiomhq/axiom-go/axiom/query"

Tabular Result Format

Query results are returned in a tabular format. Each query Result contains one or more [Table]s. Each Table contains a list of [Field]s and a list of [Column]s. All [Column]s are equally sized and there are as much [Column]s as there are [Field]s.

In case you want to work with events that are usually composed of multiple fields, you will find the values separated by Column. To aid with working with events in the tabular result format, the Table type provides the Table.Rows method that returns an iter.Iter over the [Row]s of the Table. Under the hood, each call to iter.Iter.Next composes a Row from the [Column]s of the Table. Alternatively, you can compose an iter.Iter over the [Row]s yourself using the Rows function. This allows for passing in a subset of the [Column]s of the Table to work with:

// Only build rows from the first two columns of the table. Returns an
// iterator for over the rows.
rows := query.Rows(result.Tables[0].Columns[0:2])

Keep in mind that it is preferable to alter the APL query to only return the fields you are interested in instead of working with a subset of the columns after the query has been executed.

Index

• func Rows(columns []Column) iter.Seq[Row]
• type Aggregation
• type AggregationOp
  • func (i AggregationOp) String() string
  • func (op *AggregationOp) UnmarshalJSON(b []byte) (err error)
• type Buckets
• type Column
  • func (c Column) Values() iter.Seq[any]
• type Field
• type Group
• type Option
  • func SetCursor(cursor string, include bool) Option
  • func SetEndTime(endTime time.Time) Option
  • func SetStartTime(startTime time.Time) Option
  • func SetVariable(name string, value any) Option
  • func SetVariables(variables map[string]any) Option
• type Options
• type Order
• type Range
• type Result
• type Row
  • func (r Row) Values() iter.Seq[any]
• type Source
• type Status
  • func (s *Status) UnmarshalJSON(b []byte) error
• type Table
  • func (t Table) Rows() iter.Seq[Row]

Examples

• Rows

Functions

func Rows

func Rows(columns []Column) iter.Seq[Row]

Rows returns an iterator over the rows build from the columns of a tabular query Result.

Example

package main

import (
	"fmt"
	"strings"

	"github.com/axiomhq/axiom-go/axiom/query"
)

func main() {
	columns := []query.Column{
		[]any{
			"2020-11-19T11:06:31.569475746Z",
			"2020-11-19T11:06:31.569479846Z",
		},
		[]any{
			"Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21)",
			"Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21)",
		},
		[]any{
			"93.180.71.3",
			"93.180.71.3",
		},
		[]any{
			"GET /downloads/product_1 HTTP/1.1",
			"GET /downloads/product_1 HTTP/1.1",
		},
		[]any{
			304,
			304,
		},
	}

	var buf strings.Builder
	for row := range query.Rows(columns) {
		_, _ = fmt.Fprintln(&buf, row)
	}

	fmt.Print(buf.String())
}

Output:

[2020-11-19T11:06:31.569475746Z Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21) 93.180.71.3 GET /downloads/product_1 HTTP/1.1 304]
[2020-11-19T11:06:31.569479846Z Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21) 93.180.71.3 GET /downloads/product_1 HTTP/1.1 304]

Types

type Aggregation

type Aggregation struct {
	// Op is the aggregation operation. If the aggregation is aliased, the alias
	// is stored in the parent [Field.Name].
	Op AggregationOp `json:"name"`
	// Fields specifies the names of the fields this aggregation is computed on.
	// E.g. ["players"] for "topk(players, 10)".
	Fields []string `json:"fields"`
	// Args are the non-field arguments of the aggregation, if any. E.g. "10"
	// for "topk(players, 10)".
	Args []any `json:"args"`
}

Aggregation that is applied to a Field in a Table.

type AggregationOp

type AggregationOp uint8

An AggregationOp describes the Aggregation operation applied on a Field.

const (
	OpUnknown AggregationOp = iota // unknown

	OpCount               // count
	OpCountIf             // countif
	OpDistinct            // distinct
	OpDistinctIf          // distinctif
	OpSum                 // sum
	OpSumIf               // sumif
	OpAvg                 // avg
	OpAvgIf               // avgif
	OpMin                 // min
	OpMinIf               // minif
	OpMax                 // max
	OpMaxIf               // maxif
	OpTopk                // topk
	OpPercentiles         // percentiles
	OpHistogram           // histogram
	OpStandardDeviation   // stdev
	OpStandardDeviationIf // stdevif
	OpVariance            // variance
	OpVarianceIf          // varianceif
	OpArgMin              // argmin
	OpArgMax              // argmax
	OpRate                // rate
	OpPearson             // pearson_correlation
	OpMakeSet             // makeset
	OpMakeSetIf           // makesetif
	OpMakeList            // makelist
	OpMakeListIf          // makelistif
)

All available Aggregation operations.

func (AggregationOp) String

func (i AggregationOp) String() string

func (*AggregationOp) UnmarshalJSON

func (op *AggregationOp) UnmarshalJSON(b []byte) (err error)

UnmarshalJSON implements json.Unmarshaler. It is in place to unmarshal the AggregationOp from the string representation the server returns.

type Buckets

type Buckets struct {
	// Field specifies the field used to create buckets on. Usually this would
	// be "_time".
	Field string
	// An integer or float representing the fixed bucket size.
	// When the bucket field is "_time" this value is in nanoseconds.
	Size any
}

Buckets captures information about how a grouped query is sorted into buckets. Usually buckets are created on the "_time" column.

type Column

type Column []any

Column in a Table containing the raw values of a Field.

func (Column) Values

func (c Column) Values() iter.Seq[any]

Values returns an iterator over the values of the column.

type Field

type Field struct {
	// Name of the field.
	Name string `json:"name"`
	// Type of the field. Can also be a composite type.
	Type string `json:"type"`
	// Aggregation is the aggregation applied to the field.
	Aggregation *Aggregation `json:"agg"`
}

Field in a Table.

type Group

type Group struct {
	// Name of the group.
	Name string `json:"name"`
}

Group in a Table.

type Option

type Option func(*Options)

An Option applies an optional parameter to a query.

func SetCursor

func SetCursor(cursor string, include bool) Option

SetCursor specifies the cursor of the query. If include is set to true the event that matches the cursor will be included in the result. When using this option, please make sure to use the initial query's start and end times.

func SetEndTime

func SetEndTime(endTime time.Time) Option

SetEndTime specifies the end time of the query interval. When also using SetCursor, please make sure to use the end time of the query that returned the cursor that will be used.

func SetStartTime

func SetStartTime(startTime time.Time) Option

SetStartTime specifies the start time of the query interval. When also using SetCursor, please make sure to use the start time of the query that returned the cursor that will be used.

func SetVariable

func SetVariable(name string, value any) Option

SetVariable adds a variable that can be referenced by the APL query. This option can be called multiple times to add multiple variables. If a variable with the same name already exists, it will be overwritten. Defining variables in APL using the "let" keyword takes precedence over variables provided via the query options.

func SetVariables

func SetVariables(variables map[string]any) Option

SetVariables sets the variables that can be referenced by the APL query. It will overwrite any existing variables. Defining variables in APL using the "let" keyword takes precedence over variables provided via the query options.

type Options

type Options struct {
	// StartTime for the interval to query.
	StartTime time.Time `json:"startTime,omitempty"`
	// EndTime of the interval to query.
	EndTime time.Time `json:"endTime,omitempty"`
	// Cursor to use for pagination. When used, don't specify new start and end
	// times but rather use the start and end times of the query that returned
	// the cursor that will be used.
	Cursor string `json:"cursor,omitempty"`
	// IncludeCursor specifies whether the event that matches the cursor should
	// be included in the result.
	IncludeCursor bool `json:"includeCursor,omitempty"`
	// Variables is an optional set of additional variables can be referenced by
	// the APL query. Defining variables in APL using the "let" keyword takes
	// precedence over variables provided via the query options.
	Variables map[string]any `json:"variables,omitempty"`
}

Options specifies the optional parameters for a query.

type Order

type Order struct {
	// Field is the name of the field to order by.
	Field string `json:"field"`
	// Desc is true if the order is descending. Otherwise the order is
	// ascending.
	Desc bool `json:"desc"`
}

Order of a Field in a Table.

type Range

type Range struct {
	// Field specifies the field name on which the query range was restricted.
	// Usually "_time":
	Field string
	// Start is the starting time the query is limited by. Usually the start of
	// the time window. Queries are restricted to the interval [start,end).
	Start time.Time
	// End is the ending time the query is limited by. Usually the end of the
	// time window. Queries are restricted to the interval [start,end).
	End time.Time
}

Range specifies the window a query was restricted to.

type Result

type Result struct {
	// Tables in the query result.
	Tables []Table `json:"tables"`
	// Status of the query result.
	Status Status `json:"status"`
	// TraceID is the ID of the trace that was generated by the server for this
	// results query request.
	TraceID string `json:"-"`
}

Result is the result of an APL query.

type Row

type Row []any

Row represents a single row of a tabular query Result.

func (Row) Values

func (r Row) Values() iter.Seq[any]

Values returns an iterator over the values of the row.

type Source

type Source struct {
	// Name of the source.
	Name string `json:"name"`
}

Source that was consulted in order to create a Table.

type Status

type Status struct {
	// MinCursor is the id of the oldest row, as seen server side. May be lower
	// than what the results include if the server scanned more data than
	// included in the results. Can be used to efficiently resume time-sorted
	// non-aggregating queries (i.e. filtering only).
	MinCursor string `json:"minCursor"`
	// MaxCursor is the id of the newest row, as seen server side. May be higher
	// than what the results include if the server scanned more data than
	// included in the results. Can be used to efficiently resume time-sorted
	// non-aggregating queries (i.e. filtering only).
	MaxCursor string `json:"maxCursor"`
	// ElapsedTime is the duration it took the query to execute.
	ElapsedTime time.Duration `json:"elapsedTime"`
	// RowsExamined is the amount of rows that have been examined by the query.
	RowsExamined uint64 `json:"rowsExamined"`
	// RowsMatched is the amount of rows that matched the query.
	RowsMatched uint64 `json:"rowsMatched"`
}

Status of an APL query Result.

func (*Status) UnmarshalJSON

func (s *Status) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler. It is in place to unmarshal the elapsed time into a proper time.Duration value because the server returns it in microseconds.

type Table

type Table struct {
	// Name of the table. Default name for unnamed results is "0", "1", "2", ...
	// etc.
	Name string `json:"name"`
	// Sources are the datasets that were consulted in order to create the
	// table.
	Sources []Source `json:"sources"`
	// Fields in the table matching the order of the [Columns] (e.g. the
	// [Column] at index 0 has the values for the [Field] at index 0).
	Fields []Field `json:"fields"`
	// Order of the fields in the table.
	Order []Order `json:"order"`
	// Groups are the groups of the table.
	Groups []Group `json:"groups"`
	// Range specifies the window the query was restricted to. Nil if the query
	// was not restricted to a time window.
	Range *Range `json:"range"`
	// Buckets defines if the query is bucketed (usually on the "_time" field).
	// Nil if the query returns a non-bucketed result.
	Buckets *Buckets `json:"buckets"`
	// Columns in the table matching the order of the [Fields] (e.g. the
	// [Column] at index 0 has the values for the [Field] at index 0). In case
	// of sub-groups, rows will repeat the group value.
	Columns []Column `json:"columns"`
}

Table in the Result of an APL query.

func (Table) Rows

func (t Table) Rows() iter.Seq[Row]

Rows returns an iterator over the rows build from the columns the table.