querytest

README

Query Tests

This package simulates an end-to-end query test without requiring launching a full service. This is currently achieved by dynamically replacing instances of from()---which access a storage location---with instances of fromCSV() which is a data source that takes a single CSV text argument.

Getting Started

You do not need to use fromCSV in your query. You can write your query as you normally would, or copy it from an application as-is. What you will need to provide, along with the query file, are two csv files that contain the input and output data for the query. The test framework will identify the Flux query by the file extension .flux and then locate the required input/output files using the same root filename.

As an example, consider the test simple_max which is defined by the file simple_max.flux:

simple_max.flux

from(bucket:"test") |> range(start:-5m)  |> group(columns:["_measurement"]) |> max(useRowTime:true) |> map(fn: (r) => ({max:r._value}))

The test will read in this query, as well as the associated CSV files that are in the flux-spec compatible CSV format:

simple_max_in.csv

#datatype,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,tag,tag,double
,blkid,_start,_stop,_time,_measurement,_field,_value
,0,2018-04-17T00:00:00Z,2018-04-17T00:05:00Z,2018-04-17T00:00:00Z,m1,f1,42.0
,0,2018-04-17T00:00:00Z,2018-04-17T00:05:00Z,2018-04-17T00:00:01Z,m1,f1,43.0

simple_max_out.csv

#datatype,long,dateTime:RFC3339Nano,dateTime:RFC3339Nano,dateTime:RFC3339Nano,string,double
,blkid,_start,_stop,_time,_measurement,max
,0,2018-04-17T00:00:00Z,2018-04-17T00:05:00Z,2018-04-17T00:00:01Z,m1,43

Note that all of the test files are identified by the test name, simple_max. So given a , a new test must provide the following files:

• .flux: a Flux query file
• _in.csv: input file for the query
• _out.csv: output file for the query
• .influxql: if present, this query will be transpiled and tested against the input and output csv files, but only if there is a corresponding .flux query.

Notes about text formatting

Compliant to the HTTP spec for CSV data, the output produced by the fluxd process has some specific requirements:

• UTF-8 normalized
• line endings are \r\n

The go program query/querytest/prepcsvtests/prepcsvtests.go can be run to prepare tests. A valid test case must have .flux and .in.csv files. The prepcsvtests executable will iterate over all such test cases in the user-supplied directory, run the Flux query on the input and prompt the user to approve saving the result as .out.csv. go build ./querytest/prepcsvtests/prepcsvtests.go && ./prepcsvtests query/querytests/test_cases

Optionally, you can give a CASENAME to prep the output for a single case: go build ./querytest/prepcsvtests/prepcsvtests.go && ./prepcsvtests query/querytests/test_cases CASENAME

Documentation

Overview

Package querytest contains utilities for testing the query end-to-end.

Index

• func NewQueryTestHelper(t *testing.T, tc NewQueryTestCase)
• func OperationMarshalingTestHelper(t *testing.T, data []byte, expOp *flux.Operation)
• func ReplaceOpInSpec(q *flux.Spec, replaceFn ReplaceOpInSpecFunction)
• func RunAndCheckResult(t testing.TB, querier *Querier, c flux.Compiler, d flux.Dialect, want string)
• type NewQueryTestCase
• type Querier
  • func NewQuerier() *Querier
  • func (q *Querier) Query(ctx context.Context, w io.Writer, c flux.Compiler, d flux.Dialect) (int64, error)
• type ReplaceOpInSpecFunction
• type ReplaceSpecCompiler
  • func NewReplaceSpecCompiler(compiler flux.Compiler, fn ReplaceOpInSpecFunction) *ReplaceSpecCompiler
  • func (c *ReplaceSpecCompiler) Compile(ctx context.Context) (*flux.Spec, error)

Functions

func NewQueryTestHelper

func NewQueryTestHelper(t *testing.T, tc NewQueryTestCase)

func OperationMarshalingTestHelper

func OperationMarshalingTestHelper(t *testing.T, data []byte, expOp *flux.Operation)

func ReplaceOpInSpec

func ReplaceOpInSpec(q *flux.Spec, replaceFn ReplaceOpInSpecFunction)

func RunAndCheckResult

func RunAndCheckResult(t testing.TB, querier *Querier, c flux.Compiler, d flux.Dialect, want string)

Types

type NewQueryTestCase

type NewQueryTestCase struct {
	Name    string
	Raw     string
	Want    *flux.Spec
	WantErr bool
}

type Querier

type Querier struct {
	C *controltest.Controller
}

func NewQuerier

func NewQuerier() *Querier

func (*Querier) Query

func (q *Querier) Query(ctx context.Context, w io.Writer, c flux.Compiler, d flux.Dialect) (int64, error)

type ReplaceOpInSpecFunction

type ReplaceOpInSpecFunction func(op *flux.Operation) flux.OperationSpec

ReplaceOpInSpecFunction is a function that takes an Operation and returns a new OperationSpec for substitution, or nil, if nothing has to be changed.

type ReplaceSpecCompiler

type ReplaceSpecCompiler struct {
	flux.Compiler
	// contains filtered or unexported fields
}

func NewReplaceSpecCompiler

func NewReplaceSpecCompiler(compiler flux.Compiler, fn ReplaceOpInSpecFunction) *ReplaceSpecCompiler

func (*ReplaceSpecCompiler) Compile

func (c *ReplaceSpecCompiler) Compile(ctx context.Context) (*flux.Spec, error)