ztest

Documentation

Overview

Package ztest runs formulaic tests ("ztests") that can be (1) run in-process with the compiled-ini zq code base, (2) run as a sub-process using the zq executable build artifact, or (3) run as a bash script running a sequence of arbitrary shell commands invoking any of the build artifacts. The first two cases comprise the "Zed test style" and the last case comprises the "script test style". Case (1) is easier to debug by simply running "go test" compared replicating the test using "go run". Script-style tests don't have this convenience.

In the Zed style, ztest runs a Zed program on an input and checks for an expected output.

A Zed-style test is defined in a YAML file.

zed: count()

input: |
  #0:record[i:int64]
  0:[1;]
  0:[2;]

output: |
  #0:record[count:uint64]
  0:[2;]

Input format is detected automatically and can be anything recognized by "zq -i auto" (including optional gzip compression). Output format defaults to tzng but can be set to anything accepted by "zq -f".

zed: count()

input: |
  #0:record[i:int64]
  0:[1;]
  0:[2;]

output-flags: -f table

output: |
  count
  2

Alternatively, tests can be configured to run as shell scripts. In this style of test, arbitrary bash scripts can run chaining together any of the tools in cmd/ in addition to zq. Scripts are executed by "bash -e -o pipefail", and a nonzero shell exit code causes a test failure, so any failed command generally results in a test failure. Here, the yaml sets up a collection of input files and stdin, the script runs, and the test driver compares expected output files, stdout, and stderr with data in the yaml spec. In this case, instead of specifying, "zed", "input", "output", you specify the yaml arrays "inputs" and "outputs" --- where each array element defines a file, stdin, stdout, or stderr --- and a "script" that specifies a multi-line yaml string defining the script, e.g.,

inputs:

• name: in1.tzng data: | #0:record[i:int64] 0:[1;]
• name: stdin data: | #0:record[i:int64] 0:[2;]

script: |

zq -o out.tzng in1.tzng -
zq -o count.tzng "count()" out.tzng

outputs:

• name: out.tzng data: | #0:record[i:int64] 0:[1;] 0:[2;]
• name: count.tzng data: | #0:record[count:uint64] 0:[2;]

Each input and output has a name. For inputs, a file (source) or inline data (data) may be specified. If no data is specified, then a file of the same name as the name field is looked for in the same directory as the yaml file. The source spec is a file path relative to the directory of the yaml file. For outputs, expected output is defined in the same fashion as the inputs though you can also specify a "regexp" string instead of expected data. If an output is named "stdout" or "stderr" then the actual output is taken from the stdout or stderr of the the shell script.

Ztest YAML files for a package should reside in a subdirectory named testdata/ztest.

pkg/
  pkg.go
  pkg_test.go
  testdata/
    ztest/
      test-1.yaml
      test-2.yaml
      ...

Name YAML files descriptively since each ztest runs as a subtest named for the file that defines it.

pkg_test.go should contain a Go test named TestZTest that calls Run.

func TestZTest(t *testing.T) { ztest.Run(t, "testdata/ztest") }

If the ZTEST_PATH environment variable is unset or empty and the test is not a script test, Run runs ztests in the current process and skips the script tests. Otherwise, Run runs each ztest in a separate process using the zq executable in the directories specified by ZTEST_PATH.

Tests of either style can be skipped by setting the skip field to a non-empty string. A message containing the string will be written to the test log.

Index

• func Run(t *testing.T, dirname string)
• func RunShell(dir, bindir, script string, stdin io.Reader, useenvs []string) (string, string, error)
• func ShellPath() string
• type Bundle
  • func Load(dirname string) ([]Bundle, error)
  • func (b *Bundle) RunScript(shellPath, tempDir string) error
• type File
• type ZTest
  • func FromYAMLFile(filename string) (*ZTest, error)
  • func (z *ZTest) Run(t *testing.T, path, filename string)
  • func (z *ZTest) RunInternal(path string) error
  • func (z *ZTest) RunScript(shellPath, testDir, tempDir string) error
  • func (z *ZTest) ShouldSkip(path string) string

Functions

func Run

func Run(t *testing.T, dirname string)

Run runs the ztests in the directory named dirname. For each file f.yaml in the directory, Run calls FromYAMLFile to load a ztest and then runs it in subtest named f. path is a command search path like the PATH environment variable.

func RunShell

func RunShell(dir, bindir, script string, stdin io.Reader, useenvs []string) (string, string, error)

func ShellPath

func ShellPath() string

Types

type Bundle

type Bundle struct {
	TestName string
	FileName string
	Test     *ZTest
	Error    error
}

func Load

func Load(dirname string) ([]Bundle, error)

func (*Bundle) RunScript

func (b *Bundle) RunScript(shellPath, tempDir string) error

type File

type File struct {
	// Name is the name of the file with respect to the directoy in which
	// the test script runs.  For inputs, if no data source is specified,
	// then name is also the name of a data file in the diectory containing
	// the yaml test file, which is copied to the test script directory.
	// Name can also be stdio (for inputs) or stdout or stderr (for outputs).
	Name string `yaml:"name"`
	// Data and Source represent the different ways file data can
	// be defined for this file.  Data is a string turned into the contents
	// of the file. Source is a string representing
	// the pathname of a file the repo that is read to comprise the data.
	Data   *string `yaml:"data,omitempty"`
	Source string  `yaml:"source,omitempty"`
	// Re is a regular expression describing the contents of the file,
	// which is only applicable to output files.
	Re string `yaml:"regexp,omitempty"`
}

type ZTest

type ZTest struct {
	Skip string `yaml:"skip,omitempty"`
	Tag  string `yaml:"tag,omitempty"`

	// For Zed-style tests.
	Zed         string `yaml:"zed,omitempty"`
	Input       string `yaml:"input,omitempty"`
	InputFlags  string `yaml:"input-flags,omitempty"`
	Output      string `yaml:"output,omitempty"`
	OutputFlags string `yaml:"output-flags,omitempty"`
	ErrorRE     string `yaml:"errorRE,omitempty"`
	Vector      bool   `yaml:"vector"`

	// For script-style tests.
	Script  string   `yaml:"script,omitempty"`
	Inputs  []File   `yaml:"inputs,omitempty"`
	Outputs []File   `yaml:"outputs,omitempty"`
	Env     []string `yaml:"env,omitempty"`
	// contains filtered or unexported fields
}

ZTest defines a ztest.

func FromYAMLFile

func FromYAMLFile(filename string) (*ZTest, error)

FromYAMLFile loads a ZTest from the YAML file named filename.

func (*ZTest) Run

func (z *ZTest) Run(t *testing.T, path, filename string)

func (*ZTest) RunInternal

func (z *ZTest) RunInternal(path string) error

func (*ZTest) RunScript

func (z *ZTest) RunScript(shellPath, testDir, tempDir string) error

func (*ZTest) ShouldSkip

func (z *ZTest) ShouldSkip(path string) string