testmatrix

README

TestMatrix

Test Matrix is a testing utility to run many test patterns against many test samples, providing a helpful visual output.

Given an input matrix, which can be generated using a CSV or a slice of slices:

The utility will run tests through all the permutations and provide visual output to help identify patterns in the failures:

The table is printed only with -v test flag or with at least one failing test.

Usage

The matrix of input can be a CSV file, a matrix of type [][]interface{} or a reader that responds to Read and ReadHeaders functions.

matrix := testmatrix.NewColoredTestMatrix(t *testing.T)

func (matrix testmatrix.TestMatrix) AssertSlice(data [][]interface{}, assert func(subjectText string, headerText string) bool)

func (matrix testmatrix.TestMatrix) AssertCsv(reader io.Reader, assert func(subjectText string, headerText string) bool)

func (matrix testmatrix.TestMatrix) Assert(r testmatrix.Reader, assert func(subjectText string, headerText string) bool)

//go:embed atoi.csv
var atoiMatrix string

func TestTestMatrixCsv(t *testing.T) {
  matrix := testmatrix.NewColoredTestMatrix(t)
  reader := strings.NewReader(atoiMatrix)

  matrix.AssertCsv(reader, func(input string, expectedOutput string) bool {
    actualOutput, err := strconv.Atoi(input)

    if expectedOutput == "error" {
      return err != nil
    }

    return strconv.Itoa(actualOutput) == expectedOutput
  })
}

The Assert* functions have 2 parameters:

An input test matrix (or a reader of such). The test matrix is composed of expected outputs in the first row and, from the second row and on, the sample in the first column and if the input function should return or not the expected output.

The second parameter is a function accepting an input value, the sample from the first column on the left, and an expected output value: the header at the top. The function should output if executing the subject function (Atoi in this case) on the input returns the expected output.

CSV input

The CSV should be formatted in the following way:

,expectedOutput1,expectedOutput2
input1,TRUE,FALSE
input2,FALSE,FALSE

Where TRUE and FALSE represent boolean values.

Slice input

[][]interface{}{
  {"", "10", "1", "error"},
  {"1", "FALSE", "TRUE", "FALSE"},
  {"10", "TRUE", "FALSE", "FALSE"},
  {"a", "FALSE", "FALSE", "TRUE"},
  {"0xa", "FALSE", "FALSE", "TRUE"},
  {"zz", "FALSE", "FALSE", "TRUE"},
}

Documentation

Overview

Package testmatrix helps running a matrix of tests

Index

• Variables
• func NewMatrixReader(testMatrix [][]interface{}) *sliceReader
• func StringToBool(text string) (bool, bool)
• type Headers
  • func (headers Headers) Row() []string
• type Reader
  • func NewCsvReader(r *csv.Reader) Reader
• type ResultsRow
  • func (results ResultsRow) Row(successText string, failureText string) []string
• type TestMatrix
  • func NewColoredTestMatrix(t *testing.T) TestMatrix
  • func NewTestMatrix(t *testing.T) TestMatrix
  • func (matrix TestMatrix) Assert(r Reader, assert func(subjectText string, headerText string) bool)
  • func (matrix TestMatrix) AssertCsv(reader io.Reader, assert func(subjectText string, headerText string) bool)
  • func (matrix TestMatrix) AssertSlice(data [][]interface{}, assert func(subjectText string, headerText string) bool)

Variables

var EOF error

var InvalidPositionErr error

var UnparsableErr error

Functions

func NewMatrixReader

func NewMatrixReader(testMatrix [][]interface{}) *sliceReader

func StringToBool

func StringToBool(text string) (bool, bool)

Types

type Headers

type Headers []string

func (Headers) Row

func (headers Headers) Row() []string

type Reader

type Reader interface {
	ReadHeaders() (Headers, error)
	Read() (ResultsRow, error)
}

func NewCsvReader

func NewCsvReader(r *csv.Reader) Reader

type ResultsRow

type ResultsRow struct {
	Subject  string
	Expected []bool
}

func (ResultsRow) Row

func (results ResultsRow) Row(successText string, failureText string) []string

type TestMatrix

type TestMatrix struct {
	FailureText string
	SuccessText string
	// contains filtered or unexported fields
}

func NewColoredTestMatrix

func NewColoredTestMatrix(t *testing.T) TestMatrix

func NewTestMatrix

func NewTestMatrix(t *testing.T) TestMatrix

func (TestMatrix) Assert

func (matrix TestMatrix) Assert(r Reader, assert func(subjectText string, headerText string) bool)

func (TestMatrix) AssertCsv

func (matrix TestMatrix) AssertCsv(reader io.Reader, assert func(subjectText string, headerText string) bool)

func (TestMatrix) AssertSlice

func (matrix TestMatrix) AssertSlice(data [][]interface{}, assert func(subjectText string, headerText string) bool)