perm

README

Package testing/perm

Goal of this package is to provide a small framework to simplify tests based on permutations of a test parameter. This is helpful when a test input must be checked in all known orders with differing outcomes depending on the order.

This was mainly developed to validate the setup functions of the mock framework, but may be useful in other use cases.

Standard permutation pattern

To setup a permutation test you need to define two parts:

1. A TestMap defined map[string]func(t *test.TestingT) setting up the tests that are object to permutation and,
2. An ExpectMap defined as map[string]Expect setting up a set of distinct expectation for various permutation.

The permutation in the ExpectMap key is given by a list of names using - as separator.

Example: If a TestMap defines the names a and b, than the permutation keys in the ExpectMap are written as a-b and b-a.

In a test, the TestMap will usually be defined via a function expecting a mock handler to create a permutation of the mock call setup:

func SetupPermTestABCD(mocks *mock.Mocks) *perm.Test {
    iface := mock.Get(mocks, mock.NewMockIFace)
    return perm.NewTest(mocks,
        perm.TestMap{
            "a": func(t *test.TestingT) { iface.CallA("a") },
            "b": func(t *test.TestingT) { iface.CallA("b") },
            "c": func(t *test.TestingT) {
                assert.Equal(t, "d", iface.CallB("c"))
            },
            "d": func(t *test.TestingT) {
                assert.Equal(t, "e", iface.CallB("d"))
            },
        })
}

As you can see the functions contain the necessary test assertions beside the mock calls. The ExpectMap itself can be incomplete and contain any subset of the full permutation list with the expected results. E.g.

var testPermParams = perm.ExpectMap{
    "a-b-c-d": test.ExpectSuccess,
    "a-b-d-c": test.ExpectSuccess,
    "a-d-b-c": test.ExpectSuccess,
    "b-a-c-d": test.ExpectSuccess,
    "b-a-d-c": test.ExpectSuccess,
    "b-d-a-c": test.ExpectSuccess,
    "d-a-b-c": test.ExpectSuccess,
    "d-b-a-c": test.ExpectSuccess,
}

The nice part of the permutation framework is, that it now allows to complete the permutation by defining a default value for all remaining permutations by calling testPermParams.Remain(test.ExpectSuccess) fluently. This can be than used in a parameterized test.

func TestDetach(t *testing.T) {
    for message, expect := range testPermParams.Remain(test.ExpectFailure) {
        t.Run(message, test.Run(expect, func(t *test.TestingT) {
            require.NotEmpty(t, message)

            // Given
            perm := strings.Split(message, "-")
            mockSetup := mock.Chain(
                mock.Detach(mock.None, CallA("a")),
                mock.Detach(mock.Head, CallA("b")),
                mock.Detach(mock.Tail, CallB("c", "d")),
                mock.Detach(mock.Both, CallB("d", "e")),
            )
            mock := MockSetup(t, mockSetup)

            // When
            test := SetupPermTestABCD(mock)

            // Then
            test.Test(t, perm, expect)
        }))
    }
}

Documentation

Overview

Package perm provides a small framework to simplify permutation tests, i.e. a consistent test set where conditions can be checked in all known orders with different outcome. This is very handy in combination with test(test) to validated the mock(mock) framework, but may be useful in other cases too.

Index

• type ExpectMap
  • func (perms ExpectMap) Remain(expect test.Expect) ExpectMap
• type Test
  • func NewTest(mocks *mock.Mocks, tests TestMap) *Test
  • func (p *Test) Test(t test.Test, perm []string, expect test.Expect)
  • func (p *Test) TestPerm(t test.Test, perm []string)
• type TestMap

Types

type ExpectMap

type ExpectMap map[string]test.Expect

ExpectMap defines a map of permutation tests that are expected to either fail or succeed to succeed depending on expressed expectation.

func (ExpectMap) Remain

func (perms ExpectMap) Remain(expect test.Expect) ExpectMap

Remain calculate and add the missing permutations and add it with expected result to the given permutation map.

type Test

type Test struct {
	// contains filtered or unexported fields
}

Test permutation test.

func NewTest

func NewTest(mocks *mock.Mocks, tests TestMap) *Test

NewTest creates a new permutation test with given mock and given permutation test map.

func (*Test) Test

func (p *Test) Test(t test.Test, perm []string, expect test.Expect)

Test executes a permutation test with given permutation and expected result.

func (*Test) TestPerm

func (p *Test) TestPerm(t test.Test, perm []string)

TestPerm tests a single permutation given by the string slice.

type TestMap

type TestMap map[string]func(t test.Test)

TestMap defines a map of test functions that is subject of the actual permutation.