test

package
v0.0.20 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 29, 2024 License: MIT Imports: 20 Imported by: 0

README

Package testing/test

The goal of this package is to provide a small framework to isolate the test execution and safely check whether a test succeeds or fails as expected. In combination with the mock package it ensures that a test finishes reliably and reports its failure even if a system under test is spawning go-routines.

Example usage

Use the following example to intercept and validate a panic using the isolated test environment.

func TestUnit(t *testing.T) {
    test.Run(test.Success, func(t test.Test){
        // Given
        mocks := mock.NewMocks(t).Expect(
            test.Panic("fail"),
        )

        // When
        panic("fail")
    })(t)

    // Then
    ...
}

Isolated parameterized parallel test runner

The test framework supports to run isolated, parameterized, parallel tests using a lean test runner. The runner can be instantiated with a single test parameter set (New), a slice of test parameter sets (Slice), or a map of test case name to test parameter sets (Map - preferred pattern). The test is started by Run that accepts a simple test function as input, using a test.Test interface, that is compatible with most tools, e.g. gomock.

func TestUnit(t *testing.T) {
    test.New|Slice|Map(t, testParams).
        Run(func(t test.Test, param UnitParams){
            // Given

            // When

            // Then
        })
}

This creates and starts a lean test wrapper using a common interface, that isolates test execution and intercepts all failures (including panics), to either forward or suppress them. The result is controlled by providing a test parameter of type test.Expect (name expect) that supports Failure (false) and Success (true - default).

Similar a test case name can be provided using type test.Name (name name - default value unknown-%d) or as key using a test case name to parameter set mapping.

Note: See Parallel tests requirements for more information on requirements in parallel parameterized tests.

Isolated in-test environment setup

It is also possible to isolate only a single test step by setting up a small test function that is run in isolation.

func TestUnit(t *testing.T) {
    test.Map(t, testParams).
        Run(func(t test.Test, param UnitParams){
            // Given

            // When
            test.InRun(test.Failure, func(t test.Test) {
                ...
            })(t)

            // Then
        })
}

Manual isolated test environment setup

If the above pattern is not sufficient, you can create your own customized parameterized, parallel, isolated test wrapper using the basic abstraction test.Run(test.Success|Failure, func (t test.Test) {}):

func TestUnit(t *testing.T) {
    t.Parallel()

    for name, param := range testParams {
        name, param := name, param
        t.Run(name, test.Run(param.expect, func(t test.Test) {
            t.Parallel()

            // Given

            // When

            // Then
        }))
    }
}

Or the interface of the underlying test.Tester:

func TestUnit(t *testing.T) {
    t.Parallel()

    test.NewTester(t, test.Success).Run(func(t test.Test){
        // Given

        // When

        // Then
    })(t)
}

But this should usually be unnecessary.

Isolated failure/panic validation

Besides just capturing the failure in the isolated test environment, it is also very simple possible to validate the failures/panics using the self installing validator that is tightly integrated with the mock framework.

func TestUnit(t *testing.T) {
    test.Run(func(t test.Test){
        // Given
        mock.NewMocks(t).Expect(mock.Setup(
            test.Errorf("fail"),
            test.Fatalf("fail"),
            test.FailNow(),
            test.Panic("fail"),
        ))

        // When
        t.Errorf("fail")
        ...
        // And one of the following
        t.Fatalf("fail")
        t.FailNow()
        panic("fail")

        // Then
    })(t)
}

Note: To enable panic testing, the isolated test environment is recovering from all panics by default and converting it in a fatal error message. This is often most usable and sufficient to fix the issue. If you need to discover the source of the panic, you need to spawn a new unrecovered go-routine.

Hint: gomock uses very complicated reporting patterns that are hard to recreate. Do not try it.

Out-of-the-box test pattners

Currently, the package supports the following out-of-the-box test pattern for testing of main-methods of commands.

testMainParams := map[string]test.MainParams{
    "no mocks": {
        Args:     []string{"mock"},
        Env:      []string{},
        ExitCode: 0,
    },
}

func TestMain(t *testing.T) {
    test.Map(t, testMainParams).Run(test.TestMain(main))
}

The pattern executes a the main-method in a separate process that allows to setup the command line arguments (Args) as well as to modify the environment variables (Env) and to capture and compare the exit code.

Note: the general approach can be used to test any code calling os.Exit, however, it is focused on testing methods without arguments parsing command line arguments, i.e. in particular func main() { ... }.

Documentation

Overview

Package test contains the main collection of functions and types for setting up the basic isolated test environment. It is part of the public interface and starting to get stable, however, we are still experimenting to optimize the interface and the user experience.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// Error type for unknown parameter types.
	ErrUnkownParameterType = errors.New("unknown parameter type")
)

Functions

func ConsumedCall 

func ConsumedCall[T any](
	creator func(*gomock.Controller) *T,
	method, caller, ecaller string, args ...any,
) func(Test, *mock.Mocks) mock.SetupFunc

func EqCall 

func EqCall(x any) gomock.Matcher

EqCall creates a new call matcher that allows to match calls by translating them to the string containing the core information instead of using the standard matcher using reflect.DeepEquals that fails for the contained actions.

func EqError 

func EqError(x any) gomock.Matcher

EqError creates a new error matcher that allows to match either the error or alternatively the string describing the error.

func Errorf 

func Errorf(format string, args ...any) mock.SetupFunc

Errorf creates a validation method call setup for `Errorf`.

func FailNow 

func FailNow() mock.SetupFunc

FailNow creates a validation method call setup for `FailNow`.

func Fatalf 

func Fatalf(format string, args ...any) mock.SetupFunc

Fatalf creates a validation method call setup for `Fatalf`.

func InRun 

func InRun(expect Expect, test func(Test)) func(Test)

InRun creates an isolated test environment for the given test function with given expectation. When executed via `t.Run()` it checks whether the result is matching the expectation.

func MissingCalls 

func MissingCalls(
	setups ...mock.SetupFunc,
) func(Test, *mock.Mocks) mock.SetupFunc

MissingCalls creates an expectation for all missing calls.

func NewErrUnknownParameterType 

func NewErrUnknownParameterType(value any) error

NewErrUnknownParameterType creates a new unknown parameter type error.

func Panic 

func Panic(arg any) mock.SetupFunc

Panic creates a validation method call setup for a panic. It allows to match the panic object, which usually is an error and alternatively the error string representing the error, since runtime errors may be irreproducible.

func Run 

func Run(expect Expect, test func(Test)) func(*testing.T)

Run creates an isolated (by default) parallel test environment running the given test function with given expectation. When executed via `t.Run()` it checks whether the result is matching the expectation.

func RunSeq 

func RunSeq(expect Expect, test func(Test)) func(*testing.T)

RunSeq creates an isolated, test environment for the given test function with given expectation. When executed via `t.Run()` it checks whether the result is matching the expectation.

func TestMain added in v0.0.8 

func TestMain(main func()) func(t Test, param MainParams)

TestMain creates a test function that runs the given `main`-method in a separate test process to allow capturing the exit code and checking it against the expectation. This can be applied as follows: 

 testMainParams := map[string]test.MainParams{
	 "no mocks": {
		 Args:     []string{"mock"},
		 Env:      []string{},
		 ExitCode: 0,
	 },
 }

 func TestMain(t *testing.T) {
	 test.Map(t, testMainParams).Run(test.TestMain(main))
 }

The test method is spawning a new test using the `TEST` environment variable to select the expected parameter set containing the command line arguments (`Args`) to execute in the spawned process. The test instance is also called setting the given additional environment variables (`Env`) to allow modification of the test environment.

func UnexpectedCall 

func UnexpectedCall[T any](
	creator func(*gomock.Controller) *T,
	method, caller string, args ...any,
) func(Test, *mock.Mocks) mock.SetupFunc

UnexpectedCall creates expectation for unexpected calls. We only support one unexpected call since the test execution stops in this case.

Types

type Builder added in v0.0.20 

type Builder[T any] interface {
	// Set sets the value of the field with the given name. If the name is empty,
	// and of the same type the stored target instance is replaced by the given
	// value.
	Set(name string, value any) Builder[T]
	// Get returns the value of the field with the given name. If the name is
	// empty, the stored target instance is returned.
	Get(name string) any
	// Build returns the created/modified target instance of the builder.
	Build() T
}

Builder is a generic interface that allows you to access and modify unexported fields of a (pointer) struct by field name.

func NewAccessor added in v0.0.15 

func NewAccessor[T any](target T) Builder[T]

NewAccessor creates a generic builder/accessor for a given target struct. The builder allows you to access and modify unexported fields of the struct by field name.

If the target is a pointer to a struct (template), the pointer is stored and the instance is modified directly. If the target is a struct, it is ignored and a new pointer struct is created for modification, since a struct cannot be modified directly by reflection.

func NewBuilder added in v0.0.20 

func NewBuilder[T any]() Builder[T]

NewBuilder creates a generic builder for a target struct type. The builder allows you to access and modify unexported fields of the struct by field name.

type Cleanuper 

type Cleanuper interface {
	Cleanup(cleanup func())
}

Cleanuper defines an interface to add a custom mehtod that is called after the test execution to cleanup the test environment.

type Expect 

type Expect bool

Expect the expectation whether a test will succeed or fail. 

const (
	// Success used to express that a test is supposed to succeed.
	Success Expect = true
	// Failure used to express that a test is supposed to fail.
	Failure Expect = false

	// Flag to run test by default sequential instead of parallel.
	Parallel = true
)

Constants to express test expectations.

type MainParams added in v0.0.8 

type MainParams struct {
	Args     []string
	Env      []string
	ExitCode int
}

MainParams provides the test parameters for testing a `main`-method.

type Name 

type Name string

Name represents a test case name.

type Recorder 

type Recorder struct {
	// contains filtered or unexported fields
}

Recorder a test failure validator recorder.

func (*Recorder) Errorf 

func (r *Recorder) Errorf(format string, args ...any) *gomock.Call

Errorf indicate an expected method call to `Errorf`.

func (*Recorder) FailNow 

func (r *Recorder) FailNow() *gomock.Call

FailNow indicate an expected method call to `FailNow`.

func (*Recorder) Fatalf 

func (r *Recorder) Fatalf(format string, args ...any) *gomock.Call

Fatalf indicate an expected method call to `Fatalf`.

func (*Recorder) Panic 

func (r *Recorder) Panic(arg any) *gomock.Call

Panic indicate an expected method call from panic.

type Reporter 

type Reporter interface {
	// Panic reports a panic.
	Panic(arg any)
	// Errorf reports a failure messages when a test is supposed to continue.
	Errorf(format string, args ...any)
	// Fatalf reports a fatal failure message that immediate aborts of the test
	// execution.
	Fatalf(format string, args ...any)
	// FailNow reports fatal failure notifications without log output that
	// aborts test execution immediately.
	FailNow()
}

Reporter is a minimal interface for abstracting test report methods that are needed to setup an isolated test environment for GoMock and Testify.

type Runner 

type Runner[P any] interface {
	// Cleanup register a function to be called for cleanup after all tests
	// have been finished.
	Cleanup(call func())
	// Run runs the test parameter sets (by default) parallel.
	Run(call func(t Test, param P)) Runner[P]
	// RunSeq runs the test parameter sets in a sequence.
	RunSeq(call func(t Test, param P)) Runner[P]
}

Runner is a generic test runner interface.

func Map 

func Map[P any](t *testing.T, params ...map[string]P) Runner[P]

Map creates a new parallel test runner with given test parameter sets provided as a test case name to parameter sets mapping.

func New 

func New[P any](t *testing.T, params any) Runner[P]

New creates a new parallel test runner with given parameter sets, i.e. a single test parameter set, a slice of test parameter sets, or a test case name to test parameter set map. If necessary, the test runner is looking into the parameter set for a suitable test case name.

func Slice 

func Slice[P any](t *testing.T, params []P) Runner[P]

Slice creates a new parallel test runner with given test parameter sets provided as a slice. The test runner is looking into the parameter set to find a suitable test case name.

type Test 

type Test interface {
	// Name provides the test name.
	Name() string
	// Helper declares a test helper function.
	Helper()
	// Parallel declares that the test is to be run in parallel with (and only
	// with) other parallel tests.
	Parallel()
	// TempDir creates a new temporary directory for the test.
	TempDir() string
	// Errorf handles a failure messages when a test is supposed to continue.
	Errorf(format string, args ...any)
	// Fatalf handles a fatal failure message that immediate aborts of the test
	// execution.
	Fatalf(format string, args ...any)
	// FailNow handles fatal failure notifications without log output that
	// aborts test execution immediately.
	FailNow()
	// Setenv sets an environment variable for the test.
	Setenv(key, value string)
}

Test is a minimal interface for abstracting test methods that are needed to setup an isolated test environment for GoMock and Testify.

type Tester 

type Tester struct {
	sync.Synchronizer
	// contains filtered or unexported fields
}

Tester is a test isolation environment based on the `Test` abstraction. It can be used as a drop in replacement for `testing.T` in various libraries to check for expected test failures.

func NewTester 

func NewTester(t Test, expect Expect) *Tester

NewTester creates a new minimal test context based on the given `go-test` context.

func (*Tester) Cleanup 

func (t *Tester) Cleanup(cleanup func())

Cleanup is a function called to setup test cleanup after execution. This method is allowing `gomock` to register its `finish` method that reports the missing mock calls.

func (*Tester) Errorf 

func (t *Tester) Errorf(format string, args ...any)

Errorf handles failure messages where the test is supposed to continue. On an expected success, the failure is also delegated to the parent test context. Else it delegates the request to the test reporter if available.

func (*Tester) FailNow 

func (t *Tester) FailNow()

FailNow handles fatal failure notifications without log output that aborts test execution immediately. On an expected success, it the failure handling is also delegated to the parent test context. Else it delegates the request to the test reporter if available.

func (*Tester) Fatalf 

func (t *Tester) Fatalf(format string, args ...any)

Fatalf handles a fatal failure message that immediate aborts of the test execution. On an expected success, the failure handling is also delegated to the parent test context. Else it delegates the request to the test reporter if available.

func (*Tester) Helper 

func (t *Tester) Helper()

Helper delegates request to the parent test context.

func (*Tester) Name 

func (t *Tester) Name() string

Name delegates the request to the parent test context.

func (*Tester) Panic 

func (t *Tester) Panic(arg any)

Panic handles failure notifications of panics that also abort the test execution immediately.

func (*Tester) Parallel 

func (t *Tester) Parallel()

Parallel robustly delegates request to the parent context. It can be called multiple times, since it is swallowing the panic that is raised when calling `t.Parallel()` multiple times.

func (*Tester) Reporter 

func (t *Tester) Reporter(reporter Reporter)

Reporter sets up a test failure reporter. This can be used to validate the reported failures in a test environment.

func (*Tester) Run 

func (t *Tester) Run(test func(Test), parallel bool) Test

Run executes the test function in a safe detached environment and check the failure state after the test function has finished. If the test result is not according to expectation, a failure is created in the parent test context.

func (*Tester) Setenv added in v0.0.15 

func (t *Tester) Setenv(key, value string)

Setenv delegates request to the parent context, if it is of type `*testing.T`. Else it is swallowing the request silently.

func (*Tester) TempDir 

func (t *Tester) TempDir() string

TempDir delegates the request to the parent test context.

func (*Tester) WaitGroup 

func (t *Tester) WaitGroup(wg sync.WaitGroup)

WaitGroup adds wait group to unlock in case of a failure.

type Validator 

type Validator struct {
	// contains filtered or unexported fields
}

Validator a test failure validator based on the test reporter interface.

func NewValidator 

func NewValidator(ctrl *gomock.Controller) *Validator

NewValidator creates a new test validator for validating error messages and panics created during test execution.

func (*Validator) EXPECT 

func (v *Validator) EXPECT() *Recorder

EXPECT implements the usual `gomock.EXPECT` call to request the recorder.

func (*Validator) Errorf 

func (v *Validator) Errorf(format string, args ...any)

Errorf receive expected method call to `Errorf`.

func (*Validator) FailNow 

func (v *Validator) FailNow()

FailNow receive expected method call to `FailNow`.

func (*Validator) Fatalf 

func (v *Validator) Fatalf(format string, args ...any)

Fatalf receive expected method call to `Fatalf`.

func (*Validator) Panic 

func (v *Validator) Panic(arg any)

Panic receive expected method call indicating a panic.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.