testcase

package module
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 3, 2020 License: Apache-2.0 Imports: 9 Imported by: 0

README

Table of Contents

Mentioned in Awesome Go GoDoc Build Status Go Report Card codecov

testcase

The testcase package provides tooling to apply BDD testing conventions.

Documentation

The Official Package documentation managed in godoc.

This README.md serves as a high level intro into the package, and a case study why the package was made. For package API, examples and usage details about the testcase package, please see the package godoc.

additional documentations:

Example

The examples managed in godoc, please read the documentation example section for more.

A Basic example:

package mypkg_test

import (
	"testing"

	"github.com/you/mypkg"

	"github.com/stretchr/testify/require"

	"github.com/adamluzsi/testcase"
)

func TestMyTypeIsLower(t *testing.T) {
	s := testcase.NewSpec(t)
	s.NoSideEffect()

	myType := func(t *testcase.T) *mypkg.MyType {
		return &mypkg.MyType{}
	}

	var subject = func(t *testcase.T) bool {
		return myType(t).IsLower(t.I(`input`).(string))
	}

	s.When(`input has upcase letter`, func(s *testcase.Spec) {
		s.LetValue(`input`, `UPPER`)

		s.Then(`it will be false`, func(t *testcase.T) {
			require.False(t, subject(t))
		})
	})

	s.When(`input is all lowercase letter`, func(s *testcase.Spec) {
		s.LetValue(`input`, `lower`)

		s.Then(`it will be true`, func(t *testcase.T) {
			require.True(t, subject(t))
		})
	})
}

Modules

  • httpspec
    • this spec module helps you create HTTP API Specs.

Summary

DRY

testcase provides a way to express common Arrange, Act sections for the Asserts with DRY principle in mind.

  • First you can define your Act section with a method under test as the subject of your test specification
    • The Act section invokes the method under test with the arranged parameters.
  • Then you can build the context of the Act by Arranging the inputs later with humanly explained reasons
    • The Arrange section initializes objects and sets the value of the data that is passed to the method under test.
  • And lastly you can define the test expected outcome in an Assert section.
    • The Assert section verifies that the action of the method under test behaves as expected.

Then adding an additional test edge case to the testing suite becomes easier, as it will have a concrete place where it must be placed.

And if during the creation of the specification, an edge case turns out to be YAGNI, it can be noted, so visually it will be easier to see what edge case is not specified for the given subject.

The value it gives is that to build test for a certain edge case, the required mental model size to express the context becomes smaller, as you only have to focus on one Arrange at a time, until you fully build the bigger picture.

It also implicitly visualize the required mental model of your production code by the nesting. You can read more on that in the nesting section.

Modularization

On top of the DRY convention, any time you need to Arrange a common scenario about your projects domain event, you can modularize these setup blocks in a helper functions.

This helps the readability of the test, while keeping the need of mocks to the minimum as possible for a given test. As a side effect, integration tests can become low hanging fruit for the project.

e.g.:

package mypkg_test

import (
	"testing"

	"my/project/mypkg"


	"github.com/adamluzsi/testcase"

	. "my/project/testing/pkg"
)

func TestMyTypeMyFunc(t *testing.T) {
	s := testcase.NewSpec(t)

	// high level Arrange helpers from my/project/testing/pkg
	SetupSpec(s)
	GivenWeHaveUser(s, `myuser`)
	// .. other givens

	myType := func() *mypkg.MyType { return &mypkg.MyType{} }

	s.Describe(`#MyFunc`, func(s *testcase.Spec) {
		var subject = func(t *testcase.T) { myType().MyFunc(t.I(`myuser`).(*mypkg.User)) } // Act

		s.Then(`edge case description`, func(t *testcase.T) {
			// Assert
			subject(t)
		})
	})
}

Stability

  • The package considered stable.
  • The package use rolling release conventions.
  • No breaking change is planned to the package exported API.
  • The package used for production development.
  • The package API is only extended if the practical use case proves its necessity.

Case Study Of The Package Origin

The Problem

The software architecture used in the projects where I work has decoupled responsibility. Therefore the layer that deals with managing business entities in an external resource, a layer that composes these interactors/consumer that manage business entities to form business rules and business logic, a layer that supply external resource interactions and a layer that interacts with an external interface such as HTTP later.

Because of this segregation, without a proper BDD testing approach, there would be a lot of mock tests that would reduce the maintainability of the project. To conquer this problem, the tests try to use real components instead of a mock, and each higher-level layer (by architecture meaning) that interacts with a layer below, they interact with an interactor that manages a resource to set up a test environment. This approach introduced a lot of boilerplate, as sharing such context setup added a lot of repetition in tests that use the same components.

To make this less abstract, imagine the following directory layout:

.
├── extintf
│   ├── caches               // cache external resource suppliers
│   ├── httpintf             // HTTP external interface implementation
│   ├── queues               // queue external resource suppliers
│   └── storages             // storage external resource supplier
├── services
│   ├── domain-service-1
│   ├── domain-service-2
│   └── domain-service-3
└── usecases
    ├── SignIn.go            // example use-case for sign in
    ├── SignUp.go            // example use-case for sign up
    └── SomeOtherUseCase.go  // example use case

And under the services directory, each directory represents certain domain rules and entities. So when tests needed to be made on a higher level than the domain rules, like the use cases, it was crucial to avoid any implementation level detail check in the assertions to test the high-level business rules.

So for example, if a domain service particular entity use storage to persist and retrieve domain entities, it was essential to avoid checking domain entities in the storage used by the domain rules, when we create tests for a high level in the use-cases interactor.

On lower level for creating the domain rule tests, it was a concern to create tests, that reflect the expected system behavior instead of the implementation of a specific rule, through mocking dependencies.

The Requirements

The following requirements were specified for the project to address the issues mentioned above.

  • DRY specifications for similar edge cases that enhance the maintainability aspect of the tests/specs.
  • shareable helper functions that can improve the readability of high-level tests.
  • make the feedback loop as fast as possible to allow small quick changes in the codebase
    • for example, when a test has no side effect to the program, it can be run in parallel
    • so if you don't use global variables (os Environment variables) in your currently tested code's scope, then all your tests should run on a separate core.
  • define test subjects with inputs that is not yet provided
    • This allows creating specification where each input needs to be specified explicitly, and not defined inputs can be easily seen while making the specification.
    • This allows us to setup steps like "something is done with something else by n", and then later define this value at a test context.
  • running a specification should generate a humanly readable specification that helps to build the mental model of a given code in the subject.
  • low maintainability cost on the framework side
    • stable API
    • no breaking change
  • specific edge cases can be executed alone easily
    • can be used easily with dlv
    • can be used with go test command out of the box
      • includes -run option to specify test case(s)
  • can visualize code complexity by the testing specification
  • don't build and use testing package level globals
  • the ability to use libraries like stretchr/testify
    • assertions are not necessarily part of the library until it is proven what is needed.
The Starting Point

So as a starting point, various BDD testing framework projects were checked out to see if there would be a good match. There were various frameworks already, and each covered some of the critical requirements to solve the issues at hand, but none answered them all.

Then battle-tested testing frameworks from other languages were checked for inspiration basis. The high-level approach of the rspec framework turned out to cover most of the requirements, so all that was needed is to extend these core functionality to solve the remaining requirements.

The Initial Implementation

Initially, two implementations were made.

The Spec approach meant to push the test writer to define each test variable and each edge case context with documentation. This was achieved by providing a structure that helps applying BDD conventions, through a minimal set of helper functions.

The Other approach was the Steps which was basically to build a list of function that represented testing steps, and meant to be used with nested tests through the usage of testing.T#Run function. This approach allowed to use testing package nesting strategy mainly, while composing testing hooks to express a certain testing.T#Run scope test runtime context.

A/B Testing For The Package Vision

These two approaches then was A/B tested in different projects. The A/B testing was ongoing for slightly more than 10 months.

In the end of the A/B testing, the following observations were made:

  • Spec based project tests were more maintainable from the Steps based approach.
  • Steps required less initial effort to learn it.
  • Steps often encountered cases where variable setup was not possible in isolation.
  • projects with Spec had generally faster feedback loops.
  • tests with Spec had better readability when values with teardown were needed.
  • Spec had advantage in solving testing feature needs commonly without need to change project specifications.
  • Using shared specification helpers with Steps was messy and hard to refactor.
  • Spec allowed the same or really similar conventions that community built for rspec, jasmin, rtl and similar testing frameworks.

At the end of the A/B testing, the Spec approach turned out to be more preferred in the projects in subject. You can see a usage of Spec approach in an open source project that is called toggler. The specification were initially made there with the MVP set of the Spec approach, therefore not all the latest idiom were applied.

The Current Implementation

The Spec approach was kept and will be maintained in the testcase package.

The internals of the Spec is based on the testing.T#Run function, and, as such, the essential parts maintained by the core testing package, since the Spec package only wraps it.

Tests coverage made to ensure the behavior of the Spec approach implementation. The coverage is more about the behavior, than the code execution flow, and while some implementation may overlap by tests, the behavior is defined for each edge case explicitly.

Reference Project

Documentation

Overview

Package testcase implements two approaches to help you to do nested BDD style testing in golang.

Spec Variables

in your spec, you can use the `*testcase.variables` object, for fetching values for your objects. Using them is gives you the ability to create value for them, only when you are in the right testing scope that responsible for providing an example for the expected value.

In test case scopes you will receive a structure ptr called `*testcase.variables` which will represent values that you configured for your test case with `Let`.

Values in `*testcase.variables` are safe to use during T#Parallel.

Spec Hooks

Hooks help you setup common things for each test case. For example clean ahead, clean up, mock expectation configuration, and similar things can be done in hooks, so your test case blocks with `Then` only represent the expected result(s).

In case you work with something that depends on side-effects, such as database tests, you can use the hooks, to create clean-ahead / clean-up blocks.

Also if you use gomock, you can use the spec#Around function, to set up the mock with a controller, and in the teardown function, call the gomock.Controller#Finish function, so your test cases will be only about what is the different behavior from the rest of the test cases.

It will panic if you use hooks or variable preparation in an ambiguous way, or when you try to access variable that doesn't exist in the context where you do so. It tries to panic with friendly and supportive messages, but that is highly subjective.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Spec 

type Spec struct {
	// contains filtered or unexported fields
}

Spec provides you a struct that makes building nested test context easy with the core T#Context function.

spec structure is a simple wrapping around the testing.T#Context. It doesn't use any global singleton cache object or anything like that. It doesn't force you to use global vars.

It uses the same idiom as the core go testing pkg also provide you. You can use the same way as the core testing pkg 

go run ./... -vars -run "the/name/of/the/test/it/print/out/in/case/of/failure"

It allows you to do context preparation for each test in a way, that it will be safe for use with testing.T#Parallel.

Example 
package main

import (
	"fmt"
	"strings"
	"testing"

	"github.com/stretchr/testify/require"

	"github.com/adamluzsi/testcase"
)

type RoleInterface interface {
	Say() string
}

type MyType struct {
	MyResource RoleInterface
}

func (mt *MyType) MyFunc() {}

func (mt *MyType) IsLower(s string) bool {
	return strings.ToLower(s) == s
}

func (mt *MyType) Fallible() (string, error) {
	return "", nil
}

func main() {
	var t *testing.T

	// spec do not use any global magic
	// it is just a simple abstraction around testing.T#Context
	// Basically you can easily can run it as you would any other go test
	//   -> `go run ./... -v -run "my/edge/case/nested/block/I/want/to/run/only"`
	//
	spec := testcase.NewSpec(t)

	// when you have no side effects in your testing suite,
	// you can enable Parallel execution.
	// You can Call Parallel even from nested specs to apply Parallel testing for that context and below.
	spec.Parallel()
	// or
	spec.NoSideEffect()

	// testcase.variables are thread safe way of setting up complex contexts
	// where some variable need to have different values for edge cases.
	// and I usually work with in-memory implementation for certain shared specs,
	// to make my test coverage run fast and still close to somewhat reality in terms of integration.
	// and to me, it is a necessary thing to have "T#Parallel" option safely available
	var myType = func(t *testcase.T) *MyType {
		return &MyType{}
	}

	spec.Describe(`IsLower`, func(s *testcase.Spec) {
		// it is a convention to me to always make a subject for a certain describe block
		//
		var subject = func(t *testcase.T) bool {
			return myType(t).IsLower(t.I(`input`).(string))
		}

		s.When(`input string has lower case characters`, func(s *testcase.Spec) {
			s.LetValue(`input`, `all lower case`)

			s.Before(func(t *testcase.T) {
				// here you can do setups like cleanup for DB tests
			})

			s.After(func(t *testcase.T) {
				// here you can setup a teardown
			})

			s.Around(func(t *testcase.T) func() {
				// here you can setup things that need teardown
				// such example to me is when I use gomock.Controller and mock setup

				return func() {
					// you can do teardown in this
					// this func will be defered after the test cases
				}
			})

			s.And(`the first character is capitalized`, func(s *testcase.Spec) {
				// you can add more nesting for more concrete specifications,
				// in each nested block, you work on a separate variable stack,
				// so even if you overwrite something here,
				// that has no effect outside of this scope
				s.LetValue(`input`, `First character is uppercase`)

				s.Then(`it will report false`, func(t *testcase.T) {
					require.False(t, subject(t),
						fmt.Sprintf(`it was expected that %q will be reported to be not lowercase`, t.I(`input`)))
				})

			})

			s.Then(`it will return true`, func(t *testcase.T) {
				require.True(t, subject(t),
					fmt.Sprintf(`it was expected that the %q will re reported to be lowercase`, t.I(`input`)))
			})
		})
	})

	spec.Describe(`Fallible`, func(s *testcase.Spec) {
		var subject = func(t *testcase.T) (string, error) {
			return myType(t).Fallible()
		}

		var onSuccess = func(t *testcase.T) string {
			someMeaningfulVarName, err := subject(t)
			require.Nil(t, err)
			return someMeaningfulVarName
		}

		s.When(`input is an empty string`, func(s *testcase.Spec) {
			s.LetValue(`input`, ``)

			s.Then(`it will return an empty string`, func(t *testcase.T) {
				require.Equal(t, "", onSuccess(t))
			})
		})
	})
}

Output:
Example (WhenProjectUseSharedSpecificationHelpers) 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
	// . "my/project/testing/pkg"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)
	SetupSpec(s)

	GivenWeHaveUser(s, `myuser`) // Arrange
	// .. other givens

	myType := func() *MyType { return &MyType{} }

	s.Describe(`#MyFunc`, func(s *testcase.Spec) {
		var subject = func(t *testcase.T) { myType().MyFunc() } // Act

		s.Then(`edge case description`, func(t *testcase.T) {
			// Assert
			subject(t)
		})
	})
}

/*
	------------------------------------------------------------------------
	Somewhere else in a project's testing package ("my/project/testing/pkg")
	------------------------------------------------------------------------
*/

func SetupSpec(s *testcase.Spec) {
	s.Let(`storage`, func(t *testcase.T) interface{} {
		// create new storage connection
		// t.Defer(s.Close) after the storage was used in the test
		return nil
	})
	s.Let(`user manager`, func(t *testcase.T) interface{} {
		// new user manager with storage
		return nil
	})
}

func GivenWeHaveUser(s *testcase.Spec, userLetVar string) {
	s.Let(userLetVar, func(t *testcase.T) interface{} {
		// use user manager to create random user with fixtures maybe
		return nil
	})
}

Output:
Example (WithBenchmark) 
var b *testing.B
s := testcase.NewSpec(b)

myType := func(t *testcase.T) *MyType {
	return &MyType{}
}

s.When(`something`, func(s *testcase.Spec) {
	s.Before(func(t *testcase.T) {
		t.Log(`setup`)
	})

	s.Then(`this benchmark block will be executed by *testing.B.N times`, func(t *testcase.T) {
		myType(t).IsLower(`Hello, World!`)
	})
})

Output:

func NewSpec 

func NewSpec(tb testing.TB) *Spec

NewSpec create new Spec struct that is ready for usage.

func (*Spec) After 

func (spec *Spec) After(afterBlock testCaseBlock)

After give you the ability to run a block after each test case. This is ideal for running cleanups. The received *testing.T object is the same as the Then block *testing.T object This hook applied to this scope and anything that is nested from here. All setup block is stackable.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.After(func(t *testcase.T) {
		// this will run after the test cases.
		// this hook applied to this scope and anything that is nested from here.
		// hooks can be stacked with each call.
	})
}

Output:

func (*Spec) And 

func (spec *Spec) And(desc string, testContextBlock func(s *Spec))

And is an alias for testcase#Spec.Context And is used to represent additional requirement for reaching a certain testing runtime contexts.

Example 
var t *testing.T
s := testcase.NewSpec(t)

var (
	myType  = func(t *testcase.T) *MyType { return &MyType{} }
	subject = func(t *testcase.T) bool { return myType(t).IsLower(t.I(`input`).(string)) }
)

s.When(`input has upcase letter`, func(s *testcase.Spec) {
	s.LetValue(`input`, `UPPER`)

	s.And(`mixed with lowercase letters`, func(s *testcase.Spec) {
		s.LetValue(`input`, `UPPER`)

		s.Then(`it will be false`, func(t *testcase.T) {
			require.False(t, subject(t))
		})
	})

	s.And(`input is all upcase letter`, func(s *testcase.Spec) {
		s.Then(`it will be false`, func(t *testcase.T) {
			require.False(t, subject(t))
		})
	})

	s.Then(`it will be false`, func(t *testcase.T) {
		require.False(t, subject(t))
	})
})

Output:

func (*Spec) Around 

func (spec *Spec) Around(aroundBlock hookBlock)

Around give you the ability to create "Before" setup for each test case, with the additional ability that the returned function will be deferred to run after the Then block is done. This is ideal for setting up mocks, and then return the assertion request calls in the return func. This hook applied to this scope and anything that is nested from here. All setup block is stackable.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Around(func(t *testcase.T) func() {
		// this will run before the test cases

		// this hook applied to this scope and anything that is nested from here.
		// hooks can be stacked with each call
		return func() {
			// The content of the returned func will be deferred to run after the test cases.
		}
	})
}

Output:

func (*Spec) Before 

func (spec *Spec) Before(beforeBlock testCaseBlock)

Before give you the ability to run a block before each test case. This is ideal for doing clean ahead before each test case. The received *testing.T object is the same as the Test block *testing.T object This hook applied to this scope and anything that is nested from here. All setup block is stackable.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Before(func(t *testcase.T) {
		// this will run before the test cases.
	})
}

Output:

func (*Spec) Context 

func (spec *Spec) Context(desc string, testContextBlock func(s *Spec))

Context allow you to create a sub specification for a given spec. In the sub-specification it is expected to add more contextual information to the test in a form of hook of variable setting. With Context you can set your custom test description, without any forced prefix like describe/when/and.

It is basically piggybacking the testing#T.Context and create new subspec in that nested testing#T.Context scope. It is used to add more description context for the given subject. It is highly advised to always use When + Before/Around together, in which you should setup exactly what you wrote in the When description input. You can Context as many When/And within each other, as you want to achieve the most concrete edge case you want to test.

To verify easily your state-machine, you can count the `if`s in your implementation, and check that each `if` has 2 `When` block to represent the two possible path.

Example 
var t *testing.T
s := testcase.NewSpec(t)

var (
	myType  = func(t *testcase.T) *MyType { return &MyType{} }
	subject = func(t *testcase.T) bool { return myType(t).IsLower(t.I(`input`).(string)) }
)

s.Context(`when input is in lowercase`, func(s *testcase.Spec) {
	s.LetValue(`input`, `lowercase text`)

	s.Then(`test-case`, func(t *testcase.T) {
		require.True(t, subject(t))
	})
})

Output:

func (*Spec) Describe 

func (spec *Spec) Describe(subjectTopic string, specification func(s *Spec))

Describe creates a new spec scope, where you usually describe a subject.

By convention it is highly advised to create a variable `subject` with function that share the return signature of the method you test on a structure, and take *testcase.variables as the only input value. If your method require input values, you should strictly set those values within a `When`/`And` scope. This ensures you have to think trough the possible state-machines paths that are based on the input values.

For functions where 2 value is returned, and the second one is an error, in order to avoid repetitive test cases in the `Then` I often define a `onSuccess` variable, with a function that takes `testcase#variables` as well and test error return value there with `testcase#variables.T()`.

Example 
var t *testing.T
s := testcase.NewSpec(t)

var myType = func(_ *testcase.T) *MyType {
	return &MyType{}
}

s.Describe(`IsLower`, func(s *testcase.Spec) {
	var subject = func(t *testcase.T) bool { return myType(t).IsLower(t.I(`input`).(string)) }

	s.LetValue(`input`, `Hello, world!`)

	s.Then(`test-case`, func(t *testcase.T) {
		// it will panic since `input` is not actually set at this testing scope,
		// and the testing framework will warn us about this.
		require.True(t, subject(t))
	})
})

Output:

func (*Spec) HasSideEffect added in v0.9.0 

func (spec *Spec) HasSideEffect()

HasSideEffect means that after this call things defined that has software side effect during runtime. This suggest on its own that execution should be sequential in order to avoid flaky tests.

HasSideEffect and NoSideEffect can be used together to describe a given piece of specification properties. Using them at the same location makes little sense, it was intended to be used in spec helper package where setup function handles what resource should be used in the spec variables. This allows flexibility for the developers to use side effect free variant for local development that has quick feedback loop, and replace them with the production implementation during CI/CD pipeline which less time critical.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)
	// this mark the test to contain side effects.
	// this forbids any parallel test execution to avoid flaky tests.
	//
	// Under the hood this is a syntax sugar for Sequential
	s.HasSideEffect()

	s.Test(`this will run in sequence`, func(t *testcase.T) {})

	s.Context(`some context`, func(s *testcase.Spec) {
		s.Test(`this run in sequence`, func(t *testcase.T) {})

		s.Test(`this run in sequence`, func(t *testcase.T) {})
	})
}

Output:

func (*Spec) Let 

func (spec *Spec) Let(varName string, letBlock func(t *T) interface{})

Let define a memoized helper method. The value will be cached across multiple calls in the same example but not across examples. Note that Let is lazy-evaluated, it is not evaluated until the first time the method it defines is invoked. You can force this early by accessing the value from a Before block. Let is threadsafe, the parallel running test will receive they own test variable instance.

Defining a value in a spec Context will ensure that the scope and it's nested scopes of the current scope will have access to the value. It cannot leak its value outside from the current scope. Calling Let in a nested/sub scope will apply the new value for that value to that scope and below.

It will panic if it is used after a When/And/Then scope definition, because those scopes would have no clue about the later defined variable. In order to keep the specification reading mental model requirement low, it is intentionally not implemented to handle such case. Defining test vars always expected in the beginning of a specification scope, mainly for readability reasons.

vars strictly belong to a given `Describe`/`When`/`And` scope, and configured before any hook would be applied, therefore hooks always receive the most latest version from the `Let` vars, regardless in which scope the hook that use the variable is define.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Let(`variable name`, func(t *testcase.T) interface{} {
		return "value that needs complex construction or can be mutated"
	})

	s.Then(`test case`, func(t *testcase.T) {
		t.Log(t.I(`variable name`).(string)) // -> "value"
	})
}

Output:
Example (Mock) 
//go:generate mockgen -source example_Spec_Let_mock_test.go -destination example_Spec_Let_mock_mocks_test.go -package testcase_test
package main

import (
	"testing"

	"github.com/golang/mock/gomock"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Let(`the-mock`, func(t *testcase.T) interface{} {
		ctrl := gomock.NewController(t)
		mock := NewMockInterfaceExample(ctrl)
		t.Defer(ctrl.Finish)
		return mock
	})

	s.When(`some scope where mock should behave in a certain way`, func(s *testcase.Spec) {
		s.Before(func(t *testcase.T) {
			t.I(`*MockInterfaceExample`).(*MockInterfaceExample).
				EXPECT().
				Say().
				Return(`some value but can also be a value from *testcase.variables`)
		})

		s.Then(`mock will be available in every test case and finish called afterwards`, func(t *testcase.T) {
			// ...
		})
	})
}

type InterfaceExample interface {
	Say() string
}

Output:
Example (SqlDB) 
package main

import (
	"database/sql"
	"testing"

	"github.com/stretchr/testify/require"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	// I highly recommend to use *sql.Tx when it is possible for testing.
	// it allows you to have easy teardown
	s.Let(`tx`, func(t *testcase.T) interface{} {
		// it is advised to use a persistent db connection between multiple specification runs,
		// because otherwise `go test -count $times` can receive random connection failures.
		tx, err := getDBConnection(t).Begin()
		if err != nil {
			t.Fatal(err.Error())
		}
		// testcase.T#Defer will execute the received function after the current test edge case
		// where the `tx` test variable were accessed.
		t.Defer(tx.Rollback)
		return tx
	})

	s.When(`something to be prepared in the db`, func(s *testcase.Spec) {
		s.Before(func(t *testcase.T) {
			_, err := t.I(`tx`).(*sql.Tx).Exec(`INSERT INTO "table" ("column") VALUES ($1)`, `value`)
			require.Nil(t, err)
		})

		s.Then(`something will happen`, func(t *testcase.T) {
			// ...
		})
	})

}

func getDBConnection(t testing.TB) *sql.DB {
	// logic to retrieve cached db connection in the testing environment
	return nil
}

Output:
Example (UsageWithinNestedScope) 
var t *testing.T
s := testcase.NewSpec(t)

var myType = func(t *testcase.T) *MyType { return &MyType{} }

s.Describe(`IsLower`, func(s *testcase.Spec) {
	var subject = func(t *testcase.T) bool {
		return myType(t).IsLower(t.I(`input`).(string))
	}

	s.When(`input characters are all lowercase`, func(s *testcase.Spec) {
		s.LetValue(`input`, `all lowercase`)

		s.Then(`it will report true`, func(t *testcase.T) {
			require.True(t, subject(t))
		})
	})

	s.When(`input is a capitalized`, func(s *testcase.Spec) {
		s.LetValue(`input`, "Capitalized")

		s.Then(`it will report false`, func(t *testcase.T) {
			require.False(t, subject(t))
		})
	})
})

Output:

func (*Spec) LetValue added in v0.2.1 

func (spec *Spec) LetValue(varName string, value interface{})

LetValue is a shorthand for defining immutable vars with Let under the hood. So the function blocks can be skipped, which makes tests more readable.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.LetValue(`variable name`, "value")

	s.Then(`test case`, func(t *testcase.T) {
		t.Log(t.I(`variable name`).(string)) // -> "value"
	})
}

Output:

func (*Spec) NoSideEffect added in v0.2.1 

func (spec *Spec) NoSideEffect()

NoSideEffect gives a hint to the reader of the current test that during the test execution, no side effect outside from the test specification scope is expected to be observable. It is important to note that this flag primary meant to represent the side effect possibility to the outside of the current testing specification, and not about the test specification's subject.

It is safe to state that if the subject of the test specification has no side effect, then the test specification must have no side effect as well.

If the subject of the test specification do side effect on an input value, then the test specification must have no side effect, as long Let memorization is used.

If the subject of the test specification does mutation on global variables such as OS Variable states for the current process, then it is likely, that even if the changes by the mutation is restored as part of the test specification, the test specification has side effects that would affect other test specification results, and, as such, must be executed sequentially.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)
	// this is an idiom to express that the subject in the tests here are not expected to have any side-effect.
	// this means they are safe to be executed in parallel.
	s.NoSideEffect()

	s.Test(`this will run in parallel`, func(t *testcase.T) {})

	s.Context(`some context`, func(s *testcase.Spec) {
		s.Test(`this run in parallel`, func(t *testcase.T) {})

		s.Test(`this run in parallel`, func(t *testcase.T) {})
	})
}

Output:

func (*Spec) Parallel 

func (spec *Spec) Parallel()

Parallel allows you to set all test case for the context where this is being called, and below to nested contexts, to be executed in parallel (concurrently). Keep in mind that you can call Parallel even from nested specs to apply Parallel testing for that context and below. This is useful when your test suite has no side effects at all. Using values from *vars when Parallel is safe. It is a shortcut for executing *testing.T#Parallel() for each test

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)
	s.Parallel() // tells the specs to run all test case in parallel

	s.Test(`this will run in parallel`, func(t *testcase.T) {})

	s.Context(`some context`, func(s *testcase.Spec) {
		s.Test(`this run in parallel`, func(t *testcase.T) {})

		s.Test(`this run in parallel`, func(t *testcase.T) {})
	})
}

Output:
Example (ScopedWithContext) 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Context(`context marked parallel`, func(s *testcase.Spec) {
		s.Parallel()

		s.Test(`this run in parallel`, func(t *testcase.T) {})
	})

	s.Context(`context without parallel`, func(s *testcase.Spec) {

		s.Test(`this will run in sequence`, func(t *testcase.T) {})
	})
}

Output:

func (*Spec) Sequential added in v0.9.0 

func (spec *Spec) Sequential()

Sequential allows you to set all test case for the context where this is being called, and below to nested contexts, to be executed sequentially. It will negate any testcase.Spec#Parallel call effect. This is useful when you want to create a spec helper package and there you want to manage if you want to use components side effects or not.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)
	s.Sequential() // tells the specs to run all test case in sequence

	s.Test(`this will run in sequence`, func(t *testcase.T) {})

	s.Context(`some context`, func(s *testcase.Spec) {
		s.Test(`this run in sequence`, func(t *testcase.T) {})

		s.Test(`this run in sequence`, func(t *testcase.T) {})
	})
}

Output:
Example (FromSpecHelper) 
package main

import (
	"os"
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)
	Setup(s) // setup specification with spec helper function

	// Tells that the subject of this specification should be software side effect free on its own.
	s.NoSideEffect()

	var myUseCase = func(t *testcase.T) *MyUseCaseThatHasStorageDependency {
		return &MyUseCaseThatHasStorageDependency{Storage: t.I(`storage`).(MyUseCaseStorageRoleInterface)}
	}

	s.Describe(`#SomeMethod`, func(s *testcase.Spec) {
		var subject = func(t *testcase.T) bool {
			return myUseCase(t).SomeMethod()
		}

		s.Test(`it is expected ...`, func(t *testcase.T) {
			if !subject(t) {
				t.Fatal(`assertion failed`)
			}
		})
	})
}

// in some package testing / spechelper
func Setup(s *testcase.Spec) {
	// spec helper function that is environment aware, and can decide what resource should be used in the test runtime.
	env, ok := os.LookupEnv(`TEST_DB_CONNECTION_URL`)

	if ok {
		s.Sequential()
		// or
		s.HasSideEffect()
		s.Let(`storage`, func(t *testcase.T) interface{} {
			// open database connection
			_ = env // use env to connect or something
			return &ExternalResourceBasedStorage{ /*...*/ }
		})
	} else {
		s.Let(`storage`, func(t *testcase.T) interface{} {
			return &InMemoryBasedStorage{}
		})
	}
}

type InMemoryBasedStorage struct{}

type ExternalResourceBasedStorage struct{}

type MyUseCaseThatHasStorageDependency struct {
	Storage MyUseCaseStorageRoleInterface
}

func (d *MyUseCaseThatHasStorageDependency) SomeMethod() bool {
	return false
}

type MyUseCaseStorageRoleInterface interface{}

Output:
Example (ScopedWithContext) 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Parallel() // on top level, spec marked as parallel

	s.Context(`context marked sequential`, func(s *testcase.Spec) {
		s.Sequential() // but in subcontext the test marked as sequential

		s.Test(`this run in sequence`, func(t *testcase.T) {})
	})

	s.Context(`context that inherit parallel flag`, func(s *testcase.Spec) {

		s.Test(`this will run in parallel`, func(t *testcase.T) {})
	})
}

Output:

func (*Spec) Tag added in v0.10.0 

func (spec *Spec) Tag(tags ...string)

Tag allow you to mark tests in the current and below specification scope with tags. This can be used to provide additional documentation about the nature of the testing scope. This later might be used as well to filter your test in your CI/CD pipeline to build separate testing stages like integration, e2e and so on.

To select or exclude tests with certain tags, you can provide a comma separated list to the following environment variables:

  • TESTCASE_TAG_INCLUDE to filter down to test with a certain tag
  • TESTCASE_TAG_EXCLUDE to exclude certain test from the overall testing scope.

They can be combined as well.

example usage: 

TESTCASE_TAG_INCLUDE='E2E' go test ./...
TESTCASE_TAG_EXCLUDE='E2E' go test ./...
TESTCASE_TAG_INCLUDE='E2E' TESTCASE_TAG_EXCLUDE='list,of,excluded,tags' go test ./...
Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Context(`E2E`, func(s *testcase.Spec) {
		// by tagging the spec context, we can filter tests out later in our CI/CD pipeline.
		// A comma separated list can be set with TESTCASE_TAG_INCLUDE env variable to filter down to tests with certain tags.
		// And/Or a comma separated list can be provided with TESTCASE_TAG_EXCLUDE to exclude tests tagged with certain tags.
		s.Tag(`E2E`)

		s.Test(`some E2E test`, func(t *testcase.T) {
			// ...
		})
	})
}

Output:

func (*Spec) Test 

func (spec *Spec) Test(desc string, test testCaseBlock)

Test creates a test case block where you receive the fully configured `testcase#T` object. Hook contents that meant to run before the test edge cases will run before the function the Test receives, and hook contents that meant to run after the test edge cases will run after the function is done. After hooks are deferred after the received function block, so even in case of panic, it will still be executed.

It should not contain anything that modify the test subject input. It should focuses only on asserting the result of the subject.

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Test(`my test description`, func(t *testcase.T) {
		// ...
	})
}

Output:

func (*Spec) Then 

func (spec *Spec) Then(desc string, test testCaseBlock)

Then is an alias for Test

Example 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	s.Then(`it is expected.... so this is the test description here`, func(t *testcase.T) {
		// ...
	})
}

Output:

func (*Spec) When 

func (spec *Spec) When(desc string, testContextBlock func(s *Spec))

When is an alias for testcase#Spec.Context When is used usually to represent `if` based decision reasons about your testing subject.

Example 
var t *testing.T
s := testcase.NewSpec(t)

var (
	myType  = func(t *testcase.T) *MyType { return &MyType{} }
	subject = func(t *testcase.T) bool { return myType(t).IsLower(t.I(`input`).(string)) }
)

s.When(`input has upcase letter`, func(s *testcase.Spec) {
	s.LetValue(`input`, `UPPER`)

	s.Then(`it will be false`, func(t *testcase.T) {
		require.False(t, subject(t))
	})
})

s.When(`input is all lowercase letter`, func(s *testcase.Spec) {
	s.LetValue(`input`, `lower`)

	s.Then(`it will be true`, func(t *testcase.T) {
		require.True(t, subject(t))
	})
})

Output:

type T 

type T struct {
	testing.TB
	T *testing.T
	// contains filtered or unexported fields
}

T embeds both testcase vars, and testing#T functionality. This leave place open for extension and but define a stable foundation for the hooks and test edge case function signatures

Works as a drop in replacement for packages where they depend on one of the function of testing#T

func (*T) Defer added in v0.2.1 

func (t *T) Defer(fn interface{}, args ...interface{})

Defer function defers the execution of a function until the current test case returns. Deferred functions are guaranteed to run, regardless of panics during the test case execution. Deferred function calls are pushed onto a testcase runtime stack. When an function passed to the Defer function, it will be executed as a deferred call in last-in-first-out order.

It is advised to use this inside a testcase.Spec#Let memorization function when spec variable defined that has finalizer requirements. This allow the specification to ensure the object finalizer requirements to be met, without using an testcase.Spec#After where the memorized function would be executed always, regardless of its actual need.

In a practical example, this means that if you have common vars defined with testcase.Spec#Let memorization, which needs to be Closed for example, after the test case already run. Ensuring such objects Close call in an after block would cause an initialization of the memorized object all the time, even in tests where this is not needed.

e.g.:

  • mock initialization with mock controller, where the mock controller #Finish function must be executed after each test suite.
  • sql.DB / sql.Tx
  • basically anything that has the io.Closer interface
Example 
package main

import (
	"database/sql"
	"testing"

	"github.com/stretchr/testify/require"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	const varName = `db for example is something that needs to defer an action after the test run`
	s.Let(varName, func(t *testcase.T) interface{} {
		db, err := sql.Open(`driverName`, `dataSourceName`)

		// asserting error here with the *testcase.T ensure that the test will don't have some spooky failure.
		require.Nil(t, err)

		// db.Close() will be called after the current test case reach the teardown hooks
		t.Defer(db.Close)

		// check if connection is OK
		require.Nil(t, db.Ping())

		// return the verified db instance for the caller
		// this db instance will be memorized during the runtime of the test case
		return db
	})

	s.Test(`a simple test case`, func(t *testcase.T) {
		db := t.I(varName).(*sql.DB)
		require.Nil(t, db.Ping()) // just to do something with it.
	})
}

Output:
Example (WithArgs) 
package main

import (
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	s := testcase.NewSpec(t)

	const something = `the ExampleDeferTeardownWithArgs value`

	s.Let(something, func(t *testcase.T) interface{} {
		ptr := &ExampleDeferTeardownWithArgs{}
		// T#Defer arguments copied upon pass by value
		// and then passed to the function during the execution of the deferred function call.
		//
		// This is ideal for situations where you need to guarantee that a value cannot be muta
		t.Defer(ptr.SomeTeardownWithArg, `Hello, World!`)
		return ptr
	})

	s.Test(`a simple test case`, func(t *testcase.T) {
		entity := t.I(something).(*ExampleDeferTeardownWithArgs)

		entity.DoSomething()
	})
}

type ExampleDeferTeardownWithArgs struct{}

func (*ExampleDeferTeardownWithArgs) SomeTeardownWithArg(arg string) {}

func (*ExampleDeferTeardownWithArgs) DoSomething() {}

Output:

func (*T) HasTag added in v0.10.2 

func (t *T) HasTag(tag string) bool
Example 
package main

import (
	"context"
	"database/sql"
	"testing"

	"github.com/stretchr/testify/require"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	var s = testcase.NewSpec(t)

	s.Let(`db`, func(t *testcase.T) interface{} {
		db, err := sql.Open(`driverName`, `dataSourceName`)
		require.Nil(t, err)

		if t.HasTag(`black box`) {
			// tests with black box  use http test server or similar things and high level tx management not maintainable.
			t.Defer(db.Close)
			return db
		}

		tx, err := db.BeginTx(context.Background(), nil)
		require.Nil(t, err)
		t.Defer(tx.Rollback)
		return tx
	})
}

Output:

func (*T) I 

func (t *T) I(varName string) interface{}

I will return a testcase variable. it is suggested to use interface casting right after to it, so you can work with concrete types. If there is no such value, then it will panic with a "friendly" message.

func (*T) Let added in v0.2.0 

func (t *T) Let(varName string, value interface{})

Let will allow you to define/override a spec runtime bounded variable. The idiom is that if you cannot express the variable declaration with spec level let, or if you need to override in a sub scope a let's content using the previous variable state, or a result of a multi return variable needs to be stored at spec runtime level you can utilize this Let function to achieve this.

Typical use-case to this when you want to have a context.Context, with different values or states, but you don't want to rebuild from scratch at each layer.

Example 
package main

import (
	"context"
	"testing"

	"github.com/adamluzsi/testcase"
)

func main() {
	var t *testing.T
	var s = testcase.NewSpec(t)
	s.Parallel()

	s.Let(`ctx`, func(t *testcase.T) interface{} {
		return context.Background()
	})

	s.When(`let can be manipulated during runtime hooks by simply calling *T#Let`, func(s *testcase.Spec) {
		s.Before(func(t *testcase.T) {
			t.Log(`here for example we update the test variable ctx to have a certain value to fulfil the subcontext goal`)
			t.Let(`ctx`, context.WithValue(t.I(`ctx`).(context.Context), `certain`, `value`))
		})

		s.Then(`ctx here has the value that was assigned in the before hook`, func(t *testcase.T) {
			_ = t.I(`ctx`).(context.Context)
		})
	})

	s.Then(`your ctx is in the original state without any modifications`, func(t *testcase.T) {
		_ = t.I(`ctx`).(context.Context)
	})
}

Output:

Source Files

View all Source files

Directories

Path Synopsis
docs
examples
random

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.