Affected by 
Documentation
¶
Overview ¶
Package framework implements a model for developing end-to-end test suites. The model includes a top level Framework which TestSuites can be added to. TestSuites include conditions under which the suite will run and a list of TestCase implementations to run. TestCases can be implemented with methods that run before/after each and all tests.
Writing Tests ¶
Tests follow a similar patterns as go tests. They are functions that must start with 'Test' and instead of a *testing.T argument, a *framework.F is passed and they must have a receiver that implements the TestCase interface. A crude example as follows:
// foo_test.go
type MyTestCase struct {
framework.TC
}
func (tc *MyTestCase) TestMyFoo(f *framework.F) {
f.T().Log("bar")
}
func TestCalledFromGoTest(t *testing.T){
framework.New().AddSuites(&framework.TestSuite{
Component: "foo",
Cases: []framework.TestCase{
new(MyTestCase),
},
}).Run(t)
}
Test cases should embed the TC struct which satisfies the TestCase interface. Optionally a TestCase can also implement the Name() function which returns a string to name the test case. By default the name is the name of the struct type, which in the above example would be "MyTestCase"
Test cases may also optionally implement additional interfaces to define setup and teardown logic:
BeforeEachTest AfterEachTest BeforeAllTests AfterAllTests
The test case struct allows you to setup and teardown state in the struct that can be consumed by the tests. For example:
type ComplexNomadTC struct {
framework.TC
jobID string
}
func (tc *ComplexNomadTC) BeforeEach(f *framework.F){
// Do some complex job setup with a unique prefix string
jobID, err := doSomeComplexSetup(tc.Nomad(), f.ID())
f.NoError(err)
f.Set("jobID", jobID)
}
func (tc *ComplexNomadTC) TestSomeScenario(f *framework.F){
jobID := f.Value("jobID").(string)
doTestThingWithJob(f, tc.Nomad(), jobID)
}
func (tc *ComplexNomadTC) TestOtherScenario(f *framework.F){
jobID := f.Value("jobID").(string)
doOtherTestThingWithJob(f, tc.Nomad(), jobID)
}
func (tc *ComplexNomadTC) AfterEach(f *framework.F){
jobID := f.Value("jobID").(string)
_, _, err := tc.Nomad().Jobs().Deregister(jobID, true, nil)
f.NoError(err)
}
As demonstrated in the previous example, TC also exposes functions that return configured api clients including Nomad, Consul and Vault. If Consul or Vault are not provisioned their respective getter functions will return nil.
Testify Integration ¶
Test cases expose a T() function to fetch the current *testing.T context. While this means the author is able to most other testing libraries, github.com/stretch/testify is recommended and integrated into the framework. The TC struct also embeds testify assertions that are preconfigured with the current testing context. Additionally TC comes with a Require() method that yields a testify Require if that flavor is desired.
func (tc *MyTestCase) TestWithTestify() {
err := someErrFunc()
tc.NoError(err)
// Or tc.Require().NoError(err)
}
Parallelism ¶
The test framework honors go test's parallel feature under certain conditions. A TestSuite can be created with the Parallel field set to true to enable parallel execution of the test cases of the suite. Tests within a test case will be executed sequentially unless f.T().Parallel() is called. Note that if multiple tests are to be executed in parallel, access to TC is not syncronized. The *framework.F offers a way to store state between before/after each method if desired.
func (tc *MyTestCase) BeforeEach(f *framework.F){
jobID, _ := doSomeComplexSetup(tc.Nomad(), f.ID())
f.Set("jobID", jobID)
}
func (tc *MyTestCase) TestParallel(f *framework.F){
f.T().Parallel()
jobID := f.Value("jobID").(string)
}
Since test cases have the potential to work with a shared Nomad cluster in parallel any resources created or destroyed must be prefixed with a unique identifier for each test case. The framework.F struct exposes an ID() function that will return a string that is unique with in a test. Therefore, multiple tests with in the case can reliably create unique IDs between tests and setup/teardown. The string returned is 8 alpha numeric characters.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type AfterAllTests ¶
type AfterAllTests interface {
AfterAll(*F)
}
AfterAllTests is used to define a method to be called after the execution of all tests.
type AfterEachTest ¶
type AfterEachTest interface {
AfterEach(*F)
}
AfterEachTest is used to degine a method to be called after each test.
type BeforeAllTests ¶
type BeforeAllTests interface {
BeforeAll(*F)
}
BeforeAllTests is used to define a method to be called before the execution of all tests.
type BeforeEachTest ¶
type BeforeEachTest interface {
BeforeEach(*F)
}
BeforeEachTest is used to define a method to be called before each test.
type ClusterInfo ¶
type Constraints ¶
type Constraints struct {
Provider string // Cloud provider ex. 'aws', 'azure', 'gcp'
OS string // Operating system ex. 'windows', 'linux'
Arch string // CPU architecture ex. 'amd64', 'arm64'
Environment string // Environment name ex. 'simple'
Tags []string // Generic tags that must all exist in the environment
}
Constraints that must be satisfied for a TestSuite to run
type Environment ¶
type Environment struct {
Name string
Provider string
OS string
Arch string
Tags map[string]struct{}
}
Environment includes information about the target environment the test framework is targeting. During 'go run', these fields are populated by the following flags:
-env <string> "name of the environment executing against" -env.provider <string> "cloud provider for which the environment is executing against" -env.os <string> "operating system of environment" -env.arch <string> "cpu architecture of environment" -env.tags <string> "comma delimited list of environment tags"
These flags are not needed when executing locally
type F ¶
type F struct {
*require.Assertions
// contains filtered or unexported fields
}
F is the framework context that is passed to each test. It is used to access the *testing.T context as well as testify helpers
func (*F) Assert ¶
func (f *F) Assert() *assert.Assertions
Assert fetches an assert flavor of testify assertions https://godoc.org/github.com/stretchr/testify/assert
type Framework ¶
type Framework struct {
// contains filtered or unexported fields
}
type Provisioner ¶
type Provisioner interface {
ProvisionCluster(opts ProvisionerOptions) (*ClusterInfo, error)
DestroyCluster(clusterID string) error
}
Provisioner interface is used by the test framework to provision a Nomad cluster for each TestCase
var DefaultProvisioner Provisioner = new(singleClusterProvisioner)
DefaultProvisioner is a noop provisioner that builds clients from environment variables according to the respective client configuration
type ProvisionerOptions ¶
type ProvisionerOptions struct {
Name string
ExpectConsul bool // If true, fails if a Consul client can't be configured
ExpectVault bool // If true, fails if a Vault client can't be configured
}
ProvisionerOptions defines options to be given to the Provisioner when calling ProvisionCluster
type TC ¶
type TC struct {
// contains filtered or unexported fields
}
TC is the base test case which should be embedded in TestCase implementations.
type TestCase ¶
type TestCase interface {
Name() string
// contains filtered or unexported methods
}
TestCase is the interface which an E2E test case implements. It is not meant to be implemented directly, instead the struct should embed the 'framework.TC' struct
type TestSuite ¶
type TestSuite struct {
Component string // Name of the component/system/feature tested
CanRunLocal bool // Flags if the cases are safe to run on a local nomad cluster
Cases []TestCase // Cases to run
Constraints Constraints // Environment constraints to follow
Parallel bool // If true, will run test cases in parallel
Slow bool // Slow test suites don't run by default
// API Clients
Consul bool
Vault bool
}
TestSuite defines a set of test cases and under what conditions to run them
Source Files