framework

README

Please see this wiki page for info on using the test framework.

Documentation

Index

• Variables
• func Run(t *testing.T, fn func(ctx TestContext))
• type Suite
  • func NewSuite(m *testing.M) Suite
• type SuiteContext
• type Test
  • func NewTest(t *testing.T) Test
• type TestContext
  • func NewContext(goTest *testing.T, labels ...label.Instance) TestContext

Variables

var (
	// TelemetryRetryDelay is the retry delay used in tests.
	TelemetryRetryDelay time.Duration
	// TelemetryRetryTimeout is the retry timeout used in tests.
	TelemetryRetryTimeout time.Duration
	// UseRealStackdriver controls whether to use real stackdriver backend for testing or not.
	UseRealStackdriver bool
)

var GlobalYAMLWrites = atomic.NewUint64(0)

GlobalYAMLWrites records how many YAMLs we have applied from all sources. Note: go tests are distinct binaries per test suite, so this is the suite level number of calls

Functions

func Run

func Run(t *testing.T, fn func(ctx TestContext))

Run runs the given test.

Types

type Suite

type Suite interface {
	// EnvironmentFactory sets a custom function used for creating the resource.Environment for this Suite.
	EnvironmentFactory(fn resource.EnvironmentFactory) Suite
	// Label all the tests in suite with the given labels
	Label(labels ...label.Instance) Suite
	// SkipIf skips the suite if the function returns true
	SkipIf(reason string, fn resource.ShouldSkipFn) Suite
	// Skip marks a suite as skipped with the given reason. This will prevent any setup functions from occurring.
	Skip(reason string) Suite
	// RequireMinClusters ensures that the current environment contains at least the given number of clusters.
	// Otherwise it stops test execution.
	//
	// Deprecated: Tests should not make assumptions about number of clusters.
	RequireMinClusters(minClusters int) Suite
	// RequireMaxClusters ensures that the current environment contains at least the given number of clusters.
	// Otherwise it stops test execution.
	//
	// Deprecated: Tests should not make assumptions about number of clusters.
	RequireMaxClusters(maxClusters int) Suite
	// RequireSingleCluster is a utility method that requires that there be exactly 1 cluster in the environment.
	//
	// Deprecated: All new tests should support multiple clusters.
	RequireSingleCluster() Suite
	// RequireMultiPrimary ensures that each cluster is running a control plane.
	//
	// Deprecated: All new tests should work for any control plane topology.
	RequireMultiPrimary() Suite
	// SkipExternalControlPlaneTopology skips the tests in external plane and config cluster topology
	SkipExternalControlPlaneTopology() Suite
	// RequireExternalControlPlaneTopology requires the environment to be external control plane topology
	RequireExternalControlPlaneTopology() Suite
	// RequireMinVersion validates the environment meets a minimum version
	RequireMinVersion(minorVersion uint) Suite
	// RequireMaxVersion validates the environment meets a maximum version
	RequireMaxVersion(minorVersion uint) Suite
	// RequireDualStack validates the test context has the required dual-stack setting enabled
	RequireDualStack() Suite
	// Setup runs enqueues the given setup function to run before test execution.
	Setup(fn resource.SetupFn) Suite
	Teardown(fn resource.TeardownFn) Suite
	// SetupParallel runs the given setup functions in parallel before test execution.
	SetupParallel(fns ...resource.SetupFn) Suite
	// Run the suite. This method calls os.Exit and does not return.
	Run()
}

Suite allows the test author to specify suite-related metadata and do setup in a fluent-style, before commencing execution.

func NewSuite

func NewSuite(m *testing.M) Suite

NewSuite returns a new suite instance.

type SuiteContext

type SuiteContext interface {
	resource.Context
}

SuiteContext contains suite-level items used during runtime.

type Test

type Test interface {
	// Label applies the given labels to this test.
	Label(labels ...label.Instance) Test
	// RequireIstioVersion ensures that all installed versions of Istio are at least the
	// required version for the annotated test to pass
	RequireIstioVersion(version string) Test
	// RequireKubernetesMinorVersion ensures that all Kubernetes clusters used in this test
	// are at least the required version for the annotated test to pass
	RequireKubernetesMinorVersion(minorVersion uint) Test
	// RequiresMinClusters ensures that the current environment contains at least the expected number of clusters.
	// Otherwise it stops test execution and skips the test.
	//
	// Deprecated: Tests should not make assumptions about number of clusters.
	RequiresMinClusters(minClusters int) Test
	// RequiresSingleCluster this a utility that requires the min/max clusters to both = 1.
	//
	// Deprecated: All new tests should support multiple clusters.
	RequiresSingleCluster() Test
	// RequiresDualstack ensures that test context contains DualStack configuration.
	RequiresDualStack() Test
	// RequiresLocalControlPlane ensures that clusters are using locally-deployed control planes.
	//
	// Deprecated: Tests should not make assumptions regarding control plane topology.
	RequiresLocalControlPlane() Test
	// RequiresSingleNetwork ensures that clusters are in the same network
	//
	// Deprecated: Tests should not make assumptions regarding number of networks.
	RequiresSingleNetwork() Test
	// TopLevel marks a test as a "top-level test" meaning a container test that has many subtests.
	// Resources created at this level will be in-scope for dumping when any descendant test fails.
	TopLevel() Test
	// Run the test, supplied as a lambda.
	Run(fn func(t TestContext))
	// RunParallel runs this test in parallel with other children of the same parent test/suite. Under the hood,
	// this relies on Go's t.Parallel() and will, therefore, have the same behavior.
	//
	// A parallel test will run in parallel with siblings that share the same parent test. The parent test function
	// will exit before the parallel children are executed. It should be noted that if the parent test is prevented
	// from exiting (e.g. parent test is waiting for something to occur within the child test), the test will
	// deadlock.
	//
	// Example:
	//
	// func TestParallel(t *testing.T) {
	//     framework.NewTest(t).
	//         Run(func(ctx framework.TestContext) {
	//             ctx.NewSubTest("T1").
	//                 Run(func(ctx framework.TestContext) {
	//                     ctx.NewSubTest("T1a").
	//                         RunParallel(func(ctx framework.TestContext) {
	//                             // Run in parallel with T1b
	//                         })
	//                     ctx.NewSubTest("T1b").
	//                         RunParallel(func(ctx framework.TestContext) {
	//                             // Run in parallel with T1a
	//                         })
	//                     // Exits before T1a and T1b are run.
	//                 })
	//
	//             ctx.NewSubTest("T2").
	//                 Run(func(ctx framework.TestContext) {
	//                     ctx.NewSubTest("T2a").
	//                         RunParallel(func(ctx framework.TestContext) {
	//                             // Run in parallel with T2b
	//                         })
	//                     ctx.NewSubTest("T2b").
	//                         RunParallel(func(ctx framework.TestContext) {
	//                             // Run in parallel with T2a
	//                         })
	//                     // Exits before T2a and T2b are run.
	//                 })
	//         })
	// }
	//
	// In the example above, non-parallel parents T1 and T2 contain parallel children T1a, T1b, T2a, T2b.
	//
	// Since both T1 and T2 are non-parallel, they are run synchronously: T1 followed by T2. After T1 exits,
	// T1a and T1b are run asynchronously with each other. After T1a and T1b complete, T2 is then run in the
	// same way: T2 exits, then T2a and T2b are run asynchronously to completion.
	RunParallel(fn func(t TestContext))
}

func NewTest

func NewTest(t *testing.T) Test

NewTest returns a new test wrapper for running a single test.

type TestContext

type TestContext interface {
	resource.Context
	test.Failer

	Context() context.Context

	// NewSubTest creates a new sub-test under the current running Test. The lifecycle of a sub-Test is scoped to the
	// parent. Calls to Done() will block until all children are also Done(). When Run, sub-Tests will automatically
	// create their own Golang *testing.T with the name provided.
	//
	// If this TestContext was not created by a Test or if that Test is not running, this method will panic.
	NewSubTest(name string) Test
	NewSubTestf(format string, a ...any) Test

	// WorkDir allocated for this test.
	WorkDir() string

	// CreateDirectoryOrFail creates a new sub directory with the given name in the workdir, or fails the test.
	CreateDirectoryOrFail(name string) string

	// CreateTmpDirectoryOrFail creates a new temporary directory with the given prefix in the workdir, or fails the test.
	CreateTmpDirectoryOrFail(prefix string) string

	// SkipDumping will skip dumping debug logs/configs/etc for this scope only (child scopes are not skipped).
	SkipDumping()

	Error(args ...any)
	Errorf(format string, args ...any)
	Failed() bool
	Name() string
	Skip(args ...any)
	SkipNow()
	Skipf(format string, args ...any)
	Skipped() bool
}

TestContext is a test-level context that can be created as part of test executing tests.

func NewContext

func NewContext(goTest *testing.T, labels ...label.Instance) TestContext

NewContext creates a new test context and returns. It is up to the caller to close to context by calling .Done() at the end of the test run.