e2e

README

End-to-End Testing Framework

TestCluster

A TestCluster is the structure that manages tests running against a single Kubernetes Cluster.

Its sole responsibility is to create TestInstallations.

TestInstallation

A TestInstallation is the structure that manages a group of tests that run against an installation of Gloo Gateway, within a Kubernetes Cluster.

We try to define a single TestInstallation per file in a TestCluster. This way, it is easy to identify what behaviors are expected for that installation.

Test

A Test is the smallest unit in the framework. It assumes there is a Kubernetes Cluster running, with Gloo Gateway installed. It runs a series of actions against the cluster and asserts that the updated cluster behaves as expected.

A Test is provided the TestInstallation to run against. This way, you can run the same Test, against different installations of Gloo Gateway.

Tests are executed from the file where a TestInstallation is defined. However, we define these tests in the features package.

Features

We define all tests in the features package. This is done for a variety of reasons:

1. We group the tests by feature, so it's easy to identify which behaviors we assert for a given feature.
2. We can invoke that same test against different TestInstallations.

Thanks

Inspiration

This framework was inspired by the following projects:

• Kubernetes Gateway API
• Gloo Platform

Areas of Improvement

Help Wanted: This framework is not feature complete, and we welcome any improvements to it.

Below are a set of known areas of improvement. The goal is to provide a starting point for developers looking to contribute. There are likely other improvements that are not currently captured, so please add/remove entries to this list as you see fit:

• Debug Improvements: On test failure, we should emit a report about the entire state of the cluster. This should be a CLI utility as well.
• Curl assertion: We need a re-usable way to execute Curl requests against a Pod, and assert properties of the response.
• Improved install action(s): We rely on the SoloTestHelper currently, and it would be nice if we relied directly on Helm or Glooctl.
• Cluster provisioning: We rely on the setup-kind script to provision a cluster. We should make this more flexible by providing a configurable, declarative way to do this.
• Istio action: We need a way to perform Istio actions against a cluster.
• Argo action: We need an easy utility to perform ArgoCD commands against a cluster.
• Re-usable Operations: Constructing an Operation is manual, and that can lead to verbose tests. If there is a consistent set of Operations that we construct in tests, create a utility to do this.

Documentation

Index

• type Test
• type TestCluster
  • func (c *TestCluster) PreFailHandler()
  • func (c *TestCluster) RegisterTestInstallation(name string, glooGatewayContext *gloogateway.Context) *TestInstallation
  • func (c *TestCluster) UnregisterTestInstallation(installation *TestInstallation)
• type TestExecutor
• type TestInstallation
  • func (i *TestInstallation) InstallGlooGateway(ctx context.Context, installAction actions.ClusterAction) error
  • func (i *TestInstallation) RunTest(ctx context.Context, test Test)
  • func (i *TestInstallation) UninstallGlooGateway(ctx context.Context, uninstallAction actions.ClusterAction) error

Types

type Test

type Test struct {
	// Name is a required value that uniquely identifies a test
	Name string
	// Description is an optional value that is used to provide context to developers about a test's purpose
	Description string
	// Test is the actual function that executes the test
	Test TestExecutor
}

Test represents a single end-to-end behavior that is validated against a running installation of Gloo Gateway. Tests are grouped by the feature they validate, and are defined in the test/kubernetes/e2e/features directory

type TestCluster

type TestCluster struct {
	// TestingFramework defines the framework that tests should rely on
	// Within the Gloo codebase, we rely extensively on Ginkgo and Gomega
	// The idea behind making this configurable, per suite, is that it ensures that
	// all of our tests can rely on the testing interface, instead of the explicit Ginkgo implementation
	TestingFramework testing.TB

	// TestingProgressWriter is the io.Writer used by the TestingFramework to emit feedback to whoever invoked the tests
	TestingProgressWriter io.Writer

	// RuntimeContext contains the set of properties that are defined at runtime by whoever is invoking tests
	RuntimeContext runtime.Context

	// ClusterContext contains the metadata about the Kubernetes Cluster that is used for this TestCluster
	ClusterContext *cluster.Context
	// contains filtered or unexported fields
}

TestCluster is the structure around a set of tests that run against a Kubernetes Cluster Within a TestCluster, we spin off multiple TestInstallation to test the behavior of a particular installation

func (*TestCluster) PreFailHandler

func (c *TestCluster) PreFailHandler()

PreFailHandler will execute the PreFailHandler for any of the TestInstallation that are registered with the given TestCluster. The function will be executed when a test in the TestCluster fails, but before any of the cleanup functions (AfterEach, AfterAll) are invoked. This allows us to capture relevant details about the running installation of Gloo Gateway and the Kubernetes Cluster

func (*TestCluster) RegisterTestInstallation

func (c *TestCluster) RegisterTestInstallation(name string, glooGatewayContext *gloogateway.Context) *TestInstallation

func (*TestCluster) UnregisterTestInstallation

func (c *TestCluster) UnregisterTestInstallation(installation *TestInstallation)

type TestExecutor

type TestExecutor func(ctx context.Context, suite *TestInstallation)

TestExecutor is a function that executes a test, for a given TestInstallation

type TestInstallation

type TestInstallation struct {
	*TestCluster

	// Name is a unique identifier for this TestInstallation
	Name string

	// Operator is responsible for executing operations against an installation of Gloo Gateway
	// This is meant to simulate the behaviors that a person could execute
	Operator *operations.Operator

	// Actions is the entity that creates actions that can be executed by the Operator
	Actions *provider.ActionsProvider

	// Assertions is the entity that creates assertions that can be executed by the Operator
	Assertions *assertions.Provider
}

TestInstallation is the structure around a set of tests that validate behavior for an installation of Gloo Gateway.

func (*TestInstallation) InstallGlooGateway

func (i *TestInstallation) InstallGlooGateway(ctx context.Context, installAction actions.ClusterAction) error

func (*TestInstallation) RunTest

func (i *TestInstallation) RunTest(ctx context.Context, test Test)

RunTest will execute a single Test against the installation We intentionally do not expose a RunTests method, because then we would lose the ability to randomize tests through the testing framework

func (*TestInstallation) UninstallGlooGateway

func (i *TestInstallation) UninstallGlooGateway(ctx context.Context, uninstallAction actions.ClusterAction) error