e2e

README

End-to-End Testing Framework

Testify

We rely on testify to provide the structure for our end-to-end testing. This allows us to decouple where tests are defined, from where they are run.

TestCluster

A TestCluster is the structure that manages tests running against a single Kubernetes Cluster.

Its sole responsibility is to create TestInstallations.

TestInstallation

A TestInstallation is the structure that manages a group of tests that run against an installation of Gloo Gateway, within a Kubernetes Cluster.

We try to define a single TestInstallation per file in a TestCluster. This way, it is easy to identify what behaviors are expected for that installation.

Features

We define all tests in the features package. This is done for a variety of reasons:

1. We group the tests by feature, so it's easy to identify which behaviors we assert for a given feature.
2. We can invoke that same test against different TestInstallations. This means we can test a feature against a variety of installation values, or even against OSS and Enterprise installations.

Debugging

Refer to the Debugging guide for more information on how to debug tests.

Thanks

Inspiration

This framework was inspired by the following projects:

• Kubernetes Gateway API
• Gloo Platform

Areas of Improvement

Help Wanted: This framework is not feature complete, and we welcome any improvements to it.

Below are a set of known areas of improvement. The goal is to provide a starting point for developers looking to contribute. There are likely other improvements that are not currently captured, so please add/remove entries to this list as you see fit:

• Debug Improvements: On test failure, we should emit a report about the entire state of the cluster. This should be a CLI utility as well.
• Curl assertion: We need a re-usable way to execute Curl requests against a Pod, and assert properties of the response.
• Improved install action(s): We rely on the SoloTestHelper currently, and it would be nice if we relied directly on Helm or Glooctl.
• Cluster provisioning: We rely on the setup-kind script to provision a cluster. We should make this more flexible by providing a configurable, declarative way to do this.
• Istio action: We need a way to perform Istio actions against a cluster.
• Argo action: We need an easy utility to perform ArgoCD commands against a cluster.

Documentation

Index

• Constants
• func DownloadIstio(ctx context.Context, version string) (string, error)
• func GlooDirectory() string
• func MustTestHelper(ctx context.Context, installation *TestInstallation) *helper.SoloTestHelper
• type TestCluster
  • func MustTestCluster() *TestCluster
  • func (c *TestCluster) RegisterTestInstallation(t *testing.T, glooGatewayContext *gloogateway.Context) *TestInstallation
  • func (c *TestCluster) UnregisterTestInstallation(installation *TestInstallation)
• type TestInstallation
  • func (i *TestInstallation) AddIstioctl(ctx context.Context) error
  • func (i *TestInstallation) InstallGlooGateway(ctx context.Context, installFn func(ctx context.Context) error)
  • func (i *TestInstallation) InstallIstioOperator(ctx context.Context, operatorFile string) error
  • func (i *TestInstallation) InstallMinimalIstio(ctx context.Context) error
  • func (i *TestInstallation) PreFailHandler(ctx context.Context)
  • func (i *TestInstallation) String() string
  • func (i *TestInstallation) UninstallGlooGateway(ctx context.Context, uninstallFn func(ctx context.Context) error)
  • func (i *TestInstallation) UninstallIstio() error

Constants

const (
	IstioctlVersionEnv = "ISTIOCTL_VERSION"
)

Functions

func DownloadIstio

func DownloadIstio(ctx context.Context, version string) (string, error)

Download istioctl binary from istio.io/downloadIstio and returns the path to the binary

func GlooDirectory

func GlooDirectory() string

func MustTestHelper

func MustTestHelper(ctx context.Context, installation *TestInstallation) *helper.SoloTestHelper

MustTestHelper returns the SoloTestHelper used for e2e tests The SoloTestHelper is a wrapper around `glooctl` and we should eventually phase it out in favor of using the exact tool that users rely on

Types

type TestCluster

type TestCluster struct {
	// RuntimeContext contains the set of properties that are defined at runtime by whoever is invoking tests
	RuntimeContext k8sruntime.Context

	// ClusterContext contains the metadata about the Kubernetes Cluster that is used for this TestCluster
	ClusterContext *cluster.Context
	// contains filtered or unexported fields
}

TestCluster is the structure around a set of tests that run against a Kubernetes Cluster Within a TestCluster, we spin off multiple TestInstallation to test the behavior of a particular installation

func MustTestCluster

func MustTestCluster() *TestCluster

func (*TestCluster) RegisterTestInstallation

func (c *TestCluster) RegisterTestInstallation(t *testing.T, glooGatewayContext *gloogateway.Context) *TestInstallation

func (*TestCluster) UnregisterTestInstallation

func (c *TestCluster) UnregisterTestInstallation(installation *TestInstallation)

type TestInstallation

type TestInstallation struct {
	fmt.Stringer

	// TestCluster contains the properties of the TestCluster this TestInstallation is a part of
	TestCluster *TestCluster

	// Metadata contains the properties used to install Gloo Gateway
	Metadata *gloogateway.Context

	// ResourceClients is a set of clients that can manipulate resources owned by Gloo Gateway
	ResourceClients gloogateway.ResourceClients

	// Actions is the entity that creates actions that can be executed by the Operator
	Actions *actions.Provider

	// Assertions is the entity that creates assertions that can be executed by the Operator
	Assertions *assertions.Provider

	// IstioctlBinary is the path to the istioctl binary that can be used to interact with Istio
	IstioctlBinary string
}

TestInstallation is the structure around a set of tests that validate behavior for an installation of Gloo Gateway.

func (*TestInstallation) AddIstioctl

func (i *TestInstallation) AddIstioctl(
	ctx context.Context) error

func (*TestInstallation) InstallGlooGateway

func (i *TestInstallation) InstallGlooGateway(ctx context.Context, installFn func(ctx context.Context) error)

func (*TestInstallation) InstallIstioOperator

func (i *TestInstallation) InstallIstioOperator(
	ctx context.Context,
	operatorFile string) error

func (*TestInstallation) InstallMinimalIstio

func (i *TestInstallation) InstallMinimalIstio(
	ctx context.Context) error

func (*TestInstallation) PreFailHandler

func (i *TestInstallation) PreFailHandler(ctx context.Context)

PreFailHandler is the function that is invoked if a test in the given TestInstallation fails

func (*TestInstallation) String

func (i *TestInstallation) String() string

func (*TestInstallation) UninstallGlooGateway

func (i *TestInstallation) UninstallGlooGateway(ctx context.Context, uninstallFn func(ctx context.Context) error)

func (*TestInstallation) UninstallIstio

func (i *TestInstallation) UninstallIstio() error