e2e

README

In Memory End-to-End tests

This directory contains end-to-end tests that do not require Kubernetes, and persist configuration in memory.

Note: All commands should be run from the root directory of the Gloo repository

Background

This is the most common type of end-to-end test, since it is the quickest to set up and easiest to debug. Additionally, since Gloo Edge may be run using various backing stores, these tests provide a single space to validate the translation of Gloo resources into Envoy resources, independent of where Gloo Edge is deployed. As a result, these test do not rely on Kubernetes, so if there is any Kubernetes behavior that needs to be tested, write a kubernetes end-to-end test instead.

How do the tests work?

1. Run the Gloo controllers in goroutines
2. Run Envoy either using a binary or docker container
3. Apply Gloo resources using in-memory resource clients
4. Execute requests against the Envoy proxy and confirm the expected response. This validates that the Gloo resources have been picked up by the controllers, were been translated correctly into Envoy configuration, the configuration was sent to the Envoy proxy, and the proxy behaves appropriately.

Example Test

We have an example test which outlines how these tests work. It also provides some examples for basic testing operations. If you are writing a new e2e test, we recommend that you review the example test first.

This was introduced in a pull request which includes other useful details about e2e test considerations.

CI

These tests are run by build-bot in Google Cloud as part of our CI pipeline.

If a test fails, you can retry it using the build-bot comment directives. If you do this, please make sure to include a link to the failed logs for debugging purposes.

Local Development

Setup

For these tests to run, we require Envoy be built in a docker container.

Refer to the Envoyinit README for build instructions.

Run Tests

The run-tests make target runs ginkgo with a set of useful flags. The following environment variables can be configured for this target:

Name	Default	Description
ENVOY_IMAGE_TAG	""	The tag of the gloo-envoy-wrapper-docker image built during setup
TEST_PKG	""	The path to the package of the test suite you want to run
WAIT_ON_FAIL	0	Set to 1 to prevent Ginkgo from cleaning up the Gloo Edge installation in case of failure. Useful to exec into inspect resources created by the test. A command to resume the test run (and thus clean up resources) will be logged to the output.
INVALID_TEST_REQS	fail	The behavior for tests which depend on environment conditions that aren't satisfied. Options are skip, run, fail

Example:

ENVOY_IMAGE_TAG=solo-test-image TEST_PKG=./test/e2e/... make run-tests

Debugging Tests

Use WAIT_ON_FAIL

When Ginkgo encounters a test failure it will attempt to clean up relevant resources, which includes stopping the running instance of Envoy, preventing the developer from inspecting the state of the Envoy instance for useful clues.

To avoid this clean up, run the test(s) with WAIT_ON_FAIL=1. When the test fails, it will halt execution, allowing you to inspect the state of the Envoy instance.

Once halted, use docker ps to determine the admin port for the Envoy instance, and follow the recommendations for debugging Envoy, specifically the parts around interacting with the Administration interface.

Use INVALID_TEST_REQS

Certain test require environmental conditions to be true for them to succeed. For example, certain tests will only run on a Linux machine.

By setting INVALID_TEST_REQS=skip, you can run all tests locally, and any tests which will not run in your local environment will be skipped. The default behavior is that they fail.

Additional Notes

Notes on EC2 tests

Note: these instructions are out of date, and require updating

• set up your ec2 instance
  • download a simple echo app
  • make the app executable
  • run it in the background

wget https://mitch-solo-public.s3.amazonaws.com/echoapp2
chmod +x echoapp2
sudo ./echoapp2 --port 80 &

Notes on AWS Lambda Tests (test/e2e/aws_test.go)

In addition to the configuration steps provided above, you will need to do the following to run the AWS Lambda Tests locally:

1. Obtain an AWS IAM User account that is part of the Solo.io organization
2. Create an AWS access key
   • Sign into the AWS console with the account created during step 1
   • Hover over your username at the top right of the page. Click on "My Security Credentials"
   • In the section titled "AWS IAM credentials", click "Create access key" to create an acess key
   • Save the Access key ID and the associated secret key
3. Install AWS CLI v2
   • You can check whether you have AWS CLI installed by running aws --version
   • Installation guides for various operating systems can be found here
4. Configure AWS credentials on your machine
   • You can do this by running aws configure
   • You will be asked to provide your Access Key ID and Secret Key from step 2, as well as the default region name and default output format
     • It is critical that you set the default region to us-east-1
   • This will create a credentials file at ~/.aws/credentials on Linux or macOS, or at C:\Users\USERNAME\.aws\credentials on Windows. The tests read this file in order to interact with lambdas that have been created in the Solo.io organization

Documentation

Index

• Constants
• type TestContext
  • func (c *TestContext) AfterEach()
  • func (c *TestContext) BeforeEach()
  • func (c *TestContext) Ctx() context.Context
  • func (c *TestContext) EnvoyInstance() *services.EnvoyInstance
  • func (c *TestContext) JustAfterEach()
  • func (c *TestContext) JustBeforeEach()
  • func (c *TestContext) PatchDefaultGateway(mutator func(gw *v1.Gateway) *v1.Gateway)
  • func (c *TestContext) PatchDefaultUpstream(mutator func(us *gloov1.Upstream) *gloov1.Upstream)
  • func (c *TestContext) PatchDefaultVirtualService(mutator func(vs *v1.VirtualService) *v1.VirtualService)
  • func (c *TestContext) ReadDefaultProxy() (*gloov1.Proxy, error)
  • func (c *TestContext) ResourcesToCreate() *gloosnapshot.ApiSnapshot
  • func (c *TestContext) SetRunServices(services services.What)
  • func (c *TestContext) SetRunSettings(settings *gloov1.Settings)
  • func (c *TestContext) TestClients() services.TestClients
  • func (c *TestContext) TestUpstream() *v1helpers.TestUpstream
• type TestContextFactory
  • func (f *TestContextFactory) NewTestContext(testRequirements ...helpers.Requirement) *TestContext

Constants

const (

	// DefaultHost defines the Host header that should be used to route traffic to the
	// default VirtualService that the TestContext creates
	// To make our tests more explicit we define VirtualServices with an explicit set
	// of domains (which match the `Host` header of a request), and DefaultHost
	// is the domain we use by default
	DefaultHost = "test.com"
)

Types

type TestContext

type TestContext struct {
	// contains filtered or unexported fields
}

TestContext represents the aggregate set of configuration needed to run a single e2e test It is intended to remove some boilerplate setup/teardown of tests out of the test themselves to ensure that tests are easier to read and maintain since they only contain the resource changes that we are validating

func (*TestContext) AfterEach

func (c *TestContext) AfterEach()

func (*TestContext) BeforeEach

func (c *TestContext) BeforeEach()

func (*TestContext) Ctx

func (c *TestContext) Ctx() context.Context

Ctx returns the Context maintained by the TestContext The Context is cancelled during the AfterEach portion of tests

func (*TestContext) EnvoyInstance

func (c *TestContext) EnvoyInstance() *services.EnvoyInstance

EnvoyInstance returns the wrapper for the running instance of Envoy that this test is using It contains utility methods to easily inspect the live configuration and statistics for the instance

func (*TestContext) JustAfterEach

func (c *TestContext) JustAfterEach()

func (*TestContext) JustBeforeEach

func (c *TestContext) JustBeforeEach()

func (*TestContext) PatchDefaultGateway

func (c *TestContext) PatchDefaultGateway(mutator func(gw *v1.Gateway) *v1.Gateway)

PatchDefaultGateway mutates the existing Gateway generated by the TestContext

func (*TestContext) PatchDefaultUpstream

func (c *TestContext) PatchDefaultUpstream(mutator func(us *gloov1.Upstream) *gloov1.Upstream)

PatchDefaultUpstream mutates the existing Upstream generated by the TestContext

func (*TestContext) PatchDefaultVirtualService

func (c *TestContext) PatchDefaultVirtualService(mutator func(vs *v1.VirtualService) *v1.VirtualService)

PatchDefaultVirtualService mutates the existing VirtualService generated by the TestContext

func (*TestContext) ReadDefaultProxy

func (c *TestContext) ReadDefaultProxy() (*gloov1.Proxy, error)

ReadDefaultProxy returns the Proxy object that will be generated by the resources in the TestContext

func (*TestContext) ResourcesToCreate

func (c *TestContext) ResourcesToCreate() *gloosnapshot.ApiSnapshot

ResourcesToCreate returns the ApiSnapshot of resources the TestContext maintains This snapshot is what is written to storage during the JustBeforeEach portion We return a reference to the object, so that individual tests can modify the snapshot before we write it to storage

func (*TestContext) SetRunServices

func (c *TestContext) SetRunServices(services services.What)

SetRunServices can be used to modify the services (gloo, fds, uds) which will run for a test This should be called after the TestContext.BeforeEach (when the default services are applied) and before the TestContext.JustBeforeEach (when the services are run)

func (*TestContext) SetRunSettings

func (c *TestContext) SetRunSettings(settings *gloov1.Settings)

SetRunSettings can be used to modify the runtime Settings object for a test This should be called after the TestContext.BeforeEach (when the default settings are applied) and before the TestContext.JustBeforeEach (when the settings are consumed)

func (*TestContext) TestClients

func (c *TestContext) TestClients() services.TestClients

TestClients returns the set of resource clients that can be used to perform CRUD operations on resources used by these tests Instead of using the resource clients directly, we recommend placing resources on the ResourcesToCreate object, and letting the TestContext handle the lifecycle of those objects

func (*TestContext) TestUpstream

func (c *TestContext) TestUpstream() *v1helpers.TestUpstream

TestUpstream returns the TestUpstream object that the TestContext built A TestUpstream is used to run an echo server and define the Gloo Upstream object to route to it

type TestContextFactory

type TestContextFactory struct {
	EnvoyFactory *services.EnvoyFactory
}

func (*TestContextFactory) NewTestContext

func (f *TestContextFactory) NewTestContext(testRequirements ...helpers.Requirement) *TestContext