e2e

package
v1.15.0-beta19 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2023 License: Apache-2.0 Imports: 17 Imported by: 0

README

In Memory End-to-End tests

This directory contains end-to-end tests that do not require Kubernetes, and persist configuration in memory.

Note: All commands should be run from the root directory of the Gloo repository

Background

This is the most common type of end-to-end test, since it is the quickest to set up and easiest to debug. Additionally, since Gloo Edge may be run using various backing stores, these tests provide a single space to validate the translation of Gloo resources into Envoy resources, independent of where Gloo Edge is deployed. As a result, these test do not rely on Kubernetes, so if there is any Kubernetes behavior that needs to be tested, write a kubernetes end-to-end test instead.

How do the tests work?
  1. Run the Gloo controllers in goroutines
  2. Run Envoy either using a binary or docker container
  3. Apply Gloo resources using in-memory resource clients
  4. Execute requests against the Envoy proxy and confirm the expected response. This validates that the Gloo resources have been picked up by the controllers, were been translated correctly into Envoy configuration, the configuration was sent to the Envoy proxy, and the proxy behaves appropriately.
Example Test

We have an example test which outlines how these tests work. It also provides some examples for basic testing operations. If you are writing a new e2e test, we recommend that you review the example test first.

This was introduced in a pull request which includes other useful details about e2e test considerations.

CI

These tests are run by build-bot in Google Cloud as part of our CI pipeline.

If a test fails, you can retry it using the build-bot comment directives. If you do this, please make sure to include a link to the failed logs for debugging purposes.

Local Development

Setup

For these tests to run, we require that our gateway-proxy component be previously built as a docker image.

If you have not made local changes to the component, you can rely on a previously published image and no setup is required.

However, if you have made changes to the component, refer to the Envoyinit README for build instructions.

Run Tests

The run-tests make target runs ginkgo with a set of useful flags. The following environment variables can be configured for this target:

Name Default Description
ENVOY_IMAGE_TAG "" The tag of the gloo-envoy-wrapper-docker image built during setup
TEST_PKG "" The path to the package of the test suite you want to run
WAIT_ON_FAIL 0 Set to 1 to prevent Ginkgo from cleaning up the Gloo Edge installation in case of failure. Useful to exec into inspect resources created by the test. A command to resume the test run (and thus clean up resources) will be logged to the output.
INVALID_TEST_REQS fail The behavior for tests which depend on environment conditions that aren't satisfied. Options are skip, run, fail
SERVICE_LOG_LEVEL "" The log levels used for services. See "Controlling Log Verbosity of Services" below.
Controlling Log Verbosity of Services

Multiple services (Gloo, Envoy, Discovery) are executed in parallel to run these tests. By default, these services log at the info level. To change the log level of a service, set the SERVICE_LOG_LEVEL environment variable to a comma separated list of service:level pairs.

Options for services are:

  • gateway-proxy
  • gloo
  • uds
  • fds

Options for log levels are:

  • debug
  • info
  • warn
  • error

For example, to set the log level of the Gloo service to debug and the Envoy service to error, you would set:

SERVICE_LOG_LEVEL=gloo:debug,gateway-proxy:error TEST_PKG=./test/e2e/... make test

If the same service has multiple log levels specified, we will log a warning and the last one defined will be used.

Controlling Log Verbosity of Ginkgo Runner

Ginkgo has 4 verbosity settings, whose details can be found in the Ginkgo docs

To control these settings, you must pass the flags using the GINKGO_USER_FLAGS environment variable.

For example, to set the Ginkgo runner to very verbose mode, you would set:

GINKGO_USER_FLAGS=-vv TEST_PKG=./test/e2e/... make test
Using Recently Published Image (Most Common)

This is the most common pattern. If you did not make changes to the gateway-proxy component, and do not specify an ENVOY_IMAGE_TAG our tests will identify the most recently published image (for your LTS branch) and use that version.

TEST_PKG=./test/e2e/... make test
Using Previously Published Image

If you want to specify a particular version that was previously published, you can also do that by specifying the ENVOY_IMAGE_TAG.

ENVOY_IMAGE_TAG=1.13.0 TEST_PKG=./test/e2e/... make test
Using Locally Built Image

If you have made changes to the component, you will have had to rebuild the image locally (see setup tests). After you rebuild the image, you need to supply the tag of that image when running the tests:

ENVOY_IMAGE_TAG=0.0.1-local TEST_PKG=./test/e2e/... make test
Running Tests in Parallel

It is possible to run tests in parallel locally, however it is not recommended, because debugging failures becomes more difficult. If you do want to run tests in parallel, you can do so by passing in the relevant GINKGO_USER_FLAGS values:

GINKGO_USER_FLAGS=-procs=4 TEST_PKG=./test/e2e/... make test

Note: When using Docker to run Envoy, we have seen occasional failures: Error response from daemon: dial unix docker.raw.sock: connect: connection refused

Debugging Tests
Use WAIT_ON_FAIL

When Ginkgo encounters a test failure it will attempt to clean up relevant resources, which includes stopping the running instance of Envoy, preventing the developer from inspecting the state of the Envoy instance for useful clues.

To avoid this clean up, run the test(s) with WAIT_ON_FAIL=1. When the test fails, it will halt execution, allowing you to inspect the state of the Envoy instance.

Once halted, use docker ps to determine the admin port for the Envoy instance, and follow the recommendations for debugging Envoy, specifically the parts around interacting with the Administration interface.

Use INVALID_TEST_REQS

Certain test require environmental conditions to be true for them to succeed. For example, certain tests will only run on a Linux machine.

By setting INVALID_TEST_REQS=skip, you can run all tests locally, and any tests which will not run in your local environment will be skipped. The default behavior is that they fail.

Additional Notes

Notes on EC2 tests

Note: these instructions are out of date, and require updating

  • set up your ec2 instance
    • download a simple echo app
    • make the app executable
    • run it in the background
wget https://mitch-solo-public.s3.amazonaws.com/echoapp2
chmod +x echoapp2
sudo ./echoapp2 --port 80 &
Notes on AWS Lambda Tests (test/e2e/aws_test.go)

In addition to the configuration steps provided above, you will need to do the following to run the AWS Lambda Tests locally:

  1. Obtain an AWS IAM User account that is part of the Solo.io organization
  2. Create an AWS access key
    • Sign into the AWS console with the account created during step 1
    • Hover over your username at the top right of the page. Click on "My Security Credentials"
    • In the section titled "AWS IAM credentials", click "Create access key" to create an acess key
    • Save the Access key ID and the associated secret key
  3. Install AWS CLI v2
    • You can check whether you have AWS CLI installed by running aws --version
    • Installation guides for various operating systems can be found here
  4. Configure AWS credentials on your machine
    • You can do this by running aws configure
    • You will be asked to provide your Access Key ID and Secret Key from step 2, as well as the default region name and default output format
      • It is critical that you set the default region to us-east-1
    • This will create a credentials file at ~/.aws/credentials on Linux or macOS, or at C:\Users\USERNAME\.aws\credentials on Windows. The tests read this file in order to interact with lambdas that have been created in the Solo.io organization

Documentation

Index

Constants

View Source 
const (
	WriteNamespace            = defaults.GlooSystem
	DefaultVirtualServiceName = "vs-test"
	DefaultRouteName          = "route-test"
	DefaultGatewayName        = gatewaydefaults.GatewayProxyName
	DefaultProxyName          = gatewaydefaults.GatewayProxyName
	// DefaultHost defines the Host header that should be used to route traffic to the
	// default VirtualService that the TestContext creates
	// To make our tests more explicit we define VirtualServices with an explicit set
	// of domains (which match the `Host` header of a request), and DefaultHost
	// is the domain we use by default
	DefaultHost = "test.com"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type TestContext 

type TestContext struct {
	// contains filtered or unexported fields
}

TestContext represents the aggregate set of configuration needed to run a single e2e test It is intended to remove some boilerplate setup/teardown of tests out of the test themselves to ensure that tests are easier to read and maintain since they only contain the resource changes that we are validating

func (*TestContext) AfterEach 

func (c *TestContext) AfterEach()

func (*TestContext) BeforeEach 

func (c *TestContext) BeforeEach()

func (*TestContext) Ctx 

func (c *TestContext) Ctx() context.Context

Ctx returns the Context maintained by the TestContext The Context is cancelled during the AfterEach portion of tests

func (*TestContext) EnvoyInstance 

func (c *TestContext) EnvoyInstance() *envoy.Instance

EnvoyInstance returns the wrapper for the running instance of Envoy that this test is using It contains utility methods to easily inspect the live configuration and statistics for the instance

func (*TestContext) EventuallyProxyAccepted 

func (c *TestContext) EventuallyProxyAccepted()

For tests that rely on changing an existing configuration.

func (*TestContext) GetHttpRequestBuilder 

func (c *TestContext) GetHttpRequestBuilder() *testutils.HttpRequestBuilder

GetHttpRequestBuilder returns an HttpRequestBuilder to easily build http requests used in e2e tests

func (*TestContext) GetHttpsRequestBuilder 

func (c *TestContext) GetHttpsRequestBuilder() *testutils.HttpRequestBuilder

GetHttpsRequestBuilder returns an HttpRequestBuilder to easily build https requests used in e2e tests

func (*TestContext) JustAfterEach 

func (c *TestContext) JustAfterEach()

func (*TestContext) JustBeforeEach 

func (c *TestContext) JustBeforeEach()

func (*TestContext) PatchDefaultGateway 

func (c *TestContext) PatchDefaultGateway(mutator func(*v1.Gateway) *v1.Gateway)

PatchDefaultGateway mutates the existing Gateway generated by the TestContext

func (*TestContext) PatchDefaultUpstream 

func (c *TestContext) PatchDefaultUpstream(mutator func(*gloov1.Upstream) *gloov1.Upstream)

PatchDefaultUpstream mutates the existing Upstream generated by the TestContext

func (*TestContext) PatchDefaultVirtualService 

func (c *TestContext) PatchDefaultVirtualService(mutator func(*v1.VirtualService) *v1.VirtualService)

PatchDefaultVirtualService mutates the existing VirtualService generated by the TestContext

func (*TestContext) ReadDefaultProxy 

func (c *TestContext) ReadDefaultProxy() (*gloov1.Proxy, error)

ReadDefaultProxy returns the Proxy object that will be generated by the resources in the TestContext

func (*TestContext) ResourcesToCreate 

func (c *TestContext) ResourcesToCreate() *gloosnapshot.ApiSnapshot

ResourcesToCreate returns the ApiSnapshot of resources the TestContext maintains This snapshot is what is written to storage during the JustBeforeEach portion We return a reference to the object, so that individual tests can modify the snapshot before we write it to storage

func (*TestContext) SetRunServices 

func (c *TestContext) SetRunServices(services services.What)

SetRunServices can be used to modify the services (gloo, fds, uds) which will run for a test This should be called after the TestContext.BeforeEach (when the default services are applied) and before the TestContext.JustBeforeEach (when the services are run)

func (*TestContext) SetRunSettings 

func (c *TestContext) SetRunSettings(settings *gloov1.Settings)

SetRunSettings can be used to modify the runtime Settings object for a test This should be called after the TestContext.BeforeEach (when the default settings are applied) and before the TestContext.JustBeforeEach (when the settings are consumed)

func (*TestContext) SetUpstreamGenerator 

func (c *TestContext) SetUpstreamGenerator(generator func(ctx context.Context, addr string) *v1helpers.TestUpstream)

SetUpstreamGenerator Use a different function to create a test upstream call before testContext.BeforeEach() Used for example with helpers.NewTestGrpcUpstream which has the side effect of also starting a grpc service

func (*TestContext) TestClients 

func (c *TestContext) TestClients() services.TestClients

TestClients returns the set of resource clients that can be used to perform CRUD operations on resources used by these tests Instead of using the resource clients directly, we recommend placing resources on the ResourcesToCreate object, and letting the TestContext handle the lifecycle of those objects

func (*TestContext) TestUpstream 

func (c *TestContext) TestUpstream() *v1helpers.TestUpstream

TestUpstream returns the TestUpstream object that the TestContext built A TestUpstream is used to run an echo server and define the Gloo Upstream object to route to it

type TestContextFactory 

type TestContextFactory struct {
	EnvoyFactory  envoy.Factory
	VaultFactory  *services.VaultFactory
	ConsulFactory *services.ConsulFactory
}

func (*TestContextFactory) NewTestContext 

func (f *TestContextFactory) NewTestContext(testRequirements ...testutils.Requirement) *TestContext

func (*TestContextFactory) NewTestContextWithConsul 

func (f *TestContextFactory) NewTestContextWithConsul(testRequirements ...testutils.Requirement) *TestContextWithConsul

func (*TestContextFactory) NewTestContextWithVault 

func (f *TestContextFactory) NewTestContextWithVault(testRequirements ...testutils.Requirement) *TestContextWithVault

type TestContextWithConsul 

type TestContextWithConsul struct {
	*TestContext
	// contains filtered or unexported fields
}

TestContextWithConsul represents the aggregate set of configuration needed to run a single e2e test using Consul as a service registry to route traffic to. This is used rarely in tests, so we intentionally try to separate the consul logic from the core TestContext to avoid adding complexity

func (*TestContextWithConsul) ConsulInstance 

func (c *TestContextWithConsul) ConsulInstance() *services.ConsulInstance

ConsulInstance returns the wrapper for the running instance of Vault that this test is using

func (*TestContextWithConsul) RunConsul 

func (c *TestContextWithConsul) RunConsul()

RunConsul starts running the ConsulInstance and blocks until it has successfully started

type TestContextWithVault 

type TestContextWithVault struct {
	*TestContext
	// contains filtered or unexported fields
}

TestContextWithVault represents the aggregate set of configuration needed to run a single e2e test using Vault as a secret store. This is used rarely in tests, so we intentionally try to separate the vault logic from the core TestContext to avoid adding complexity

func (*TestContextWithVault) RunVault 

func (v *TestContextWithVault) RunVault()

RunVault starts running the VaultInstance and blocks until it has successfully started

func (*TestContextWithVault) VaultInstance 

func (v *TestContextWithVault) VaultInstance() *services.VaultInstance

VaultInstance returns the wrapper for the running instance of Vault that this test is using

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.