kube2e

package
v1.15.31 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 6, 2024 License: Apache-2.0 Imports: 35 Imported by: 0

README

Kubernetes End-to-End tests

This directory contains tests that install each of the 3 Gloo Edge flavors (gateway, ingress, and knative) in a Kubernetes cluster, and run end-to-end tests against them.

Note: All commands should be run from the root directory of the Gloo repository

Background

Kubernetes may be relied on for scheduling, persistence or security. These tests validate that Gloo Edge can successfully operate within a Kubernetes cluster.

How do the tests work?
  1. Install Gloo Edge in Kubernetes cluster using Helm
  2. Apply Gloo resources using Kubernetes resource clients
  3. Execute requests against the Envoy proxy and confirm the expected response. This validates that the Gloo resources have been picked up by the controllers, were been translated correctly into Envoy configuration, the configuration was sent to the Envoy proxy, and the proxy behaves appropriately.

CI

These tests are run by a GitHub action as part of our CI pipeline.

If a test fails, you can retry it from a browser window. If you do this, please make sure to comment on the Pull Request with a link to the failed logs for debugging purposes.

Nightly runs

Tests are also run on a schedule via another GitHub action. The nightly tests use the latest release - specified with the RELEASED_VERSION environment variable.

Extra considerations for running from released builds

The GetTestHelper util method handles installing gloo from either a local or released build. When testing released builds, tests that interact directly with the helm chart need to download the chart using the version stored in testHelper.GetChartVersion()

Adding new tests

The list of tests to run during CI and nightly builds is provided in kube-e2e-test-type matrices in the github workflows. A new test can be added to one or both lists of tests.

Local Development

Setup (Previously Released Assets)

It is possible to run these tests against a previously released version of Gloo Edge. This is useful for testing a release candidate, or a nightly build.

There is no setup required for this option, as the test suite will download the helm chart archive and glooctl binary from the specified release. You will use the RELEASED_VERSION environment variable when running the tests. See the variable definition for more details.

Setup (Locally Build Assets)

For these tests to run, we require the following conditions:

  • Gloo Edge Helm chart archive be present in the _test folder,
  • glooctl be built in the_output folder
  • kind cluster set up and loaded with the images to be installed by the helm chart
Use the CI Install Script

ci/kind/setup-kind.sh gets run in CI to setup the test environment for the above requirements. It accepts a number of environment variables, to control the creation of a kind cluster and deployment of Gloo resources to that kind cluster.

Name Default Description
CLUSTER_NAME kind The name of the cluster that will be generated
CLUSTER_NODE_VERSION v1.25.3 The version of the Node Docker image to use for booting the cluster
VERSION 1.0.0-ci The version used to tag Gloo images that are deployed to the cluster
KUBE2E_TESTS gateway Name of the test suite to be run. Options: 'gateway', 'gloo', 'ingress', 'helm', 'gloomtls', 'glooctl', 'upgrade'
SKIP_DOCKER false Skip building docker images (used when testing a release version)

Example:

CLUSTER_NAME=solo-test-cluster CLUSTER_NODE_VERSION=v1.25.3 VERSION=v1.0.0-solo-test ci/kind/setup-kind.sh
Verify Your Setup

Before running your tests, it's worthwhile to verify that a cluster was created, and the proper images have been loaded.

CLUSTER_NAME=solo-test-cluster make kind-list-images

You should see the list of images in the cluster, including the ones you just uploaded

Common Setup Errors

Error: validation: chart.metadata.version "solo" is invalid
In newer versions of helm (>3.5), the version used to build the helm chart (ie the VERSION env variable), needs to respect semantic versioning. This error implies that the version provided does not.

Prepend a valid semver to avoid the error. (ie kind can become 0.0.0-kind1)

Run Tests

To run the regression tests, your kubeconfig file must point to a running Kubernetes cluster. kubectl config current-context should run kind-<CLUSTER_NAME>

Use the Make Target

Use the same command that CI relies on:

KUBE2E_TESTS=<test-to-run> make run-kube-e2e-tests
Test Environment Variables

The below table contains the environment variables that can be used to configure the test execution.

Name Default Description
KUBE2E_TESTS gateway Name of the test suite to be run. Options: 'gateway', 'gloo', 'ingress', 'helm', 'gloomtls', 'glooctl', 'upgrade'
DEBUG 0 Set to 1 for debug log output
WAIT_ON_FAIL 0 Set to 1 to prevent Ginkgo from cleaning up the Gloo Edge installation in case of failure. Useful to exec into inspect resources created by the test. A command to resume the test run (and thus clean up resources) will be logged to the output.
TEAR_DOWN false Set to true to uninstall Gloo after the test suite completes
RELEASED_VERSION '' Used by nightlies to tests a specific released version. 'LATEST' will find the latest release
Common Test Errors

getting Helm chart version: expected a single entry with name [gloo], found: 5
The test helm charts are written to the _test directory, with the index.yaml file containing references to all available charts. The tests require that this file contain only 1 entry. Delete the other entries manually, or run make clean to delete this folder entirely, and then re-build the test helm chart.

Documentation

Index

Constants

View Source 
const SimpleTestRunnerHttpResponse = `` /* 851-byte string literal not displayed */
View Source 
const SimpleTestRunnerHttpResponseArm = `` /* 807-byte string literal not displayed */
View Source 
const (
	// UniqueTestResourceLabel can be assigned to the resources used by kube2e tests
	// This unique label per test run ensures that the generated snapshot is different on subsequent runs
	// We have previously seen flakes where a resource is deleted and re-created with the same hash and thus
	// the emitter can miss the update
	UniqueTestResourceLabel = "gloo-kube2e-test-id"
)

Variables

This section is empty.

Functions

func EventuallyReachesConsistentState added in v1.5.0 

func EventuallyReachesConsistentState(installNamespace string)

func GetHttpEchoImage added in v1.9.25 

func GetHttpEchoImage() string

func GetSimpleTestRunnerHttpResponse added in v1.9.25 

func GetSimpleTestRunnerHttpResponse() string

https://github.com/solo-io/gloo/issues/4043#issuecomment-772706604 We should move tests away from using the testrunner, and instead depend on EphemeralContainers. The default response changed in later kube versions, which caused this value to change. Ideally the test utilities used by Gloo are maintained in the Gloo repo, so I opted to move this constant here. This response is given by the testrunner when the SimpleServer is started

func GetTestHelper added in v1.12.48 

func GetTestHelper(ctx context.Context, namespace string) (*helper.SoloTestHelper, error)

func GetTestReleasedVersion added in v1.12.48 

func GetTestReleasedVersion(ctx context.Context, repoName string) string

For nightly runs, we want to install a released version rather than using a locally built chart To do this, set the environment variable RELEASED_VERSION with either a version name or "LATEST" to get the last release

func GlooctlCheckEventuallyHealthy added in v1.3.16 

func GlooctlCheckEventuallyHealthy(offset int, testHelper *helper.SoloTestHelper, timeoutInterval string)

GlooctlCheckEventuallyHealthy will run up until proved timeoutInterval or until gloo is reported as healthy

func MustKubeClient added in v0.11.2 

func MustKubeClient() kubernetes.Interface

func ToFile added in v1.12.21 

func ToFile(content string) string

func UpdateAlwaysAcceptSetting added in v1.5.0 

func UpdateAlwaysAcceptSetting(ctx context.Context, alwaysAccept bool, installNamespace string)

enable/disable strict validation

func UpdateDisableTransformationValidationSetting added in v1.8.3 

func UpdateDisableTransformationValidationSetting(ctx context.Context, shouldDisable bool, installNamespace string)

func UpdateReplaceInvalidRoutes added in v1.8.3 

func UpdateReplaceInvalidRoutes(ctx context.Context, replaceInvalidRoutes bool, installNamespace string)

func UpdateRestEdsSetting added in v1.7.0 

func UpdateRestEdsSetting(ctx context.Context, enableRestEds bool, installNamespace string)

func UpdateSettings added in v1.5.0 

func UpdateSettings(ctx context.Context, updateSettings func(settings *v1.Settings), installNamespace string)

func UpdateSettingsWithPropagationDelay added in v1.8.3 

func UpdateSettingsWithPropagationDelay(updateSettings func(settings *v1.Settings), waitForSettingsToPropagate func(), ctx context.Context, installNamespace string)

Types

type KubeResourceClientSet added in v1.12.21 

type KubeResourceClientSet struct {
	// contains filtered or unexported fields
}

KubeResourceClientSet is a set of ResourceClients

func NewDefaultKubeResourceClientSet added in v1.15.0 

func NewDefaultKubeResourceClientSet(ctx context.Context) (*KubeResourceClientSet, error)

func NewKubeResourceClientSet added in v1.9.25 

func NewKubeResourceClientSet(ctx context.Context, cfg *rest.Config) (*KubeResourceClientSet, error)

func (KubeResourceClientSet) ArtifactClient added in v1.12.21 

func (k KubeResourceClientSet) ArtifactClient() gloov1.ArtifactClient

func (KubeResourceClientSet) AuthConfigClient added in v1.14.0 

func (k KubeResourceClientSet) AuthConfigClient() extauthv1.AuthConfigClient

func (KubeResourceClientSet) GatewayClient added in v1.12.21 

func (k KubeResourceClientSet) GatewayClient() gatewayv1.GatewayClient

func (KubeResourceClientSet) HttpGatewayClient added in v1.12.21 

func (k KubeResourceClientSet) HttpGatewayClient() gatewayv1.MatchableHttpGatewayClient

func (KubeResourceClientSet) KubeClients added in v1.12.21 

func (k KubeResourceClientSet) KubeClients() *kubernetes.Clientset

func (KubeResourceClientSet) ProxyClient added in v1.12.21 

func (k KubeResourceClientSet) ProxyClient() gloov1.ProxyClient

func (KubeResourceClientSet) RateLimitConfigClient added in v1.12.21 

func (k KubeResourceClientSet) RateLimitConfigClient() externalrl.RateLimitConfigClient

func (KubeResourceClientSet) RouteOptionClient added in v1.12.21 

func (k KubeResourceClientSet) RouteOptionClient() gatewayv1.RouteOptionClient

func (KubeResourceClientSet) RouteTableClient added in v1.12.21 

func (k KubeResourceClientSet) RouteTableClient() gatewayv1.RouteTableClient

func (KubeResourceClientSet) SecretClient added in v1.12.21 

func (k KubeResourceClientSet) SecretClient() gloov1.SecretClient

func (KubeResourceClientSet) ServiceClient added in v1.12.21 

func (k KubeResourceClientSet) ServiceClient() skkube.ServiceClient

func (KubeResourceClientSet) SettingsClient added in v1.12.21 

func (k KubeResourceClientSet) SettingsClient() gloov1.SettingsClient

func (KubeResourceClientSet) TcpGatewayClient added in v1.14.4 

func (k KubeResourceClientSet) TcpGatewayClient() gatewayv1.MatchableTcpGatewayClient

func (KubeResourceClientSet) UpstreamClient added in v1.12.21 

func (k KubeResourceClientSet) UpstreamClient() gloov1.UpstreamClient

func (KubeResourceClientSet) UpstreamGroupClient added in v1.12.21 

func (k KubeResourceClientSet) UpstreamGroupClient() gloov1.UpstreamGroupClient

func (KubeResourceClientSet) VirtualHostOptionClient added in v1.12.21 

func (k KubeResourceClientSet) VirtualHostOptionClient() gatewayv1.VirtualHostOptionClient

func (KubeResourceClientSet) VirtualServiceClient added in v1.12.21 

func (k KubeResourceClientSet) VirtualServiceClient() gatewayv1.VirtualServiceClient

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.