e2e

Documentation

Overview

Package e2e and its sub-packages define go e2e tests for Nomos.

Running the Tests

A) With make, rebuilding the Nomos image:

$ make go-e2e-test

B) With make, without rebuilding the image:

$ make go-e2e-test-nobuild

C) Directly, without rebuilding:

$ go test ./e2e/... --e2e

Running the tests directly requires running scripts.docker-registry.sh as a one-time setup step.

You can use all of the normal `go test` flags. The `--e2e` is required or else the e2e tests won't run. This lets you run go test ./... to just run unit/integration tests.

Debugging

Use --debug to use the debug mode for tests. In this mode, on failure the test does not destroy the kind cluster and delete the temporary directory. Instead, it prints out where the temporary directory is and how to connect to the kind cluster.

The temporary directory includes:

1. All manifests used to install ConfigSync
2. The private/public SSH keys to connect to git-server
3. The local repository(ies), already configured to talk to git server. Just remember to port-forward to the git-server Pod if you want to read from/write to it.

If you want to stop the test at any time, just use t.FailNow().

Package e2e defines e2e-test-specific imports and flags for use in e2e testing.

Index

• Constants
• Variables
• func EnableParallel(t *testing.T)
• func NumParallel() int
• func RunInParallel() bool

Constants

const (
	// Kind indicates creating a Kind cluster for testing.
	Kind = "kind"
	// GKE indicates using an existing GKE cluster for testing.
	GKE = "gke"
	// Kubeconfig provides the context via KUBECONFIG for testing.
	Kubeconfig = "kube-config"
	// DefaultGKEChannel is the default GKE release channel to use when creating a cluster
	DefaultGKEChannel = "regular"
	// DefaultGKEMachineType is the default GKE machine type to use when creating a cluster
	DefaultGKEMachineType = "n2-standard-8"
	// DefaultGKENumNodes is the default number of nodes to use when creating a GKE cluster
	DefaultGKENumNodes = 1
	// DefaultGKEDiskType is the default disk type to use when creating a cluster
	DefaultGKEDiskType = "pd-ssd"
	// DefaultGKEDiskSize is the default disk size to use when creating a GKE cluster
	DefaultGKEDiskSize = "50Gb"
	// CreateClustersEnabled indicates that clusters should be created and error
	// if the cluster already exists.
	CreateClustersEnabled = "true"
	// CreateClustersLazy indicates to use clusters that exist and create them
	// if they don't
	CreateClustersLazy = "lazy"
	// CreateClustersDisabled indicates to not create clusters
	CreateClustersDisabled = "false"
	// DestroyClustersEnabled indicates to destroy clusters
	DestroyClustersEnabled = "true"
	// DestroyClustersAuto indicates to only destroy clusters if they were created
	// by the test framework
	DestroyClustersAuto = "auto"
	// DestroyClustersDisabled indicates to not destroy clusters
	DestroyClustersDisabled = "false"
)

const (
	// Local indicates using a local git-test-server.
	Local = "local"
	// Bitbucket indicates using Bitbucket to host the repositories.
	Bitbucket = "bitbucket"
	// Github indicates using GitHub to host the repositories.
	Github = "github"
	// GitLab indicates using GitLab to host the repositories.
	GitLab = "gitlab"
	// CSR indicates using Google Cloud Source Repositories to host the repositories.
	CSR = "csr"
	// ArtifactRegistry indicates using Google Artifact Registry to host the repositories.
	ArtifactRegistry = "gar"
)

const DefaultImagePrefix = "localhost:5000"

DefaultImagePrefix points to the local docker registry.

Variables

var ClusterNames = newStringListFlag("cluster-names", util.EnvList("E2E_CLUSTER_NAMES", nil),
	"List of cluster names to use for the tests. If specified without create-clusters, assumes the clusters were pre-provisioned.")

ClusterNames is a list of cluster names to use for the tests. If specified without create-clusters, assumes the clusters were pre-provisioned.

var ClusterPrefix = flag.String("cluster-prefix", util.EnvString("E2E_CLUSTER_PREFIX", ""),
	"Prefix to use when naming clusters. An index is appended to the prefix with the pattern '<prefix>-<index>', starting at 0. If unset, defaults to cs-e2e-<UNIX_TIME>. cluster-names takes precedence, if provided.")

ClusterPrefix is the prefix to use when naming clusters. An index is appended to the prefix with the pattern '<prefix>-<index>', starting at 0. If unset, defaults to cs-e2e-<UNIX_TIME>. cluster-names takes precedence, if provided.

var CreateClusters = flag.String("create-clusters", util.EnvString("E2E_CREATE_CLUSTERS", CreateClustersDisabled),
	fmt.Sprintf("Whether to create clusters, otherwise will assume pre-provisioned clusters. Allowed values: [%s]",
		strings.Join(CreateClustersAllowedValues, ", ")))

CreateClusters indicates the test framework should create clusters.

var CreateClustersAllowedValues = []string{CreateClustersEnabled, CreateClustersLazy, CreateClustersDisabled}

CreateClustersAllowedValues is a list of allowed values for the create-clusters parameter

var Debug = flag.Bool("debug", false,
	"If true, do not destroy cluster and clean up temporary directory after test.")

Debug enables running the test in debug mode. In debug mode:

1. Test execution immediately stops on a call to t.Fatal.
2. The test prints the absolute path to the test temporary directory, and not delete it.
3. The test prints out how to connect to the kind cluster.

var DestroyClusters = flag.String("destroy-clusters", util.EnvString("E2E_DESTROY_CLUSTERS", DestroyClustersAuto),
	fmt.Sprintf("Whether to destroy clusters that were created by the test suite after the tests. Allowed Values: [%s]",
		strings.Join(DestroyClustersAllowedValues, ", ")))

DestroyClusters indicates whether to destroy clusters that were created by the test suite after the tests.

var DestroyClustersAllowedValues = []string{DestroyClustersEnabled, DestroyClustersAuto, DestroyClustersDisabled}

DestroyClustersAllowedValues is a list of allowed values for the destroy-clusters parameter

var E2E = flag.Bool("e2e", false,
	"If true, run end-to-end tests.")

E2E enables running end-to-end tests.

var GCPCluster = flag.String("gcp-cluster", util.EnvString("GCP_CLUSTER", ""),
	"GCP Cluster (GKE) to use when running tests. Defaults to GCP_CLUSTER env var.")

GCPCluster is the GCP Cluster (GKE) to use when running tests.

var GCPNetwork = flag.String("gcp-network", util.EnvString("GCP_NETWORK", ""),
	"GCP Network to use when creating GKE clusters. Defaults to GCP_SUBNETWORK env var,")

GCPNetwork is the GCP network to use when creating GKE clusters.

var GCPProject = flag.String("gcp-project", util.EnvString("GCP_PROJECT", ""),
	"GCP Project to use when running tests. Defaults to GCP_PROJECT env var.")

GCPProject is the GCP project to use when running tests.

var GCPRegion = flag.String("gcp-region", util.EnvString("GCP_REGION", ""),
	"GCP Region to use when running tests. Only one of gcp-region and gcp-zone must be set. Defaults to GCP_REGION env var.")

GCPRegion is the GCP Region to use when running tests.

var GCPSubNetwork = flag.String("gcp-subnetwork", util.EnvString("GCP_SUBNETWORK", ""),
	"GCP Subnetwork to use when creating clusters. Defaults to GCP_SUBNETWORK env var.")

GCPSubNetwork is the GCP subnetwork to use when creating GKE clusters.

var GCPZone = flag.String("gcp-zone", util.EnvString("GCP_ZONE", ""),
	"GCP Zone to use when running tests. Only one of gcp-region and gcp-zone must be set. Defaults to GCP_ZONE env var.")

GCPZone is the GCP Zone to use when running tests.

var GKEAutopilot = flag.Bool("gke-autopilot", util.EnvBool("GKE_AUTOPILOT", false),
	"Whether to create GKE clusters with autopilot enabled.")

GKEAutopilot indicates whether to enable autopilot when creating GKE clusters.

var GKEClusterVersion = flag.String("gke-cluster-version", util.EnvString("GKE_CLUSTER_VERSION", ""),
	"GKE cluster version to use when creating GKE clusters. Defaults to GKE_CLUSTER_VERSION env var.")

GKEClusterVersion is the GKE cluster version to use when creating GKE clusters.

var GKEDiskSize = flag.String("gke-disk-size", util.EnvString("GKE_DISK_SIZE", DefaultGKEDiskSize),
	"GKE disk size to use when creating GKE clusters. Defaults to GKE_DISK_SIZE env var.")

GKEDiskSize is the GKE disk size to use when creating GKE clusters.

var GKEDiskType = flag.String("gke-disk-type", util.EnvString("GKE_DISK_TYPE", DefaultGKEDiskType),
	"GKE disk type to use when creating GKE clusters. Defaults to GKE_DISK_TYPE env var.")

GKEDiskType is the GKE disk type to use when creating GKE clusters.

var GKEMachineType = flag.String("gke-machine-type", util.EnvString("GKE_MACHINE_TYPE", DefaultGKEMachineType),
	"GKE machine type to use when creating GKE clusters. Defaults to GKE_MACHINE_TYPE env var.")

GKEMachineType is the GKE machine type to use when creating GKE clusters.

var GKENumNodes = flag.Int("gke-num-nodes", util.EnvInt("GKE_NUM_NODES", DefaultGKENumNodes),
	"Number of node to use when creating GKE clusters. Defaults to GKE_NUM_NODES env var.")

GKENumNodes is the number of nodes to use when creating GKE clusters.

var GKEReleaseChannel = flag.String("gke-release-channel", util.EnvString("GKE_RELEASE_CHANNEL", DefaultGKEChannel),
	"GKE release channel to use when creating GKE clusters. Defaults to GKE_RELEASE_CHANNEL env var.")

GKEReleaseChannel is the GKE release channel to use when creating GKE clusters.

var GceNode = flag.Bool("gcenode", false,
	"If true, run test with 'gcenode' auth type.")

GceNode enables running the e2e tests for 'gcenode' auth type

var GitProvider = newStringEnum("git-provider", util.EnvString("E2E_GIT_PROVIDER", Local),
	"The git provider that hosts the Git repositories. Defaults to Local.",
	[]string{Local, Bitbucket, GitLab, CSR})

GitProvider is the provider that hosts the Git repositories.

var HelmProvider = newStringEnum("helm-provider", util.EnvString("E2E_HELM_PROVIDER", Local),
	"The registry provider that hosts the helm repositories. Defaults to Local.",
	[]string{Local, ArtifactRegistry})

HelmProvider is the provider that hosts the helm repositories.

var KCC = flag.Bool("kcc", false,
	"If true, run kcc tests.")

KCC enables running the e2e tests for kcc resources.

var KubernetesVersion = flag.String("kubernetes-version", "1.23",
	"The version of Kubernetes to create")

KubernetesVersion is the version of Kubernetes to test against. Only has effect when testing against test-created Kind clusters.

var Manual = flag.Bool("manual", false,
	"Specify that the test is being run manually.")

Manual indicates the test is being run manually. Some tests are not yet safe to be run automatically.

var NumClusters = flag.Int("num-clusters", util.EnvInt("E2E_NUM_CLUSTERS", 1),
	"Number of parallel test threads to run. Also dictates the number of clusters which will be created in parallel. Overrides the -test.parallel flag.")

NumClusters is the number of clusters to run tests on. Each cluster only has a single test running on it at a given time. The number of clusters equals the number of test threads which can run concurrently.

var OCIProvider = newStringEnum("oci-provider", util.EnvString("E2E_OCI_PROVIDER", Local),
	"The registry provider that hosts the OCI repositories. Defaults to Local.",
	[]string{Local, ArtifactRegistry})

OCIProvider is the provider that hosts the OCI repositories.

var ShareTestEnv = flag.Bool("share-test-env", false,
	"Specify that the test is using a shared test environment instead of fresh installation per test case.")

ShareTestEnv indicates whether to share the test env for all test cases. If it is true, we only install nomos once before all tests and tear it down until all tests complete.

var Stress = flag.Bool("stress", false,
	"If true, run stress tests.")

Stress enables running of stress tests.

var TestCluster = flag.String("test-cluster", Kind,
	fmt.Sprintf("The cluster config used for testing. Allowed values are: %s and %s. "+
		"If --test-cluster=%s, create a Kind cluster. Otherwise use the GKE context specified in %s.",
		GKE, Kind, Kind, Kubeconfig))

TestCluster specifies the cluster config used for testing.

var TestFeatures = flag.String("test-features", "",
	"A list of features to run, separated by comma. Defaults to empty, which should run all tests.")

TestFeatures is the list of features to run.

var Usage = flag.Bool("usage", false, "Print usage and exit.")

Usage indicates to print usage and exit. This is a workaround for the builtin help command of `go test`

Functions

func EnableParallel

func EnableParallel(t *testing.T)

EnableParallel allows parallel execution of test functions that call t.Parallel if test.parallel is greater than 1.

func NumParallel

func NumParallel() int

NumParallel returns the number of parallel test threads

func RunInParallel

func RunInParallel() bool

RunInParallel indicates whether the test is running in parallel.