e2e

README

e2e tests: Invoke gcloud and the wrapper binaries using their public APIs.

Overview

Each directory adjacent to this README is a test suite. At the time of writing, there are four test suites:

gce_image_import_export
gce_ovf_export
gce_ovf_import
gce_windows_upgrade

We build, deploy, and invoke the e2e tests using Docker images, with one image per test suite. The Docker image contains:

1. Tests, as a compiled go binary
2. One or more wrappers, as a compiled go binary.
3. gcloud

The Dockerfiles that generate these images are in the root of this repository. At the time of writing, they are:

gce_image_import_export_tests.Dockerfile
gce_ovf_export_tests.Dockerfile
gce_ovf_import_tests.Dockerfile
gce_windows_upgrade_tests.Dockerfile

Quick Run

To run e2e tests, we recommend using the script run_e2e.sh from the root of this repository. The header of that file contains usage information.

Details about building

Build e2e Dockerfiles using docker build -f <docker file name>.

This example builds the tests associated with gce_image_import_export_tests.Dockerfile, and stores the image in a tag gce_image_import_export_tests.

From the root of this repository, execute:

gcloud auth login
gcloud auth configure-docker
docker build -f gce_image_import_export_tests.Dockerfile . -t gce_image_import_export_tests

Notes:

1. The first two gcloud commands ensure that your local docker installation can read the gcr.io/compute-image-tools-test/wrapper-with-gcloud:latest image.

Use the same syntax to build other e2e Dockerfiles.

Details about running

Run e2e Docker images using docker run <tag name>.

This example runs a subset of the tests from the image built in the prior step.

export PROJECT=your-project
gcloud auth application-default login
docker run --env GOOGLE_APPLICATION_CREDENTIALS= \
           --env CLOUDSDK_CONFIG=/root/.config/gcloud \
           -v $HOME/.config/gcloud:/root/.config/gcloud \
           gce_image_import_export_tests \
           -test_project_id $PROJECT \
           -test_zone us-central1-a \
           -test_suite_filter=^ImageImport$ \
           -test_case_filter=ubuntu

Notes:

1. The first gcloud command activates application default credentials.
2. The docker run command erases the GOOGLE_APPLICATION_CREDENTIALS, and sets CLOUDSDK_CONFIG. This ensures that gcloud will use the credentials from step [1], which are mounted into a volume using -v. Prow uses GOOGLE_APPLICATION_CREDENTIALS to inject a service account; using CLOUDSDK_CONFIG ensures that you don't have to download those keys to your machine.
3. -test_project_id and -test_zone are required. Instead of using the test pool project, you can run the tests in your own test project. This example uses the test pool, since the test suites assumes resources from that project will be present.
4. -test_suite_filter and -test_case_filter are optional. These specify which tests to execute; if empty, all tests are executed. See launcher.go for the filter's implementation. To find values for filtering, search for to junitxml.NewTestSuite and junitxml.NewTestCase.
5. The example uses your test project. You may find that thet tests depend on resources that were manually created in the project compute-import-test-pool-001. If so, you'll need to re-create those resources in your test project.

Beyond these flags, some test suites have further customization via a -variables flag. Search for -variables in the prow configuration to see what's currently used.

Documentation

Index

• func CLITestSuite(ctx context.Context, tswg *sync.WaitGroup, testSuites chan *junitxml.TestSuite, ...)
• func ContainsAll(arr []string, subarr []string, junit *junitxml.TestCase, logger *log.Logger, ...)
• func ContainsNone(arr []string, subarr []string, junit *junitxml.TestCase, logger *log.Logger, ...)
• func ContainsSubString(strs []string, s string) bool
• func Failure(testCase *junitxml.TestCase, logger *log.Logger, msg string)
• func GcloudAuth(logger *log.Logger, testCase *junitxml.TestCase) bool
• func GcloudUpdate(logger *log.Logger, testCase *junitxml.TestCase, latest bool) bool
• func GuestOSFeatures(requiredGuestOsFeatures []string, notAllowedGuestOsFeatures []string, ...)
• func RunCliTool(logger *log.Logger, testCase *junitxml.TestCase, cmdString string, ...) error
• func RunTestCommand(cmd string, args []string, logger *log.Logger, testCase *junitxml.TestCase) bool
• func RunTestCommandAssertErrorMessage(cmd string, args []string, expectedErrorMessage string, logger *log.Logger, ...) bool
• func RunTestCommandAsync(cmd string, args []string, logger *log.Logger, testCase *junitxml.TestCase) *exec.Cmd
• func RunTestCommandIgnoringError(cmd string, args []string, logger *log.Logger, testCase *junitxml.TestCase) bool
• func RunTestForTestType(cmd string, args []string, testType CLITestType, logger *log.Logger, ...) bool
• type CLITestType
• type ServiceAccountTestVariables
  • func GetServiceAccountTestVariables(argMap map[string]string, isDefaultServiceAccountDisabled bool) (v ServiceAccountTestVariables, ok bool)

Functions

func CLITestSuite

func CLITestSuite(ctx context.Context, tswg *sync.WaitGroup, testSuites chan *junitxml.TestSuite,
	logger *log.Logger, testSuiteRegex, testCaseRegex *regexp.Regexp,
	testProjectConfig *testconfig.Project, testSuiteName string, testsMap map[CLITestType]map[*junitxml.TestCase]func(
		context.Context, *junitxml.TestCase, *log.Logger, *testconfig.Project, CLITestType))

CLITestSuite executes given test suite.

func ContainsAll

func ContainsAll(arr []string, subarr []string, junit *junitxml.TestCase, logger *log.Logger, failureMessage string)

ContainsAll asserts all given strings in subarr exists in arr

func ContainsNone

func ContainsNone(arr []string, subarr []string, junit *junitxml.TestCase, logger *log.Logger, failureMessage string)

ContainsNone asserts some given strings in subarr exists in arr

func ContainsSubString

func ContainsSubString(strs []string, s string) bool

ContainsSubString checks whether the string slice contains a substring anywhere.

func Failure

func Failure(testCase *junitxml.TestCase, logger *log.Logger, msg string)

Failure logs failure message to both test case output and logger.

func GcloudAuth

func GcloudAuth(logger *log.Logger, testCase *junitxml.TestCase) bool

GcloudAuth runs "gcloud auth"

func GcloudUpdate

func GcloudUpdate(logger *log.Logger, testCase *junitxml.TestCase, latest bool) bool

GcloudUpdate runs "gcloud update" to pull either latest or prod version

func GuestOSFeatures

func GuestOSFeatures(requiredGuestOsFeatures []string, notAllowedGuestOsFeatures []string, actualGuestOSFeatures []*compute.GuestOsFeature, junit *junitxml.TestCase, logger *log.Logger)

GuestOSFeatures asserts expected & unexpected guestOSFeatures.

func RunCliTool

func RunCliTool(logger *log.Logger, testCase *junitxml.TestCase, cmdString string, args []string) error

RunCliTool runs a cli tool with given args

func RunTestCommand

func RunTestCommand(cmd string, args []string, logger *log.Logger, testCase *junitxml.TestCase) bool

RunTestCommand runs given test command

func RunTestCommandAssertErrorMessage

func RunTestCommandAssertErrorMessage(cmd string, args []string, expectedErrorMessage string, logger *log.Logger, testCase *junitxml.TestCase) bool

RunTestCommandAssertErrorMessage runs given test command and assert given error message appears.

func RunTestCommandAsync

func RunTestCommandAsync(cmd string, args []string, logger *log.Logger, testCase *junitxml.TestCase) *exec.Cmd

RunTestCommandAsync runs given test command asynchronously

func RunTestCommandIgnoringError

func RunTestCommandIgnoringError(cmd string, args []string, logger *log.Logger, testCase *junitxml.TestCase) bool

RunTestCommandIgnoringError runs given test command. The test case won't be marked as fail even error happens.

func RunTestForTestType

func RunTestForTestType(cmd string, args []string, testType CLITestType, logger *log.Logger, testCase *junitxml.TestCase) bool

RunTestForTestType runs test for given test type

Types

type CLITestType

type CLITestType string

CLITestType defines which type of test is going to be executed

const (
	Wrapper                       CLITestType = "1 wrapper"
	GcloudBetaProdWrapperLatest   CLITestType = "2 gcloud(beta)-prod wrapper-latest"
	GcloudBetaLatestWrapperLatest CLITestType = "3 gcloud(beta)-latest wrapper-latest"
	GcloudGaLatestWrapperRelease  CLITestType = "4 gcloud(ga)-latest wrapper-release"
)

List all test types here

type ServiceAccountTestVariables

type ServiceAccountTestVariables struct {
	ProjectID              string
	ComputeServiceAccount  string
	InstanceServiceAccount string
}

ServiceAccountTestVariables contains service-account related test variables.

func GetServiceAccountTestVariables

func GetServiceAccountTestVariables(argMap map[string]string, isDefaultServiceAccountDisabled bool) (v ServiceAccountTestVariables, ok bool)

GetServiceAccountTestVariables extract extra test variables related to service account from input variable map.