e2e

README

boundary-e2e-tests

This test suite tests Boundary in an end-to-end setting, utilizing both the Boundary CLI and the Boundary Go API to exercise Boundary through various user workflows. It was designed to be run in a variety of environments as long as the appropriate environment variables are set. The test suite itself uses the standard go test library.

One method for setting up an environment is utilizing Enos to create the desired infrastructure.

Getting Started

Enos

Setup Enos as described here

Then, use the following commands to run tests

cd enos
enos scenario list

# `Run` executes the tests and destroys the associated infrastructure in one command
enos scenario run e2e_{scenario} builder:local

# `Launch` executes the tests, but leaves the infrastructure online for debugging purposes
enos scenario launch e2e_{scenario} builder:local
enos scenario output  # displays any defined enos output
enos scenario destroy  # destroys infra

Enos scenarios set up the infrastructure, set the appropriate environment variables, and run the specified tests in its scenario file.

Note: To run the e2e_host_aws scenario, you will need access to the boundary team's test AWS account.

Local

Set the appropriate environment variables...

export E2E_TESTS=true  # This is needed for any e2e test. Otherwise, the test is skipped

# For e2e/tests/static
export BOUNDARY_ADDR=  # e.g. http://127.0.0.1:9200
export E2E_PASSWORD_AUTH_METHOD_ID=  # e.g. ampw_1234567890
export E2E_PASSWORD_ADMIN_LOGIN_NAME=  # e.g. "admin"
export E2E_PASSWORD_ADMIN_PASSWORD=  # e.g. "password"

export E2E_TARGET_IP=  # e.g. 192.168.0.1
export E2E_SSH_KEY_PATH=  # e.g. /Users/username/key.pem
export E2E_SSH_USER=  # e.g. ubuntu

# For e2e/tests/static_with_vault
export BOUNDARY_ADDR=  # e.g. http://127.0.0.1:9200
export E2E_PASSWORD_AUTH_METHOD_ID=  # e.g. ampw_1234567890
export E2E_PASSWORD_ADMIN_LOGIN_NAME=  # e.g. "admin"
export E2E_PASSWORD_ADMIN_PASSWORD=  # e.g. "password"

export VAULT_ADDR=  # e.g. http://127.0.0.1:8200
export VAULT_TOKEN=
export E2E_TARGET_IP=  # e.g. 192.168.0.1
export E2E_SSH_KEY_PATH=  # e.g. /Users/username/key.pem
export E2E_SSH_USER=  # e.g. ubuntu

# For e2e/tests/aws
export BOUNDARY_ADDR=  # e.g. http://127.0.0.1:9200
export E2E_PASSWORD_AUTH_METHOD_ID=  # e.g. ampw_1234567890
export E2E_PASSWORD_ADMIN_LOGIN_NAME=  # e.g. "admin"
export E2E_PASSWORD_ADMIN_PASSWORD=  # e.g. "password"

export E2E_AWS_ACCESS_KEY_ID=
export E2E_AWS_SECRET_ACCESS_KEY=
export E2E_AWS_HOST_SET_FILTER=  # e.g. "tag:testtag=true"
export E2E_AWS_HOST_SET_IPS=  # e.g. "[\"1.2.3.4\", \"2.3.4.5\"]"
export E2E_AWS_HOST_SET_FILTER2=  # e.g. "tag:testtagtwo=test"
export E2E_AWS_HOST_SET_IPS2=  # e.g. "[\"1.2.3.4\"]
export E2E_SSH_KEY_PATH=  # e.g. /Users/username/key.pem
export E2E_SSH_USER=  # e.g. ubuntu

# For e2e/tests/database
export E2E_AWS_ACCESS_KEY_ID=
export E2E_AWS_SECRET_ACCESS_KEY=
export E2E_AWS_HOST_SET_FILTER=  # e.g. "tag:testtag=true"
export VAULT_ADDR=  # e.g. http://127.0.0.1:8200
export VAULT_TOKEN=

Then, run...

go test github.com/hashicorp/boundary/testing/e2e/target // run target tests
go test ./target/ // run target tests if running from this directory
go test github.com/hashicorp/boundary/testing/e2e/target -v // verbose
go test github.com/hashicorp/boundary/testing/e2e/target -v -run '^TestCreateTargetApi$' // run a specific test

Adding Tests

Tests live in the tests/ directory. Additional tests can be added to an existing go package or a new one can be created. If a new package is created, a new enos scenario would also need to be created.

Enos is comprised of scenarios, where a scenario is the environment you want the tests to operate in. In one scenario, there may be a boundary cluster and a target. Another scenario might involve a boundary cluster and a vault instance. Scenarios can be found in boundary/enos

To run these tests in CI, the enos-run.yml github action workflow must be updated to include the new scenario (see the matrix).

Development

To assist with iterating on tests on enos launched infrastructure, you can perform the following...

Launch an enos scenario and print out the environment variables

cd enos
enos scenario launch e2e_{scenario} builder:local
enos scenario output
cd .enos
ls -ltr
cd <most_recent_directory> # bottom of list
terraform show -json terraform.tfstate | jq -r '.values.root_module.child_modules[].resources[] | select(.address=="module.run_e2e_test.enos_local_exec.run_e2e_test") | .values.environment | to_entries[] | "export \(.key)=\(.value)"'

Take the printed environment variable information and export them into another terminal session

export BOUNDARY_ADDR=
export E2E_PASSWORD_AUTH_METHOD_ID=
...

Run your tests

go test -v {go package}

Documentation

Index

• Constants
• func MaybeSkipTest(t testing.TB)
• type CommandResult
  • func RunCommand(ctx context.Context, command string, opt ...Option) *CommandResult
• type Option
  • func WithArgs(args ...string) Option
  • func WithEnv(name string, value string) Option

Constants

const EnvToCheckSkip = "E2E_TESTS"

Functions

func MaybeSkipTest

func MaybeSkipTest(t testing.TB)

MaybeSkipTest is a check used at the start of the test to determine if the test should run

Types

type CommandResult

type CommandResult struct {
	Stdout   []byte
	Stderr   []byte
	ExitCode int
	Err      error
}

CommandResult captures the output from running an external command

func RunCommand

func RunCommand(ctx context.Context, command string, opt ...Option) *CommandResult

RunCommand executes external commands on the system. Returns the results of running the provided command.

RunCommand(context.Background(), "ls")
RunCommand(context.Background(), "ls", WithArgs("-al", "/path"))

CommandResult is always valid even if there is an error.

type Option

type Option func(*options)

Option is a func that sets optional attributes for a call. This does not need to be used directly, but instead option arguments are built from the functions in this package. WithX options set a value to that given in the argument; DefaultX options indicate that the value should be set to its default. When an API call is made options are processed in ther order they appear in the function call, so for a given argument X, a succession of WithX or DefaultX calls will result in the last call taking effect.

func WithArgs

func WithArgs(args ...string) Option

WithArgs is an option to RunCommand that allows the user to specify arguments for the provided command. This option can be used multiple times in one command.

RunCommand(context.Background(), "ls", WithArgs("-al"))

func WithEnv

func WithEnv(name string, value string) Option

WithEnv is an option to RunCommand that allows the user to specify environment variables to be set when running the command. This option can be used multiple times in one command.

RunCommand(context.Background(), "ls", WithEnv("NAME", "VALUE"), WithEnv("NAME", "VALUE"))