e2e

package
v0.50.0-rc.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 20, 2023 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package e2e provides the API to manage environments and organize E2E tests.

Here is a small example of E2E tests. E2E tests use testify Suite and it is strongly recommended to read the documentation of testify Suite if you are not familiar with it. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
)

type vmSuite struct {
	e2e.Suite[e2e.VMEnv]
}

func TestVMSuite(t *testing.T) {
	e2e.Run[e2e.VMEnv](t, &vmSuite{}, e2e.EC2VMStackDef())
}

func (v *vmSuite) TestBasicVM() {
	v.Env().VM.Execute("ls")
}

To write an E2E test:

1. Define your own suite type with the embedded e2e.Suite struct. 

type vmSuite struct {
	e2e.Suite[e2e.VMEnv]
}

e2e.VMEnv defines the components available in your stack. See "Using existing stack definition" section for more information.

2. Write a regular Go test function that runs the test suite using e2e.Run. 

func TestVMSuite(t *testing.T) {
	e2e.Run[e2e.VMEnv](t, &vmSuite{}, e2e.EC2VMStackDef())
}

The first argument of e2e.Run is an instance of type *testing.T.

The second argument is a pointer to an empty instance of the previous defined structure (&vmSuite{} in our example)

The third parameter defines the environment. See "Using existing stack definition" section for more information about a environment definition.

3. Write a test function 

func (v *vmSuite) TestBasicVM() {
	v.Env().VM.Execute("ls")
}

e2e.Suite.Env gives access to the components in your environment.

Depending on your stack definition, e2e.Suite.Env can provide the following objects:

  • client.VM: A virtual machine where you can execute commands.
  • client.Agent: A struct that provides methods to run datadog agent commands.
  • client.Fakeintake: A struct that provides methods to run queries to a fake instance of Datadog intake.

Using an existing stack definition

The stack definition defines the components available in your environment. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
)

type vmSuite struct {
	e2e.Suite[e2e.VMEnv]
}

func TestVMSuite(t *testing.T) {
	e2e.Run[e2e.VMEnv](t, &vmSuite{}, e2e.EC2VMStackDef())
}

In this example, the components available are defined by the struct e2e.VMEnv which contains a virtual machine. The generic type of e2e.Suite must match the type of the stack definition. In our example, e2e.EC2VMStackDef returns an instance of *e2e.StackDefinition[e2e.VMEnv].

e2e.EC2VMStackDef

e2e.EC2VMStackDef creates an environment with a virtual machine. The available options are located in the ec2params package. 

import (
	"testing"
	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
	"github.com/DataDog/test-infra-definitions/components/os"
	"github.com/DataDog/test-infra-definitions/scenarios/aws/vm/ec2os"
	"github.com/DataDog/test-infra-definitions/scenarios/aws/vm/ec2params"
)

type vmSuite struct {
	e2e.Suite[e2e.VMEnv]
}

func TestVMSuite(t *testing.T) {
	e2e.Run[e2e.VMEnv](t, &vmSuite{}, e2e.EC2VMStackDef(
		ec2params.WithImageName("ami-0a0c8eebcdd6dcbd0", os.ARM64Arch, ec2os.UbuntuOS),
		ec2params.WithName("My-instance"),
	))
}

func (v *vmSuite) TestBasicVM() {
	v.Env().VM.Execute("ls")
}

e2e.AgentStackDef

e2e.AgentStackDef creates an environment with an Agent installed on a virtual machine. The available options are located in the agentparams package. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
	"github.com/DataDog/test-infra-definitions/components/datadog/agentparams"
	"github.com/DataDog/test-infra-definitions/components/os"
	"github.com/DataDog/test-infra-definitions/scenarios/aws/vm/ec2os"
	"github.com/DataDog/test-infra-definitions/scenarios/aws/vm/ec2params"
	"github.com/stretchr/testify/require"
)

type agentSuite struct {
	e2e.Suite[e2e.AgentEnv]
}

func TestVMSuite(t *testing.T) {
	e2e.Run[e2e.AgentEnv](t, &agentSuite{}, e2e.AgentStackDef(
		[]ec2params.Option{
			ec2params.WithImageName("ami-0a0c8eebcdd6dcbd0", os.ARM64Arch, ec2os.UbuntuOS),
			ec2params.WithName("My-instance"),
		},
		agentparams.WithAgentConfig("log_level: debug"),
		agentparams.WithTelemetry(),
	))
}

func (v *agentSuite) TestBasicAgent() {
	config := v.Env().Agent.Config()
	require.Contains(v.T(), config, "log_level: debug")
}

Defining your stack definition

In some special cases, you have to define a custom environment. Here is an example of an environment with Docker installed on a virtual machine. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e/client"
	"github.com/DataDog/test-infra-definitions/components/datadog/agent/docker"
	"github.com/DataDog/test-infra-definitions/scenarios/aws/vm/ec2vm"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

type dockerSuite struct {
	e2e.Suite[e2e.VMEnv]
}

func TestDockerSuite(t *testing.T) {
	e2e.Run[e2e.VMEnv](t, &dockerSuite{}, e2e.EnvFactoryStackDef(dockerEnvFactory))
}

func dockerEnvFactory(ctx *pulumi.Context) (*e2e.VMEnv, error) {
	vm, err := ec2vm.NewUnixEc2VM(ctx)
	if err != nil {
		return nil, err
	}

	_, err = docker.NewAgentDockerInstaller(vm.UnixVM, docker.WithAgent(docker.WithAgentImageTag("7.42.0")))

	if err != nil {
		return nil, err
	}

	return &e2e.VMEnv{
		VM: client.NewVM(vm),
	}, nil
}

func (docker *dockerSuite) TestDocker() {
	docker.Env().VM.Execute("docker container ls")
}

e2e.EnvFactoryStackDef is used to define a custom environment. Here is a non exhaustive list of components that can be used to create a custom environment:

  • EC2 VM: Provide methods to create a virtual machine on EC2.
  • Agent: Provide methods to install the Agent on a virtual machine
  • File Manager: Provide methods to manipulate files and folders

Organizing your tests

The execution order for tests in testify Suite is IMPLEMENTATION SPECIFIC UNLIKE REGULAR GO TESTS.

Having a single environment

In the simple case, there is a single environment and each test checks one specific thing. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
)

type singleEnvSuite struct {
	e2e.Suite[e2e.AgentEnv]
}

func TestSingleEnvSuite(t *testing.T) {
	e2e.Run[e2e.AgentEnv](t, &singleEnvSuite{}, e2e.AgentStackDef())
}

func (suite *singleEnvSuite) Test1() {
	// Check feature 1
}

func (suite *singleEnvSuite) Test2() {
	// Check feature 2
}

func (suite *singleEnvSuite) Test3() {
	// Check feature 3
}

Having different environments

In this scenario, the environment is different for each test (or for most of them). e2e.Suite.UpdateEnv is used to update the environment. Keep in mind that using e2e.Suite.UpdateEnv to update virtual machine settings can destroy the current virtual machine and create a new one when updating the operating system for example.

Note: Calling twice e2e.Suite.UpdateEnv with the same argument does nothing. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
	"github.com/DataDog/test-infra-definitions/components/datadog/agentparams"
	"github.com/stretchr/testify/require"
)

type multipleEnvSuite struct {
	e2e.Suite[e2e.AgentEnv]
}

func TestMultipleEnvSuite(t *testing.T) {
	e2e.Run[e2e.AgentEnv](t, &multipleEnvSuite{}, e2e.AgentStackDef())
}

func (suite *multipleEnvSuite) TestLogDebug() {
	suite.UpdateEnv(e2e.AgentStackDef(e2e.WithAgentParams(agentparams.WithAgentConfig("log_level: debug"))))
	config := suite.Env().Agent.Config()
	require.Contains(suite.T(), config, "log_level: debug")
}

func (suite *multipleEnvSuite) TestLogInfo() {
	suite.UpdateEnv(e2e.AgentStackDef(e2e.WithAgentParams(agentparams.WithAgentConfig("log_level: info"))))
	config := suite.Env().Agent.Config()
	require.Contains(suite.T(), config, "log_level: info")
}

Having few environments

You may sometime have few environments but several tests for each on them. You can still use e2e.Suite.UpdateEnv as explained in the previous section but using Subtests is an alternative solution. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
	"github.com/DataDog/test-infra-definitions/components/datadog/agentparams"
)

type subTestSuite struct {
	e2e.Suite[e2e.AgentEnv]
}

func TestSubTestSuite(t *testing.T) {
	e2e.Run[e2e.AgentEnv](t, &subTestSuite{}, e2e.AgentStackDef())
}

func (suite *subTestSuite) TestLogDebug() {
	suite.UpdateEnv(e2e.AgentStackDef(e2e.WithAgentParams(agentparams.WithAgentConfig("log_level: debug"))))
	suite.T().Run("MySubTest1", func(t *testing.T) {
		// Sub test 1
	})
	suite.T().Run("MySubTest2", func(t *testing.T) {
		// Sub test 2
	})
}

func (suite *subTestSuite) TestLogInfo() {
	suite.UpdateEnv(e2e.AgentStackDef(e2e.WithAgentParams(agentparams.WithAgentConfig("log_level: info"))))
	suite.T().Run("MySubTest1", func(t *testing.T) {
		// Sub test 1
	})
	suite.T().Run("MySubTest2", func(t *testing.T) {
		// Sub test 2
	})
}

WithDevMode

When writing a new e2e test, it is important to iterate quickly until your test succeeds. You can use params.WithDevMode to not destroy the environment when the test finishes. For example it allows you to not create a new virtual machine each time you run a test. Note: params.WithDevMode is ignored when the test runs on the CI but it should be removed when you finish the writing of the test. 

import (
	"testing"

	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e"
	"github.com/DataDog/datadog-agent/test/new-e2e/pkg/utils/e2e/params"
)

type vmSuite struct {
	e2e.Suite[e2e.VMEnv]
}

func TestVMSuite(t *testing.T) {
	e2e.Run[e2e.VMEnv](t, &vmSuite{}, e2e.EC2VMStackDef(), params.WithDevMode())
}

func (v *vmSuite) TestBasicVM() {
	v.Env().VM.Execute("ls")
}

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Run 

func Run[Env any, T suiteConstraint[Env]](t *testing.T, e2eSuite T, stackDef *StackDefinition[Env], options ...params.Option)

Run runs the tests defined in e2eSuite

t is an instance of type *testing.T.

e2eSuite is a pointer to a structure with a e2e.Suite embbeded struct.

stackDef defines the stack definition.

options is an optional list of options like [DevMode], [SkipDeleteOnFailure] or [WithStackName]. 

type vmSuite struct {
	e2e.Suite[e2e.VMEnv]
}
// ...
e2e.Run(t, &vmSuite{}, e2e.EC2VMStackDef())

func WithAgentClientParams 

func WithAgentClientParams(options ...agentclientparams.Option) func(*AgentStackDefParam) error

WithAgentClientParams sets Agent client parameters See agentclientparams.Params for available options for agentParams

func WithAgentParams 

func WithAgentParams(options ...agentparams.Option) func(*AgentStackDefParam) error

WithAgentParams sets Agent parameters See agent.Params for available options for agentParams.

func WithFakeIntakeParams added in v0.50.0 

func WithFakeIntakeParams(options ...fakeintakeparams.Option) func(*AgentStackDefParam) error

WithFakeIntakeParams sets FakeIntake parameters See fakeintakeparams.Params for available options for fakeIntakeParams

func WithVMParams 

func WithVMParams(options ...ec2params.Option) func(*AgentStackDefParam) error

WithVMParams sets VM parameters See ec2vm.Params for available options for vmParams.

Types

type AgentEnv 

type AgentEnv struct {
	VM    client.VM
	Agent client.Agent
}

AgentEnv contains an Agent VM environment

type AgentStackDefParam 

type AgentStackDefParam struct {
	// contains filtered or unexported fields
}

AgentStackDefParam defines the parameters for a stack with a VM and the Datadog Agent installed. The AgentStackDefParam configuration uses the Functional options pattern.

The available options are:

type DockerEnv 

type DockerEnv struct {
	Docker *client.Docker
}

DockerEnv contains an environment with Docker

type FakeIntakeEnv 

type FakeIntakeEnv struct {
	VM         client.VM
	Agent      client.Agent
	Fakeintake *client.Fakeintake
}

FakeIntakeEnv contains an environment with the Agent installed on a VM and a dedicated fakeintake

type StackDefinition 

type StackDefinition[Env any] struct {
	// contains filtered or unexported fields
}

StackDefinition contains a Pulumi stack definition

func AgentStackDef 

func AgentStackDef(options ...func(*AgentStackDefParam) error) *StackDefinition[AgentEnv]

AgentStackDef creates a stack definition containing a virtual machine and an Agent.

See ec2vm.Params for available options for vmParams.

See agent.Params for available options for agentParams.

See agentclientparams.Params for available options for agentParams

func AgentStackDefWithDefaultVMAndAgentClient 

func AgentStackDefWithDefaultVMAndAgentClient(options ...agentparams.Option) *StackDefinition[AgentEnv]

AgentStackDefWithDefaultVMAndAgentClient creates a stack definition containing an Ubuntu virtual machine and an Agent. The Agent is awaited at TestSuite setup time, any subtest runs with an healthy agent

See agent.Params for available options.

func CustomEC2VMStackDef 

func CustomEC2VMStackDef[T any](fct func(vm.VM) (T, error), options ...ec2params.Option) *StackDefinition[VMEnv]

CustomEC2VMStackDef creates a custom stack definition containing a virtual machine

func DockerStackDef 

func DockerStackDef(params ...dockerparams.Option) *StackDefinition[DockerEnv]

DockerStackDef creates a stack definition for Docker.

See dockerparams.Params for available options for params.

func EC2VMStackDef 

func EC2VMStackDef(options ...ec2params.Option) *StackDefinition[VMEnv]

EC2VMStackDef creates a stack definition containing a virtual machine. See ec2vm.Params for available options.

func EnvFactoryStackDef 

func EnvFactoryStackDef[Env any](envFactory func(ctx *pulumi.Context) (*Env, error)) *StackDefinition[Env]

EnvFactoryStackDef creates a custom stack definition

func FakeIntakeStackDef 

func FakeIntakeStackDef(options ...func(*AgentStackDefParam) error) *StackDefinition[FakeIntakeEnv]

FakeIntakeStackDef creates a stack definition containing a virtual machine the Agent and the fake intake.

See ec2vm.Params for available options for vmParams.

See agent.Params for available options for agentParams.

See agentclientparams.Params for available options for agentParams

func FakeIntakeStackDefWithDefaultVMAndAgentClient 

func FakeIntakeStackDefWithDefaultVMAndAgentClient(options ...agentparams.Option) *StackDefinition[FakeIntakeEnv]

FakeIntakeStackDefWithDefaultVMAndAgentClient creates a stack definition containing an Ubuntu virtual machine, an Agent and a fake Datadog intake. The Agent is awaited at TestSuite setup time, any subtest runs with an healthy agent

See agent.Params for available options.

func NewStackDef 

func NewStackDef[Env any](envFactory func(ctx *pulumi.Context) (*Env, error), configMap runner.ConfigMap) *StackDefinition[Env]

NewStackDef creates a custom definition

type Suite 

type Suite[Env any] struct {
	suite.Suite
	// contains filtered or unexported fields
}

Suite manages the environment creation and runs E2E tests.

func (*Suite[Env]) AfterTest 

func (suite *Suite[Env]) AfterTest(suiteName, testName string)

AfterTest is executed right after the test finishes and receives the suite and test names as input. This function is called by testify Suite.

If you override AfterTest in your custom test suite type, the function must call e2e.Suite.AfterTest.

func (*Suite[Env]) BeforeTest 

func (suite *Suite[Env]) BeforeTest(suiteName, testName string)

BeforeTest is executed right before the test starts and receives the suite and test names as input. This function is called by testify Suite.

If you override BeforeTest in your custom test suite type, the function must call e2e.Suite.BeforeTest.

func (*Suite[Env]) Env 

func (suite *Suite[Env]) Env() *Env

Env returns the current environment. In order to improve the efficiency, this function behaves as follow:

  • It creates the default environment if no environment exists.
  • It restores the default environment if e2e.Suite.UpdateEnv was not already called during this test. This avoid having to restore the default environment for each test even if suite.UpdateEnv immedialy overrides the environment.

func (*Suite[Env]) SetupSuite 

func (suite *Suite[Env]) SetupSuite()

SetupSuite method will run before the tests in the suite are run. This function is called by testify Suite.

If you override SetupSuite in your custom test suite type, the function must call e2e.Suite.SetupSuite.

func (*Suite[Env]) TearDownSuite 

func (suite *Suite[Env]) TearDownSuite()

TearDownSuite run after all the tests in the suite have been run. This function is called by testify Suite.

If you override TearDownSuite in your custom test suite type, the function must call e2e.Suite.TearDownSuite.

func (*Suite[Env]) UpdateEnv 

func (suite *Suite[Env]) UpdateEnv(stackDef *StackDefinition[Env])

UpdateEnv updates the environment. This affects only the test that calls this function. Test functions that don't call UpdateEnv have the environment defined by e2e.Run.

type VMEnv 

type VMEnv struct {
	VM client.VM
}

VMEnv contains a VM environment

Source Files

View all Source files

Directories

Path Synopsis
Package client contains clients used to communicate with the remote service
 Package client contains clients used to communicate with the remote service
agentclientparams
Package agentclientparams implements function parameters for [e2e.Agent]
 Package agentclientparams implements function parameters for [e2e.Agent]
executeparams
Package executeparams implements function parameters for [client.vmClient]
 Package executeparams implements function parameters for [client.vmClient]
Package params implements function parameters for [e2e.Suite]
 Package params implements function parameters for [e2e.Suite]

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.