e2e

package
v0.0.0-...-8fd29ad Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 29, 2021 License: Apache-2.0 Imports: 15 Imported by: 0

README

Importable End-To-End Test Suite

Virtual Kubelet (VK) provides an importable end-to-end (E2E) test suite containing a set of common integration tests. As a provider, you can import the test suite and use it to validate your VK implementation.

Prerequisite

To run the E2E test suite, three things are required:

  • A local Kubernetes cluster (we have tested with Docker for Mac and Minikube);
  • Your kubeconfig default context points to the local Kubernetes cluster;
  • skaffold

The test suite is based on VK 1.0. If your VK implementation is based on legacy VK library (< v1.0.0), you will have to upgrade it to VK 1.0 using virtual-kubelet/node-cli.

Skaffold Folder

Before running the E2E test suite, you will need to copy the ./hack folder containing Skaffold-related files such as Dockerfile, manifests, and certificates to your VK project root. Skaffold essentially helps package your virtual kubelet into a container based on the given Dockerfile and deploy it as a pod (see pod.yml) to your Kubernetes test cluster. In summary, you will likely need to modify the VK name in those files, customize the VK configuration file, and the API server certificates (<vk-name>-crt.pem and <vk-name>-key.pem) before running the test suite.

Makefile.e2e

Also, you will need to copy Makefile.e2e to your VK project root. It contains necessary make commands to run the E2E test suite. Do not forget to add include Makefile.e2e in your Makefile.

File Structure

A minimal VK provider should now have a file structure similar to the one below:

.
├── Makefile
├── Makefile.e2e
├── README.md
├── cmd
│   └── virtual-kubelet
│       └── main.go
├── go.mod
├── go.sum
├── hack
│   └── skaffold
│       └── virtual-kubelet
│           ├── Dockerfile
│           ├── base.yml
│           ├── pod.yml
│           ├── skaffold.yml
│           ├── vkubelet-provider-0-cfg.json
│           ├── vkubelet-provider-0-crt.pem
│           └── vkubelet-provider-0-key.pem
├── test
│   └── e2e
│       └── main_test.go # import and run the E2E test suite here
├── provider.go # provider-specific VK implementation
├── provider_test.go # unit test

Importing the Test Suite

The test suite can be easily imported in your test files (e.g. ./test/e2e/main_test.go) with the following import statement:

import (
	vke2e "github.com/virtual-kubelet/virtual-kubelet/test/e2e"
)
Test Suite Customization

The test suite allows providers to customize the test suite using EndToEndTestSuiteConfig:

// EndToEndTestSuiteConfig is the config passed to initialize the testing framework and test suite.
type EndToEndTestSuiteConfig struct {
	// Kubeconfig is the path to the kubeconfig file to use when running the test suite outside a Kubernetes cluster.
	Kubeconfig string
	// Namespace is the name of the Kubernetes namespace to use for running the test suite (i.e. where to create pods).
	Namespace string
	// NodeName is the name of the virtual-kubelet node to test.
	NodeName string
	// WatchTimeout is the duration for which the framework watch a particular condition to be satisfied (e.g. watches a pod  becoming ready)
	WatchTimeout time.Duration
	// Setup is a function that sets up provider-specific resource in the test suite
	Setup suite.SetUpFunc
	// Teardown is a function that tears down provider-specific resources from the test suite
	Teardown suite.TeardownFunc
	// ShouldSkipTest is a function that determines whether the test suite should skip certain tests
	ShouldSkipTest suite.ShouldSkipTestFunc
}

Setup() is invoked before running the E2E test suite, and Teardown() is invoked after all the E2E tests are finished.

You will need an EndToEndTestSuiteConfig to create an EndToEndTestSuite using NewEndToEndTestSuite. After that, invoke Run from EndToEndTestSuite to start the test suite. The code snippet below is a minimal example of how to import and run the test suite in your test file.

package e2e

import (
	"flag"
	"fmt"
	"testing"
	"time"

	vke2e "github.com/virtual-kubelet/virtual-kubelet/test/e2e"
)

var (
	kubeconfig string
	namespace  string
	nodeName   string
)

var defaultNamespace = "default"
var defaultNodeName = "default-node"

// Read the following variables from command-line flags
func init() {
	flag.StringVar(&kubeconfig, "kubeconfig", "", "path to the kubeconfig file to use when running the test suite outside a kubernetes cluster")
	flag.StringVar(&namespace, "namespace", defaultNamespace, "the name of the kubernetes namespace to use for running the test suite (i.e. where to create pods)")
	flag.StringVar(&nodeName, "node-name", defaultNodeName, "the name of the virtual-kubelet node to test")
	flag.Parse()
}

func setup() error {
	fmt.Println("Setting up end-to-end test suite...")
	return nil
}

func teardown() error {
	fmt.Println("Tearing down end-to-end test suite...")
	return nil
}

func shouldSkipTest(testName string) bool {
	// Skip the test 'TestGetStatsSummary'
	return testName == "TestGetStatsSummary"
}

// TestEndToEnd runs the e2e tests against a previously configured cluster
func TestEndToEnd(t *testing.T) {
	config := vke2e.EndToEndTestSuiteConfig{
		Kubeconfig:     kubeconfig,
		Namespace:      namespace,
		NodeName:       nodeName,
		Setup:          setup,
		Teardown:       teardown,
		ShouldSkipTest: shouldSkipTest,
		WatchTimeout:   5 * time.Minute,
	}
	ts := vke2e.NewEndToEndTestSuite(config)
	ts.Run(t)
}

Running the Test Suite

Since our CI uses Minikube, we describe below how to run E2E on top of it.

To create a Minikube cluster, run the following command after installing Minikube:

minikube start

To run the E2E test suite, you can run the following command:

make e2e

You can see from the console output whether the tests in the test suite pass or not.

...
=== RUN   TestEndToEnd
Setting up end-to-end test suite for mock provider...
    suite.go:62: True
=== RUN   TestEndToEnd/TestCreatePodWithMandatoryInexistentConfigMap
=== RUN   TestEndToEnd/TestCreatePodWithMandatoryInexistentSecrets
=== RUN   TestEndToEnd/TestCreatePodWithOptionalInexistentConfigMap
=== RUN   TestEndToEnd/TestCreatePodWithOptionalInexistentSecrets
=== RUN   TestEndToEnd/TestGetPods
    basic.go:40: Created pod: nginx-testgetpods-g9s42
    basic.go:46: Pod nginx-testgetpods-g9s42 ready
=== RUN   TestEndToEnd/TestGetStatsSummary
=== RUN   TestEndToEnd/TestNodeCreateAfterDelete
=== RUN   TestEndToEnd/TestPodLifecycleForceDelete
    basic.go:208: Created pod: nginx-testpodlifecycleforcedelete-wrjgk
    basic.go:214: Pod nginx-testpodlifecycleforcedelete-wrjgk ready
    basic.go:247: Force deleted pod:  nginx-testpodlifecycleforcedelete-wrjgk
    basic.go:264: Pod ended as phase: Running
=== RUN   TestEndToEnd/TestPodLifecycleGracefulDelete
    basic.go:135: Created pod: nginx-testpodlifecyclegracefuldelete-tp49x
    basic.go:141: Pod nginx-testpodlifecyclegracefuldelete-tp49x ready
    basic.go:168: Deleted pod: nginx-testpodlifecyclegracefuldelete-tp49x
Tearing down end-to-end test suite for mock provider...
--- PASS: TestEndToEnd (11.75s)
    --- PASS: TestEndToEnd/TestCreatePodWithMandatoryInexistentConfigMap (0.04s)
    --- PASS: TestEndToEnd/TestCreatePodWithMandatoryInexistentSecrets (0.03s)
    --- PASS: TestEndToEnd/TestCreatePodWithOptionalInexistentConfigMap (0.73s)
    --- PASS: TestEndToEnd/TestCreatePodWithOptionalInexistentSecrets (1.00s)
    --- PASS: TestEndToEnd/TestGetPods (0.80s)
    --- PASS: TestEndToEnd/TestGetStatsSummary (0.80s)
    --- PASS: TestEndToEnd/TestNodeCreateAfterDelete (5.25s)
    --- PASS: TestEndToEnd/TestPodLifecycleForceDelete (2.05s)
    --- PASS: TestEndToEnd/TestPodLifecycleGracefulDelete (1.05s)
PASS
ok      github.com/virtual-kubelet/virtual-kubelet/internal/test/e2e    12.298s
?       github.com/virtual-kubelet/virtual-kubelet/internal/test/e2e/framework  [no test files]
...

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type EndToEndTestSuite 

type EndToEndTestSuite struct {
	// contains filtered or unexported fields
}

EndToEndTestSuite holds the setup, teardown, and shouldSkipTest functions for a specific provider

func NewEndToEndTestSuite 

func NewEndToEndTestSuite(cfg EndToEndTestSuiteConfig) *EndToEndTestSuite

NewEndToEndTestSuite returns a new EndToEndTestSuite given a test suite configuration, setup, and teardown functions from provider

func (*EndToEndTestSuite) Run 

func (ts *EndToEndTestSuite) Run(t *testing.T)

Run runs tests registered in the test suite

func (*EndToEndTestSuite) Setup 

func (ts *EndToEndTestSuite) Setup(t *testing.T)

Setup runs the setup function from the provider and other procedures before running the test suite

func (*EndToEndTestSuite) ShouldSkipTest 

func (ts *EndToEndTestSuite) ShouldSkipTest(testName string) bool

ShouldSkipTest returns true if a provider wants to skip running a particular test

func (*EndToEndTestSuite) Teardown 

func (ts *EndToEndTestSuite) Teardown()

Teardown runs the teardown function from the provider and other procedures after running the test suite

func (*EndToEndTestSuite) TestCreatePodWithMandatoryInexistentConfigMap 

func (ts *EndToEndTestSuite) TestCreatePodWithMandatoryInexistentConfigMap(t *testing.T)

TestCreatePodWithMandatoryInexistentConfigMap tries to create a pod referencing inexistent secrets. It then verifies that the pod is not created.

func (*EndToEndTestSuite) TestCreatePodWithMandatoryInexistentSecrets 

func (ts *EndToEndTestSuite) TestCreatePodWithMandatoryInexistentSecrets(t *testing.T)

TestCreatePodWithMandatoryInexistentSecrets tries to create a pod referencing inexistent secrets. It then verifies that the pod is not created.

func (*EndToEndTestSuite) TestCreatePodWithOptionalInexistentConfigMap 

func (ts *EndToEndTestSuite) TestCreatePodWithOptionalInexistentConfigMap(t *testing.T)

TestCreatePodWithOptionalInexistentConfigMap tries to create a pod referencing optional, inexistent config map. It then verifies that the pod is created successfully.

func (*EndToEndTestSuite) TestCreatePodWithOptionalInexistentSecrets 

func (ts *EndToEndTestSuite) TestCreatePodWithOptionalInexistentSecrets(t *testing.T)

TestCreatePodWithOptionalInexistentSecrets tries to create a pod referencing optional, inexistent secrets. It then verifies that the pod is created successfully.

func (*EndToEndTestSuite) TestGetPods 

func (ts *EndToEndTestSuite) TestGetPods(t *testing.T)

TestGetPods tests that the /pods endpoint works, and only returns pods for our kubelet

func (*EndToEndTestSuite) TestGetStatsSummary 

func (ts *EndToEndTestSuite) TestGetStatsSummary(t *testing.T)

TestGetStatsSummary creates a pod having two containers and queries the /stats/summary endpoint of the virtual-kubelet. It expects this endpoint to return stats for the current node, as well as for the aforementioned pod and each of its two containers.

func (*EndToEndTestSuite) TestNodeCreateAfterDelete 

func (ts *EndToEndTestSuite) TestNodeCreateAfterDelete(t *testing.T)

TestNodeCreateAfterDelete makes sure that a node is automatically recreated if it is deleted while VK is running.

func (*EndToEndTestSuite) TestPodLifecycleForceDelete 

func (ts *EndToEndTestSuite) TestPodLifecycleForceDelete(t *testing.T)

TestPodLifecycleForceDelete creates one podsand verifies that the provider has created them and put them in the running lifecycle. It then does a force delete on the pod, and verifies the provider has deleted it.

func (*EndToEndTestSuite) TestPodLifecycleGracefulDelete 

func (ts *EndToEndTestSuite) TestPodLifecycleGracefulDelete(t *testing.T)

TestPodLifecycleGracefulDelete creates a pod and verifies that the provider has been asked to create it. Then, it deletes the pods and verifies that the provider has been asked to delete it. These verifications are made using the /stats/summary endpoint of the virtual-kubelet, by checking for the presence or absence of the pods. Hence, the provider being tested must implement the PodMetricsProvider interface.

type EndToEndTestSuiteConfig 

type EndToEndTestSuiteConfig struct {
	// Kubeconfig is the path to the kubeconfig file to use when running the test suite outside a Kubernetes cluster.
	Kubeconfig string
	// Namespace is the name of the Kubernetes namespace to use for running the test suite (i.e. where to create pods).
	Namespace string
	// NodeName is the name of the virtual-kubelet node to test.
	NodeName string
	// WatchTimeout is the duration for which the framework watch a particular condition to be satisfied (e.g. watches a pod  becoming ready)
	WatchTimeout time.Duration
	// Setup is a function that sets up provider-specific resource in the test suite
	Setup suite.SetUpFunc
	// Teardown is a function that tears down provider-specific resources from the test suite
	Teardown suite.TeardownFunc
	// ShouldSkipTest is a function that determines whether the test suite should skip certain tests
	ShouldSkipTest suite.ShouldSkipTestFunc
}

EndToEndTestSuiteConfig is the config passed to initialize the testing framework and test suite.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.