tests

package module
v0.0.0-...-cae6a2d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 28, 2021 License: Apache-2.0 Imports: 19 Imported by: 0

README

Kata Containers tests

This repository contains various types of tests and utilities (called "content" from now on) for testing the Kata Containers code repositories.

Getting the code

$ go get -d github.com/kata-containers/tests

Test Content

We provide several tests to ensure Kata-Containers run on different scenarios and with different container managers.

  1. Integration tests to ensure compatibility with:
  2. Stability tests
  3. Metrics
  4. VFIO

CI Content

This repository contains a number of scripts that run from under a "CI" (Continuous Integration) system.

Centralised scripts

The CI scripts in this repository are used to test changes to the content of this repository. These scripts are also used by the other Kata Containers code repositories.

The advantages of this approach are:

  • Functionality is defined once.

    • Easy to make changes affecting all code repositories centrally.

  • Assurance that all the code repositories are tested in this same way.

CI scripts also provide a convenient way for other Kata repositories to install software. The preferred way to use these scripts is to invoke make with the corresponding install- target. For example, to install CRI-O you would use:

$ make -C <path-to-this-repo> install-crio

Use make list-install-targets to retrieve all the available install targets.

CI setup

WARNING:

The CI scripts perform a lot of setup before running content under a CI. Some of this setup runs as the root user and could break your developer's system. See Developer Mode.

Controlling the CI
GitHub Actions

Kata Containers uses GitHub Actions in the Kata Containers repos. All those actions, apart from the one to test kata-deploy, are automatically triggered when a pull request is submitted. The trigger phrase for testing kata-deploy is /test_kata_deploy.

Jenkins

The Jenkins configuration and most documentation is kept in the CI repository. Jenkins is setup to trigger a CI run on all the slaves/nodes when a /test comment is added to a pull request. However, there are some specific comments that are defined for specific CI slaves/nodes which are defined in the Jenkins config.xml files in the <triggerPhase> XML element in the CI repository.

Specific Jenkins job triggers

Some jobs like a particular distro, feature or architecture can be triggered individually, the specific job triggers information can be found in the Community repository.

Detecting a CI system

The strategy to check if the tests are running under a CI system is to see if the CI variable is set to the value true. For example, in shell syntax:

if [ "$CI" = true ]; then
    : # Assumed to be running in a CI environment
else
    : # Assumed to NOT be running in a CI environment
fi
Breaking Compatibility

In case the patch you submit breaks the CI because it needs to be tested together with a patch from another kata-containers repository, you have to specify which repository and which pull request it depends on.

Using a simple tag Depends-on: in your commit message will allow the CI to run properly. Notice that this tag is parsed from the latest commit of the pull request.

For example:

	Subsystem: Change summary

	Detailed explanation of your changes.

	Fixes: #nnn

	Depends-on:github.com/kata-containers/kata-containers#999

	Signed-off-by: <contributor@foo.com>

In this example, we tell the CI to fetch the pull request 999 from the kata-containers repository and use that rather than the main branch when testing the changes contained in this pull request.

CLI tools

This repository contains a number of command line tools. They are used by the CI tests but may be useful for user to run stand alone.

Developer Mode

Developers need a way to run as much test content as possible locally, but as explained in CI Setup, running all the content in this repository could be dangerous.

The recommended approach to resolve this issue is to set the following variable to any non-blank value before using any content from this repository:

export KATA_DEV_MODE=true

Setting this variable has the following effects:

  • Disables content that might not be safe for developers to run locally.
  • Ignores the effect of the CI variable being set (for extra safety).

You should be aware that setting this variable provides a safe subset of functionality; it is still possible that PRs raised for code repositories will still fail under the automated CI systems since those systems are running all possible tests.

Write a new Unit Test

See the unit test advice documentation.

Run the Kata Containers tests

Requirements to run Kata Containers tests

You need to install the following to run Kata Containers tests:

Prepare an environment

The recommended method to set up Kata Containers is to use the official and latest stable release. You can find the official documentation to do this in the Kata Containers installation user guides.

To try the latest commits of Kata use the CI scripts, which build and install from the kata-containers repositories, with the following steps:

Warning: This may replace/delete packages and configuration that you already have. Please use these steps only on a testing environment.

Add the $GOPATH/bin directory to the PATH:

$ export PATH=${GOPATH}/bin:${PATH}

Clone the kata-container/tests repository:

$ go get -d github.com/kata-containers/tests

Go to the tests repo directory:

$ cd $GOPATH/src/github.com/kata-containers/tests

Execute the setup script:

$ .ci/setup.sh

Limitation: If the script fails for a reason and it is re-executed, it will execute all steps from the beginning and not from the failed step.

Run the tests

If you have already installed the Kata Containers packages and a container manager (i.e. Kubernetes), and you want to execute the content for all the tests, run the following:

$ export RUNTIME=kata-runtime
$ export KATA_DEV_MODE=true
$ sudo -E PATH=$PATH make test

You can also execute a single test suite. For example, if you want to execute the Kubernetes integration tests, run the following:

$ sudo -E PATH=$PATH make kubernetes

A list of available test suite make targets can be found by running the following:

$ make help
Running subsets of tests

Individual tests or subsets of tests can be selected to be run. The method of test selection depends on which type of test framework the test is written with. Most of the Kata Containers test suites are written using Bats files.

Running Bats based tests

The Bats based tests are shell scripts, starting with the line:

#!/usr/bin/env bats

This allows the Bats files to be executed directly. Before executing the file, ensure you have Bats installed. The Bats files should be executed from the root directory of the tests repository to ensure they can locate all other necessary components. An example of how a Bats test is run from the Makefile looks like:

kubernetes:
        bash -f .ci/install_bats.sh
        bash -f integration/kubernetes/run_kubernetes_tests.sh

Metrics tests

See the metrics documentation.

Kata Admission controller webhook

See the webhook documentation.

Using Vagrant to test your code changes

It is strongly recommended that you test your changes locally before opening a pull request as this can save people's time and CI resources. Because testing Kata Containers involve complex build and setup instructions, scripts on the .ci directory are created to ease and provide a reproducible process; but they are meant to run on CI environments that can be discarded after use. Therefore, developers have noticed dangerous side effects from running those scripts on a workstation or development environment.

That said, we provide in this repository a Vagrantfile which allows developers to use the vagrant tool to create a VM with the setup as close as as possible to the environments where CI jobs will run the tests. Thus, allowing to reproduce a CI job locally.

Currently it is only able to create a Fedora 32 or Ubuntu 20.04 VM. And your workstation must be capable of running VMs with:

  • 8GB of system memory
  • ~45GB and ~20GB of disk space for the VM images (Fedora and Ubuntu, respectively) on the Libvirt's storage pool

Besides having vagrant installed in your host, it is needed the vagrant libvirt plug-in (Libvirt is the provider currently used), QEMU and rsync (needed to copy files between the host and guest).

For example, to install the required software on Fedora host:

$ sudo dnf install -y qemu-kvm libvirt vagrant vagrant-libvirt rsync

Note: ensure that you don't have Kata Container's built QEMU overwritten the distro's in your host, otherwise Vagrant will not work.

Use the vagrant up [fedora|ubuntu] command to bring up the VM. Vagrant is going to pull (unless cached) the base VM image, provision it and then bootstrap the Kata Containers environment (essentially by sourcing environment variables and running the .ci/setup.sh script). For example:

$ cd ${GOPATH}/src/github.com/kata-containers/tests
$ vagrant up fedora

The following repositories are automatically copied to the guest:

  • ${GOPATH}/src/github.com/kata-containers/tests
  • ${GOPATH}/src/github.com/kata-containers/kata-containers

If you want to reproduce a specific CI job, ensure that you have the CI_JOB environment variable exported on your host environment before you run vagrant up. For the possible CI_JOB values, see the .ci/ci_job_flags.sh file. For example, the following will setup the VM to run CRI-O + Kubernetes job:

$ cd $GOPATH/src/github.com/kata-containers/tests
$ export CI_JOB="CRIO_K8S"
$ vagrant up fedora

At this point, if everything went well, you have a fully functional environment with Kata Containers built and installed. To connect in the VM and run the tests:

$ vagrant ssh fedora
$ .ci/run.sh

In theory you could export CI_JOB with a different value and re-provision the same VM (vagrant provision [fedora|ubuntu]), however this is not recommended because our CI scripts are meant for a single-shot execution. So if you need to run a different job locally, you should destroy the VM with the vagrant destroy [fedora|ubuntu] command then start the process again.

Documentation

Index

Constants

View Source 
const (
	// DefaultHypervisor default hypervisor
	DefaultHypervisor = "qemu"

	// FirecrackerHypervisor is firecracker
	FirecrackerHypervisor = "firecracker"

	// CloudHypervisor is cloud-hypervisor
	CloudHypervisor = "clh"

	// DefaultProxy default proxy
	DefaultProxy = "kata"

	// DefaultAgent default agent
	DefaultAgent = "kata"

	// DefaultShim default shim
	DefaultShim = "kata"

	// DefaultKataConfigPath is the default path to the kata configuration file
	DefaultKataConfigPath = "/usr/share/defaults/kata-containers/configuration.toml"
)

Variables

View Source 
var Hypervisor string

Hypervisor is the hypervisor currently being used with Kata

View Source 
var KataHypervisor string
View Source 
var Runtime string

Runtime is the path of a Kata Containers Runtime

View Source 
var Timeout int

Timeout specifies the time limit in seconds for each test

Functions

func HypervisorRunning 

func HypervisorRunning(containerID string) bool

HypervisorRunning returns true if the hypervisor is still running, otherwise false

func KataInit 

func KataInit()

KataInit initializes the kata test suite. This function should be called as soon as possible preferably from `TestMain`

func LogIfFail 

func LogIfFail(format string, args ...interface{})

LogIfFail will output the message online if the test fails. This can be used for information that would be useful to debug a failure.

func ProxyRunning 

func ProxyRunning(containerID string) bool

ProxyRunning returns true if the proxy is still running, otherwise false

func RandID 

func RandID(n int) string

RandID returns a random string

func ShimRunning 

func ShimRunning(containerID string) bool

ShimRunning returns true if the shim is still running, otherwise false

Types

type Bundle 

type Bundle struct {
	// Config represents the config.json
	Config *spec.Spec

	// Path to the bundle
	Path string
}

Bundle represents the root directory where config.json and rootfs are

func NewBundle 

func NewBundle(workload []string) (*Bundle, error)

NewBundle creates a new bundle

func (*Bundle) Remove 

func (b *Bundle) Remove() error

Remove the bundle files and directories

func (*Bundle) Save 

func (b *Bundle) Save() error

Save to disk the Config

type Command 

type Command struct {

	// Timeout is the time limit of seconds of the command
	Timeout time.Duration
	// contains filtered or unexported fields
}

Command contains the information of the command to run

func NewCommand 

func NewCommand(path string, args ...string) *Command

NewCommand returns a new instance of Command

func (*Command) Run 

func (c *Command) Run() (string, string, int)

Run runs a command returning its stdout, stderr and exit code

func (*Command) RunWithPipe 

func (c *Command) RunWithPipe(stdin *bytes.Buffer) (string, string, int)

RunWithPipe runs a command with stdin as an input and returning its stdout, stderr and exit code

type Container 

type Container struct {
	// Bundle contains the container information
	// if nil then try to run the container without --bundle option
	Bundle *Bundle

	// Console pty slave path
	// if nil then try to run the container without --console option
	Console *string

	// PidFile where process id is written
	// if nil then try to run the container without --pid-file option
	PidFile *string

	// LogFile where debug information is written
	// if nil then try to run the container without --log option
	LogFile *string

	// ID of the container
	// if nil then try to run the container without container ID
	ID *string

	// Detach allows to run the process detached from the shell
	Detach bool
}

Container represents a kata container

func NewContainer 

func NewContainer(workload []string, detach bool) (*Container, error)

NewContainer returns a new Container

func (*Container) Delete 

func (c *Container) Delete(force bool) (string, string, int)

Delete the container calls to delete command returning its stdout, stderr and exit code

func (*Container) Exec 

func (c *Container) Exec(process Process) (string, string, int)

Exec the container calls into exec command returning its stdout, stderr and exit code

func (*Container) Exist 

func (c *Container) Exist() bool

Exist returns true if any of next cases is true: - list command shows the container - the process id specified in the pid file is running (cc-shim) - the VM is running (qemu) - the proxy is running - the shim is running else false is returned

func (*Container) Kill 

func (c *Container) Kill(all bool, signal interface{}) (string, string, int)

Kill the container calls to kill command returning its stdout, stderr and exit code

func (*Container) List 

func (c *Container) List(format string, quiet bool, all bool) (string, string, int)

List the containers calls to list command returning its stdout, stderr and exit code

func (*Container) RemoveOption 

func (c *Container) RemoveOption(option string) error

RemoveOption removes a specific option container will run without the specific option

func (*Container) Run 

func (c *Container) Run() (string, string, int)

Run the container calls to run command returning its stdout, stderr and exit code

func (*Container) SetWorkload 

func (c *Container) SetWorkload(workload []string) error

SetWorkload sets a workload for the container

func (*Container) State 

func (c *Container) State() (string, string, int)

State returns the state of the container calls into state command returning its stdout, stderr and exit code

func (*Container) Teardown 

func (c *Container) Teardown() error

Teardown deletes the container if it is running, ensures the container is not running and removes any file created by the container

type KataConfiguration 

type KataConfiguration struct {
	Hypervisor map[string]hypervisor
	Proxy      map[string]proxy
	Shim       map[string]shim
	Agent      map[string]agent
	Runtime    runtime
}

KataConfiguration is the runtime configuration 

var KataConfig KataConfiguration

KataConfig is the runtime configuration

type Process 

type Process struct {
	ContainerID *string
	Console     *string
	Tty         *string
	Workload    []string
	Detach      bool
}

Process describes a process to be executed on a running container.

Source Files

View all Source files

Directories

Path Synopsis
cmd
check-markdown
checkcommits
checkmetrics
github-labels
log-parser
integration
docker

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.