Rook Test Framework
The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of Kubernetes.
The framework also provides scripts for starting Kubernetes using kubeadm
or minikube
so users can
quickly spin up a Kubernetes cluster.The Test framework is designed to install Rook, run tests, and uninstall Rook.
Requirements
- Docker version => 1.2 && < 17.0 (for other alternatives, see below)
- Ubuntu 18 (the framework has only been tested on this version)
- kernel >= 4.15
- Kubernetes with kubectl configured
- Rook
Instructions
Setup
Container Runtime
By default, the test suite tries to use the docker
command line tool. To use
an alternative container runtime tool, set the DOCKERCMD
environment variable
to your command line tool of choice.
Install Kubernetes
You can choose any Kubernetes flavor of your choice. The test framework only depends on kubectl being configured.
The framework also provides scripts to install Kubernetes. There are two scripts to start the cluster:
- Kubeadm (recommended for Ubuntu): run kubeadm.sh to setup a single-node K8s cluster using kubeadm
- Minikube (recommended for non-Ubuntu): Run minikube.sh to setup a single-node Minikube Kubernetes.
Minikube scripts only support "docker" and "none" as vm-driver.
Minikube (recommended for non-Ubuntu)
Starting the cluster on Minikube is as simple as running:
tests/scripts/minikube.sh up
To copy Rook images generated from your local build into the Minikube VM, run the following commands after minikube.sh up
succeeded:
tests/scripts/minikube.sh update
tests/scripts/minikube.sh helm
Stopping the cluster and destroying the Minikube VM can be done with:
tests/scripts/minikube.sh clean
Kubeadm (recommended for Ubuntu)
Starting the cluster using kubeadm
is as simple as running:
tests/scripts/kubeadm.sh up
kubeadm.sh
starts Kubernetes on the host, so all the images generated by your local build should be available to the cluster without any additional commands.
You can change the Kubernetes version (default is v1.15.12) with KUBE_VERSION
environment variable. Here is the example of using Kubernetes v1.19.0.
export KUBE_VERSION=v1.19.0
tests/scripts/kubeadm.sh up
Please note that this script may downgrade the following deb packages.
- kubernetes-cni
- kubelet
- kubeadm
Stopping the cluster can be done with:
tests/scripts/kubeadm.sh clean
Alternate Kubernetes Versions
These two scripts can install any version of Kubernetes you wish based on the KUBE_VERSION
environment variable.
To use an alternate version, simply set this variable before running the relevant up
command from above.
For example, if you wanted to use v1.12.5
, you would run export KUBE_VERSION=v1.12.5
first before running up
.
Run Tests
From the project root do the following:
1. Build rook:
Run make build
2. Start Kubernetes
Using one of the following:
tests/scripts/kubeadm.sh up
export KUBECONFIG=~/admin.conf
tests/scripts/helm.sh up
tests/scripts/minikube.sh up
tests/scripts/minikube.sh helm
tests/scripts/helm.sh up
3. Run integration tests:
Some settings are available to run the tests under different environments. The settings are all configured with environment variables.
See environment.go for the available environment variables.
At least you should set the following variables.
export TEST_BASE_DIR=WORKING_DIR
export TEST_SCRATCH_DEVICE=<block device> # for example, TEST_SCRATCH_DEVICE=/dev/sdb
Please note that the integration test erases the contents of TEST_SCRATCH_DEVICE.
To run all integration tests:
go test -v -timeout 7200s github.com/rook/cassandra/tests/integration
After running tests, you can get test logs under "tests/integration/_output".