tests/

README

Rook Test Framework

The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of kubernetes. The framework also provides scripts for starting kubernetes using kubeadm or minikube so users can quickly spin up a kubernetes cluster.The Test framework is designed to install Rook, run tests, and uninstall Rook.

Requirements

1. Docker version => 1.2 && < 17.0
2. Ubuntu 16 (the framework has only been tested on this version)
3. Kubernetes with kubectl configured
4. Rook

Instructions

Setup

Install Kubernetes

You can choose any kubernetes flavor of your choice. The test framework only depends on kubectl being configured. The framework also provides scripts to install Kubernetes. There are two scripts to start the cluster:

1. Minikube (recommended for MacOS): Run minikube.sh to setup a single-node Minikube kubernetes.
   1. Minikube v0.21.0 and higher is supported. Older minikube versions do not have cephfs or rbd tools installed.
   2. If using Kubernetes v1.7.3 or higher, then Minikube v0.22.2 or higher is required.
2. Kubeadm (recommended for ubuntu): run kubeadm.sh to setup a single-node K8s cluster using kubeadm

These start-up scripts are designed to start any version of kubernetes, Based on KUBE_VERSION environment variable By default they start kubernetes version v1.7.5. e.g. To kubernetes version v1.6.7

export KUBE_VERSION=v1.6.7
tests/scripts/kubeadm.sh up

To stop the cluster : tests/scripts/kubeadm.sh clean or tests/scripts/miniube.sh clean

To copy rook images generated by build into minikube vm, run the following commands after minikube.sh up

tests/scripts/minikube.sh update
tests/scripts/minikube.sh helm

kubeadm.sh starts kubernetes on the host, so all the images generated by build should be available to the cluster.

Install Helm

Use helm.sh to install helm and set up rook charts defined under _output/charts (generated by build).

• To install and set up Helm charts for rook run tests/scripts/helm.sh up
• To clean up tests/scripts/helm.sh clean

Note: kubeadm.sh, minikube.sh and helm.sh scripts depend on some artifacts under _output dir generated during build time, these scripts should be run from project root. eg tests/script/kubeadm.sh up

Run Tests

From the project root do the following:

1. Build rook:

run make build

2. Start kubernetes

using one of the following:

• using Kubeadm

 tests/scripts/kubeadm.sh up
 tests/scripts/helm.sh up  

• using minikube

tests/scripts/minikube.sh up
tests/scripts/minikube.sh helm
tests/scripts/helm.sh up

3. Run integration tests:

Integration tests can be run using tests binary _output/tests/${platform}/integration that is generated during build time eg ~/integration -test.v

Test parameters

In addition to standard go tests parameters, the following custom parameters are available while running tests

Parameter	Description	Possible values	Default
rook_platform	platform rook needs to be installed on	kubernetes	kubernetes
k8s_version	version of Kubernetes to be installed	v1.6+	v1.8.5
rook_image	rook image name to be installed	valid image name	rook/rook
toolbox_image	toolbox image name to be installed	valid image name	rook/toolbox
skip_install_rook	skips installing rook (if already installed)	true or false	false

Running Tests with parameters.

To run all integration tests run

go test -v -timeout 1800s github.com/rook/rook/tests/integration

To run a specific suite (uses regex)

go test -v -timeout 1800s -run SmokeSuite github.com/rook/rook/tests/integration

To run specific tests inside a suite:

go test -v -timeout 1800s -run SmokeSuite github.com/rook/rook/tests/integration -testify.m TestRookClusterInstallation_smokeTest

To run specific without installing rook

go test -v -timeout 1800s -run SmokeSuite github.com/rook/rook/tests/integration --skip_install_rook

If the skip_install_rook flag is set to true, then rook is not uninstalled either.

Run Longhaul Tests

Longhaul tests are integration tests that run for extended period of time. A load profile can be configured using the following load test flags

Parameter	Description	Possible values	Default
load_parallel_runs	performs concurrent operations	any number	20
load_volumes	number of volumes	>1	1
load_time	number of seconds to run	>1	1800
load_size	size of load profile (3M, 10M, or 50M per thread)	small, medium, or large	medium
enable_chaos	kill random pods in rook cluster	true or false	false

e.g.

go test -run TestObjectLongHaul github.com/rook/rook/tests/longhaul --load_parallel_runs=20 --load_time 1800 --load_size small --load_volumes 3

The Longhaul test just like other test is going to install rook if it's not already installed, but it is not going to clean up test data or uninstall rook after the run. Longhaul test is designed to run multiple times on the same setup and installation of rook to tests its stability. Test Data and rook should be cleaned up manually after the test.

You can measure memory, cpu, IOPS, throughput, and other settings on a cluster using Prometheus. The metrics collected during load test can be visualize using Grafana. Here a couple of helpful links to get prometheus and grafana started and collect metrics: kube-prometheus and cluster-deploy-script

Prerequisites :

• Go installed and GO_PATH set
• Dep installed
• when running tests locally, make sure kubectl is accessible globally as the test framework uses kubectl