e2e

README

E2E Tests

These tests are adapted from the Service test suite in the Kubernetes core E2E tests.

Pre-requisite

You need to have ginkgo installed and configured.

Running the tests locally

If you already have a cluster and wish to run the tests on your existing cluster.

An example of cloud-provider.yaml - See provider-config-example.yaml For accessing you cluster's kubeconfig refer organize-cluster-access-kubeconfig

NOTE: Test suite will fail if executed behind a $HTTP_PROXY that returns a 200 OK response upon failure to connect.

Define the environment variables as explained here in your shell environment and run below command -

source hack/existing-standalone-cluster-env-template.sh

Then run

make run-ccm-e2e-tests-local

Additional option to specify test image pull repo

The tests use below images -

• nginx:stable-alpine
• agnhost:2.6
• centos:latest
• busybox:latest

By default, public images are used. But if your Cluster's environment cannot access above public images then below option can be used to specify an accessible repo.

export IMAGE_PULL_REPO="accessiblerepo.com/repo/path/" 
make run-ccm-e2e-tests-local

Note: Above listed : should be available in the provider repo path and should be accessible.

Running FSS in-transit encryption tests

The statically provisioned FSS in-transit encryption tests need the oci-fss-utils package to be installed on the nodes running the tests. The package is needed to be downloaded from here and installed on the applicable nodes. In addition to this, the below label is to be added using the command for each of the nodes which have the package installed to help the test suite recognise the applicable nodes.

kubectl label nodes <node-name> oke.oraclecloud.com/e2e.oci-fss-util=installed

Please refer FSS Network Setup for FSS setup.

Running Block Volume expansion tests

The BV expand tests need the cluster version and nodepool version to be 1.19+

Running Ultra High Performance tests (UHP)

Please refer Block Volume Ultra High Performance Doc for details on UHP Block Volumes.

Pre-requisite for UHP:

1. Nodes should have 16+ cores. Please refer Shapes Block Details Doc for exact compute shape requirements for the node.
2. Enabling Block Volume Management on Existing Instances

To run UHP E2Es set the environment variable RUN_UHP_E2E=true here. Then run the E2Es.

Set FOCUS=[UHP] to run only the UHP E2Es.

Additional Debug Options when running tests on existing cluster

Additional seclist count based sanity checks can be applied during e2e testing by providing the appropriate seclist ocids. Both must be supplied. If you wish to use the tests in debug mode and want to look at the namespaces created by tests you can provide the option to keep the namespaces

--delete-namespaces=false

By default the namespaces will be deleted. So if you use above option, then you should delete the resources manually after investigation is complete. These values can be specified as command line parameters.

$ ginkgo -v -progress test/e2e/cloud-provider-oci -- \
    --clusterkubeconfig=$CLUSTER_KUBECONFIG \
    --delete-namespace=false \
    --cloud-config=$CLOUD_CONFIG \
    --adlocation=$ADLOCATION \
    --ccm-seclist-id=ocid1.securitylist.$ccmloadblancerid \
    --k8s-seclist-id=ocid1.securitylist.$k8sworkerid \
    --cmek-kms-key=${CMEK_KMS_KEY} \
    --nsg-ocids=${NSG_OCIDS} \
    --reserved-ip=${RESERVED_IP}\
    --volume-handle=${VOLUME_HANDLE}

---

Running subsets of the cloud-provider-oci-e2e-tests (focused tests)

You can run a subset of the tests manually by setting the 'FOCUS' environment variable to be the regular expressions matching the 'Ginkgo' descriptions of the test you want to run.

Please see 'test/e2e/cloud-provider-oci' to see what description tags are available. The broad category of tags is "ccm" and "storage" look at list of tests

export FOCUS=\[cloudprovider\]

They can be passed directly to the make target as well:

FOCUS="\[cloudprovider\]" make run-ccm-e2e-tests-local

Another way you can run a subset of the tests manually is by setting the 'FOCUS' environment variable to be the regular expression for the test files you want to run. In this case you must also set an environment variable 'FILES=true' so Gingko knows to interperate the expression in that way.

export FOCUS="instances.go"
export FILES="true"

They can be passed directly to the make target as well:

make run-ccm-e2e-tests-local FOCUS="load_*" FILES="true"

---

Documentation

Index

• Constants
• func CreateHealthCheckScript(healthCheckNodePort int, ips []string, path string, nodeIndex int) string
• func TestCMEKAttachmentTypeAndEncryptionType(f *framework.CloudProviderFramework, expectedAttachmentType string, ...)
• func TestEncryptionType(f *framework.CloudProviderFramework, mountOptions []string)

Constants

const (
	WriteCommand                    = "echo 'Hello World' > /usr/share/nginx/html/testdata.txt; while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"
	WriteCommandBlock               = "echo 'Hello World' > /var/test.txt; dd if=/var/test.txt of=/dev/xvda count=8; while true; do sleep 5; done"
	KeepAliveCommand                = "while true; do echo 'hello world' >> /usr/share/nginx/html/out.txt; sleep 5; done"
	KeepAliveCommandBlock           = "while true; do echo 'hello world' >> /var/newpod.txt; sleep 5; done"
	BVDriverName                    = "blockvolume.csi.oraclecloud.com"
	BindingModeWaitForFirstConsumer = "WaitForFirstConsumer"
	ReclaimPolicyDelete             = "Delete"
	ReclaimPolicyRetain             = "Retain"
)

Functions

func CreateHealthCheckScript

func CreateHealthCheckScript(healthCheckNodePort int, ips []string, path string, nodeIndex int) string

ips is the list of private IPs of the nodes, the path is the endpoint at which health is checked, and nodeIndex is the node which has the current pod

func TestCMEKAttachmentTypeAndEncryptionType

func TestCMEKAttachmentTypeAndEncryptionType(f *framework.CloudProviderFramework, expectedAttachmentType string, isRawBlockVolume bool)

func TestEncryptionType

func TestEncryptionType(f *framework.CloudProviderFramework, mountOptions []string)