e2e

package
v0.0.0-...-d19fc40 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 29, 2020 License: Apache-2.0 Imports: 25 Imported by: 0

README

E2E Tests

Prow will run ./e2e-tests.sh with authentication mechanism using Kubernetes Secrets and ./e2e-wi-tests.sh with authentication mechanism using Workload Identity.

Adding E2E Tests

E2E tests are tagged with // +build e2e but tagging a Go file this way will prevent the compiler from compiling the test code. To work around this, for the test code we separate them into different files:

test/e2e
├── e2e_test.go
└── test_xxx.go
  • e2e_test.go is the testing file entry point (tagged with e2e).
  • test_xxx.go are the test implementations (not tagged with e2e).

We leverage the test library in Eventing as much as possible for implementing the e2e tests. Logic specific to knative-gcp should be added under knative-gcp e2e test lib.

Setup a test cluster

Run the following command:

SKIP_TESTS=true ./test/e2e-tests.sh --skip-teardowns

SKIP_TESTS will skip the actual tests so only the cluster initialization is run. --skip-teardowns tells the script to not tear down the cluster. This command runs the cluster initialization logic and leaves the cluster in a state that's ready to run tests.

If you run into Something went wrong: error creating deployer: --gcp-zone and --gcp-region cannot both be set, you may have set the ZONE environment variable. Try unset ZONE.

Running E2E Tests on an existing cluster

Prerequisites

There are two ways to set up authentication mechanism.

  • (GKE Specific) If you want to run E2E tests with Workload Identity as the authentication mechanism, please follow below instructions to configure the authentication mechanism with Workload Identity.
  • If you want to run E2E tests with Kubernetes Secrets as the authentication mechanism, please follow below instructions to configure the authentication mechanism with Kubernetes Secrets.

  1. A running Kubernetes cluster with knative-gcp installed and configured.

  2. Create a Service Account for the Data Plane. Download a credential file and set GOOGLE_APPLICATION_CREDENTIALS env var. This is used by some tests(e.g., TestSmokePullSubscription) to authorize the Google SDK clients.

    cred_file=$(pwd)/cre-dataplane.json
gcloud iam service-accounts keys create ${cred_file} --iam-account=cre-dataplane@$PROJECT_ID.iam.gserviceaccount.com
export GOOGLE_APPLICATION_CREDENTIALS=${cred_file}

  3. Install GCP Broker.

  4. Broker with Pub/Sub Channel installed.

  5. CloudSchedulerSource Prerequisites. Note that you only need to:

    1. Create with an App Engine application in your project.

  6. CloudStorageSource Prerequisites. Note that you only need to:

    1. Enable the Cloud Storage API on your project.
    2. Give Google Cloud Storage permissions to publish to GCP Pub/Sub.

  7. A docker repo containing the test images. Remember to specify the build tag e2e.

  8. (Optional) Note that if you plan on running metrics-related E2E tests using the StackDriver backend, you need to give your Service Account the monitoring.metricWriter role on your Google Cloud project:

    gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member=serviceAccount:${your_service_account}@$PROJECT_ID.iam.gserviceaccount.com \
  --role roles/monitoring.metricWriter

    If you also plan on running tracing-related E2E tests using the StackDriver backend, your Service Account needs additional cloudtrace.admin role:

    gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member=serviceAccount:"${your_service_account}"@$PROJECT_ID.iam.gserviceaccount.com \
  --role roles/cloudtrace.admin

  9. (Optional) Note that if plan on running tracing-related E2E tests using the Zipkin backend, you need to install zipkin-in-mem and patch the configmap config-tracing in the knative-eventing namespace to use the Zipkin backend as the with patch-config-tracing-configmap-with-zipkin.yaml.

    kubectl patch configmap config-tracing -n knative-eventing --patch "\$(cat patch-config-tracing-configmap-with-zipkin.yaml)"
Running E2E tests
Running E2E tests with authentication mechanism using Kubernetes Secrets
E2E_PROJECT_ID=<project name> \
  go test --tags=e2e ./test/e2e/...

And count is supported too:

E2E_PROJECT_ID=<project name> \
  go test --tags=e2e ./test/e2e/... --count=3

If you want to run a specific test:

E2E_PROJECT_ID=<project name> \
  go test --tags=e2e ./test/e2e/... -run NameOfTest

For example, to run TestPullSubscription:

E2E_PROJECT_ID=<project name> \
  go test --tags=e2e ./test/e2e/... -run TestPullSubscription
Running E2E tests with authentication mechanism using Workload Identity.

First, you'll have to modify clusterDefaults in ConfigMap config-gcp-auth.

You can directly edit the ConfigMap by:

kubectl edit configmap config-gcp-auth -n cloud-run-events

and replace the default-auth-config: part with:

  default-auth-config: |
    clusterDefaults:
      serviceAccountName: test-default-ksa
      workloadIdentityMapping:
        test-default-ksa: $PUBSUB_SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com

$PUBSUB_SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com is the Pub/Sub enabled Google Cloud Service Account.

Then, add -workloadIndentity=true and -serviceAccountName=test-default-ksa to the go test command.

For example,

E2E_PROJECT_ID=<project name> go test --tags=e2e ./test/e2e/... \
  -workloadIndentity=true \
  -serviceAccountName=test-default-ksa \
  -run TestPullSubscription

Running E2E Tests on an new cluster

Prerequisites

  1. Enable necessary APIs:

    gcloud services enable compute.googleapis.com
gcloud services enable container.googleapis.com

  2. Install kubetest. (Note this is just a workaround because of kubernetes issue

  3. Set the project you want to run E2E tests to be the default one with:

    export PROJECT=<REPLACE_ME>
gcloud config set core/project $PROJECT
Running E2E tests

If you want to run E2E tests with authentication mechanism using Kubernetes Secrets:

./test/e2e-tests.sh

If you want to run E2E tests with authentication mechanism using Workload Identity:

./test/e2e-wi-tests.sh

Test images

Building the test images

Note: this is only required when you run e2e tests locally with go test commands. Running tests through e2e-tests.sh will publish the images automatically.

The upload-test-images.sh script can be used to build and push the test images used by the e2e tests. It requires:

For images deployed in GCR, a docker tag is mandatory to avoid issues with using latest tag:

./test/upload-test-images.sh ./test/test_images e2e
sed -i 's@ko://knative.dev/eventing/test/test_images@ko://github.com/aavarghese/knative-gcp/vendor/knative.dev/eventing/test/test_images@g' vendor/knative.dev/eventing/test/test_images/*/*.yaml
./test/upload-test-images.sh ./vendor/knative.dev/eventing/test/test_images/ e2e

To run the script for all end to end test images:

./test/upload-test-images.sh ./test/test_images
sed -i 's@ko://knative.dev/eventing/test/test_images@ko://github.com/aavarghese/knative-gcp/vendor/knative.dev/eventing/test/test_images@g' vendor/knative.dev/eventing/test/test_images/*/*.yaml
./test/upload-test-images.sh ./vendor/knative.dev/eventing/test/test_images/
Adding new test images

New test images should be placed in ./test/test_images. For each image create a new sub-folder and include a Go file that will be an entry point to the application. This Go file should use the package main and include the function main(). It is a good practice to include a README file as well. When uploading test images, ko will build an image from this folder and upload to the Docker repository configured as KO_DOCKER_REPO.

Troubleshooting E2E Tests

Prow

Each PR will trigger E2E tests. For failed tests, follow the prow links on the PR page. Such links are in the format of https://prow.knative.dev/view/gcs/knative-prow/pr-logs/pull/google_knative-gcp/[PR ID]/[TEST NAME]/[TEST ID] , e.g. https://prow.knative.dev/view/gcs/knative-prow/pr-logs/pull/google_knative-gcp/1153/pull-google-knative-gcp-integration-tests/1267481606424104960 .

If the prow page doesn't provide any useful information, check out the full logs dump.

  • The control plane pods (in cloud-run-events namespace) logs dump are at https://console.cloud.google.com/storage/browser/knative-prow/pr-logs/pull/google_knative-gcp/[PR ID]/[TEST NAME]/[TEST ID]/artifacts/controller-logs/ .
  • The data plane pods logs dump are at https://console.cloud.google.com/storage/browser/knative-prow/pr-logs/pull/google_knative-gcp/[PR ID]/[TEST NAME]/[TEST ID]/artifacts/pod-logs/ .
Local

Add CI=trueto the go test command.

  • The data plane pods logs dump are at $GOPATH/src/github.com/aavarghese/knative-gcp/test/e2e/artifacts/pod-logs .

For example:

CI=true E2E_PROJECT_ID=$PROJECT_ID \
 go test --tags=e2e ./test/e2e/...

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AuditLogsSourceBrokerWithGCPBrokerTestImpl 

func AuditLogsSourceBrokerWithGCPBrokerTestImpl(t *testing.T, authConfig lib.AuthConfig)

func AuditLogsSourceBrokerWithPubSubChannelTestImpl 

func AuditLogsSourceBrokerWithPubSubChannelTestImpl(t *testing.T, authConfig lib.AuthConfig)

func BrokerWithPubSubChannelTestImpl 

func BrokerWithPubSubChannelTestImpl(t *testing.T, authConfig lib.AuthConfig)

func CloudAuditLogsSourceWithTargetTestImpl 

func CloudAuditLogsSourceWithTargetTestImpl(t *testing.T, authConfig lib.AuthConfig)

func CloudBuildSourceWithTargetTestImpl 

func CloudBuildSourceWithTargetTestImpl(t *testing.T, authConfig lib.AuthConfig)

CloudBuildSourceWithTargetTestImpl tests we can receive an event from a CloudBuildSource.

func CloudPubSubSourceWithTargetTestImpl 

func CloudPubSubSourceWithTargetTestImpl(t *testing.T, assertMetrics bool, authConfig lib.AuthConfig)

CloudPubSubSourceWithTargetTestImpl tests we can receive an event from a CloudPubSubSource. If assertMetrics is set to true, we also assert for StackDriver metrics.

func CloudSchedulerSourceWithTargetTestImpl 

func CloudSchedulerSourceWithTargetTestImpl(t *testing.T, authConfig lib.AuthConfig)

CloudSchedulerSourceWithTargetTestImpl injects a scheduler event and checks if it is in the log of the receiver.

func CloudStorageSourceWithTargetTestImpl 

func CloudStorageSourceWithTargetTestImpl(t *testing.T, assertMetrics bool, authConfig lib.AuthConfig)

func GCPBrokerMetricsTestImpl 

func GCPBrokerMetricsTestImpl(t *testing.T, authConfig lib.AuthConfig)

func GCPBrokerTestImpl 

func GCPBrokerTestImpl(t *testing.T, authConfig lib.AuthConfig)

func GCPBrokerTracingTestImpl 

func GCPBrokerTracingTestImpl(t *testing.T, authConfig lib.AuthConfig)

func PubSubSourceBrokerWithPubSubChannelTestImpl 

func PubSubSourceBrokerWithPubSubChannelTestImpl(t *testing.T, authConfig lib.AuthConfig)

func PubSubSourceWithGCPBrokerTestImpl 

func PubSubSourceWithGCPBrokerTestImpl(t *testing.T, authConfig lib.AuthConfig)

func PullSubscriptionWithTargetTestImpl 

func PullSubscriptionWithTargetTestImpl(t *testing.T, authConfig lib.AuthConfig)

PullSubscriptionWithTargetTestImpl tests we can receive an event from a PullSubscription.

func SchedulerSourceBrokerWithPubSubChannelTestImpl 

func SchedulerSourceBrokerWithPubSubChannelTestImpl(t *testing.T, authConfig lib.AuthConfig)

func SchedulerSourceWithGCPBrokerTestImpl 

func SchedulerSourceWithGCPBrokerTestImpl(t *testing.T, authConfig lib.AuthConfig)

func SmokeCloudAuditLogsSourceTestHelper 

func SmokeCloudAuditLogsSourceTestHelper(t *testing.T, authConfig lib.AuthConfig, cloudAuditLogsSourceVersion string)

SmokeCloudAuditLogsSourceTestHelper tests if a CloudAuditLogsSource object can be created to ready state.

func SmokeCloudAuditLogsSourceWithDeletionTestImpl 

func SmokeCloudAuditLogsSourceWithDeletionTestImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeCloudAuditLogsSourceWithDeletionTestImpl tests if a CloudAuditLogsSource object can be created to ready state and delete a CloudAuditLogsSource resource and its underlying resources..

func SmokeCloudBuildSourceTestHelper 

func SmokeCloudBuildSourceTestHelper(t *testing.T, authConfig lib.AuthConfig, cloudBuildSourceVersion string)

SmokeCloudBuildSourceWithDeletionTestHelper tests we can create a CloudBuildSource to ready state.

func SmokeCloudBuildSourceWithDeletionTestImpl 

func SmokeCloudBuildSourceWithDeletionTestImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeCloudBuildSourceWithDeletionTestImpl tests we can create a CloudBuildSource to ready state and we can delete a CloudBuildSource and its underlying resources.

func SmokeCloudPubSubSourceTestHelper 

func SmokeCloudPubSubSourceTestHelper(t *testing.T, authConfig lib.AuthConfig, cloudPubSubSourceVersion string)

SmokeCloudPubSubSourceTestHelper tests we can create a CloudPubSubSource to ready state.

func SmokeCloudPubSubSourceWithDeletionTestImpl 

func SmokeCloudPubSubSourceWithDeletionTestImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeCloudPubSubSourceWithDeletionTestImpl tests we can create a CloudPubSubSource to ready state and we can delete a CloudPubSubSource and its underlying resources.

func SmokeCloudSchedulerSourceTestHelper 

func SmokeCloudSchedulerSourceTestHelper(t *testing.T, authConfig lib.AuthConfig, cloudSchedulerSourceVersion string)

SmokeCloudSchedulerSourceTestHelper tests if a CloudSchedulerSource object can be created to ready state.

func SmokeCloudSchedulerSourceWithDeletionTestImpl 

func SmokeCloudSchedulerSourceWithDeletionTestImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeCloudSchedulerSourceWithDeletionTestImpl tests if a CloudSchedulerSource object can be created to ready state and delete a CloudSchedulerSource resource and its underlying resources..

func SmokeCloudStorageSourceTestHelper 

func SmokeCloudStorageSourceTestHelper(t *testing.T, authConfig lib.AuthConfig, cloudStorageSourceVersion string)

SmokeCloudStorageSourceTestHelper tests we can create a CloudStorageSource to ready state.

func SmokeCloudStorageSourceWithDeletionTestImpl 

func SmokeCloudStorageSourceWithDeletionTestImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeCloudStorageSourceWithDeletionTestImpl tests if a CloudStorageSource object can be created to ready state and delete a CloudStorageSource resource and its underlying resources..

func SmokeGCPBrokerTestImpl 

func SmokeGCPBrokerTestImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeGCPBrokerTestImpl tests we can create a GCPBroker to ready state and we can delete a GCPBroker and its underlying resources.

func SmokePullSubscriptionTestHelper 

func SmokePullSubscriptionTestHelper(t *testing.T, authConfig lib.AuthConfig, pullsubscriptionVersion string)

SmokePullSubscriptionTestHelper tests we can create a pull subscription to ready state.

func SmokeTestChannelImpl 

func SmokeTestChannelImpl(t *testing.T, authConfig lib.AuthConfig)

SmokeTestChannelImpl makes sure we can run tests.

func StorageSourceBrokerWithPubSubChannelTestImpl 

func StorageSourceBrokerWithPubSubChannelTestImpl(t *testing.T, authConfig lib.AuthConfig)

func StorageSourceWithGCPBrokerTestImpl 

func StorageSourceWithGCPBrokerTestImpl(t *testing.T, authConfig lib.AuthConfig)

Types

This section is empty.

Source Files

View all Source files

Directories

Path Synopsis
lib
Utility functions used for testing
 Utility functions used for testing
helpers
metrics
operations
resources

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.