test

package
v0.41.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2022 License: Apache-2.0 Imports: 83 Imported by: 12

README

Tests

To run tests:

Unit tests and build tests (those run by presubmits) run against your Pipelines clone:

# Unit tests
go test ./...
# Build tests
./test/presubmit-tests.sh --build-tests

E2E tests run test cases in your local Pipelines clone against the Pipelines installation on your current kube cluster. To ensure your local changes are reflected on your cluster, you must first build and install them with ko apply -R -f ./config/.

# Integration tests
go test -v -count=1 -tags=e2e -timeout=20m ./test

# Conformance tests
go test -v -count=1 -tags=conformance -timeout=10m ./test

By running the commands above, you start the tests on the cluster of current-context in local kubeconfig file (~/.kube/config by default) in you local machine.

Sometimes local tests pass but presubmit tests fail, one possible reason is the difference of running environments. The envs that our presubmit test uses are stored in ./*.env files. Specifically,

Unit tests

Unit tests live side by side with the code they are testing and can be run with:

go test ./...

By default go test will not run the end to end tests, which need -tags=e2e to be enabled.

Unit testing Controllers

Kubernetes client-go provides a number of fake clients and objects for unit testing. The ones we are using are:

  1. Fake Kubernetes client: Provides a fake REST interface to interact with Kubernetes API
  2. Fake pipeline client : Provides a fake REST PipelineClient Interface to interact with Pipeline CRDs.

You can create a fake PipelineClient for the Controller under test like this:

import (
    fakepipelineclientset "github.com/tektoncd/pipeline/pkg/client/clientset/versioned/fake
)
pipelineClient := fakepipelineclientset.NewSimpleClientset()

This pipelineClient is initialized with no runtime objects. You can also initialize the client with Kubernetes objects and can interact with them using the pipelineClient.Pipeline()

import (
    v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

obj := &v1alpha1.PipelineRun {
    ObjectMeta: metav1.ObjectMeta {
        Name:      "name",
        Namespace: "namespace",
    },
    Spec: v1alpha1.PipelineRunSpec {
        PipelineRef: v1alpha1.PipelineRef {
            Name:       "test-pipeline",
            APIVersion: "a1",
        },
    }
}
pipelineClient := fakepipelineclientset.NewSimpleClientset(obj)
objs := pipelineClient.Pipeline().PipelineRuns("namespace").List(v1.ListOptions{})
// You can verify if List was called in your test like this
action :=  pipelineClient.Actions()[0]
if action.GetVerb() != "list" {
    t.Errorf("expected list to be called, found %s", action.GetVerb())
}

To test the Controller of CRD (CustomResourceDefinitions), you need to add the CRD to the informers so that the listers can get the access.

For example, the following code will test PipelineRun

pipelineClient := fakepipelineclientset.NewSimpleClientset()
sharedInfomer := informers.NewSharedInformerFactory(pipelineClient, 0)
pipelineRunsInformer := sharedInfomer.Pipeline().V1alpha1().PipelineRuns()

obj := &v1alpha1.PipelineRun {
    ObjectMeta: metav1.ObjectMeta {
        Name:      "name",
        Namespace: "namespace",
    },
    Spec: v1alpha1.PipelineRunSpec {
        PipelineRef: v1alpha1.PipelineRef {
            Name:       "test-pipeline",
            APIVersion: "a1",
        },
    }
}
pipelineRunsInformer.Informer().GetIndexer().Add(obj)

End to end tests

Setup

Environment variables used by end to end tests:

  • KO_DOCKER_REPO - Set this to an image registry your tests can push images to

  • GCP_SERVICE_ACCOUNT_KEY_PATH - Tests that need to interact with GCS buckets will use the json credentials at this path to authenticate with GCS.

  • SYSTEM_NAMESPACE - Set this to your Tekton deployment namespace like tekton-pipelines. Without this setting, the E2E test will use knative-testing as default namespace.

  • In Kaniko e2e test, setting GCP_SERVICE_ACCOUNT_KEY_PATH as the path of the GCP service account JSON key which has permissions to push to the registry specified in KO_DOCKER_REPO will enable Kaniko to use those credentials when pushing an image.

  • In GCS taskrun test, GCP service account JSON key file at path GCP_SERVICE_ACCOUNT_KEY_PATH, if present, is used to generate Kubernetes secret to access GCS bucket.

  • In Storage artifact bucket test, the GCP_SERVICE_ACCOUNT_KEY_PATH JSON key is used to create/delete a bucket which will be used for output to input linking by the PipelineRun controller.

To create a service account usable in the e2e tests:

PROJECT_ID=your-gcp-project
ACCOUNT_NAME=service-account-name
# gcloud configure project
gcloud config set project $PROJECT_ID

# create the service account
gcloud iam service-accounts create $ACCOUNT_NAME --display-name $ACCOUNT_NAME
EMAIL=$(gcloud iam service-accounts list | grep $ACCOUNT_NAME | awk '{print $2}')

# add the storage.admin policy to the account so it can push containers
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:$EMAIL --role roles/storage.admin

# create the JSON key
gcloud iam service-accounts keys create config.json --iam-account=$EMAIL

export GCP_SERVICE_ACCOUNT_KEY_PATH="$PWD/config.json"

export SYSTEM_NAMESPACE=tekton-pipelines
Running

End to end tests live in this directory. To run these tests, you must provide go with -tags=e2e. By default the tests run against your current kubeconfig context, but you can change that and other settings with the flags:

go test -v -count=1 -tags=e2e -timeout=20m ./test
go test -v -count=1 -tags=e2e -timeout=20m ./test --kubeconfig ~/special/kubeconfig --cluster myspecialcluster

If tests are applied to the cluster with hardware architecture different to the base one (for instance go test starts on amd64 architecture and --kubeconfig points to s390x Kubernetes cluster), use TEST_RUNTIME_ARCH environment variable to specify the target hardware architecture(amd64, s390x, ppc64le, arm, arm64, etc)

You can also use all of flags defined in knative/pkg/test.

To include tests for Windows, you need to specify the windows_e2e build tag. For example:

go test -v -count=1 -tags=e2e,windows_e2e -timeout=20m ./test

Please note that in order to run Windows tests there must be at least one Windows node available in the target Kubernetes cluster.

Flags
  • By default the e2e tests against the current cluster in ~/.kube/config using the environment specified in your environment variables.
  • Since these tests are fairly slow, running them with logging enabled is recommended (-v).
  • Using --logverbose to see the verbose log output from test as well as from k8s libraries.
  • Using -count=1 is the idiomatic way to disable test caching.
  • The end to end tests take a long time to run so a value like -timeout=20m can be useful depending on what you're running
  • TestKanikoTaskRun requires containers to run with root user. Using -skipRootUserTests=true skips it.

You can use test flags to control the environment your tests run against, i.e. override your environment variables:

go test -v -tags=e2e -count=1 ./test --kubeconfig ~/special/kubeconfig --cluster myspecialcluster

Tests importing github.com/tektoncd/pipeline/test recognize the flags added by knative/pkg/test.

Tests are run in a new random namespace prefixed with the word arendelle-. Unless you set the TEST_KEEP_NAMESPACES environment variable they will get automatically cleaned up after running the test.

Running specific test cases

To run all the test cases with their names starting with the same letters, e.g. TestTaskRun, use the -run flag with go test:

go test -v -tags=e2e -count=1 ./test -run ^TestTaskRun
Running YAML tests

To run the YAML e2e tests, run the following command:

go test -v -count=1 -tags=examples -timeout=20m ./test/

To limit parallelism of tests, use -parallel=n where n is the number of tests to run in parallel.

Running upgrade tests

There are two scenarios in upgrade tests. One is to install the previous release, upgrade to the current release, and validate whether the Tekton pipeline works. The other is to install the previous release, create the pipelines and tasks, upgrade to the current release, and validate whether the Tekton pipeline works.

To run the upgrade tests, run the following command:

./test/e2e-tests-upgrade.sh
Adding integration tests

In the test dir you will find several libraries in the test package you can use in your tests.

This library exists partially in this directory and partially in knative/pkg/test.

The libs in this dir can:

All integration tests must be marked with the e2e build constraint so that go test ./... can be used to run only the unit tests, i.e.:

// +build e2e
Get access to client objects

To initialize client objects use the command line flags which describe the environment:

func setup(t *testing.T) *test.Clients {
    clients, err := test.NewClients(kubeconfig, cluster, namespaceName)
    if err != nil {
        t.Fatalf("Couldn't initialize clients: %v", err)
    }
    return clients
}

The Clients struct contains initialized clients for accessing:

For example, to create a Pipeline:

_, err = clients.PipelineClient.Pipelines.Create(test.Route(namespaceName, pipelineName))

And you can use the client to clean up resources created by your test (e.g. in your test cleanup):

func tearDown(clients *test.Clients) {
    if clients != nil {
        clients.Delete([]string{routeName}, []string{configName})
    }
}

See clients.go.

Generate random names

You can use the function GenerateName() to append a random string for crds or anything else, so that your tests can use unique names each time they run.

import "github.com/tektoncd/pipeline/pkg/names"

namespace := names.SimpleNameGenerator.GenerateName("arendelle")
Poll Pipeline resources

After creating Pipeline resources or making changes to them, you will need to wait for the system to realize those changes. You can use polling methods to check the resources reach the desired state.

The WaitFor* functions use the Kubernetes wait package. For polling they use PollImmediate behind the scene. And the callback function is ConditionFunc, which returns a bool to indicate if the function should stop, and an error to indicate if there was an error.

For example, you can poll a TaskRun until having a Status.Condition:

err = WaitForTaskRunState(c, hwTaskRunName, func(tr *v1alpha1.TaskRun) (bool, error) {
    if len(tr.Status.Conditions) > 0 {
        return true, nil
    }
    return false, nil
}, "TaskRunHasCondition")

Metrics will be emitted for these Wait methods tracking how long test poll for.

Conformance tests

Conformance tests live in this directory. These tests are used to check API specs of Pipelines. To run these tests, you must provide go with -tags=conformance. By default, the tests run against your current kubeconfig context, but you can change that and other settings with the flags like the end to end tests:

go test -v -count=1 -tags=conformace -timeout=10m ./test
go test -v -count=1 -tags=conformace -timeout=10m ./test --kubeconfig ~/special/kubeconfig --cluster myspecialcluster

Flags that could be set in conformance tests are exactly the same as flags in end to end tests. Just note that the build tags should be -tags=conformance.

Presubmit tests

presubmit-tests.sh is the entry point for all tests run on presubmit by Prow.

You can run this locally with:

test/presubmit-tests.sh
test/presubmit-tests.sh --build-tests
test/presubmit-tests.sh --unit-tests

Prow is configured in the knative config.yaml in tektoncd/plumbing via the sections for tektoncd/pipeline.

Running presubmit integration tests

The presubmit integration tests entrypoint will run:

When run using Prow, integration tests will try to get a new cluster using boskos and these hardcoded GKE projects, which only the tektoncd/plumbing OWNERS have access to.

If you would like to run the integration tests against your cluster, you can use the current context in your kubeconfig, provide KO_DOCKER_REPO (as specified in the DEVELOPMENT.md), use e2e-tests.sh directly and provide the --run-tests argument:

export KO_DOCKER_REPO=gcr.io/my_docker_repo
test/e2e-tests.sh --run-tests

Or you can set $PROJECT_ID to a GCP project and rely on kubetest to setup a cluster for you:

export PROJECT_ID=my_gcp_project
test/presubmit-tests.sh --integration-tests

Documentation

Overview

Package test holds the project's end-to-end tests (e2e).

Index

Constants

This section is empty.

Variables

View Source 
var (
	// DefaultObjectAnnotationMapper does the "right" thing by conforming to the Tekton Bundle spec.
	DefaultObjectAnnotationMapper = func(obj runtime.Object) map[string]string {
		return map[string]string{
			tkremote.TitleAnnotation:      getObjectName(obj),
			tkremote.KindAnnotation:       strings.TrimSuffix(strings.ToLower(obj.GetObjectKind().GroupVersionKind().Kind), "s"),
			tkremote.APIVersionAnnotation: obj.GetObjectKind().GroupVersionKind().Version,
		}
	}
)

Functions

func AddToInformer added in v0.14.0 

func AddToInformer(t *testing.T, store cache.Store) func(ktesting.Action) (bool, runtime.Object, error)

AddToInformer returns a function to add ktesting.Actions to the cache store

func CollectPodLogs added in v0.5.0 

func CollectPodLogs(ctx context.Context, c *clients, podName, namespace string, logf logging.FormatLogger)

CollectPodLogs will get the logs for all containers in a Pod

func CreateImage added in v0.12.0 

func CreateImage(ref string, objs ...runtime.Object) (string, error)

CreateImage will push a new OCI image artifact with the provided raw data object as a layer and return the full image reference with a digest to fetch the image. Key must be specified as [lowercase kind]/[object name]. The image ref with a digest is returned.

func CreateImageWithAnnotations added in v0.24.0 

func CreateImageWithAnnotations(ref string, mapper ObjectAnnotationMapper, objs ...runtime.Object) (string, error)

CreateImageWithAnnotations is the base form of #CreateImage which accepts an ObjectAnnotationMapper to map an object to the annotations for it.

func EnsureConfigurationConfigMapsExist added in v0.41.0 

func EnsureConfigurationConfigMapsExist(d *Data)

EnsureConfigurationConfigMapsExist makes sure all the configmaps exists.

func GetEmbeddedStatus added in v0.35.0 

func GetEmbeddedStatus(ctx context.Context, t *testing.T, kubeClient kubernetes.Interface) string

GetEmbeddedStatus gets the current value for the "embedded-status" feature flag. If the flag is not set, it returns the default value.

func PrependResourceVersionReactor added in v0.14.0 

func PrependResourceVersionReactor(f *ktesting.Fake)

PrependResourceVersionReactor will instrument a client-go testing Fake with a reactor that simulates resourceVersion changes on mutations. This does not work with patches.

func SeedTestData 

func SeedTestData(t *testing.T, ctx context.Context, d Data) (Clients, Informers)

SeedTestData returns Clients and Informers populated with the given Data. nolint: revive

func WaitForDeploymentState added in v0.9.0 

func WaitForDeploymentState(ctx context.Context, c *clients, name string, namespace string, inState func(d *appsv1.Deployment) (bool, error), desc string) error

WaitForDeploymentState polls the status of the Deployment called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForPipelineRunState 

func WaitForPipelineRunState(ctx context.Context, c *clients, name string, polltimeout time.Duration, inState ConditionAccessorFn, desc string) error

WaitForPipelineRunState polls the status of the PipelineRun called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForPodState 

func WaitForPodState(ctx context.Context, c *clients, name string, namespace string, inState func(r *corev1.Pod) (bool, error), desc string) error

WaitForPodState polls the status of the Pod called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForRunState added in v0.28.0 

func WaitForRunState(ctx context.Context, c *clients, name string, polltimeout time.Duration, inState ConditionAccessorFn, desc string) error

WaitForRunState polls the status of the Run called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForServiceExternalIPState 

func WaitForServiceExternalIPState(ctx context.Context, c *clients, namespace, name string, inState func(s *corev1.Service) (bool, error), desc string) error

WaitForServiceExternalIPState polls the status of the a k8s Service called name from client every interval until an external ip is assigned indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForTaskRunState 

func WaitForTaskRunState(ctx context.Context, c *clients, name string, inState ConditionAccessorFn, desc string) error

WaitForTaskRunState polls the status of the TaskRun called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

Types

type Assets added in v0.10.0 

type Assets struct {
	Logger     *zap.SugaredLogger
	Controller *controller.Impl
	Clients    Clients
	Informers  Informers
	Recorder   *record.FakeRecorder
	Ctx        context.Context
}

Assets holds references to the controller, logs, clients, and informers.

type Clients 

type Clients struct {
	Pipeline           *fakepipelineclientset.Clientset
	Resource           *fakeresourceclientset.Clientset
	Kube               *fakekubeclientset.Clientset
	CloudEvents        cloudeventclient.CEClient
	ResolutionRequests *fakeresolutionclientset.Clientset
}

Clients holds references to clients which are useful for reconciler tests.

type ConditionAccessorFn added in v0.11.0 

type ConditionAccessorFn func(ca apis.ConditionAccessor) (bool, error)

ConditionAccessorFn is a condition function used polling functions

func Chain added in v0.18.0 

func Chain(fns ...ConditionAccessorFn) ConditionAccessorFn

Chain allows multiple ConditionAccessorFns to be chained together, checking the condition of each in order.

func Failed added in v0.11.0 

func Failed(name string) ConditionAccessorFn

Failed provides a poll condition function that checks if the ConditionAccessor resource has failed or not.

func FailedWithMessage added in v0.11.0 

func FailedWithMessage(message, name string) ConditionAccessorFn

FailedWithMessage provides a poll function that checks if the ConditionAccessor resource has failed with the TimeoudOut reason

func FailedWithReason added in v0.11.0 

func FailedWithReason(reason, name string) ConditionAccessorFn

FailedWithReason provides a poll function that checks if the ConditionAccessor resource has failed with the TimeoudOut reason

func PipelineRunFailed 

func PipelineRunFailed(name string) ConditionAccessorFn

PipelineRunFailed provides a poll condition function that checks if the PipelineRun has failed.

func PipelineRunPending added in v0.21.0 

func PipelineRunPending(name string) ConditionAccessorFn

PipelineRunPending provides a poll condition function that checks if the PipelineRun has been marked pending by the Tekton controller.

func PipelineRunSucceed 

func PipelineRunSucceed(name string) ConditionAccessorFn

PipelineRunSucceed provides a poll condition function that checks if the PipelineRun has successfully completed.

func Running added in v0.11.0 

func Running(name string) ConditionAccessorFn

Running provides a poll condition function that checks if the ConditionAccessor resource is currently running.

func Succeed added in v0.11.0 

func Succeed(name string) ConditionAccessorFn

Succeed provides a poll condition function that checks if the ConditionAccessor resource has successfully completed or not.

func TaskRunFailed 

func TaskRunFailed(name string) ConditionAccessorFn

TaskRunFailed provides a poll condition function that checks if the TaskRun has failed.

func TaskRunSucceed 

func TaskRunSucceed(name string) ConditionAccessorFn

TaskRunSucceed provides a poll condition function that checks if the TaskRun has successfully completed.

type Data 

type Data struct {
	PipelineRuns       []*v1beta1.PipelineRun
	Pipelines          []*v1beta1.Pipeline
	TaskRuns           []*v1beta1.TaskRun
	Tasks              []*v1beta1.Task
	ClusterTasks       []*v1beta1.ClusterTask
	PipelineResources  []*resourcev1alpha1.PipelineResource
	Runs               []*v1alpha1.Run
	Pods               []*corev1.Pod
	Namespaces         []*corev1.Namespace
	ConfigMaps         []*corev1.ConfigMap
	ServiceAccounts    []*corev1.ServiceAccount
	LimitRange         []*corev1.LimitRange
	ResolutionRequests []*resolutionv1alpha1.ResolutionRequest
}

Data represents the desired state of the system (i.e. existing resources) to seed controllers with.

type Informers 

type Informers struct {
	PipelineRun       informersv1beta1.PipelineRunInformer
	Pipeline          informersv1beta1.PipelineInformer
	TaskRun           informersv1beta1.TaskRunInformer
	Run               informersv1alpha1.RunInformer
	Task              informersv1beta1.TaskInformer
	ClusterTask       informersv1beta1.ClusterTaskInformer
	PipelineResource  resourceinformersv1alpha1.PipelineResourceInformer
	Pod               coreinformers.PodInformer
	ConfigMap         coreinformers.ConfigMapInformer
	ServiceAccount    coreinformers.ServiceAccountInformer
	LimitRange        coreinformers.LimitRangeInformer
	ResolutionRequest resolutioninformersv1alpha1.ResolutionRequestInformer
}

Informers holds references to informers which are useful for reconciler tests.

type ObjectAnnotationMapper added in v0.24.0 

type ObjectAnnotationMapper func(object runtime.Object) map[string]string

ObjectAnnotationMapper is a func alias that maps a runtime Object to the Tekton Bundle annotations map.

type Requester added in v0.35.0 

type Requester struct {
	// The resolved resource object to return when a request is
	// submitted.
	ResolvedResource resolution.ResolvedResource
	// An error to return when a request is submitted.
	SubmitErr error
	// Params that should match those on the request in order to return the resolved resource
	Params []pipelinev1beta1.Param
}

Requester implements resolution.Requester and makes it easier to mock the outcome of a remote pipelineRef or taskRef resolution.

func NewRequester added in v0.35.0 

func NewRequester(resource resolution.ResolvedResource, err error) *Requester

NewRequester creates a mock requester that resolves to the given resource or returns the given error on Submit().

func (*Requester) Submit added in v0.35.0 

func (r *Requester) Submit(ctx context.Context, resolverName resolution.ResolverName, req resolution.Request) (resolution.ResolvedResource, error)

Submit implements resolution.Requester, accepting the name of a resolver and a request for a specific remote file, and then returns whatever mock data was provided on initialization.

type ResolvedResource added in v0.35.0 

type ResolvedResource struct {
	// The resolved bytes to return when resolution is complete.
	ResolvedData []byte
	// An error to return instead of the resolved bytes after
	// resolution completes.
	DataErr error
	// Annotations to return when resolution is complete.
	ResolvedAnnotations map[string]string
	// ResolvedSource to return the source reference of the remote data
	ResolvedSource *pipelinev1beta1.ConfigSource
}

ResolvedResource implements resolution.ResolvedResource and makes it easier to mock the resolved content of a fetched pipeline or task.

func NewResolvedResource added in v0.35.0 

func NewResolvedResource(data []byte, annotations map[string]string, dataErr error) *ResolvedResource

NewResolvedResource creates a mock resolved resource that is populated with the given data and annotations or returns the given error from its Data() method.

func (*ResolvedResource) Annotations added in v0.35.0 

func (r *ResolvedResource) Annotations() map[string]string

Annotations implements resolution.ResolvedResource and returns the mock annotations given to it on initialization.

func (*ResolvedResource) Data added in v0.35.0 

func (r *ResolvedResource) Data() ([]byte, error)

Data implements resolution.ResolvedResource and returns the mock data and/or error given to it on initialization.

func (*ResolvedResource) Source added in v0.41.0 

func (r *ResolvedResource) Source() *pipelinev1beta1.ConfigSource

Source is the source reference of the remote data that records where the remote file came from including the url, digest and the entrypoint.

type ResourceVersionReactor added in v0.14.0 

type ResourceVersionReactor struct {
	// contains filtered or unexported fields
}

ResourceVersionReactor is an implementation of Reactor for our tests

func (*ResourceVersionReactor) Handles added in v0.14.0 

func (r *ResourceVersionReactor) Handles(action ktesting.Action) bool

Handles returns whether our test reactor can handle a given ktesting.Action

func (*ResourceVersionReactor) React added in v0.14.0 

func (r *ResourceVersionReactor) React(action ktesting.Action) (handled bool, ret runtime.Object, err error)

React is noop-function

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.