test

README

Test

This directory contains tests and testing docs.

• Test library contains code you can use in your knative tests
• Flags added by the test library
• Unit tests currently reside in the codebase alongside the code they test

Running unit tests

To run all unit tests:

go test ./...

Test library

You can use the test library in this dir to:

• Use common test flags
• Output logs
• Emit metrics
• Ensure test cleanup

Use common test flags

These flags are useful for running against an existing cluster, making use of your existing environment setup.

By importing github.com/knative/pkg/test you get access to a global variable called test.Flags which holds the values of the command line flags.

logger.Infof("Using namespace %s", test.Flags.Namespace)

See e2e_flags.go.

Output logs

When tests are run with --logverbose option, debug logs will be emitted to stdout.

We are using the common e2e logging library that uses the Knative logging library for structured logging. It is built on top of zap. Tests should initialize the global logger to use a test specifc context with logging.GetContextLogger:

// The convention is for the name of the logger to match the name of the test.
logging.GetContextLogger("TestHelloWorld")

Logs can then be emitted using the logger object which is required by many functions in the test library. To emit logs:

logger.Infof("Creating a new Route %s and Configuration %s", route, configuration)
logger.Debugf("The LogURL is %s, not yet verifying", logURL)

See logging.go.

Emit metrics

You can emit metrics from your tests using the opencensus library, which is being used inside Knative as well. These metrics will be emitted by the test if the test is run with the --emitmetrics option.

You can record arbitrary metrics with stats.Record or measure latency with trace.StartSpan:

ctx, span := trace.StartSpan(context.Background(), "MyMetric")

• These traces will be emitted automatically by the generic crd polling functions.
• The traces are emitted by a custom metric exporter that uses the global logger instance.

Metric format

When a trace metric is emitted, the format is metric <name> <startTime> <endTime> <duration>. The name of the metric is arbitrary and can be any string. The values are:

• metric - Indicates this log is a metric
• <name> - Arbitrary string indentifying the metric
• <startTime> - Unix time in nanoseconds when measurement started
• <endTime> - Unix time in nanoseconds when measurement ended
• <duration> - The difference in ms between the startTime and endTime

For example:

metric WaitForConfigurationState/prodxiparjxt/ConfigurationUpdatedWithRevision 1529980772357637397 1529980772431586609 73.949212ms

The Wait methods (which poll resources) will prefix the metric names with the name of the function, and if applicable, the name of the resource, separated by /. In the example above, WaitForConfigurationState is the name of the function, and prodxiparjxt is the name of the configuration resource being polled. ConfigurationUpdatedWithRevision is the string passed to WaitForConfigurationState by the caller to identify what state is being polled for.

Check Knative Serving resources

WARNING: this code also exists in knative/serving.

After creating Knative Serving resources or making changes to them, you will need to wait for the system to realize those changes. You can use the Knative Serving CRD check and polling methods to check the resources are either in or reach the desired state.

The WaitFor* functions use the kubernetes wait package. To poll they use PollImmediate and the return values of the function you provide behave the same as ConditionFunc: a bool to indicate if the function should stop or continue polling, and an error to indicate if there has been an error.

For example, you can poll a Configuration object to find the name of the Revision that was created for it:

var revisionName string
err := test.WaitForConfigurationState(clients.ServingClient, configName, func(c *v1alpha1.Configuration) (bool, error) {
    if c.Status.LatestCreatedRevisionName != "" {
        revisionName = c.Status.LatestCreatedRevisionName
        return true, nil
    }
    return false, nil
}, "ConfigurationUpdatedWithRevision")

Metrics will be emitted for these Wait method tracking how long test poll for.

See kube_checks.go.

Ensure test cleanup

To ensure your test is cleaned up, you should defer cleanup to execute after your test completes and also ensure the cleanup occurs if the test is interrupted:

defer tearDown(clients)
test.CleanupOnInterrupt(func() { tearDown(clients) })

See cleanup.go.

Flags

Importing the test library adds flags that are useful for end to end tests that need to run against a cluster.

Tests importing github.com/knative/pkg/test recognize these flags:

• --kubeconfig
• --cluster
• --namespace
• --logverbose
• --emitmetrics

Specifying kubeconfig

By default the tests will use the kubeconfig file at ~/.kube/config. If there is an error getting the current user, it will use kubeconfig instead as the default value. You can specify a different config file with the argument --kubeconfig.

To run tests with a non-default kubeconfig file:

go test ./test --kubeconfig /my/path/kubeconfig

Specifying cluster

The --cluster argument lets you use a different cluster than your specified kubeconfig's active context. This will default to the value of your K8S_CLUSTER_OVERRIDE environment variable if not specified.

go test ./test --cluster your-cluster-name

The current cluster names can be obtained by running:

kubectl config get-clusters

Specifying namespace

The --namespace argument lets you specify the namespace to use for the tests. By default, tests will use serving-tests.

go test ./test --namespace your-namespace-name

Output verbose logs

The --logverbose argument lets you see verbose test logs and k8s logs.

go test ./test --logverbose

Metrics flag

Running tests with the --emitmetrics argument will cause latency metrics to be emitted by the tests.

go test ./test --emitmetrics

• To add additional metrics to a test, see emitting metrics.
• For more info on the format of the metrics, see metric format.

---

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License.

Documentation

Index

• Variables
• func BuildClientConfig(kubeConfigPath string, clusterName string) (*rest.Config, error)
• func CleanupOnInterrupt(cleanup func(), logger *logging.BaseLogger)
• func EventuallyMatchesBody(expected string) spoof.ResponseChecker
• func MatchesAny(_ *spoof.Response) (bool, error)
• func MatchesBody(expected string) spoof.ResponseChecker
• func NewSpoofingClient(client *KubeClient, logger *logging.BaseLogger, domain string, resolvable bool) (*spoof.SpoofingClient, error)
• func Retrying(rc spoof.ResponseChecker, codes ...int) spoof.ResponseChecker
• func WaitForDeploymentState(client *KubeClient, name string, ...) error
• func WaitForEndpointState(kubeClient *KubeClient, logger *logging.BaseLogger, domain string, ...) (*spoof.Response, error)
• func WaitForPodListState(client *KubeClient, inState func(p *corev1.PodList) (bool, error), desc string, ...) error
• type EnvironmentFlags
• type KubeClient
  • func NewKubeClient(configPath string, clusterName string) (*KubeClient, error)
  • func (client *KubeClient) UpdateConfigMap(name string, configName string, values map[string]string) error

Variables

var Flags = initializeFlags()

Flags holds the command line flags or defaults for settings in the user's environment. See EnvironmentFlags for a list of supported fields.

Functions

func BuildClientConfig

func BuildClientConfig(kubeConfigPath string, clusterName string) (*rest.Config, error)

BuildClientConfig builds the client config specified by the config path and the cluster name

func CleanupOnInterrupt

func CleanupOnInterrupt(cleanup func(), logger *logging.BaseLogger)

CleanupOnInterrupt will execute the function cleanup if an interrupt signal is caught

func EventuallyMatchesBody

func EventuallyMatchesBody(expected string) spoof.ResponseChecker

EventuallyMatchesBody checks that the response body *eventually* matches the expected body. TODO(#1178): Delete me. We don't want to need this; we should be waiting for an appropriate Status instead.

func MatchesAny

func MatchesAny(_ *spoof.Response) (bool, error)

MatchesAny is a NOP matcher. This is useful for polling until a 200 is returned.

func MatchesBody

func MatchesBody(expected string) spoof.ResponseChecker

MatchesBody checks that the *first* response body matches the "expected" body, otherwise failing.

func NewSpoofingClient

func NewSpoofingClient(client *KubeClient, logger *logging.BaseLogger, domain string, resolvable bool) (*spoof.SpoofingClient, error)

NewSpoofingClient returns a spoofing client to make requests

func Retrying

func Retrying(rc spoof.ResponseChecker, codes ...int) spoof.ResponseChecker

Retrying modifies a ResponseChecker to retry certain response codes.

func WaitForDeploymentState

func WaitForDeploymentState(client *KubeClient, name string, inState func(d *apiv1beta1.Deployment) (bool, error), desc string, namespace string) error

WaitForDeploymentState polls the status of the Deployment called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForEndpointState

func WaitForEndpointState(kubeClient *KubeClient, logger *logging.BaseLogger, domain string, inState spoof.ResponseChecker, desc string, resolvable bool) (*spoof.Response, error)

WaitForEndpointState will poll an endpoint until inState indicates the state is achieved. If resolvableDomain is false, it will use kubeClientset to look up the ingress and spoof the domain in the request headers, otherwise it will make the request directly to domain. desc will be used to name the metric that is emitted to track how long it took for the domain to get into the state checked by inState. Commas in `desc` must be escaped.

func WaitForPodListState

func WaitForPodListState(client *KubeClient, inState func(p *corev1.PodList) (bool, error), desc string, namespace string) error

WaitForPodListState polls the status of the PodList from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took to get into the state checked by inState.

Types

type EnvironmentFlags

type EnvironmentFlags struct {
	Cluster     string // K8s cluster (defaults to $K8S_CLUSTER_OVERRIDE)
	Kubeconfig  string // Path to kubeconfig (defaults to ./kube/config)
	Namespace   string // K8s namespace (blank by default, to be overwritten by test suite)
	LogVerbose  bool   // Enable verbose logging
	EmitMetrics bool   // Emit metrics
}

EnvironmentFlags define the flags that are needed to run the e2e tests.

type KubeClient

type KubeClient struct {
	Kube *kubernetes.Clientset
}

KubeClient holds instances of interfaces for making requests to kubernetes client.

func NewKubeClient

func NewKubeClient(configPath string, clusterName string) (*KubeClient, error)

NewKubeClient instantiates and returns several clientsets required for making request to the kube client specified by the combination of clusterName and configPath. Clients can make requests within namespace.

func (*KubeClient) UpdateConfigMap

func (client *KubeClient) UpdateConfigMap(name string, configName string, values map[string]string) error

UpdateConfigMap updates the config map for specified @name with values