test

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2018 License: Apache-2.0 Imports: 30 Imported by: 107

README

Test

This directory contains tests and testing docs for Knative Serving:

The conformance tests are a subset of the end to end test with more strict requirements around what can be tested.

If you want to add more tests, see adding_tests.md.

Presubmit tests

presubmit-tests.sh is the entry point for both the end-to-end tests and the conformance tests

This script, and consequently, the e2e and conformance tests will be run before every code submission. You can run these tests manually with:

test/presubmit-tests.sh

Note that to run presubmit-tests.sh or e2e-tests.sh scripts, you'll need kubernetes kubetest installed:

go get -u k8s.io/test-infra/kubetest

Running unit tests

To run all unit tests:

go test ./...

By default go test will not run the e2e tests, which need -tags=e2e to be enabled.

Running end to end tests

To run the e2e tests and the conformance tests, you need to have a running environment that meets the e2e test environment requirements, and you need to specify the build tag e2e.

go test -v -tags=e2e -count=1 ./test/conformance
go test -v -tags=e2e -count=1 ./test/e2e
One test case

To run one e2e test case, e.g. TestAutoscaleUpDownUp, use the -run flag with go test:

go test -v -tags=e2e -count=1 ./test/e2e -run ^TestAutoscaleUpDownUp$
Environment requirements

These tests require:

  1. A running Knative Serving cluster.
  2. The namespaces pizzaplanet and noodleburg: 
    kubectl create namespace pizzaplanet
kubectl create namespace noodleburg
  3. A docker repo containing the test images
Flags

You can use test flags to control the environment your tests run against, i.e. override your environment variables:

go test -v -tags=e2e -count=1 ./test/conformance --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo

If you are running against an environment with no loadbalancer for the ingress, at the moment your only option is to use a domain which will resolve to the IP of the running node (see #609):

go test -v -tags=e2e -count=1 ./test/conformance --resolvabledomain
go test -v -tags=e2e -count=1 ./test/e2e --resolvabledomain

Test images

Building the test images

The upload-test-images.sh script can be used to build and push the test images used by the conformance and e2e tests. It requires:

To run the script for all end to end test images:

./test/upload-test-images.sh ./test/e2e/test_images
./test/upload-test-images.sh ./test/conformance/test_images
Adding new test images

New test images should be placed in their own subdirectories. Be sure to to include a Dockerfile for building and running the test image.

The new test images will also need to be uploaded to the e2e tests Docker repo. You will need one of the owners found in /test/OWNERS to do this.

Flags

These flags are useful for running against an existing cluster, making use of your existing environment setup.

Tests importing github.com/knative/serving/test recognize these flags:

Specifying kubeconfig

By default the tests will use the kubeconfig file at ~/.kube/config. You can specify a different config file with the argument --kubeconfig.

To run the tests with a non-default kubeconfig file:

go test -v -tags=e2e -count=1 ./test/conformance --kubeconfig /my/path/kubeconfig
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig /my/path/kubeconfig
Specifying cluster

The --cluster argument lets you use a different cluster than your specified kubeconfig's active context. This will default to the value of your K8S_CLUSTER_OVERRIDE environment variable if not specified.

go test -v -tags=e2e -count=1 ./test/conformance --cluster your-cluster-name
go test -v -tags=e2e -count=1 ./test/e2e --cluster your-cluster-name

The current cluster names can be obtained by running:

kubectl config get-clusters
Specifying namespace

The --namespace argument lets you specify the namespace to use for the tests. By default, conformance will use noodleburg and e2e will use pizzaplanet.

go test -v -tags=e2e -count=1 ./test/conformance --namespace your-namespace-name
go test -v -tags=e2e -count=1 ./test/e2e --namespace your-namespace-name
Overridding docker repo

The --dockerrepo argument lets you specify the docker repo from which images used by your tests should be pulled. This will default to the value of your DOCKER_REPO_OVERRIDE environment variable if not specified.

go test -v -tags=e2e -count=1 ./test/conformance --dockerrepo gcr.myhappyproject
go test -v -tags=e2e -count=1 ./test/e2e --dockerrepo gcr.myhappyproject
Using a resolvable domain

If you set up your cluster using the getting started docs, Routes created in the test will use the domain example.com, unless the route has label app=prod in which case they will use the domain prod-domain.com. Since these domains will not be resolvable to deployments in your test cluster, in order to make a request against the endpoint, the test use the IP assigned to the service knative-ingressgateway in the namespace istio-system and spoof the Host in the header.

If you have configured your cluster to use a resolvable domain, you can use the --resolvabledomain flag to indicate that the test should make requests directly against Route.Status.Domain and does not need to spoof the Host.

Output verbose logs

The --logverbose argument lets you see verbose test logs and k8s logs.

go test -v -tags=e2e -count=1 ./test/e2e --logverbose
Emit metrics

Running tests with the --emitmetrics argument will cause latency metrics to be emitted by the tests.

Documentation

Index

Constants

View Source 
const (
	// VerboseLogLevel defines verbose log level as 10
	VerboseLogLevel glog.Level = 10
)

Variables

View Source 
var Flags = initializeFlags()

Flags holds the command line flags or defaults for settings in the user's environment. See EnvironmentFlags for a list of supported fields.

View Source 
var Logger *zap.SugaredLogger

Logger is to be used by test cases for logging and is also used for emitting metrics.

Functions

func AllRouteTrafficAtRevision 

func AllRouteTrafficAtRevision(names ResourceNames) func(r *v1alpha1.Route) (bool, error)

AllRouteTrafficAtRevision will check the revision that route r is routing traffic to and return true if 100% of the traffic is routing to revisionName.

func AppendRandomString 

func AppendRandomString(prefix string, logger *zap.SugaredLogger) string

AppendRandomString will generate a random string that begins with prefix. This is useful if you want to make sure that your tests can run at the same time against the same environment without conflicting. This method will seed rand with the current time when called for the first time.

func BlueGreenRoute 

func BlueGreenRoute(namespace string, names, blue, green ResourceNames) *v1alpha1.Route

BlueGreenRoute returns a Route object in namespace using the route and configuration names in names. Traffic is split evenly between blue and green.

func CheckConfigurationState 

func CheckConfigurationState(client servingtyped.ConfigurationInterface, name string, inState func(r *v1alpha1.Configuration) (bool, error)) error

CheckConfigurationState verifies the status of the Configuration called name from client is in a particular state by calling `inState` and expecting `true`. This is the non-polling variety of WaitForConfigurationState

func CheckRevisionState 

func CheckRevisionState(client servingtyped.RevisionInterface, name string, inState func(r *v1alpha1.Revision) (bool, error)) error

CheckRevisionState verifies the status of the Revision called name from client is in a particular state by calling `inState` and expecting `true`. This is the non-polling variety of WaitForRevisionState

func CheckRouteState 

func CheckRouteState(client servingtyped.RouteInterface, name string, inState func(r *v1alpha1.Route) (bool, error)) error

CheckRouteState verifies the status of the Route called name from client is in a particular state by calling `inState` and expecting `true`. This is the non-polling variety of WaitForRouteState

func CheckServiceState 

func CheckServiceState(client servingtyped.ServiceInterface, name string, inState func(s *v1alpha1.Service) (bool, error)) error

CheckServiceState verifies the status of the Service called name from client is in a particular state by calling `inState` and expecting `true`. This is the non-polling variety of WaitForServiceState

func CleanupOnInterrupt 

func CleanupOnInterrupt(cleanup func(), logger *zap.SugaredLogger)

CleanupOnInterrupt will execute the function cleanup if an interrupt signal is caught

func Configuration 

func Configuration(namespace string, names ResourceNames, imagePath string) *v1alpha1.Configuration

Configuration returns a Configuration object in namespace with the name names.Config that uses the image specifed by imagePath.

func EventuallyMatchesBody 

func EventuallyMatchesBody(expected string) spoof.ResponseChecker

EventuallyMatchesBody checks that the response body *eventually* matches the expected body. TODO(#1178): Delete me. We don't want to need this; we should be waiting for an appropriate Status instead.

func IsRevisionReady 

func IsRevisionReady(r *v1alpha1.Revision) (bool, error)

IsRevisionReady will check the status conditions of the revision and return true if the revision is ready to serve traffic. It will return false if the status indicates a state other than deploying or being ready. It will also return false if the type of the condition is unexpected.

func IsRouteReady 

func IsRouteReady(r *v1alpha1.Route) (bool, error)

IsRouteReady will check the status conditions of the route and return true if the route is ready.

func IsServiceReady 

func IsServiceReady(s *v1alpha1.Service) (bool, error)

IsServiceReady will check the status conditions of the service and return true if the service is ready. This means that its configurations and routes have all reported ready.

func LatestService 

func LatestService(namespace string, names ResourceNames, imagePath string) *v1alpha1.Service

LatestService returns a RunLatest Service object in namespace with the name names.Service that uses the image specifed by imagePath.

func MatchesAny 

func MatchesAny(_ *spoof.Response) (bool, error)

MatchesAny is a NOP matcher. This is useful for polling until a 200 is returned.

func MatchesBody 

func MatchesBody(expected string) spoof.ResponseChecker

MatchesBody checks that the *first* response body matches the "expected" body, otherwise failing.

func Route 

func Route(namespace string, names ResourceNames) *v1alpha1.Route

Route returns a Route object in namespace using the route and configuration names in names.

func TODO_RouteTrafficToRevisionWithInClusterDNS 

func TODO_RouteTrafficToRevisionWithInClusterDNS(r *v1alpha1.Route) (bool, error)

RouteTrafficToRevisionWithInClusterDNS will check the revision that route r is routing traffic using in cluster DNS and return true if the revision received the request.

func TODO_ServiceTrafficToRevisionWithInClusterDNS 

func TODO_ServiceTrafficToRevisionWithInClusterDNS(s *v1alpha1.Service) (bool, error)

ServiceTrafficToRevisionWithInClusterDNS will check the revision that route r is routing traffic using in cluster DNS and return true if the revision received the request.

func WaitForConfigurationState 

func WaitForConfigurationState(client servingtyped.ConfigurationInterface, name string, inState func(c *v1alpha1.Configuration) (bool, error), desc string) error

WaitForConfigurationState polls the status of the Configuration called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForDeploymentState 

func WaitForDeploymentState(client v1beta1.DeploymentInterface, name string, inState func(d *apiv1beta1.Deployment) (bool, error), desc string) error

WaitForDeploymentState polls the status of the Deployment called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForEndpointState 

func WaitForEndpointState(kubeClientset *kubernetes.Clientset, logger *zap.SugaredLogger, resolvableDomain bool, domain string, inState spoof.ResponseChecker, desc string) error

WaitForEndpointState will poll an endpoint until inState indicates the state is achieved. If resolvableDomain is false, it will use kubeClientset to look up the ingress and spoof the domain in the request headers, otherwise it will make the request directly to domain. desc will be used to name the metric that is emitted to track how long it took for the domain to get into the state checked by inState. Commas in `desc` must be escaped.

func WaitForIngressState 

func WaitForIngressState(client v1beta1.IngressInterface, name string, inState func(r *apiv1beta1.Ingress) (bool, error), desc string) error

WaitForIngressState polls the status of the Ingress called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForPodListState 

func WaitForPodListState(client v1.PodInterface, inState func(p *corev1.PodList) (bool, error), desc string) error

WaitForPodListState polls the status of the PodList from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took to get into the state checked by inState.

func WaitForRevisionState 

func WaitForRevisionState(client servingtyped.RevisionInterface, name string, inState func(r *v1alpha1.Revision) (bool, error), desc string) error

WaitForRevisionState polls the status of the Revision called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForRouteState 

func WaitForRouteState(client servingtyped.RouteInterface, name string, inState func(r *v1alpha1.Route) (bool, error), desc string) error

WaitForRouteState polls the status of the Route called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForServiceState 

func WaitForServiceState(client servingtyped.ServiceInterface, name string, inState func(s *v1alpha1.Service) (bool, error), desc string) error

WaitForServiceState polls the status of the Service called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

Types

type Clients 

type Clients struct {
	Kube      *kubernetes.Clientset
	Routes    servingtyped.RouteInterface
	Configs   servingtyped.ConfigurationInterface
	Revisions servingtyped.RevisionInterface
	Services  servingtyped.ServiceInterface
}

Clients holds instances of interfaces for making requests to Knative Serving.

func NewClients 

func NewClients(configPath string, clusterName string, namespace string) (*Clients, error)

NewClients instantiates and returns several clientsets required for making request to the Knative Serving cluster specified by the combination of clusterName and configPath. Clients can make requests within namespace.

func (*Clients) Delete 

func (clients *Clients) Delete(routes []string, configs []string) error

Delete will delete all Routes and Configs with the names routes and configs, if clients has been successfully initialized.

type EnvironmentFlags 

type EnvironmentFlags struct {
	Cluster          string // K8s cluster (defaults to $K8S_CLUSTER_OVERRIDE)
	DockerRepo       string // Docker repo (defaults to $DOCKER_REPO_OVERRIDE)
	Kubeconfig       string // Path to kubeconfig (defaults to ./kube/config)
	Namespace        string // K8s namespace (blank by default, to be overwritten by test suite)
	ResolvableDomain bool   // Resolve Route controller's `domainSuffix`
	LogVerbose       bool   // Enable verbose logging
	EmitMetrics      bool   // Emit metrics
}

type ResourceNames 

type ResourceNames struct {
	Config        string
	Route         string
	Revision      string
	Service       string
	TrafficTarget string
}

ResourceNames holds names of various resources.

type ZapMetricExporter 

type ZapMetricExporter struct {
	// contains filtered or unexported fields
}

ZapMetricExporter is a stats and trace exporter that logs the exported data to the provided (probably test specific) zap logger. It conforms to the view.Exporter and trace.Exporter interfaces.

func (*ZapMetricExporter) ExportSpan 

func (e *ZapMetricExporter) ExportSpan(vd *trace.SpanData)

ExportSpan will emit the trace data to the zap logger.

func (*ZapMetricExporter) ExportView 

func (e *ZapMetricExporter) ExportView(vd *view.Data)

ExportView will emit the view data vd (i.e. the stats that have been recorded) to the zap logger.

Source Files

View all Source files

Directories

Path Synopsis
conformance
test_images/pizzaplanetv1
test_images/pizzaplanetv2
e2e
test_images/autoscale
test_images/helloworld

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.