upgrade

README

Upgrade Tests

In order to get coverage for the upgrade process from an operator’s perspective, we need an additional suite of tests that perform a complete knative upgrade. Running these tests on every commit will ensure that we don’t introduce any non-upgradeable changes, so every commit should be releasable.

This is inspired by kubernetes upgrade testing.

These tests are a pretty big hammer in that they cover more than just version changes, but it’s one of the only ways to make sure we don’t accidentally make breaking changes for now.

Flow

We’d like to validate that the upgrade doesn’t break any resources (they still respond to requests) and doesn't break our installation (we can still update resources).

At a high level, we want to do this:

1. Install the latest knative release.
2. Create some resources.
3. Install knative at HEAD.
4. Test those resources, verify that we didn’t break anything.

To achieve that, we utilize the upgrade test framework. The framework runs tests in the following phases:

1. Install the latest release from GitHub.
2. Run the PreUpgrade tests.
3. Start the Continual tests.
4. Install from the HEAD of the default branch.
5. Run the PostUpgrade tests.
6. Install the latest release from GitHub.
7. Run the PostDowngrade tests.
8. End the Continual tests and collect results.

Tests

PreUpgrade

Create a Service pointing to image1, ensure it responds correctly.

PostUpgrade

Ensure the Service still responds correctly after upgrading. Update it to point to image2, ensure it responds correctly. Ensure that a new service can be created after downgrade.

PostDowngrade

Ensure the Service still responds correctly after downgrading. Update it to point back to image1, ensure it responds correctly. Ensure that a new service can be created after downgrade.

Continual

In order to verify that we don't have data-plane unavailability during our control-plane outages (when we're upgrading the knative/serving installation), we run a prober test that continually sends requests to a service during the entire upgrade process. When the upgrade completes, we make sure that none of those requests failed. We also run autoscale tests to ensure that autoscaling works correctly during upgrades/downgrades.

Documentation

Index

• func AutoscaleSustainingTest() pkgupgrade.BackgroundOperation
• func AutoscaleSustainingWithTBCTest() pkgupgrade.BackgroundOperation
• func BYORevisionPostUpgradeTest() pkgupgrade.Operation
• func BYORevisionPreUpgradeTest() pkgupgrade.Operation
• func CRDStoredVersionPostUpgradeTest() pkgupgrade.Operation
• func ContinualTests() []pkgupgrade.BackgroundOperation
• func CreateNewServicePostDowngradeTest() pkgupgrade.Operation
• func CreateNewServicePostUpgradeTest() pkgupgrade.Operation
• func InitialScalePostUpgradeTest() pkgupgrade.Operation
• func InitialScalePreUpgradeTest() pkgupgrade.Operation
• func ProbeTest() pkgupgrade.BackgroundOperation
• func ServicePostDowngradeTest() pkgupgrade.Operation
• func ServicePostUpgradeFromScaleToZeroTest() pkgupgrade.Operation
• func ServicePostUpgradeTest() pkgupgrade.Operation
• func ServicePreUpgradeAndScaleToZeroTest() pkgupgrade.Operation
• func ServicePreUpgradeTest() pkgupgrade.Operation
• func ServingPostDowngradeTests() []pkgupgrade.Operation
• func ServingPostUpgradeTests() []pkgupgrade.Operation
• func ServingPreUpgradeTests() []pkgupgrade.Operation

Functions

func AutoscaleSustainingTest

func AutoscaleSustainingTest() pkgupgrade.BackgroundOperation

AutoscaleSustainingTest checks that when traffic increases a knative app scales up and sustains the scale as long as the traffic sustains, despite whether it is switching modes between normal and panic.

func AutoscaleSustainingWithTBCTest

func AutoscaleSustainingWithTBCTest() pkgupgrade.BackgroundOperation

AutoscaleSustainingWithTBCTest checks that when traffic increases and the activator is in the path a knative app scales up and sustains the scale.

func BYORevisionPostUpgradeTest

func BYORevisionPostUpgradeTest() pkgupgrade.Operation

BYORevisionPostUpgradeTest attempts to update the RouteSpec of a Service using BYO Revision name. This test is meant to catch new defaults that break the immutability of BYO Revision name.

func BYORevisionPreUpgradeTest

func BYORevisionPreUpgradeTest() pkgupgrade.Operation

BYORevisionPreUpgradeTest creates a Service that uses the BYO Revision name functionality. This test is meant to catch new defaults that break bring your own revision name immutability.

func CRDStoredVersionPostUpgradeTest

func CRDStoredVersionPostUpgradeTest() pkgupgrade.Operation

func ContinualTests

func ContinualTests() []pkgupgrade.BackgroundOperation

ContinualTests is an umbrella function for grouping all Serving continual/background tests.

func CreateNewServicePostDowngradeTest

func CreateNewServicePostDowngradeTest() pkgupgrade.Operation

CreateNewServicePostDowngradeTest verifies creating a new service after downgrade.

func CreateNewServicePostUpgradeTest

func CreateNewServicePostUpgradeTest() pkgupgrade.Operation

CreateNewServicePostUpgradeTest verifies creating a new service after upgrade.

func InitialScalePostUpgradeTest

func InitialScalePostUpgradeTest() pkgupgrade.Operation

InitialScalePostUpgradeTest verifies that the service is ready after upgrade despite the fact that it does not receive any requests.

func InitialScalePreUpgradeTest

func InitialScalePreUpgradeTest() pkgupgrade.Operation

InitialScalePreUpgradeTest creates a service and lets it scale down to zero without sending any requests to it.

func ProbeTest

func ProbeTest() pkgupgrade.BackgroundOperation

ProbeTest sends requests to Knative services while performing an upgrade and verifies the expected success rate.

func ServicePostDowngradeTest

func ServicePostDowngradeTest() pkgupgrade.Operation

ServicePostDowngradeTest verifies an existing service after downgrade.

func ServicePostUpgradeFromScaleToZeroTest

func ServicePostUpgradeFromScaleToZeroTest() pkgupgrade.Operation

ServicePostUpgradeFromScaleToZeroTest verifies a scaled-to-zero service after upgrade.

func ServicePostUpgradeTest

func ServicePostUpgradeTest() pkgupgrade.Operation

ServicePostUpgradeTest verifies an existing service after upgrade.

func ServicePreUpgradeAndScaleToZeroTest

func ServicePreUpgradeAndScaleToZeroTest() pkgupgrade.Operation

ServicePreUpgradeAndScaleToZeroTest creates a new service before upgrade and wait for it to scale to zero.

func ServicePreUpgradeTest

func ServicePreUpgradeTest() pkgupgrade.Operation

ServicePreUpgradeTest creates a service before upgrade.

func ServingPostDowngradeTests

func ServingPostDowngradeTests() []pkgupgrade.Operation

ServingPostDowngradeTests is an umbrella function for grouping all Serving post-downgrade tests.

func ServingPostUpgradeTests

func ServingPostUpgradeTests() []pkgupgrade.Operation

ServingPostUpgradeTests is an umbrella function for grouping all Serving post-upgrade tests.

func ServingPreUpgradeTests

func ServingPreUpgradeTests() []pkgupgrade.Operation

ServingPreUpgradeTests is an umbrella function for grouping all Serving pre-upgrade tests.