imagetest

package module
v0.0.0-...-07b211d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 9, 2024 License: Apache-2.0 Imports: 24 Imported by: 0

README

Cloud Image Tests

The Cloud Image Tests are a testing framework and a set of test suites used for testing GCE Images.

Invocation

Testing components are built into a container image. The entrypoint is /manager, which supports the following options:

Usage:
  -filter string
        only run tests matching filter
  -images string
        comma separated list of images to test
  -out_path string
        junit xml path (default "junit.xml")
  -parallel_count int
        TestParallelCount (default 5)
  -print
        print out the parsed test workflows and exit
  -project string
        project to be used for tests
  -validate
        validate all the test workflows and exit
  -zone string
        zone to be used for tests

It can be invoked via docker as:

$ images="projects/debian-cloud/global/images/family/debian-10,"
$ images+="projects/debian-cloud/global/images/family/debian-9"
$ docker run gcr.io/gcp-guest/cloud-image-tests --project $PROJECT \
  --zone $ZONE --images $images
Credentials

The test manager is designed to be run in a Google Cloud environment, and will use application default credentials. If you are not in a Google Cloud environment and need to specify the credentials to use, you can provide them as a docker volume and specify the path with the GOOGLE_APPLICATION_CREDENTIALS environment variable.

Assuming your application default or service account credentials are in a file named credentials.json:

$ docker run -v /path/to/local/creds:/creds \
  -e GOOGLE_APPLICATION_CREDENTIALS=/creds/credentials.json \
  gcr.io/gcp-guest/cloud-image-tests -project $PROJECT \
  -zone $ZONE -images $images

The manager will exit with 0 if all tests completed successfully, 1 otherwise. JUnit format XML will also be output.

Writing tests

Tests are organized into go packages in the test_suites directory and are written in go. Each package must at a minimum contain a setup file (by conventioned named setup.go) and at least one test file (by convention named $packagename_test.go). Due to golang style conventions, the package name cannot contain an underscore. Thus, for the test suite name to match the packagename, the name of the test suite should not contain an underscore. For example, if a new test suite was created to test image licenses, it should be called imagelicensing, not image_licensing.

The setup.go file describes the workflow to run including the VMs and other GCE resources to create, any necessary configuration for those resources, which specific tests to run, etc.. It is here where you can also skip an entire test package based on inputs e.g. image, zone or compute endpoint or other conditions.

Tests themselves are written in the test file(s) as go unit tests. Tests may use any of the test fixtures provided by the standard testing package. These will be packaged into a binary and run on the test VMs created during setup using the Google Compute Engine startup script runner. The test files must specify a 'cit' build constraint in order to prevent the presubmits on this repository from invoking the image tests.

When writing tests to run against both Linux and Windows, it is preferred to use separate functions within the same test where appropriate based on differences between OSes (ex. powershell vs bash commands). This makes the test definitions easier to read and maintain.

For example, if the test TestSomeCondition() needs to run different commands to achieve similar results (and the test is located in the directory "mydefaulttest"):


package mydefaulttest

import (
    "runtime"
    "testing"
)

func RunTestConditionWindows() {
    //Test something in Windows
}
func RunTestCondition() {
    //Test something in Linux
}

func TestSomeCondition(t *testing.T) {
    if runtime.GOOS == "windows" {
    RunTestConditionWindows()
    } else {
        RunTestCondition()
    }
}

Note that there are also functions available in the test_utils.go for some OS level abstractions such as running a Windows powershell command or checking if a Linux binary exists.

It is suggested to start by copying an existing test package. Do not forget to add your test to the relevant setup.go file in order to add the test to the test suite.

Modifying test behavior based on image properties

For tests that need to behave different based on whether an image is arm or x86, or linux or windows, it is preferred to use compute API properties rather than relying on image naming conventions. These properties can be found on the testworkflow Image value. The list of values can be found in the compute API documentation here. Some examples are in the following code snippet.

func Setup(t *imagetest.Testworkflow) {
	if t.Image.Architecture == "ARM64" {
	//...
	} else if utils.HasFeature(t.Image, "GVNIC") {
	//...
	}
}
Testing features in compute beta API

Tests that need to run against features in the beta API can do so by creating TestVMs using CreateTestVMBeta or CreateTestVMFromInstanceBeta to use the beta instance API. However, due to limitation with daisy's create instances step, if one instance in a TestWorkflow uses the beta API all instances in that workflow must use the beta API.

Building the container image

From the root directory of this repository:

$ docker build -t cloud-image-tests -f imagetest/Dockerfile .

Testing on a local machine

From the imagetest directory of this repository, where outspath is the folder where test outputs are stored:

$ local_build.sh -o $outspath

By default, all test suites are built. To build only one test suite:

$ local_build.sh -o $outspath -s $test_suite_name

To build from a directory other than imagetest

$ local_build.sh -o $outspath -i $path_to_imagetest

To run the tests, cd into $outspath, set the shell variables and run

$ manager -zone $ZONE -project $PROJECT -images $images -filter $test_suite_name -local_path .

What is being tested

The tests are a combination of various types - end to end tests on certain software components, image validations and feature validations, etc. The collective whole represents the quality assurance bar for releasing a [supported GCE Image][gce-images], and the test suites here must all pass before Google engineers will release a new GCE image.

The tests are documented in the test_suites directory.

Documentation

Index

Constants

View Source 
const (

	// ShouldRebootDuringTest is a local map key to indicate that the
	// test will reboot and relies on results from the second boot.
	ShouldRebootDuringTest = "shouldRebootDuringTest"
	// DefaultSourceRange is the RFC-1918 range used in default rules.
	DefaultSourceRange = "10.128.0.0/9"

	// DefaultMTU is the default MTU set for a network.
	DefaultMTU = 1460

	// JumboFramesMTU is the maximum MTU settable for a network.
	JumboFramesMTU = 8896

	// DefaultMachineType is the default machine type when machine type isn't specified.
	DefaultMachineType = "n1-standard-1"
)
View Source 
const (
	// PdStandard disktype string
	PdStandard = "pd-standard"
	// PdSsd disktype string
	PdSsd = "pd-ssd"
	// PdBalanced disktype string
	PdBalanced = "pd-balanced"
	// PdExtreme disktype string
	PdExtreme = "pd-extreme"
	// HyperdiskExtreme disktype string
	HyperdiskExtreme = "hyperdisk-extreme"
	// HyperdiskThroughput disktype string
	HyperdiskThroughput = "hyperdisk-throughput"
	// HyperdiskBalanced disktype string
	HyperdiskBalanced = "hyperdisk-balanced"
)

Variables

This section is empty.

Functions

func PrintTests 

func PrintTests(ctx context.Context, storageClient *storage.Client, testWorkflows []*TestWorkflow, project, zone, gcsPath, localPath string)

PrintTests prints all test workflows.

func ValidateTests 

func ValidateTests(ctx context.Context, storageClient *storage.Client, testWorkflows []*TestWorkflow, project, zone, gcsPath, localPath string) error

ValidateTests validates all test workflows.

Types

type Network 

type Network struct {
	// contains filtered or unexported fields
}

Network represent network used by vm in setup.go.

func (*Network) CreateFirewallRule 

func (n *Network) CreateFirewallRule(firewallName, protocol string, ports, ranges []string) error

CreateFirewallRule create firewall rule.

func (*Network) CreateSubnetwork 

func (n *Network) CreateSubnetwork(name string, ipRange string) (*Subnetwork, error)

CreateSubnetwork creates custom subnetwork. Using AddCustomNetwork method provided by TestVM to config network on vm

func (*Network) SetMTU 

func (n *Network) SetMTU(mtu int)

SetMTU sets the MTU of the network. The MTU must be between 1460 and 8896, inclusively.

type Subnetwork 

type Subnetwork struct {
	// contains filtered or unexported fields
}

Subnetwork represent subnetwork used by vm in setup.go.

func (Subnetwork) AddSecondaryRange 

func (s Subnetwork) AddSecondaryRange(rangeName, ipRange string)

AddSecondaryRange add secondary IP range to Subnetwork

func (*Subnetwork) SetPurpose 

func (s *Subnetwork) SetPurpose(purpose string)

SetPurpose sets the subnetwork purpose

func (*Subnetwork) SetRegion 

func (s *Subnetwork) SetRegion(region string)

SetRegion sets the subnetwork region

func (*Subnetwork) SetRole 

func (s *Subnetwork) SetRole(role string)

SetRole sets the subnetwork role

type TestSuites 

type TestSuites struct {
	XMLName   xml.Name     `xml:"testsuites"`
	Name      string       `xml:"name,attr"`
	Errors    int          `xml:"errors,attr"`
	Failures  int          `xml:"failures,attr"`
	Disabled  int          `xml:"disabled,attr"`
	Skipped   int          `xml:"skipped,attr"`
	Tests     int          `xml:"tests,attr"`
	Time      float64      `xml:"time,attr"`
	TestSuite []*testSuite `xml:"testsuite"`
}

TestSuites represent a collection of test cases.

func RunTests 

func RunTests(ctx context.Context, storageClient *storage.Client, testWorkflows []*TestWorkflow, project, zone, gcsPath, localPath string, parallelCount int, parallelStagger string, testProjects []string) (*TestSuites, error)

RunTests runs all test workflows.

type TestVM 

type TestVM struct {
	// contains filtered or unexported fields
}

TestVM is a test VM.

func (*TestVM) AddAliasIPRanges 

func (t *TestVM) AddAliasIPRanges(aliasIPRange, rangeName string) error

AddAliasIPRanges add alias ip range to current test VMs.

func (*TestVM) AddCustomNetwork 

func (t *TestVM) AddCustomNetwork(network *Network, subnetwork *Subnetwork) error

AddCustomNetwork add current test VMs in workflow using provided network and subnetwork. If subnetwork is empty, not using subnetwork, in this case network has to be in auto mode VPC.

func (*TestVM) AddMetadata 

func (t *TestVM) AddMetadata(key, value string)

AddMetadata adds the specified key:value pair to metadata during VM creation.

func (*TestVM) AddScope 

func (t *TestVM) AddScope(scope string)

AddScope adds the specified auth scope to the service account on the VM.

func (*TestVM) AddUser 

func (t *TestVM) AddUser(user, publicKey string)

AddUser add user public key to metadata ssh-keys.

func (*TestVM) EnableConfidentialInstance 

func (t *TestVM) EnableConfidentialInstance()

EnableConfidentialInstance enabled CVM features for the instance.

func (*TestVM) EnableSecureBoot 

func (t *TestVM) EnableSecureBoot()

EnableSecureBoot make the current test VMs in workflow with secure boot.

func (*TestVM) ForceMachineType 

func (t *TestVM) ForceMachineType(machinetype string)

ForceMachineType sets the machine type for the test VM. This will override the machine_type flag in the CIT wrapper, and should only be used when a test absolutely requires a specific machine shape.

func (*TestVM) ForceZone 

func (t *TestVM) ForceZone(z string)

ForceZone sets the zone for the test vm. This will override the zone option from the CIT wrapper and and should only be used when a test requires a specific zone.

func (*TestVM) Reboot 

func (t *TestVM) Reboot() error

Reboot stops the VM, waits for it to shutdown, then starts it again. Your test package must handle being run twice.

func (*TestVM) ResizeDiskAndReboot 

func (t *TestVM) ResizeDiskAndReboot(diskSize int) error

ResizeDiskAndReboot resize the disk of the current test VMs and reboot

func (*TestVM) Resume 

func (t *TestVM) Resume() error

Resume waits for the vm to be SUSPENDED, then resumes it. It does not handle suspension.

func (*TestVM) RunTests 

func (t *TestVM) RunTests(runtest string)

RunTests runs only the named tests on the testVM.

From go help test: 

-run regexp
 Run only those tests and examples matching the regular expression.
 For tests, the regular expression is split by unbracketed slash (/)
 characters into a sequence of regular expressions, and each part
 of a test's identifier must match the corresponding element in
 the sequence, if any. Note that possible parents of matches are
 run too, so that -run=X/Y matches and runs and reports the result
 of all tests matching X, even those without sub-tests matching Y,
 because it must run them to look for those sub-tests.

func (*TestVM) SetMinCPUPlatform 

func (t *TestVM) SetMinCPUPlatform(minCPUPlatform string)

SetMinCPUPlatform sets the minimum CPU platform of the instance.

func (*TestVM) SetNetworkPerformanceTier 

func (t *TestVM) SetNetworkPerformanceTier(tier string) error

SetNetworkPerformanceTier sets the performance tier of the VM. The tier must be one of "DEFAULT" or "TIER_1"

func (*TestVM) SetPrivateIP 

func (t *TestVM) SetPrivateIP(network *Network, networkIP string) error

SetPrivateIP set IPv4 internal IP address for target network to the current test VMs.

func (*TestVM) SetShutdownScript 

func (t *TestVM) SetShutdownScript(script string)

SetShutdownScript sets the `shutdown-script` metadata key for a non-Windows VM.

func (*TestVM) SetShutdownScriptURL 

func (t *TestVM) SetShutdownScriptURL(script string) error

SetShutdownScriptURL sets the`shutdown-script-url` metadata key for a non-Windows VM.

func (*TestVM) SetStartupScript 

func (t *TestVM) SetStartupScript(script string)

SetStartupScript sets the `startup-script` metadata key for a VM. On Windows VMs, this script does not run on startup: different guest attributes must be set.

func (*TestVM) SetWindowsShutdownScript 

func (t *TestVM) SetWindowsShutdownScript(script string)

SetWindowsShutdownScript sets the `windows-shutdown-script-ps1` metadata key for a Windows VM.

func (*TestVM) SetWindowsShutdownScriptURL 

func (t *TestVM) SetWindowsShutdownScriptURL(script string) error

SetWindowsShutdownScriptURL sets the`windows-shutdown-script-url` metadata key for a Windows VM.

func (*TestVM) SetWindowsStartupScript 

func (t *TestVM) SetWindowsStartupScript(script string)

SetWindowsStartupScript sets the `windows-startup-script-ps1` metadata key for a VM.

func (*TestVM) UseGVNIC 

func (t *TestVM) UseGVNIC()

UseGVNIC sets the type of vNIC to be used to GVNIC

type TestWorkflow 

type TestWorkflow struct {
	Name string
	// Client is a shared client for the compute service.
	Client daisycompute.Client
	// Image is the image under test
	Image *compute.Image
	// ImageURL will be the partial URL of a GCE image.
	ImageURL string
	// MachineType is the machine type to be used for the test. This can be overridden by individual test suites.
	MachineType *compute.MachineType
	Project     *compute.Project
	Zone        *compute.Zone
	// contains filtered or unexported fields
}

TestWorkflow defines a test workflow which creates at least one test VM.

func NewTestWorkflow 

func NewTestWorkflow(client daisycompute.Client, computeEndpointOverride, name, image, timeout, project, zone, x86Shape string, arm64Shape string) (*TestWorkflow, error)

NewTestWorkflow returns a new TestWorkflow.

func (*TestWorkflow) AddSSHKey 

func (t *TestWorkflow) AddSSHKey(user string) (string, error)

AddSSHKey generate ssh key pair and return public key.

func (*TestWorkflow) CreateNetwork 

func (t *TestWorkflow) CreateNetwork(networkName string, autoCreateSubnetworks bool) (*Network, error)

CreateNetwork creates custom network. Using AddCustomNetwork method provided by TestVM to config network on vm

func (*TestWorkflow) CreateTestVM 

func (t *TestWorkflow) CreateTestVM(name string) (*TestVM, error)

CreateTestVM adds the necessary steps to create a VM with the specified name to the workflow.

func (*TestWorkflow) CreateTestVMBeta 

func (t *TestWorkflow) CreateTestVMBeta(name string) (*TestVM, error)

CreateTestVMBeta adds the necessary steps to create a VM with the specified name from the compute beta API to the workflow.

func (*TestWorkflow) CreateTestVMFromInstanceBeta 

func (t *TestWorkflow) CreateTestVMFromInstanceBeta(i *daisy.InstanceBeta, disks []*compute.Disk) (*TestVM, error)

CreateTestVMFromInstanceBeta creates a test vm struct to run CIT suites on from the given daisy instancebeta and adds it to the test workflow.

func (*TestWorkflow) CreateTestVMMultipleDisks 

func (t *TestWorkflow) CreateTestVMMultipleDisks(disks []*compute.Disk, instanceParams *daisy.Instance) (*TestVM, error)

CreateTestVMMultipleDisks adds the necessary steps to create a VM with the specified name to the workflow.

func (*TestWorkflow) LockProject 

func (t *TestWorkflow) LockProject()

LockProject indicates this test modifies project-level data and must have exclusive use of the project.

func (*TestWorkflow) Skip 

func (t *TestWorkflow) Skip(message string)

Skip marks a test workflow to be skipped.

func (*TestWorkflow) SkippedMessage 

func (t *TestWorkflow) SkippedMessage() string

SkippedMessage returns the skip reason message for the workflow.

func (*TestWorkflow) WaitForDisksQuota 

func (t *TestWorkflow) WaitForDisksQuota(qa *daisy.QuotaAvailable) error

WaitForDisksQuota appends a list of quotas to the wait for disk quota step. Quotas with a blank region will be populated with the region corresponding to the workflow zone.

func (*TestWorkflow) WaitForVMQuota 

func (t *TestWorkflow) WaitForVMQuota(qa *daisy.QuotaAvailable) error

WaitForVMQuota appends a list of quotas to the wait for vm quota step. Quotas with a blank region will be populated with the region corresponding to the workflow zone.

Source Files

View all Source files

Directories

Path Synopsis
cmd
manager
wrapper
test_suites
cvm
disk
guestagent
hostnamevalidation
hotattach
imageboot
licensevalidation
livemigrate
Package livemigrate tests standard live migration, not confidential vm live migrate.
 Package livemigrate tests standard live migration, not confidential vm live migrate.
loadbalancer
mdsmtls
metadata
network
networkperf
oslogin
packagevalidation
security
shapevalidation
sql
ssh
storageperf
suspendresume
windowscontainers
winrm

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.