e2e

README

This directory contains end to end tests of LUCI CV.

These tests are designed to run locally with fakes instead of Gerrit, Datastore, Cloud Task Queue.

How to run?

The usual Go way, e.g. go test ./... or go test go.chromium.org/luci/cv/tests/e2e/...

How to debug?

Select failing test, e.g. TestXYZ. Run just this test in verbose mode, which emits copious amounts of logging, as if you are looking at production CV app, except log lines not attributed nor grouped to individual task queue executions.

go test -v -run TestXYZ

To make sense of logs, it may be easiest to save them to a file:

go test -v -run TestXYZ 2>&1 >out.log

If test seems to hang, save your time:

go test -v -run TestXYZ -test.timeout=1s 2>&1 >out.log

If test is flaky, try:

go test -v -run TestXYZ -test.count=1000 -test.failfast 2>&1 >out.log

Guidance on writing new tests

• Actual tests should be added into ..._test.go files, as usual.
• Prefer re-using existing project config setups, instead of generating new ones.
• Mark each TestXYZ func with t.Parallel().
  • Corollary: in rare cases where your test really must not be run in parallel, create a dedicated sub-package for just your test and change global vars as necessary. e2e package is intentionally exportable.
• Helper functions should be added to e2e.go. They should be public (see above).
• Use exactly 1 Convey block inside each TestXYZ function.
• Avoid deeply nested Convey blocks. Ideally there is 1 top level only.
  • Rational: easier to debug, since each section may produce substantial amount of logs, which are harder to attribute to individual sub-blocks.
  • Corollary: moderate copy-pasta is fine.
• Tests should be fast, finishing with 1s even on under-powered laptops.

Advanced: run with flaky Datastore:

Use special -cv.dsflakiness flag, e.g. go test ./... -cv.dsflakiness=0.06. See flakifyDS function for explanation of what 0.06 value actually means and the shortcomings of the simulation.

Documentation

Overview

Package e2e contains all CV end-to-end tests.

Index

• func MakeCfgCombinable(cgName, gHost, gRepo, gRef string) *cfgpb.Config
• func MakeCfgSingular(cgName, gHost, gRepo, gRef string) *cfgpb.Config
• type Test
  • func (t *Test) EarliestCreatedRunOf(ctx context.Context, lProject string) *run.Run
  • func (t *Test) ExportedBQAttemptsCount() int
  • func (t *Test) LastMessage(gHost string, gChange int64) *gerritpb.ChangeMessageInfo
  • func (t *Test) LatestRunWithGerritCL(ctx context.Context, lProject, gHost string, gChange int64) *run.Run
  • func (t *Test) LoadCL(ctx context.Context, id common.CLID) *changelist.CL
  • func (t *Test) LoadGerritCL(ctx context.Context, gHost string, gChange int64) *changelist.CL
  • func (t *Test) LoadGerritRuns(ctx context.Context, gHost string, gChange int64, lProject string) []*run.Run
  • func (t *Test) LoadProject(ctx context.Context, lProject string) *prjmanager.Project
  • func (t *Test) LoadRun(ctx context.Context, id common.RunID) *run.Run
  • func (t *Test) LoadRunsOf(ctx context.Context, lProject string) []*run.Run
  • func (t *Test) LogPhase(ctx context.Context, format string, args ...interface{})
  • func (t *Test) MaxCQVote(ctx context.Context, gHost string, gChange int64) int32
  • func (t *Test) MaxVote(ctx context.Context, gHost string, gChange int64, gLabel string) int32
  • func (t *Test) MigrationContext(ctx context.Context) context.Context
  • func (t *Test) MigrationFetchActiveRuns(ctx context.Context, project string) []*migrationpb.ActiveRun
  • func (t *Test) MustCQD(ctx context.Context, luciProject string) *cqdfake.CQDFake
  • func (t *Test) Now() time.Time
  • func (t *Test) RunUntil(ctx context.Context, stopIf func() bool)
  • func (t *Test) SetUp() (ctx context.Context, deferme func())

Functions

func MakeCfgCombinable

func MakeCfgCombinable(cgName, gHost, gRepo, gRef string) *cfgpb.Config

MakeCfgCombinable return project config with a combinable ConfigGroup.

func MakeCfgSingular

func MakeCfgSingular(cgName, gHost, gRepo, gRef string) *cfgpb.Config

MakeCfgSingular return project config with a single ConfigGroup.

Types

type Test

type Test struct {
	*cvtesting.Test // auto-initialized if nil
	// CVDev if true sets e2e test to use `cv-dev` GAE app.
	// Defaults to `cv` GAE app.
	CVDev bool

	PMNotifier  *prjmanager.Notifier
	RunNotifier *run.Notifier

	AdminServer     adminpb.AdminServer
	MigrationServer migrationpb.MigrationServer
	// contains filtered or unexported fields
}

Test encapsulates e2e setup for a CV test.

Embeds cvtesting.Test, which sets CV's dependencies and some simple CV components (e.g. TreeClient), while this Test focuses on setup of CV's own components.

Typical use:

ct := Test{CVDev: true}
ctx, cancel := ct.SetUp()
defer cancel()
...
ct.RunUntil(ctx, func() bool { return len(ct.LoadRunsOf("project")) > 0 })

func (*Test) EarliestCreatedRunOf

func (t *Test) EarliestCreatedRunOf(ctx context.Context, lProject string) *run.Run

EarliestCreatedRun returns the earliest created Run in a project.

If there are several such runs, may return any one of them.

Returns nil if there are no Runs.

func (*Test) ExportedBQAttemptsCount

func (t *Test) ExportedBQAttemptsCount() int

ExportedBQAttemptsCount returns number of exported CQ Attempts.

func (*Test) LastMessage

func (t *Test) LastMessage(gHost string, gChange int64) *gerritpb.ChangeMessageInfo

LastMessage returns the last message posted on a Gerrit CL from Gerrit fake.

Returns nil if there are no messages. Panics if the CL doesn't exist.

func (*Test) LatestRunWithGerritCL

func (t *Test) LatestRunWithGerritCL(ctx context.Context, lProject, gHost string, gChange int64) *run.Run

LatestRunWithCL returns the latest created Run containing given CL.

If there are several, returns the one with latest .StartTime. Returns nil if there is such Runs, including if Gerrit CL isn't yet in DS.

func (*Test) LoadCL

func (t *Test) LoadCL(ctx context.Context, id common.CLID) *changelist.CL

LoadCL returns CL entity or nil if not exists.

func (*Test) LoadGerritCL

func (t *Test) LoadGerritCL(ctx context.Context, gHost string, gChange int64) *changelist.CL

LoadGerritCL returns CL entity or nil if not exists.

func (*Test) LoadGerritRuns

func (t *Test) LoadGerritRuns(ctx context.Context, gHost string, gChange int64, lProject string) []*run.Run

LoadGerritRuns loads all Runs from Datastore which include a Gerrit CL.

func (*Test) LoadProject

func (t *Test) LoadProject(ctx context.Context, lProject string) *prjmanager.Project

LoadProject returns Project entity or nil if not exists.

func (*Test) LoadRun

func (t *Test) LoadRun(ctx context.Context, id common.RunID) *run.Run

LoadRun returns Run entity or nil if not exists.

func (*Test) LoadRunsOf

func (t *Test) LoadRunsOf(ctx context.Context, lProject string) []*run.Run

LoadRunsOf loads all Runs of a project from Datastore.

func (*Test) LogPhase

func (t *Test) LogPhase(ctx context.Context, format string, args ...interface{})

LogPhase emits easy to recognize log like =========================== PHASE: .... ===========================

func (*Test) MaxCQVote

func (t *Test) MaxCQVote(ctx context.Context, gHost string, gChange int64) int32

MaxCQVote returns max CQ vote of a Gerrit CL loaded from Gerrit fake.

Returns 0 if there are no votes. Panics if CL doesn't exist.

func (*Test) MaxVote

func (t *Test) MaxVote(ctx context.Context, gHost string, gChange int64, gLabel string) int32

MaxVote returns max vote of a Gerrit CL loaded from Gerrit fake.

Returns 0 if there are no votes. Panics if CL doesn't exist.

func (*Test) MigrationContext

func (t *Test) MigrationContext(ctx context.Context) context.Context

MigrationContext returns context authorized to call to the Migration API.

func (*Test) MigrationFetchActiveRuns

func (t *Test) MigrationFetchActiveRuns(ctx context.Context, project string) []*migrationpb.ActiveRun

func (*Test) MustCQD

func (t *Test) MustCQD(ctx context.Context, luciProject string) *cqdfake.CQDFake

MustCQD returns a CQDaemon fake for the given project, starting a new one if necessary, in which case it's lifetime is limited by the given context.

func (*Test) Now

func (t *Test) Now() time.Time

Now returns test clock time in UTC.

func (*Test) RunUntil

func (t *Test) RunUntil(ctx context.Context, stopIf func() bool)

RunUntil runs TQ tasks, while stopIf returns false.

If `dsFlakinessFlag` is set, uses flaky datastore for running TQ tasks. If `tqParallelFlag` is set, runs TQ tasks concurrently.

Not goroutine safe.

func (*Test) SetUp

func (t *Test) SetUp() (ctx context.Context, deferme func())

SetUp sets up the end to end test.

Must be called exactly once.