testresults

package
v0.0.0-...-a8f2654 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 3, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package testresults contains methods for accessing test results in Spanner.

Index

Constants

View Source
const GerritHostnameSuffix = "-review.googlesource.com"

The suffix used for all gerrit hostnames.

Variables

View Source
var (
	// minTimestamp is the minimum Timestamp value in Spanner.
	// https://cloud.google.com/spanner/docs/reference/standard-sql/data-types#timestamp_type
	MinSpannerTimestamp = time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC)
	// maxSpannerTimestamp is the max Timestamp value in Spanner.
	// https://cloud.google.com/spanner/docs/reference/standard-sql/data-types#timestamp_type
	MaxSpannerTimestamp = time.Date(9999, time.December, 31, 23, 59, 59, 999999999, time.UTC)
)
View Source
var QueryTestsQueryTmpl = template.Must(template.New("QueryTestsQuery").Parse(`
  @{USE_ADDITIONAL_PARALLELISM=TRUE}
	WITH Tests as (
		SELECT DISTINCT TestId, SubRealm IN UNNEST(@subRealms) as HasAccess
		FROM TestRealms
		WHERE
			Project = @project
				AND TestId > @paginationTestId
				AND TestId LIKE @testIdPattern
	)
	SELECT TestId FROM Tests
	WHERE HasAccess
	ORDER BY TestId ASC
	{{if .hasLimit}}
		LIMIT @limit
	{{end}}
`))

The query is written in a way to force spanner NOT to put `SubRealm IN UNNEST(@subRealms)` check in Filter Scan seek condition, which can significantly increase the time it takes to scan the table.

View Source
var TestRealmSaveCols = []string{"Project", "TestId", "SubRealm", "LastIngestionTime"}

TestRealmSaveCols is the set of columns written to in a test variant realm save. Allocated here once to avoid reallocating on every save.

View Source
var TestResultSaveCols = []string{
	"Project", "TestId", "PartitionTime", "VariantHash",
	"IngestedInvocationId", "RunIndex", "ResultIndex",
	"IsUnexpected", "RunDurationUsec", "Status",
	"ExonerationReasons",
	"SourceRefHash", "SourcePosition",
	"ChangelistHosts", "ChangelistChanges", "ChangelistPatchsets",
	"ChangelistOwnerKinds",
	"HasDirtySources",
	"SubRealm", "IsFromBisection",
}

TestResultSaveCols is the set of columns written to in a test result save. Allocated here once to avoid reallocating on every test result save.

View Source
var TestVariantRealmSaveCols = []string{
	"Project", "TestId", "VariantHash", "SubRealm",
	"Variant", "LastIngestionTime",
}

TestVariantRealmSaveCols is the set of columns written to in a test variant realm save. Allocated here once to avoid reallocating on every save.

Functions

func CompressHost

func CompressHost(host string) string

CompressHost transforms a gerrit hostname into its compressed database representation.

func CreateQueryFailureRateTestData

func CreateQueryFailureRateTestData(ctx context.Context) error

CreateQueryFailureRateTestData creates test data in Spanner for testing QueryFailureRate.

func DecompressHost

func DecompressHost(host string) string

DecompressHost recovers a gerrit hostname from its compressed database representation.

func OwnerKindFromDB

func OwnerKindFromDB(value string) pb.ChangelistOwnerKind

OwnerKindFromDB decodes owner kind from its database representation.

func OwnerKindToDB

func OwnerKindToDB(value pb.ChangelistOwnerKind) string

OwnerKindToDB encodes owner kind to its database representation.

func QueryFailureRate

QueryFailureRate queries the failure rate of nominated test variants.

Must be called in a Spanner transactional context. Context must support multiple reads (i.e. NOT spanner.Single()) as request may batched over multiple reads.

func QueryFailureRateSampleRequest

func QueryFailureRateSampleRequest() (project string, asAtTime time.Time, testVariants []*pb.TestVariantIdentifier)

func QueryFailureRateSampleResponse

func QueryFailureRateSampleResponse() *pb.QueryTestVariantFailureRateResponse

QueryFailureRateSampleResponse returns expected response data from QueryFailureRate after being invoked with QueryFailureRateSampleRequest. It is assumed test data was setup with CreateQueryFailureRateTestData.

func QueryTests

func QueryTests(ctx context.Context, project, testIDSubstring string, opts QueryTestsOptions) (testIDs []string, nextPageToken string, err error)

QueryTests finds all the test IDs with the specified testIDSubstring from the spanner database. Must be called in a spanner transactional context.

func ReadTestHistory

func ReadTestHistory(ctx context.Context, opts ReadTestHistoryOptions) (verdicts []*pb.TestVerdict, nextPageToken string, err error)

ReadTestHistory reads verdicts from the spanner database. Must be called in a spanner transactional context.

func ReadTestHistoryStats

func ReadTestHistoryStats(ctx context.Context, opts ReadTestHistoryOptions) (groups []*pb.QueryTestHistoryStatsResponse_Group, nextPageToken string, err error)

ReadTestHistoryStats reads stats of verdicts grouped by UTC dates from the spanner database. Must be called in a spanner transactional context.

func ReadTestRealms

func ReadTestRealms(ctx context.Context, keys spanner.KeySet, fn func(tr *TestRealm) error) error

ReadTestRealms read test variant realms from the TestRealms table. Must be called in a spanner transactional context.

func ReadTestResults

func ReadTestResults(ctx context.Context, keys spanner.KeySet, fn func(tr *TestResult) error) error

ReadTestResults reads test results from the TestResults table. Must be called in a spanner transactional context.

func ReadTestVariantRealms

func ReadTestVariantRealms(ctx context.Context, keys spanner.KeySet, fn func(tvr *TestVariantRealm) error) error

ReadTestVariantRealms read test variant realms from the TestVariantRealms table. Must be called in a spanner transactional context.

func ReadVariants

func ReadVariants(ctx context.Context, project, testID string, opts ReadVariantsOptions) (variants []*pb.QueryVariantsResponse_VariantInfo, nextPageToken string, err error)

ReadVariants reads all the variants of the specified test from the spanner database. Must be called in a spanner transactional context.

func SortChangelists

func SortChangelists(cls []Changelist)

SortChangelists sorts a slice of changelists to be in ascending lexicographical order by (host, change, patchset).

func ValidateGerritHostname

func ValidateGerritHostname(host string) error

ValidateGerritHostname validates the given gerrit hostname.

Types

type BQChangelist

type BQChangelist struct {
	Host      bigquery.NullString
	Change    bigquery.NullInt64
	Patchset  bigquery.NullInt64
	OwnerKind bigquery.NullString
}

type BQGitiles

type BQGitiles struct {
	Host    bigquery.NullString
	Project bigquery.NullString
	Ref     bigquery.NullString
}

type BQRef

type BQRef struct {
	Gitiles *BQGitiles
}

type BQTestVerdict

type BQTestVerdict struct {
	TestID                string
	VariantHash           string
	RefHash               string
	InvocationID          string
	Status                string
	PartitionTime         time.Time
	PassedAvgDurationUsec bigquery.NullFloat64
	Changelists           []*BQChangelist
	// Whether the caller has access to this test verdict.
	HasAccess bool
}

type Changelist

type Changelist struct {
	// Host is the gerrit hostname. E.g. chromium-review.googlesource.com.
	Host      string
	Change    int64
	Patchset  int64
	OwnerKind pb.ChangelistOwnerKind
}

Changelist represents a gerrit changelist.

type CommitWithVerdicts

type CommitWithVerdicts struct {
	// Source position of this commit.
	Position int64
	// Commit hash of this commit.
	CommitHash string
	// Represent a branch in the source control.
	Ref *BQRef
	// Realm of test verdicts at this commit.
	Realm string
	// Returns at most 20 test verdicts at this commit.
	TestVerdicts []*BQTestVerdict
}

CommitWithVerdicts represents a commit with test verdicts.

type FakeReadClient

type FakeReadClient struct {
	CommitsWithVerdicts []*CommitWithVerdicts
	SourceVerdicts      []SourceVerdict
}

FakeReadClient represents a fake implementation of the client to read test verdicts from BigQuery, for testing.

func (*FakeReadClient) ReadSourceVerdicts

func (f *FakeReadClient) ReadSourceVerdicts(ctx context.Context, options ReadSourceVerdictsOptions) ([]SourceVerdict, error)

func (*FakeReadClient) ReadTestVerdictsPerSourcePosition

func (f *FakeReadClient) ReadTestVerdictsPerSourcePosition(ctx context.Context, options ReadTestVerdictsPerSourcePositionOptions) ([]*CommitWithVerdicts, error)

ReadTestVerdictsPerSourcePosition reads test verdicts per source position.

type QueryFailureRateOptions

type QueryFailureRateOptions struct {
	// Project is the LUCI Project to query.
	Project string
	// SubRealms are the realms (of the form "ci", NOT "chromium:ci")
	// within the project to query.
	SubRealms []string
	// TestVariants are the test variants to query.
	TestVariants []*pb.TestVariantIdentifier
	// AsAtTime is latest parititon time to include in the results;
	// outside of testing contexts, this should be the current time.
	// QueryTestVariants returns data for the 5 * 24 weekday hour
	// period leading up to this time.
	AsAtTime time.Time
}

QueryFailureRateOptions specifies options for QueryFailureRate().

type QueryTestsOptions

type QueryTestsOptions struct {
	SubRealms []string
	PageSize  int
	PageToken string
}

QueryTestsOptions specifies options for QueryTests().

type ReadClient

type ReadClient struct {
	// contains filtered or unexported fields
}

ReadClient represents a client to read test results table from BigQuery.

func NewReadClient

func NewReadClient(ctx context.Context, gcpProject string) (*ReadClient, error)

NewReadClient creates a new client for reading test results BigQuery table.

func (*ReadClient) Close

func (c *ReadClient) Close() error

Close releases any resources held by the client.

func (*ReadClient) ReadSourceVerdicts

func (c *ReadClient) ReadSourceVerdicts(ctx context.Context, options ReadSourceVerdictsOptions) ([]SourceVerdict, error)

ReadSourceVerdicts reads source verdicts on the specified test variant branch, for a nominated source position range.

func (*ReadClient) ReadTestVerdictsPerSourcePosition

func (c *ReadClient) ReadTestVerdictsPerSourcePosition(ctx context.Context, options ReadTestVerdictsPerSourcePositionOptions) ([]*CommitWithVerdicts, error)

ReadTestVerdictsPerSourcePosition returns commits with test verdicts in source position ascending order. Only return commits within the last 90 days.

type ReadSourceVerdictsOptions

type ReadSourceVerdictsOptions struct {
	Project     string
	TestID      string
	VariantHash string
	RefHash     string
	// Only test verdicts with allowed invocation subrealms can be returned.
	AllowedSubrealms []string
	// The maximum source position to return, inclusive.
	StartSourcePosition int64
	// The minimum source position to return, exclusive.
	EndSourcePosition int64
	// The last partition time to include in the results, exclusive.
	EndPartitionTime time.Time
}

type ReadTestHistoryOptions

type ReadTestHistoryOptions struct {
	Project                 string
	TestID                  string
	SubRealms               []string
	VariantPredicate        *pb.VariantPredicate
	SubmittedFilter         pb.SubmittedFilter
	TimeRange               *pb.TimeRange
	ExcludeBisectionResults bool
	PageSize                int
	PageToken               string
}

ReadTestHistoryOptions specifies options for ReadTestHistory().

type ReadTestVerdictsPerSourcePositionOptions

type ReadTestVerdictsPerSourcePositionOptions struct {
	Project     string
	TestID      string
	VariantHash string
	RefHash     string
	// Only test verdicts with allowed invocation realms can be returned.
	AllowedRealms []string
	// All returned commits has source position greater than PositionMustGreater.
	PositionMustGreater int64
	// The maximum number of commits to return.
	NumCommits int64
}

type ReadVariantsOptions

type ReadVariantsOptions struct {
	SubRealms        []string
	VariantPredicate *pb.VariantPredicate
	PageSize         int
	PageToken        string
}

ReadVariantsOptions specifies options for ReadVariants().

type RunStatus

type RunStatus int64
const (
	Unexpected RunStatus = iota
	Flaky
	Expected
)

type SourceVerdict

type SourceVerdict struct {
	// The source position.
	Position int64
	// Test verdicts at the position. Limited to 20.
	Verdicts []SourceVerdictTestVerdict
}

SourceVerdict aggregates all test results at a source position.

type SourceVerdictTestVerdict

type SourceVerdictTestVerdict struct {
	// The invocation for which the verdict is.
	InvocationID string
	// Partition time of the test verdict.
	PartitionTime time.Time
	// Status is one of SKIPPED, EXPECTED, UNEXPECTED, FLAKY.
	Status string
	// Changelists tested by the verdict.
	Changelists []BQChangelist
}

SourceVerdictTestVerdict is a test verdict that is part of a source verdict.

type Sources

type Sources struct {
	// 8-byte hash of the source reference (e.g. git branch) tested.
	// This refers to the base commit/version tested, before any changelists
	// are applied.
	RefHash []byte
	// The position along the source reference that was tested.
	// This refers to the base commit/version tested, before any changelists
	// are applied.
	Position int64
	// The gerrit changelists applied on top of the base version/commit.
	// At most 10 changelists should be specified here, if there are more
	// then limit to 10 and set HasDirtySources to true.
	Changelists []Changelist
	// Whether other modifications were made to the sources, not described
	// by the fields above. For example, a package was upreved in the build.
	// If this is set, then the source information is approximate: suitable
	// for plotting results by source position the UI but not good enough
	// for change point analysis.
	IsDirty bool
}

Sources captures information about the code sources that were tested by a test result.

func CopySources

func CopySources(sources Sources) Sources

CopySources makes a deep copy of the given code sources.

type TestRealm

type TestRealm struct {
	Project           string
	TestID            string
	SubRealm          string
	LastIngestionTime time.Time
}

TestVariantRealm represents a row in the TestVariantRealm table.

func (*TestRealm) SaveUnverified

func (tvr *TestRealm) SaveUnverified() *spanner.Mutation

SaveUnverified creates a mutation to save the test realm into the TestRealms table. The test realm is not verified. Must be called in spanner RW transactional context.

type TestResult

type TestResult struct {
	Project              string
	TestID               string
	PartitionTime        time.Time
	VariantHash          string
	IngestedInvocationID string
	RunIndex             int64
	ResultIndex          int64
	IsUnexpected         bool
	RunDuration          *time.Duration
	Status               pb.TestResultStatus
	// Properties of the test verdict (stored denormalised) follow.
	ExonerationReasons []pb.ExonerationReason
	Sources            Sources
	// Properties of the invocation (stored denormalised) follow.
	SubRealm        string
	IsFromBisection bool
}

TestResult represents a row in the TestResults table.

func (*TestResult) SaveUnverified

func (tr *TestResult) SaveUnverified() *spanner.Mutation

SaveUnverified prepare a mutation to insert the test result into the TestResults table. The test result is not validated.

type TestResultBuilder

type TestResultBuilder struct {
	// contains filtered or unexported fields
}

TestResultBuilder provides methods to build a test result for testing.

func NewTestResult

func NewTestResult() TestResultBuilder

func (TestResultBuilder) Build

func (b TestResultBuilder) Build() *TestResult

func (TestResultBuilder) WithExonerationReasons

func (b TestResultBuilder) WithExonerationReasons(exonerationReasons ...pb.ExonerationReason) TestResultBuilder

func (TestResultBuilder) WithIngestedInvocationID

func (b TestResultBuilder) WithIngestedInvocationID(invID string) TestResultBuilder

func (TestResultBuilder) WithIsFromBisection

func (b TestResultBuilder) WithIsFromBisection(value bool) TestResultBuilder

func (TestResultBuilder) WithIsUnexpected

func (b TestResultBuilder) WithIsUnexpected(unexpected bool) TestResultBuilder

func (TestResultBuilder) WithPartitionTime

func (b TestResultBuilder) WithPartitionTime(partitionTime time.Time) TestResultBuilder

func (TestResultBuilder) WithProject

func (b TestResultBuilder) WithProject(project string) TestResultBuilder

func (TestResultBuilder) WithResultIndex

func (b TestResultBuilder) WithResultIndex(resultIndex int64) TestResultBuilder

func (TestResultBuilder) WithRunDuration

func (b TestResultBuilder) WithRunDuration(duration time.Duration) TestResultBuilder

func (TestResultBuilder) WithRunIndex

func (b TestResultBuilder) WithRunIndex(runIndex int64) TestResultBuilder

func (TestResultBuilder) WithSources

func (b TestResultBuilder) WithSources(sources Sources) TestResultBuilder

func (TestResultBuilder) WithStatus

func (TestResultBuilder) WithSubRealm

func (b TestResultBuilder) WithSubRealm(subRealm string) TestResultBuilder

func (TestResultBuilder) WithTestID

func (b TestResultBuilder) WithTestID(testID string) TestResultBuilder

func (TestResultBuilder) WithVariantHash

func (b TestResultBuilder) WithVariantHash(variantHash string) TestResultBuilder

func (TestResultBuilder) WithoutExoneration

func (b TestResultBuilder) WithoutExoneration() TestResultBuilder

func (TestResultBuilder) WithoutRunDuration

func (b TestResultBuilder) WithoutRunDuration() TestResultBuilder

type TestVariantRealm

type TestVariantRealm struct {
	Project           string
	TestID            string
	VariantHash       string
	SubRealm          string
	Variant           *pb.Variant
	LastIngestionTime time.Time
}

TestVariantRealm represents a row in the TestVariantRealm table.

func (*TestVariantRealm) SaveUnverified

func (tvr *TestVariantRealm) SaveUnverified() *spanner.Mutation

SaveUnverified creates a mutation to save the test variant realm into the TestVariantRealms table. The test variant realm is not verified. Must be called in spanner RW transactional context.

type TestVerdictBuilder

type TestVerdictBuilder struct {
	// contains filtered or unexported fields
}

TestVerdictBuilder provides methods to build a test variant for testing.

func NewTestVerdict

func NewTestVerdict() *TestVerdictBuilder

func (*TestVerdictBuilder) Build

func (b *TestVerdictBuilder) Build() []*TestResult

func (*TestVerdictBuilder) WithBaseTestResult

func (b *TestVerdictBuilder) WithBaseTestResult(testResult *TestResult) *TestVerdictBuilder

WithBaseTestResult specifies a test result to use as the template for the test variant's test results.

func (*TestVerdictBuilder) WithPassedAvgDuration

func (b *TestVerdictBuilder) WithPassedAvgDuration(duration *time.Duration) *TestVerdictBuilder

WithPassedAvgDuration specifies the average duration to use for passed test results. If setting to a non-nil value, make sure to set the result status as passed on the base test result if using this option.

func (*TestVerdictBuilder) WithRunStatus

func (b *TestVerdictBuilder) WithRunStatus(runStatuses ...RunStatus) *TestVerdictBuilder

WithRunStatus specifies the status of runs of the test verdict.

func (*TestVerdictBuilder) WithStatus

WithStatus specifies the status of the test verdict.

Directories

Path Synopsis
Package exporter contains methods to export test results to BigQuery.
Package exporter contains methods to export test results to BigQuery.
Package lowlatency contains methods for accessing the low-latency test results table in Spanner.
Package lowlatency contains methods for accessing the low-latency test results table in Spanner.
Package stability implements the test stability analysis used by the QueryStability RPC.
Package stability implements the test stability analysis used by the QueryStability RPC.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL