testresults

package
v0.0.0-...-add0f8d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 22, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package testresults contains methods for accessing test results in Spanner.

Index

Constants

View Source 
const GerritHostnameSuffix = "-review.googlesource.com"

The suffix used for all gerrit hostnames.

Variables

View Source 
var (
	// minTimestamp is the minimum Timestamp value in Spanner.
	// https://cloud.google.com/spanner/docs/reference/standard-sql/data-types#timestamp_type
	MinSpannerTimestamp = time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC)
	// maxSpannerTimestamp is the max Timestamp value in Spanner.
	// https://cloud.google.com/spanner/docs/reference/standard-sql/data-types#timestamp_type
	MaxSpannerTimestamp = time.Date(9999, time.December, 31, 23, 59, 59, 999999999, time.UTC)
)
View Source 
var QueryTestsQueryTmpl = template.Must(template.New("QueryTestsQuery").Parse(`
  @{USE_ADDITIONAL_PARALLELISM=TRUE}
	WITH Tests as (
		SELECT DISTINCT TestId, SubRealm IN UNNEST(@subRealms) as HasAccess
		FROM TestRealms
		WHERE
			Project = @project
				AND TestId > @paginationTestId
				AND TestId LIKE @testIdPattern
	)
	SELECT TestId FROM Tests
	WHERE HasAccess
	ORDER BY TestId ASC
	{{if .hasLimit}}
		LIMIT @limit
	{{end}}
`))

The query is written in a way to force spanner NOT to put `SubRealm IN UNNEST(@subRealms)` check in Filter Scan seek condition, which can significantly increase the time it takes to scan the table.

View Source 
var TestRealmSaveCols = []string{"Project", "TestId", "SubRealm", "LastIngestionTime"}

TestRealmSaveCols is the set of columns written to in a test variant realm save. Allocated here once to avoid reallocating on every save.

View Source 
var TestResultSaveCols = []string{
	"Project", "TestId", "PartitionTime", "VariantHash",
	"IngestedInvocationId", "RunIndex", "ResultIndex",
	"IsUnexpected", "RunDurationUsec", "Status",
	"ExonerationReasons",
	"SourceRefHash", "SourcePosition",
	"ChangelistHosts", "ChangelistChanges", "ChangelistPatchsets",
	"ChangelistOwnerKinds",
	"HasDirtySources",
	"SubRealm", "IsFromBisection",
}

TestResultSaveCols is the set of columns written to in a test result save. Allocated here once to avoid reallocating on every test result save.

View Source 
var TestVariantRealmSaveCols = []string{
	"Project", "TestId", "VariantHash", "SubRealm",
	"Variant", "LastIngestionTime",
}

TestVariantRealmSaveCols is the set of columns written to in a test variant realm save. Allocated here once to avoid reallocating on every save.

Functions

func CompressHost 

func CompressHost(host string) string

CompressHost transforms a gerrit hostname into its compressed database representation.

func CreateQueryFailureRateTestData 

func CreateQueryFailureRateTestData(ctx context.Context) error

CreateQueryFailureRateTestData creates test data in Spanner for testing QueryFailureRate.

func DecompressHost 

func DecompressHost(host string) string

DecompressHost recovers a gerrit hostname from its compressed database representation.

func OwnerKindFromDB 

func OwnerKindFromDB(value string) pb.ChangelistOwnerKind

OwnerKindFromDB decodes owner kind from its database representation.

func OwnerKindToDB 

func OwnerKindToDB(value pb.ChangelistOwnerKind) string

OwnerKindToDB encodes owner kind to its database representation.

func QueryFailureRate 

func QueryFailureRate(ctx context.Context, opts QueryFailureRateOptions) (*pb.QueryTestVariantFailureRateResponse, error)

QueryFailureRate queries the failure rate of nominated test variants.

Must be called in a Spanner transactional context. Context must support multiple reads (i.e. NOT spanner.Single()) as request may batched over multiple reads.

func QueryFailureRateSampleRequest 

func QueryFailureRateSampleRequest() (project string, asAtTime time.Time, testVariants []*pb.TestVariantIdentifier)

func QueryFailureRateSampleResponse 

func QueryFailureRateSampleResponse() *pb.QueryTestVariantFailureRateResponse

QueryFailureRateSampleResponse returns expected response data from QueryFailureRate after being invoked with QueryFailureRateSampleRequest. It is assumed test data was setup with CreateQueryFailureRateTestData.

func QueryTests 

func QueryTests(ctx context.Context, project, testIDSubstring string, opts QueryTestsOptions) (testIDs []string, nextPageToken string, err error)

QueryTests finds all the test IDs with the specified testIDSubstring from the spanner database. Must be called in a spanner transactional context.

func ReadTestHistory 

func ReadTestHistory(ctx context.Context, opts ReadTestHistoryOptions) (verdicts []*pb.TestVerdict, nextPageToken string, err error)

ReadTestHistory reads verdicts from the spanner database. Must be called in a spanner transactional context.

func ReadTestHistoryStats 

func ReadTestHistoryStats(ctx context.Context, opts ReadTestHistoryOptions) (groups []*pb.QueryTestHistoryStatsResponse_Group, nextPageToken string, err error)

ReadTestHistoryStats reads stats of verdicts grouped by UTC dates from the spanner database. Must be called in a spanner transactional context.

func ReadTestRealms 

func ReadTestRealms(ctx context.Context, keys spanner.KeySet, fn func(tr *TestRealm) error) error

ReadTestRealms read test variant realms from the TestRealms table. Must be called in a spanner transactional context.

func ReadTestResults 

func ReadTestResults(ctx context.Context, keys spanner.KeySet, fn func(tr *TestResult) error) error

ReadTestResults reads test results from the TestResults table. Must be called in a spanner transactional context.

func ReadTestVariantRealms 

func ReadTestVariantRealms(ctx context.Context, keys spanner.KeySet, fn func(tvr *TestVariantRealm) error) error

ReadTestVariantRealms read test variant realms from the TestVariantRealms table. Must be called in a spanner transactional context.

func ReadVariants 

func ReadVariants(ctx context.Context, project, testID string, opts ReadVariantsOptions) (variants []*pb.QueryVariantsResponse_VariantInfo, nextPageToken string, err error)

ReadVariants reads all the variants of the specified test from the spanner database. Must be called in a spanner transactional context.

func SortChangelists 

func SortChangelists(cls []Changelist)

SortChangelists sorts a slice of changelists to be in ascending lexicographical order by (host, change, patchset).

func ValidateGerritHostname 

func ValidateGerritHostname(host string) error

ValidateGerritHostname validates the given gerrit hostname.

Types

type BQChangelist 

type BQChangelist struct {
	Host      bigquery.NullString
	Change    bigquery.NullInt64
	Patchset  bigquery.NullInt64
	OwnerKind bigquery.NullString
}

type BQGitiles 

type BQGitiles struct {
	Host    bigquery.NullString
	Project bigquery.NullString
	Ref     bigquery.NullString
}

type BQRef 

type BQRef struct {
	Gitiles *BQGitiles
}

type BQTestVerdict 

type BQTestVerdict struct {
	TestID                string
	VariantHash           string
	RefHash               string
	InvocationID          string
	Status                string
	PartitionTime         time.Time
	PassedAvgDurationUsec bigquery.NullFloat64
	Changelists           []*BQChangelist
	// Whether the caller has access to this test verdict.
	HasAccess bool
}

type Changelist 

type Changelist struct {
	// Host is the gerrit hostname. E.g. chromium-review.googlesource.com.
	Host      string
	Change    int64
	Patchset  int64
	OwnerKind pb.ChangelistOwnerKind
}

Changelist represents a gerrit changelist.

type CommitWithVerdicts 

type CommitWithVerdicts struct {
	// Source position of this commit.
	Position int64
	// Commit hash of this commit.
	CommitHash string
	// Represent a branch in the source control.
	Ref *BQRef
	// Realm of test verdicts at this commit.
	Realm string
	// Returns at most 20 test verdicts at this commit.
	TestVerdicts []*BQTestVerdict
}

CommitWithVerdicts represents a commit with test verdicts.

type FakeReadClient 

type FakeReadClient struct {
	CommitsWithVerdicts []*CommitWithVerdicts
	SourceVerdicts      []SourceVerdict
}

FakeReadClient represents a fake implementation of the client to read test verdicts from BigQuery, for testing.

func (*FakeReadClient) ReadSourceVerdicts 

func (f *FakeReadClient) ReadSourceVerdicts(ctx context.Context, options ReadSourceVerdictsOptions) ([]SourceVerdict, error)

func (*FakeReadClient) ReadTestVerdictsPerSourcePosition 

func (f *FakeReadClient) ReadTestVerdictsPerSourcePosition(ctx context.Context, options ReadTestVerdictsPerSourcePositionOptions) ([]*CommitWithVerdicts, error)

ReadTestVerdictsPerSourcePosition reads test verdicts per source position.

type QueryFailureRateOptions 

type QueryFailureRateOptions struct {
	// Project is the LUCI Project to query.
	Project string
	// SubRealms are the realms (of the form "ci", NOT "chromium:ci")
	// within the project to query.
	SubRealms []string
	// TestVariants are the test variants to query.
	TestVariants []*pb.TestVariantIdentifier
	// AsAtTime is latest parititon time to include in the results;
	// outside of testing contexts, this should be the current time.
	// QueryTestVariants returns data for the 5 * 24 weekday hour
	// period leading up to this time.
	AsAtTime time.Time
}

QueryFailureRateOptions specifies options for QueryFailureRate().

type QueryTestsOptions 

type QueryTestsOptions struct {
	SubRealms []string
	PageSize  int
	PageToken string
}

QueryTestsOptions specifies options for QueryTests().

type ReadClient 

type ReadClient struct {
	// contains filtered or unexported fields
}

ReadClient represents a client to read test results table from BigQuery.

func NewReadClient 

func NewReadClient(ctx context.Context, gcpProject string) (*ReadClient, error)

NewReadClient creates a new client for reading test results BigQuery table.

func (*ReadClient) Close 

func (c *ReadClient) Close() error

Close releases any resources held by the client.

func (*ReadClient) ReadSourceVerdicts 

func (c *ReadClient) ReadSourceVerdicts(ctx context.Context, options ReadSourceVerdictsOptions) ([]SourceVerdict, error)

ReadSourceVerdicts reads source verdicts on the specified test variant branch, for a nominated source position range.

func (*ReadClient) ReadTestVerdictsPerSourcePosition 

func (c *ReadClient) ReadTestVerdictsPerSourcePosition(ctx context.Context, options ReadTestVerdictsPerSourcePositionOptions) ([]*CommitWithVerdicts, error)

ReadTestVerdictsPerSourcePosition returns commits with test verdicts in source position ascending order. Only return commits within the last 90 days.

type ReadSourceVerdictsOptions 

type ReadSourceVerdictsOptions struct {
	Project     string
	TestID      string
	VariantHash string
	RefHash     string
	// Only test verdicts with allowed invocation subrealms can be returned.
	AllowedSubrealms []string
	// The maximum source position to return, inclusive.
	StartSourcePosition int64
	// The minimum source position to return, exclusive.
	EndSourcePosition int64
	// The last partition time to include in the results, exclusive.
	EndPartitionTime time.Time
}

type ReadTestHistoryOptions 

type ReadTestHistoryOptions struct {
	Project                 string
	TestID                  string
	SubRealms               []string
	VariantPredicate        *pb.VariantPredicate
	SubmittedFilter         pb.SubmittedFilter
	TimeRange               *pb.TimeRange
	ExcludeBisectionResults bool
	PageSize                int
	PageToken               string
}

ReadTestHistoryOptions specifies options for ReadTestHistory().

type ReadTestVerdictsPerSourcePositionOptions 

type ReadTestVerdictsPerSourcePositionOptions struct {
	Project     string
	TestID      string
	VariantHash string
	RefHash     string
	// Only test verdicts with allowed invocation realms can be returned.
	AllowedRealms []string
	// All returned commits has source position greater than PositionMustGreater.
	PositionMustGreater int64
	// The maximum number of commits to return.
	NumCommits int64
}

type ReadVariantsOptions 

type ReadVariantsOptions struct {
	SubRealms        []string
	VariantPredicate *pb.VariantPredicate
	PageSize         int
	PageToken        string
}

ReadVariantsOptions specifies options for ReadVariants().

type RunStatus 

type RunStatus int64
const (
	Unexpected RunStatus = iota
	Flaky
	Expected
)

type SourceVerdict 

type SourceVerdict struct {
	// The source position.
	Position int64
	// Test verdicts at the position. Limited to 20.
	Verdicts []SourceVerdictTestVerdict
}

SourceVerdict aggregates all test results at a source position.

type SourceVerdictTestVerdict 

type SourceVerdictTestVerdict struct {
	// The invocation for which the verdict is.
	InvocationID string
	// Partition time of the test verdict.
	PartitionTime time.Time
	// Status is one of SKIPPED, EXPECTED, UNEXPECTED, FLAKY.
	Status string
	// Changelists tested by the verdict.
	Changelists []BQChangelist
}

SourceVerdictTestVerdict is a test verdict that is part of a source verdict.

type Sources 

type Sources struct {
	// 8-byte hash of the source reference (e.g. git branch) tested.
	// This refers to the base commit/version tested, before any changelists
	// are applied.
	RefHash []byte
	// The position along the source reference that was tested.
	// This refers to the base commit/version tested, before any changelists
	// are applied.
	Position int64
	// The gerrit changelists applied on top of the base version/commit.
	// At most 10 changelists should be specified here, if there are more
	// then limit to 10 and set HasDirtySources to true.
	Changelists []Changelist
	// Whether other modifications were made to the sources, not described
	// by the fields above. For example, a package was upreved in the build.
	// If this is set, then the source information is approximate: suitable
	// for plotting results by source position the UI but not good enough
	// for change point analysis.
	IsDirty bool
}

Sources captures information about the code sources that were tested by a test result.

func CopySources 

func CopySources(sources Sources) Sources

CopySources makes a deep copy of the given code sources.

type TestRealm 

type TestRealm struct {
	Project           string
	TestID            string
	SubRealm          string
	LastIngestionTime time.Time
}

TestVariantRealm represents a row in the TestVariantRealm table.

func (*TestRealm) SaveUnverified 

func (tvr *TestRealm) SaveUnverified() *spanner.Mutation

SaveUnverified creates a mutation to save the test realm into the TestRealms table. The test realm is not verified. Must be called in spanner RW transactional context.

type TestResult 

type TestResult struct {
	Project              string
	TestID               string
	PartitionTime        time.Time
	VariantHash          string
	IngestedInvocationID string
	RunIndex             int64
	ResultIndex          int64
	IsUnexpected         bool
	RunDuration          *time.Duration
	Status               pb.TestResultStatus
	// Properties of the test verdict (stored denormalised) follow.
	ExonerationReasons []pb.ExonerationReason
	Sources            Sources
	// Properties of the invocation (stored denormalised) follow.
	SubRealm        string
	IsFromBisection bool
}

TestResult represents a row in the TestResults table.

func (*TestResult) SaveUnverified 

func (tr *TestResult) SaveUnverified() *spanner.Mutation

SaveUnverified prepare a mutation to insert the test result into the TestResults table. The test result is not validated.

type TestResultBuilder 

type TestResultBuilder struct {
	// contains filtered or unexported fields
}

TestResultBuilder provides methods to build a test result for testing.

func NewTestResult 

func NewTestResult() TestResultBuilder

func (TestResultBuilder) Build 

func (b TestResultBuilder) Build() *TestResult

func (TestResultBuilder) WithExonerationReasons 

func (b TestResultBuilder) WithExonerationReasons(exonerationReasons ...pb.ExonerationReason) TestResultBuilder

func (TestResultBuilder) WithIngestedInvocationID 

func (b TestResultBuilder) WithIngestedInvocationID(invID string) TestResultBuilder

func (TestResultBuilder) WithIsFromBisection 

func (b TestResultBuilder) WithIsFromBisection(value bool) TestResultBuilder

func (TestResultBuilder) WithIsUnexpected 

func (b TestResultBuilder) WithIsUnexpected(unexpected bool) TestResultBuilder

func (TestResultBuilder) WithPartitionTime 

func (b TestResultBuilder) WithPartitionTime(partitionTime time.Time) TestResultBuilder

func (TestResultBuilder) WithProject 

func (b TestResultBuilder) WithProject(project string) TestResultBuilder

func (TestResultBuilder) WithResultIndex 

func (b TestResultBuilder) WithResultIndex(resultIndex int64) TestResultBuilder

func (TestResultBuilder) WithRunDuration 

func (b TestResultBuilder) WithRunDuration(duration time.Duration) TestResultBuilder

func (TestResultBuilder) WithRunIndex 

func (b TestResultBuilder) WithRunIndex(runIndex int64) TestResultBuilder

func (TestResultBuilder) WithSources 

func (b TestResultBuilder) WithSources(sources Sources) TestResultBuilder

func (TestResultBuilder) WithStatus 

func (b TestResultBuilder) WithStatus(status pb.TestResultStatus) TestResultBuilder

func (TestResultBuilder) WithSubRealm 

func (b TestResultBuilder) WithSubRealm(subRealm string) TestResultBuilder

func (TestResultBuilder) WithTestID 

func (b TestResultBuilder) WithTestID(testID string) TestResultBuilder

func (TestResultBuilder) WithVariantHash 

func (b TestResultBuilder) WithVariantHash(variantHash string) TestResultBuilder

func (TestResultBuilder) WithoutExoneration 

func (b TestResultBuilder) WithoutExoneration() TestResultBuilder

func (TestResultBuilder) WithoutRunDuration 

func (b TestResultBuilder) WithoutRunDuration() TestResultBuilder

type TestVariantRealm 

type TestVariantRealm struct {
	Project           string
	TestID            string
	VariantHash       string
	SubRealm          string
	Variant           *pb.Variant
	LastIngestionTime time.Time
}

TestVariantRealm represents a row in the TestVariantRealm table.

func (*TestVariantRealm) SaveUnverified 

func (tvr *TestVariantRealm) SaveUnverified() *spanner.Mutation

SaveUnverified creates a mutation to save the test variant realm into the TestVariantRealms table. The test variant realm is not verified. Must be called in spanner RW transactional context.

type TestVerdictBuilder 

type TestVerdictBuilder struct {
	// contains filtered or unexported fields
}

TestVerdictBuilder provides methods to build a test variant for testing.

func NewTestVerdict 

func NewTestVerdict() *TestVerdictBuilder

func (*TestVerdictBuilder) Build 

func (b *TestVerdictBuilder) Build() []*TestResult

func (*TestVerdictBuilder) WithBaseTestResult 

func (b *TestVerdictBuilder) WithBaseTestResult(testResult *TestResult) *TestVerdictBuilder

WithBaseTestResult specifies a test result to use as the template for the test variant's test results.

func (*TestVerdictBuilder) WithPassedAvgDuration 

func (b *TestVerdictBuilder) WithPassedAvgDuration(duration *time.Duration) *TestVerdictBuilder

WithPassedAvgDuration specifies the average duration to use for passed test results. If setting to a non-nil value, make sure to set the result status as passed on the base test result if using this option.

func (*TestVerdictBuilder) WithRunStatus 

func (b *TestVerdictBuilder) WithRunStatus(runStatuses ...RunStatus) *TestVerdictBuilder

WithRunStatus specifies the status of runs of the test verdict.

func (*TestVerdictBuilder) WithStatus 

func (b *TestVerdictBuilder) WithStatus(status pb.TestVerdictStatus) *TestVerdictBuilder

WithStatus specifies the status of the test verdict.

Source Files

View all Source files

Directories

Path Synopsis
Package exporter contains methods to export test results to BigQuery.
 Package exporter contains methods to export test results to BigQuery.
Package lowlatency contains methods for accessing the low-latency test results table in Spanner.
 Package lowlatency contains methods for accessing the low-latency test results table in Spanner.
Package stability implements the test stability analysis used by the QueryStability RPC.
 Package stability implements the test stability analysis used by the QueryStability RPC.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.