changepoints

package
v0.0.0-...-bfbb315 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 21, 2024 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Overview

Package changepoints handles change point detection and analysis. See go/luci-test-variant-analysis-design for details.

It implements two ingestion paths:

  • Low latency ingestion: Ingestion for test results that are not testing gerrit changelists. This occurs on the result-ingestion pipeline. It is called low-latency because the result-ingestion pipeline is generally triggered immediately upon invocation finalization.
  • High latency ingestion: Ingestion for test results testing gerrit changelists. These results are only eligible for ingestion if the presubmit run suceeded and was submitted. This occurs on the verdict-ingestion pipeline because this is the only pipeline that waits until the LUCI CV run has completed before it ingests. This is known as the high-latency pipeline.

Index

Constants

View Source 
const RegressionRangeOverlapPrecThreshold = 0.4

RegressionRangeOverlapPrecThreshold decides whether two changepoints can be grouped together. The regression range overlap percentage is calculated by (# of overlapped commits/# of commits in the narrower regression range).

View Source 
const TestIDGroupingThreshold = 64

TestIDGroupingThreshold is the threshold to partition changepoints by test ID. TODO: Set this threshold dynamically base on the total number of unique tests in the requested LUCI project and the number of regressions in this period. Because the significance of seeing a gap of 64 in test ID number depends on the above two factors. A possible formula to derive this threshold is (total tests / # of regressions in period) * coefficient.

Variables

View Source 
var (
	RunCounter = metric.NewCounter(
		"analysis/changepoints/analyze/runs",
		"The number of runs processed by changepoint analysis, classified by project and status.",
		nil,

		field.String("project"),

		field.String("status"),
	)
)

Functions

func Analyze 

func Analyze(ctx context.Context, tvs []*rdbpb.TestVariant, payload *taskspb.IngestTestVerdicts, sourcesMap map[string]*pb.Sources, exporter *bqexporter.Exporter) error

Analyze performs change point analyses based on incoming test verdicts. sourcesMap contains the information about the source code being tested.

func AnalyzeRun 

func AnalyzeRun(ctx context.Context, tvs []*rdbpb.RunTestVerdict, opts AnalysisOptions, exporter *bqexporter.Exporter) error

AnalyzeRun performs change point analyses based on an incoming test run.

func ClaimInvocationMutation 

func ClaimInvocationMutation(project string, invocationID string, rootInvocationID string) *spanner.Mutation

ClaimInvocationMutation creates a mutation to claim an invocation for the given root invocation.

func CompareTestVariantBranchChangepoint 

func CompareTestVariantBranchChangepoint(cpi, cpj *ChangepointDetailRow) bool

CompareTestVariantBranchChangepoint returns whether element at i is smaller than element at j by comparing TestIDNum, VariantHash, RefHash, NominalStartPosition.

func FetchTestVariantBranches 

func FetchTestVariantBranches(ctx context.Context) ([]*testvariantbranch.Entry, error)

func GroupChangepoints 

func GroupChangepoints(ctx context.Context, rows []*ChangepointDetailRow) [][]*ChangepointDetailRow

GroupChangepoints returns a 2D array where each row represents a group of changepoints. The grouping result is deterministic, which means same input always results in same groups. The groups are generated with the following steps.

  1. partition changepoints base on test ID
  2. For changepoints in each partition, group base on percentage regression range overlap.

func QueryStatsForClustering 

func QueryStatsForClustering(ctx context.Context, tvs []*rdbpb.TestVariant, project string, partitionTime time.Time, sourcesMap map[string]*pb.Sources) ([]*clusteringpb.TestVariantBranch, error)

QueryStatsForClustering reads selected statistics for the test variant branch of nominated test verdicts. The statistics are those tracked in the clustered_failures table.

The result slice will contain result items in 1:1 correspondance to the provided test verdicts (i.e. result[i] corresponds to tvs[i]). If no source information is available for some or all of the verdicts, the corresponding item in the response slice will be nil.

func StartOfWeek 

func StartOfWeek(t time.Time) time.Time

StartOfWeek returns the start of the week for the given time. A week refers to the period from Sunday at 00:00:00 AM UTC (inclusive) to the following Sunday at 00:00:00 AM UTC (exclusive).

Types

type AnalysisOptions 

type AnalysisOptions struct {
	// The LUCI Project the test results should be ingested into.
	Project string
	// The ResultDB host from which the invocations are from.
	ResultDBHost string
	// The export root under which the test results are being ingested.
	RootInvocationID string
	// The invocation being ingested.
	InvocationID string
	// The sources tested by tests run in invocation InvocationID,
	// as viewed from export root RootInvocationID.
	Sources *pb.Sources
	// The start of the retention period.
	PartitionTime time.Time
}

type ChangepointDetailRow 

type ChangepointDetailRow struct {
	Project string
	// TestIDNum is the alphabetical ranking of the test ID in this LUCI project.
	TestIDNum   int64
	TestID      string
	VariantHash string
	Variant     bigquery.NullJSON
	// Point to a branch in the source control.
	Ref     *Ref
	RefHash string
	// The source verdict unexpected rate before the changepoint.
	UnexpectedSourceVerdictRateBefore float64
	// The source verdict unexpected rate after the changepoint.
	UnexpectedSourceVerdictRateAfter float64
	// The current source verdict unexpected rate.
	UnexpectedSourceVerdictRateCurrent float64
	// The nominal start hour of the segment after the changepoint.
	StartHour                  time.Time
	LowerBound99th             int64
	UpperBound99th             int64
	NominalStartPosition       int64
	PreviousNominalEndPosition int64
}

ChangepointDetailRow represents a changepoint of a test variant branch with more details including TestIDNum and source verdict rates.

type ChangepointRow 

type ChangepointRow struct {
	Project     string
	TestID      string
	VariantHash string
	Variant     bigquery.NullJSON
	// Point to a branch in the source control.
	Ref     *Ref
	RefHash string
	// The nominal start hour of the segment after the changepoint.
	StartHour                  time.Time
	LowerBound99th             int64
	UpperBound99th             int64
	NominalStartPosition       int64
	PreviousNominalEndPosition int64
}

ChangepointRow represents a changepoint of a test variant branch.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client to read LUCI Analysis changepoints.

func NewClient 

func NewClient(ctx context.Context, gcpProject string) (*Client, error)

NewClient creates a new client for reading changepints.

func (*Client) Close 

func (c *Client) Close() error

Close releases any resources held by the client.

func (*Client) ReadChangepointGroupSummaries 

func (c *Client) ReadChangepointGroupSummaries(ctx context.Context, opts ReadChangepointGroupSummariesOptions) (groups []*GroupSummary, nextPageToken string, err error)

ReadChangepointGroupSummaries reads summaries of changepoint groups started at a week which is within the last 90 days.

func (*Client) ReadChangepoints 

func (c *Client) ReadChangepoints(ctx context.Context, project string, week time.Time) (changepoints []*ChangepointDetailRow, err error)

ReadChangepoints reads changepoints of a certain week from BigQuery. The week parameter can be at any time of that week. A week is defined by Sunday to Satureday in UTC.

func (*Client) ReadChangepointsInGroup 

func (c *Client) ReadChangepointsInGroup(ctx context.Context, opts ReadChangepointsInGroupOptions) (changepoints []*ChangepointRow, err error)

ReadChangepointsInGroup read changepoints in the same group as the changepoint specified in option. At most 1000 changepoints are returned, ordered by NominalStartPosition DESC, TestID, VariantHash, RefHash.

func (*Client) ReadChangepointsRealtime 

func (c *Client) ReadChangepointsRealtime(ctx context.Context, week time.Time) (changepoints []*ChangepointDetailRow, err error)

ReadChangepointsRealtime reads changepoints of a certain week directly from BigQuery exports of changepoint analysis. A week in this context refers to the period from Sunday at 00:00:00 AM UTC (inclusive) to the following Sunday at 00:00:00 AM UTC (exclusive). The week parameter MUST be a timestamp representing the start of a week (Sunday at 00:00:00 AM UTC) and within the last 90 days.

type Gitiles 

type Gitiles struct {
	Host    bigquery.NullString
	Project bigquery.NullString
	Ref     bigquery.NullString
}

type GroupSummary 

type GroupSummary struct {
	CanonicalChangepoint               ChangepointRow
	Total                              int64
	UnexpectedSourceVerdictRateBefore  RateDistribution
	UnexpectedSourceVerdictRateAfter   RateDistribution
	UnexpectedSourceVerdictRateCurrent RateDistribution
	UnexpectedSourveVerdictRateChange  RateChangeDistribution
}

type RateChangeDistribution 

type RateChangeDistribution struct {
	Increase0to20percent   int64
	Increase20to50percent  int64
	Increase50to100percent int64
}

type RateDistribution 

type RateDistribution struct {
	Mean                    float64
	Less5Percent            int64
	Above5LessThan95Percent int64
	Above95Percent          int64
}

type ReadChangepointGroupSummariesOptions 

type ReadChangepointGroupSummariesOptions struct {
	Project       string
	TestIDContain string
	PageSize      int
	PageToken     string
}

type ReadChangepointsInGroupOptions 

type ReadChangepointsInGroupOptions struct {
	Project       string
	TestIDContain string
	// TestID, VariantHash, RefHash and StartPosition define a changepoint.
	TestID        string
	VariantHash   string
	RefHash       string
	StartPosition int64
}

type Ref 

type Ref struct {
	Gitiles *Gitiles
}

Source Files

View all Source files

Directories

Path Synopsis
Package analyzer converts the input buffer into segments using changepoint analysis, and synthesises those segments with the segments in the output buffer to produce a logical segementation of the test history.
 Package analyzer converts the input buffer into segments using changepoint analysis, and synthesises those segments with the segments in the output buffer to produce a logical segementation of the test history.
Package bayesian implements bayesian analysis for detecting change points.
 Package bayesian implements bayesian analysis for detecting change points.
Package bqexporter handles the export of test variant analysis results to BigQuery.
 Package bqexporter handles the export of test variant analysis results to BigQuery.
Package bqupdator handles the update of bigquery tables related to changepoint analysis.
 Package bqupdator handles the update of bigquery tables related to changepoint analysis.
Package groupexporter provides functions to export grouped changepoints to BigQuery.
 Package groupexporter provides functions to export grouped changepoints to BigQuery.
Package groupscheduler schedules group changepoints tasks.
 Package groupscheduler schedules group changepoints tasks.
Package inputbuffer handles the input buffer of change point analysis.
 Package inputbuffer handles the input buffer of change point analysis.
Package model contains common types used in changepoint analysis.
 Package model contains common types used in changepoint analysis.
Package sorbet implements analysis for test changepoint culprits.
 Package sorbet implements analysis for test changepoint culprits.
Package sources handles sources information.
 Package sources handles sources information.
Package testutil contains utility functions for tests.
 Package testutil contains utility functions for tests.
Package testvariantbranch handles test variant branch of change point analysis.
 Package testvariantbranch handles test variant branch of change point analysis.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.