internal

package
v0.0.0-...-0d3e22f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2024 License: BSD-3-Clause Imports: 33 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	Functional  = "Functional"
	Performance = "Performance"
)

Variables

View Source 
var MidpointHandlerContextKey = &MidpointHandlerKey{}

Functions

func BuildWorkflow 

func BuildWorkflow(ctx workflow.Context, params workflows.BuildParams) (*workflows.Build, error)

BuildWorkflow is a Workflow definition that creates a build artifact.

func CheckCombinedCommitEqualActivity 

func CheckCombinedCommitEqualActivity(ctx context.Context, lower, higher *common.CombinedCommit) (bool, error)

CheckCombinedCommitEqualActivity checks whether two combined commits are equal.

func CollectAndUploadWorkflow 

func CollectAndUploadWorkflow(ctx workflow.Context, req *CollectAndUploadParams) error

CollectAndUploadWorkflow coordinates fetching the test result artifacts from all Swarming tasks triggered by the RunTestAndExportWorkflow, gathering results from those executions by chart, and uplodaing those sample values to BQ.

func CollectValuesActivity 

func CollectValuesActivity(ctx context.Context, run *workflows.TestRun, benchmark, chart, aggMethod string) ([]float64, error)

CollectValuesActivity is an activity to collect sampled values from a single test run.

func ComparePairwiseActivity 

func ComparePairwiseActivity(ctx context.Context, valuesA, valuesB []float64, dir compare.ImprovementDir) (*compare.ComparePairwiseResult, error)

ComparePairwiseActivity wraps compare.ComparePairwise as a temporal activity

func FindAvailableBotsActivity 

func FindAvailableBotsActivity(ctx context.Context, botConfig string, seed int64) ([]string, error)

FindAvailableBotsActivity fetches a list of free, alive and non quarantined bots per provided bot configuration for eg: android-go-wembley-perf

The function makes a swarming API call internally to fetch the desired bots. If successful, a slice of bot ids is returned

func FindMidCommitActivity 

func FindMidCommitActivity(ctx context.Context, lower, higher *common.CombinedCommit) (*common.CombinedCommit, error)

FindMidCommitActivity is an Activity that finds the middle point of two commits.

func PairwiseWorkflow 

func PairwiseWorkflow(ctx workflow.Context, p *workflows.PairwiseParams) (*pinpoint_proto.PairwiseExecution, error)

func PostBugCommentActivity 

func PostBugCommentActivity(ctx context.Context, issueID int64, comment string) (bool, error)

PostBugCommentActivity wraps the call to Issuetracker's PostComment.

func PostBugCommentWorkflow 

func PostBugCommentWorkflow(ctx workflow.Context, issueID int64, comment string) (bool, error)

TODO(sunxiaodi): Add a unit test for this workflow

func ReportStatusActivity 

func ReportStatusActivity(ctx context.Context, issueID int, culprits []*pinpoint_proto.CombinedCommit) error

ReportStatusActivity wraps the call to IssueTracker to report culprits. TODO(sunxiaodi@): Update this activity for culprit verification

func RunBenchmarkPairwiseWorkflow 

func RunBenchmarkPairwiseWorkflow(ctx workflow.Context, firstRBP, secondRBP *RunBenchmarkParams, first workflows.PairwiseOrder) (*workflows.PairwiseTestRun, error)

RunBenchmarkPairwiseWorkflow is a Workflow definition that schedules a pairwise of tasks, polls and retrieves the CAS for the RunBenchmarkParams defined. TODO(b/340247044): connect mutex lock to this workflow and lock the swarming resource from the same pinpoint job and other pinpoint jobs. After swarming tasks have scheduled, the mutex lock can be released and the rest of the workflow can proceed. This workflow will also not schedule swarming tasks until it obtains the lock on the swarming resource. TODO(sunxiaodi@): Convert this workflow to accept slice and replace RunBenchmarkWorkflow with this workflow.

func RunBenchmarkWorkflow 

func RunBenchmarkWorkflow(ctx workflow.Context, p *RunBenchmarkParams) (*workflows.TestRun, error)

RunBenchmarkWorkflow is a Workflow definition that schedules a single task, polls and retrieves the CAS for the RunBenchmarkParams defined.

func RunTestAndExportWorkflow 

func RunTestAndExportWorkflow(ctx workflow.Context, req *TestAndExportParams) error

RunTestAndExportWorkflow will build chrome, run tests and export the swarming task ids to BQ.

func UploadResultsActivity 

func UploadResultsActivity(ctx context.Context, project, dataset, tableName string, results []*TestResult) error

Types

type BisectExecution 

type BisectExecution struct {
	JobId string
	// TODO(b/322203189): replace Culprits with DetailedCulprits. This field is used by the
	// catapult bisect UI write.
	Culprits []*pinpoint_proto.CombinedCommit
	// DetailedCulprits stores a list of culprits and the commit prior to the culprit.
	// Without this field, culprit verification would not know the commit prior to any culprit
	// and be unable to verify the correct pair of commits.
	DetailedCulprits []*pinpoint_proto.Culprit
	CreateTime       *timestamppb.Timestamp
	Comparisons      []*CombinedResults
	RunData          []*BisectRun
}

BisectExecution is a mirror of pinpoint_proto.BisectExecution, with additional raw data.

When this BisectExecution embeds pinpoint_proto.BisectExecution, it fails to store CommitPairValues and BisectRuns, which are used to curate the information for Catapult Pinpoint. TODO(b/322203189) - This is a temporary solution for backwards compatibilty to the Catapult UI and should be removed when the catapult package is deprecated.

func BisectWorkflow 

func BisectWorkflow(ctx workflow.Context, p *workflows.BisectParams) (be *BisectExecution, wkErr error)

BisectWorkflow is a Workflow definition that takes a range of git hashes and finds the culprit.

type BisectRun 

type BisectRun struct {
	CommitRun
	ScheduledRuns []scheduledRun
}

BisectRun tracks current scheduled SingleCommitRun's and merges from other runs.

This is not thread-safe. Schedule and Update usually happens in an I/O nonblocking manner where they are invoked in the same thread vis select.

type BisectRunIndex 

type BisectRunIndex int

bisectRunTracker stores all the running bisect runs.

It keeps track of all the runs by indexes. The BisectRun will be updated from different future fulfillment. One can wait for a bisect run that's already triggered by a different bisection. This usually happens when the mid commit is computed and the result will be used for comparisions from both sides. This can also happen when one comparision requires more runs, and part of them is already triggered by another comparision.

type BuildActivity 

type BuildActivity struct{}

BuildActivity wraps Build Activities.

func (*BuildActivity) CleanupBuildActivity 

func (bca *BuildActivity) CleanupBuildActivity(ctx context.Context, buildID int64, status buildbucketpb.Status, project string) error

CleanupBuildActivity wraps BuildChromeClient.CancelBuild

func (*BuildActivity) RetrieveBuildArtifactActivity 

func (bca *BuildActivity) RetrieveBuildArtifactActivity(ctx context.Context, buildID int64, target, project string) (*apipb.CASReference, error)

RetrieveBuildArtifactActivity gets build artifacts.

func (*BuildActivity) SearchOrBuildActivity 

func (bca *BuildActivity) SearchOrBuildActivity(ctx context.Context, params workflows.BuildParams) (int64, error)

SearchOrBuildActivity searches for an existing build to reuse, or triggers a new one.

func (*BuildActivity) WaitBuildCompletionActivity 

func (bca *BuildActivity) WaitBuildCompletionActivity(ctx context.Context, buildID int64, project string) (buildbucketpb.Status, error)

WaitBuildCompletionActivity polls the build and waits until it is completed or errors.

type CollectAndUploadParams 

type CollectAndUploadParams struct {
	// Project is the Google Cloud Project (ie/ chromeperf), used for creating the
	// fully qualified BQ table name.
	Project string
	// Dataset is the BQ Dataset. Format is {project}.{dataset}.
	// Used for creating the fully qualified BQ table name.
	Dataset string
	// TableName is the name of the table within Dataset. Format is
	// {project}.{dataset}.{tableName}. Used for creating the fully qualified BQ
	// table name.
	TableName string
	// Benchmark is the benchmark name, ie/ Speedometer3.
	Benchmark string
	// WorkflowID is the ID of the Temporal Workflow.
	WorkflowId string
}

CollectAndUploadParams contains required paramers for the CollectAndUploadWorkflow. This is the minimum set of parameters to collect results of the Swarming task runs, process them and upload to BigQuery.

type CombinedResults 

type CombinedResults struct {
	Result           *compare.CompareResults
	OtherResult      *compare.CompareResults // record the other comparison
	ResultType       string                  // either Functional or Performance
	CommitPairValues CommitPairValues
}

func CompareActivity 

func CompareActivity(ctx context.Context, allValues CommitPairValues, magnitude, errRate float64, direction compare.ImprovementDir) (*CombinedResults, error)

CompareActivity wraps compare.ComparePerformance and compare.CompareFunctional as activity

commitA and commitB are passed in to make it easier to see on the Temporal UI what two commits are being tested. Errors are recorded in the activity but the ErrorVerdict is not passed back to the main workflow.

type CommitPairValues 

type CommitPairValues struct {
	Lower  CommitValues
	Higher CommitValues
}

func GetAllDataForCompareLocalActivity 

func GetAllDataForCompareLocalActivity(ctx context.Context, lbr *BisectRun, hbr *BisectRun, chart string) (*CommitPairValues, error)

TODO(sunxiaodi@): Change GetAllDataForCompareLocalActivity to a regular function

type CommitRangeTracker 

type CommitRangeTracker struct {
	Lower  BisectRunIndex
	Higher BisectRunIndex
}

CommitRangeTracker stores a commit range as [Lower, Higher].

It stores bisect run as indexes as it needs to be serialized. The indexes are stable within the workflow thru bisectRunTracker.

func (CommitRangeTracker) CloneWithHigher 

func (t CommitRangeTracker) CloneWithHigher(higher BisectRunIndex) CommitRangeTracker

CloneWithHigher clones itself with the overriden higher index.

func (CommitRangeTracker) CloneWithLower 

func (t CommitRangeTracker) CloneWithLower(lower BisectRunIndex) CommitRangeTracker

CloneWithHigher clones itself with the overriden lower index.

type CommitRun 

type CommitRun struct {
	// The Chrome build associated with the commit.
	Build *workflows.Build

	// All the benchmark runs using the build.
	Runs []*workflows.TestRun
}

CommitRun stores benchmark tests runs for a single commit

func SingleCommitRunner 

func SingleCommitRunner(ctx workflow.Context, sc *SingleCommitRunnerParams) (*CommitRun, error)

SingleCommitRunner is a Workflow definition.

SingleCommitRunner builds, runs and collects benchmark sampled values from one single commit.

func (*CommitRun) AllErrorValues 

func (cr *CommitRun) AllErrorValues(chart string) []float64

func (*CommitRun) AllValues 

func (cr *CommitRun) AllValues(chart string) []float64

type CommitValues 

type CommitValues struct {
	Commit      *common.CombinedCommit
	Values      []float64
	ErrorValues []float64
}

type MidpointHandlerKey 

type MidpointHandlerKey struct{}

type PairwiseCommitsRunnerParams 

type PairwiseCommitsRunnerParams struct {
	SingleCommitRunnerParams

	// LeftCommit and RightCommit specify the two commits the pairwise runner will compare.
	// SingleCommitRunnerParams includes a field for only one commit.
	LeftCommit, RightCommit *common.CombinedCommit

	// The random seed used to generate pairs.
	Seed int64
}

PairwiseCommitsRunnerParams defines the parameters for PairwiseCommitsRunner workflow.

type PairwiseRun 

type PairwiseRun struct {
	Left, Right CommitRun
	// Order represents the order of the runs between Left and Right.
	// 0 means Left went first and 1 means Right went first.
	// The order is needed to handle pair failures. If a pair fails,
	// another pair that went in the other order needs to be tossed
	// from the data analysis to ensure balancing.
	Order []workflows.PairwiseOrder
}

PairwiseRun is the output of the PairwiseCommitsRunnerWorkflow TODO(b/321306427): This struct assumes that the i-th Left and Right CommitRuns are part of the same pair. If this assumption breaks, consider refactoring this struct and the subsequent workflows to instead store a list of PairwiseTestRuns. Another potential reason for refactoring is if len(Order) != len(Left) or len(Right).

func PairwiseCommitsRunnerWorkflow 

func PairwiseCommitsRunnerWorkflow(ctx workflow.Context, pc PairwiseCommitsRunnerParams) (*PairwiseRun, error)

PairwiseCommitsRunnerWorkflow is a Workflow definition.

PairwiseCommitsRunner builds, runs and collects benchmark sampled values from several commits. It runs the tests in pairs to reduces sample noises.

type RunBenchmarkActivity 

type RunBenchmarkActivity struct {
}

RunBenchmarkActivity wraps RunBenchmarkWorkflow in Activities

func (*RunBenchmarkActivity) CleanupBenchmarkRunActivity 

func (rba *RunBenchmarkActivity) CleanupBenchmarkRunActivity(ctx context.Context, taskID string, state run_benchmark.State) error

CleanupActivity wraps run_benchmark.Cancel

func (*RunBenchmarkActivity) IsTaskPairContinuousActivity 

func (rba *RunBenchmarkActivity) IsTaskPairContinuousActivity(ctx context.Context, botID, taskID1, taskID2 string) (bool, error)

func (*RunBenchmarkActivity) IsTaskPairOrderedActivity 

func (rba *RunBenchmarkActivity) IsTaskPairOrderedActivity(ctx context.Context, taskID1, taskID2 string) (bool, error)

func (*RunBenchmarkActivity) RetrieveTestCASActivity 

func (rba *RunBenchmarkActivity) RetrieveTestCASActivity(ctx context.Context, taskID string) (*apipb.CASReference, error)

RetrieveTestCASActivity wraps retrieves task artifacts from CAS

func (*RunBenchmarkActivity) ScheduleTaskActivity 

func (rba *RunBenchmarkActivity) ScheduleTaskActivity(ctx context.Context, rbp *RunBenchmarkParams) (string, error)

ScheduleTaskActivity wraps run_benchmark.Run

func (*RunBenchmarkActivity) WaitTaskAcceptedActivity 

func (rba *RunBenchmarkActivity) WaitTaskAcceptedActivity(ctx context.Context, taskID string) (run_benchmark.State, error)

WaitTaskAcceptedActivity polls the task until Swarming schedules the task. If the task is not scheduled, then it returns NO_RESOURCE. Note that there are other causes for NO_RESOURCE, but the solution is generally the same: schedule the run on a different, available bot. This activity is intended to only be used by pairwise workflow.

func (*RunBenchmarkActivity) WaitTaskFinishedActivity 

func (rba *RunBenchmarkActivity) WaitTaskFinishedActivity(ctx context.Context, taskID string) (run_benchmark.State, error)

WaitTaskFinishedActivity polls the task until it finishes or errors. Returns the status if the task finishes regardless of task success

func (*RunBenchmarkActivity) WaitTaskPendingActivity 

func (rba *RunBenchmarkActivity) WaitTaskPendingActivity(ctx context.Context, taskID string) (run_benchmark.State, error)

WaitTaskPendingActivity polls the task until it is no longer pending. Returns the status if the task stops pending regardless of task success

type RunBenchmarkParams 

type RunBenchmarkParams struct {
	// the Pinpoint job id
	JobID string
	// the swarming instance and cas digest hash and bytes location for the build
	BuildCAS *apipb.CASReference
	// commit hash
	Commit *common.CombinedCommit
	// device configuration
	BotConfig string
	// benchmark to test
	Benchmark string
	// story to test
	Story string
	// story tags for the test
	StoryTags string
	// additional dimensions for bot selection
	Dimensions map[string]string
	// iteration for the benchmark run. A few workflows have multiple iterations of
	// benchmark runs and this param comes in handy to get additional info of a specific run.
	// This is for debugging/informational purposes only.
	IterationIdx int32
	// Chart is a story histogram in a Benchmark.
	Chart string
	// AggregationMethod is method to aggregate sampled values.
	// If empty, then the original values are returned.
	AggregationMethod string
}

RunBenchmarkParams are the Temporal Workflow params for the RunBenchmarkWorkflow.

type SingleCommitRunnerParams 

type SingleCommitRunnerParams struct {
	// PinpointJobID is the Job ID to associate with the run.
	PinpointJobID string

	// BotConfig is the name of the device configuration, e.g. "linux-perf".
	BotConfig string

	// Benchmark is the name of the benchmark test.
	Benchmark string

	// Chart is a story histogram in a Benchmark.
	Chart string

	// Story is a story in a Benchmark.
	Story string

	// StoryTags is for the story in a Benchmark.
	StoryTags string

	// AggregationMethod is method to aggregate sampled values.
	// If empty, then the original values are returned.
	AggregationMethod string

	// The commit with optional deps override
	// Note: This field is only used in bisect
	// TODO(b/326352320): move CombinedCommit to a common package or rename midpoint
	CombinedCommit *common.CombinedCommit

	// The number of benchmark tests to run.
	// Note the collected sampled values can be more than iterations as each iteration produce
	// more than one samples.
	Iterations int32

	// Finished number of iterations of benchmark test
	// In the bisect jobs, one commit run will start with an initial number of iteration
	// If the compare result is not significant enough, then extra number of iteration will be added to that commit
	// We need this attribute to record the finished number of iterations
	// Note: This field is only used in bisect
	FinishedIteration int32

	// Available bot list
	BotIds []string
}

SingleCommitRunnerParams defines the parameters for SingleCommitRunner workflow.

type TestAndExportParams 

type TestAndExportParams struct {
	// WorkflowID is the ID of the Temporal Workflow.
	WorkflowID string
	// Benchmark is the benchmark name, ie/ speedometer3.
	Benchmark string
	// Bot, also referred to as bot_configuration, is usually a bot name.
	Bot string
	// Git Hash associated with chromium/src to build.
	GitHash string
	// Iterations is the number of runs to trigger.
	// Note: Temporal will struggle and crash with more than 10,000 steps. Each
	// activity generates 3, so be cautious when running more than 1,000 iterations.
	// If you need to run more than 1,000, you can run several of these Test
	// AndExportWorkflows.
	Iterations int
	// Project is the Google Cloud Project (ie/ chromeperf), used for creating the
	// fully qualified BQ table name.
	Project string
	// Dataset is the BQ Dataset. Format is {project}.{dataset}.
	// Used for creating the fully qualified BQ table name.
	Dataset string
	// TableName is the name of the table within Dataset. Format is
	// {project}.{dataset}.{tableName}. Used for creating the fully qualified BQ
	// table name.
	TableName string
}

TestAndExportParams contains information needed to build a version of Chrome and trigger the tests on Swarming.

type TestResult 

type TestResult struct {
	// WorkflowID is the ID of the Temporal Workflow.
	WorkflowID string `bigquery:"workflow_id"`
	// Bot, also referred to as bot_configuration, is usually a bot name.
	// See bot_configs/internal.json or external.json for the fully supported list.
	Bot string `bigquery:"bot"`
	// SwarmingTaskID is the ID of the swarming task associated with the test run.
	SwarmingTaskID string `bigquery:"swarming_task_id"`
	// Benchmark is the benchmark name, ie/ Speedometer3.
	Benchmark string `bigquery:"benchmark"`
	// Story is the story associated with the benchmark. Not all benchmarks have
	// stories (ie/ Speedometer3)
	Story string `bigquery:"story"`
	// Chart is the sub module to the benchmark.
	Chart string `bigquery:"chart"`
	// SampleValues is the list of values generated by the benchmark run.
	SampleValues []float64 `bigquery:"sample_values"`
	// TaskFailed indicates whether the benchmark run failed.
	TaskFailed bool `bigquery:"task_failed"`
	// PGOEnabled indicates whether PGO was enabled for the Chrome build.
	PGOEnabled bool `bigquery:"pgo_enabled"`
	// CreateTime is when the row data is generated.
	CreateTime time.Time `bigquery:"create_time"`
}

TestResult represents the columns used by the BQ table.

func FetchAllSwarmingTasksActivity 

func FetchAllSwarmingTasksActivity(ctx context.Context, project, dataset, tableName, workflowId string) ([]*TestResult, error)

FetchAllSwarmingTasksActivity retrieves all Swarming task IDs for the workflow ID. It's expected that RunTestAndExportWorkflow has been run prior, where a Temporal workflow has triggered Swarming task IDs and has uploaded this information to BQ.

func GetAllSampleValuesActivity 

func GetAllSampleValuesActivity(ctx context.Context, benchmark string, task *TestResult) ([]*TestResult, error)

GetAllSampleValuesActivity goes through all Swarming tasks, finds the test result artifacts, and reads the values. The sample values are broken down by chart, so for one Swarming task (one benchmark execution), there will be multiple TestResult objects.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.