runs

Documentation

Index

• Variables
• func Create(ctx context.Context, r *ReclusteringRun) error
• func SetRunsForTesting(ctx context.Context, t testing.TB, rs []*ReclusteringRun) error
• func UpdateProgress(ctx context.Context, projectID string, attemptTimestamp time.Time, ...) error
• type ReclusteringProgress
  • func ReadReclusteringProgress(ctx context.Context, project string) (*ReclusteringProgress, error)
  • func ReadReclusteringProgressUpTo(ctx context.Context, project string, upToAttemptTimestamp time.Time) (*ReclusteringProgress, error)
  • func (p *ReclusteringProgress) IncorporatesRulesVersion(rulesVersion time.Time) bool
  • func (p *ReclusteringProgress) IsReclusteringToNewAlgorithms() bool
  • func (p *ReclusteringProgress) IsReclusteringToNewConfig() bool
• type ReclusteringRun
  • func Read(ctx context.Context, projectID string, attemptTimestamp time.Time) (*ReclusteringRun, error)
  • func ReadLastCompleteUpTo(ctx context.Context, projectID string, upToAttemptTimestamp time.Time) (*ReclusteringRun, error)
  • func ReadLastUpTo(ctx context.Context, projectID string, upToAttemptTimestamp time.Time) (*ReclusteringRun, error)
  • func ReadLastWithProgressUpTo(ctx context.Context, projectID string, upToAttemptTimestamp time.Time) (*ReclusteringRun, error)
• type ReclusteringTarget
• type RunBuilder
  • func NewRun(uniqifier int) *RunBuilder
  • func (b *RunBuilder) Build() *ReclusteringRun
  • func (b *RunBuilder) WithAlgorithmsVersion(value int64) *RunBuilder
  • func (b *RunBuilder) WithAttemptTimestamp(attemptTimestamp time.Time) *RunBuilder
  • func (b *RunBuilder) WithCompletedProgress() *RunBuilder
  • func (b *RunBuilder) WithConfigVersion(value time.Time) *RunBuilder
  • func (b *RunBuilder) WithNoReportedProgress() *RunBuilder
  • func (b *RunBuilder) WithProject(project string) *RunBuilder
  • func (b *RunBuilder) WithReportedProgress(progress int) *RunBuilder
  • func (b *RunBuilder) WithRulesVersion(value time.Time) *RunBuilder
  • func (b *RunBuilder) WithShardCount(count int64) *RunBuilder

Variables

var MaxAttemptTimestamp = time.Date(9999, 12, 31, 23, 59, 0, 0, time.UTC)

MaxAttemptTimestamp can be passed to any Read....() method to return data up to the last attempt.

var NotFound = errors.New("reclustering run row not found")

NotFound is the error returned by Read if the row could not be found.

var StartingEpoch = shards.StartingEpoch

StartingEpoch is the earliest valid run attempt time.

Functions

func Create

func Create(ctx context.Context, r *ReclusteringRun) error

Create inserts a new reclustering run.

func SetRunsForTesting

func SetRunsForTesting(ctx context.Context, t testing.TB, rs []*ReclusteringRun) error

SetRunsForTesting replaces the set of stored runs to match the given set.

func UpdateProgress

func UpdateProgress(ctx context.Context, projectID string, attemptTimestamp time.Time, shardsReported, progress int64) error

UpdateProgress sets the progress of a particular run.

Types

type ReclusteringProgress

type ReclusteringProgress struct {
	// ProgressPerMille is the progress of the current re-clustering run,
	// measured in thousandths (per mille).
	ProgressPerMille int `json:"progressPerMille"`
	// Next is the goal of the current re-clustering run. (For which
	// ProgressPerMille is specified.)
	Next ReclusteringTarget `json:"next"`
	// Last is the goal of the last completed re-clustering run.
	Last ReclusteringTarget `json:"last"`
}

ReclusteringProgress captures the progress re-clustering a given LUCI project's test results using specific rules versions or algorithms versions.

func ReadReclusteringProgress

func ReadReclusteringProgress(ctx context.Context, project string) (*ReclusteringProgress, error)

ReadReclusteringProgress reads the re-clustering progress for the given LUCI project.

func ReadReclusteringProgressUpTo

func ReadReclusteringProgressUpTo(ctx context.Context, project string, upToAttemptTimestamp time.Time) (*ReclusteringProgress, error)

ReadReclusteringProgressUpTo reads the re-clustering progress for the given LUCI project up to the given reclustering attempt timestamp.

For the latest re-clustering progess, pass MaxAttemptTimestamp. For re-clustering as it was at a given attempt minute in the past, pass the timestamp for that minute. Reading past reclustering progress is useful for understanding the versions of rules and algorithms reflected in outputs of BigQuery jobs which ran using data some minutes old.

When reading historic progress, data for reclustering progress as it was part way through a minute is not available; this is also why the passed timestamp is called 'upToAttemptTimestamp' not an asAtTime.

func (*ReclusteringProgress) IncorporatesRulesVersion

func (p *ReclusteringProgress) IncorporatesRulesVersion(rulesVersion time.Time) bool

IncorporatesRulesVersion returns returns whether LUCI Analysis clustering output incorporates all rule changes up to the given predicate last updated time. Later changes may also be included, in full or in part.

func (*ReclusteringProgress) IsReclusteringToNewAlgorithms

func (p *ReclusteringProgress) IsReclusteringToNewAlgorithms() bool

IsReclusteringToNewAlgorithms returns whether LUCI Analysis's clustering output is being updated to use a newer standard of algorithms and is not yet stable. The algorithms version LUCI Analysis is re-clustering to is accessible via LatestAlgorithmsVersion.

func (*ReclusteringProgress) IsReclusteringToNewConfig

func (p *ReclusteringProgress) IsReclusteringToNewConfig() bool

IsReclusteringToNewConfig returns whether LUCI Analysis's clustering output is in the process of being updated to a later configuration standard and is not yet stable. The configuration version LUCI Analysis is re-clustering to is accessible via LatestConfigVersion. Clients using re-clustering output should verify they are using the configuration version defined by LatestConfigVersion when interpreting the output.

type ReclusteringRun

type ReclusteringRun struct {
	// The LUCI Project for which this rule is defined.
	Project string
	// The attempt. This is the time the orchestrator run ends.
	AttemptTimestamp time.Time
	// The minimum algorithms version this reclustering run is trying
	// to achieve. Chunks with an AlgorithmsVersion less than this
	// value are eligible to be re-clustered.
	AlgorithmsVersion int64
	// The minimum config version the reclustering run is trying to achieve.
	// Chunks with a ConfigVersion less than this value are eligible to be
	// re-clustered.
	ConfigVersion time.Time
	// The minimum rules version the reclustering run is trying to achieve.
	// Chunks with a RulesVersion less than this value are eligible to be
	// re-clustered.
	RulesVersion time.Time
	// The number of shards created for this run (for this LUCI project).
	ShardCount int64
	// The number of shards that have reported progress (at least once).
	// When this is equal to ShardCount, readers can have confidence Progress
	// is a reasonable reflection of the progress made reclustering
	// this project. Until then, it is a loose lower-bound.
	ShardsReported int64
	// The progress. This is a value between 0 and 1000*ShardCount.
	Progress int64
}

ReclusteringRun contains the details of a runs used to re-cluster test results.

func Read

func Read(ctx context.Context, projectID string, attemptTimestamp time.Time) (*ReclusteringRun, error)

Read reads the run with the given attempt timestamp in the given LUCI project. If the row does not exist, the error NotFound is returned.

func ReadLastCompleteUpTo

func ReadLastCompleteUpTo(ctx context.Context, projectID string, upToAttemptTimestamp time.Time) (*ReclusteringRun, error)

ReadLastCompleteUpTo reads the last run that completed in the given LUCI project up to the given attempt timestamp. If no row exists, a fake row is returned; see ReadLast for details.

func ReadLastUpTo

func ReadLastUpTo(ctx context.Context, projectID string, upToAttemptTimestamp time.Time) (*ReclusteringRun, error)

ReadLastUpTo reads the last run in the given LUCI project up to the given attempt timestamp. If no row exists, a fake run is returned with the following details: - Project matching the requested Project ID. - AttemptTimestamp of StartingEpoch. - AlgorithmsVersion of 1. - ConfigVersion of clusteringcfg.StartingEpoch. - RulesVersion of rules.StartingEpoch. - ShardCount and ShardsReported of 1. - Progress of 1000.

func ReadLastWithProgressUpTo

func ReadLastWithProgressUpTo(ctx context.Context, projectID string, upToAttemptTimestamp time.Time) (*ReclusteringRun, error)

ReadLastWithProgress reads the last run with progress in the given LUCI project up to the given attempt timestamp.

If no row exists, a fake row is returned; see ReadLast for details.

type ReclusteringTarget

type ReclusteringTarget struct {
	// RulesVersion is the rules version the re-clustering run is attempting
	// to achieve.
	RulesVersion time.Time `json:"rulesVersion"`
	// ConfigVersion is the config version the re-clustering run is attempting
	// to achieve.
	ConfigVersion time.Time `json:"configVersion"`
	// AlgorithmsVersion is the algorithms version the re-clustering run is
	// attempting to achieve.
	AlgorithmsVersion int64 `json:"algorithmsVersion"`
}

ReclusteringTarget captures the rules and algorithms a re-clustering run is re-clustering to.

type RunBuilder

type RunBuilder struct {
	// contains filtered or unexported fields
}

RunBuilder provides methods to build a reclustering run for testing.

func NewRun

func NewRun(uniqifier int) *RunBuilder

NewRun starts building a new Run, for testing.

func (*RunBuilder) Build

func (b *RunBuilder) Build() *ReclusteringRun

func (*RunBuilder) WithAlgorithmsVersion

func (b *RunBuilder) WithAlgorithmsVersion(value int64) *RunBuilder

WithAlgorithmsVersion specifies the algorithms version to use on the run.

func (*RunBuilder) WithAttemptTimestamp

func (b *RunBuilder) WithAttemptTimestamp(attemptTimestamp time.Time) *RunBuilder

WithAttemptTimestamp specifies the attempt timestamp to use on the run.

func (*RunBuilder) WithCompletedProgress

func (b *RunBuilder) WithCompletedProgress() *RunBuilder

WithCompletedProgress sets that all shards have reported, and the run has completed.

func (*RunBuilder) WithConfigVersion

func (b *RunBuilder) WithConfigVersion(value time.Time) *RunBuilder

WithRulesVersion specifies the config version to use on the run.

func (*RunBuilder) WithNoReportedProgress

func (b *RunBuilder) WithNoReportedProgress() *RunBuilder

WithNoProgress sets that no shards reported and no progress has been made.

func (*RunBuilder) WithProject

func (b *RunBuilder) WithProject(project string) *RunBuilder

WithProject specifies the project to use on the run.

func (*RunBuilder) WithReportedProgress

func (b *RunBuilder) WithReportedProgress(progress int) *RunBuilder

WithReportedProgress sets that all shards have reported, and some progress has been made. (progress is a value out of 1000 that is scaled to the number of shards).

func (*RunBuilder) WithRulesVersion

func (b *RunBuilder) WithRulesVersion(value time.Time) *RunBuilder

WithRulesVersion specifies the rules version to use on the run.

func (*RunBuilder) WithShardCount

func (b *RunBuilder) WithShardCount(count int64) *RunBuilder

WithShardCount specifies the number of shards to use on the run.