web

Documentation

Index

• Constants
• type ClusterDiffRequest
• type Handlers
  • func NewHandlers(conf HandlersConfig, val validateFields, alogin alogin.Login) (*Handlers, error)
  • func (wh *Handlers) AddIgnoreRule(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) BaselineHandlerV2(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ByBlameHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ChangelistSearchRedirect(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ChangelistSummaryHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ChangelistsHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ClusterDiffHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) CommitsHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) DeleteIgnoreRule(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) DetailsHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) DiffHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) DigestListHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) GroupingsHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ImageHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) KnownHashesHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) LatestPositiveDigestHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ListIgnoreRules2(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ListTestsHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) ParamsHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) PatchsetsAndTryjobsForCL2(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) PositiveDigestsByGroupingIDHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) SearchHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) StartCacheWarming(ctx context.Context)
  • func (wh *Handlers) StartKnownHashesCacheProcess(ctx context.Context)
  • func (wh *Handlers) StatusHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) TriageHandlerV2(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) TriageHandlerV3(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) TriageLogHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) TriageUndoHandler(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) UpdateIgnoreRule(w http.ResponseWriter, r *http.Request)
  • func (wh *Handlers) Whoami(w http.ResponseWriter, r *http.Request)
• type HandlersConfig

Constants

const (
	// FullFrontEnd means all fields should be set
	FullFrontEnd validateFields = iota
	// BaselineSubset means just the fields needed for BaselineV2Response Server should be set.
	BaselineSubset
)

const (

	// RPCCallCounterMetric is the metric that should be used when counting how many times a given
	// RPC route is called from clients.
	RPCCallCounterMetric = "gold_rpc_call_counter"
)

Types

type ClusterDiffRequest

type ClusterDiffRequest struct {
	Corpus                  string
	Filters                 paramtools.ParamSet
	IncludePositiveDigests  bool
	IncludeNegativeDigests  bool
	IncludeUntriagedDigests bool
	// TODO(kjlubick) the frontend does not yet support these yet.
	ChangelistID       string
	CodeReviewSystemID string
	PatchsetID         string
}

ClusterDiffRequest contains the options that the frontend provides to the clusterdiff RPC.

type Handlers

type Handlers struct {
	HandlersConfig
	// contains filtered or unexported fields
}

Handlers represents all the handlers (e.g. JSON endpoints) of Gold. It should be created by clients using NewHandlers.

func NewHandlers

func NewHandlers(conf HandlersConfig, val validateFields, alogin alogin.Login) (*Handlers, error)

NewHandlers returns a new instance of Handlers.

func (*Handlers) AddIgnoreRule

func (wh *Handlers) AddIgnoreRule(w http.ResponseWriter, r *http.Request)

AddIgnoreRule is for adding a new ignore rule.

func (*Handlers) BaselineHandlerV2

func (wh *Handlers) BaselineHandlerV2(w http.ResponseWriter, r *http.Request)

BaselineHandlerV2 returns a JSON representation of that baseline including baselines for a options issue. It can respond to requests like these:

/json/expectations
/json/expectations?issue=123456

The "issue" parameter indicates the changelist ID for which we would like to retrieve the baseline. In that case the returned options will be a blend of the master baseline and the baseline defined for the changelist (usually based on tryjob results).

func (*Handlers) ByBlameHandler

func (wh *Handlers) ByBlameHandler(w http.ResponseWriter, r *http.Request)

ByBlameHandler takes the response from the SQL backend's GetBlamesForUntriagedDigests and converts it into the same format that the legacy version (v1) produced.

func (*Handlers) ChangelistSearchRedirect

func (wh *Handlers) ChangelistSearchRedirect(w http.ResponseWriter, r *http.Request)

ChangelistSearchRedirect redirects the user to a search page showing the search results for a given CL. It will do a (hopefully) quick scan of the untriaged digests - if it finds some, it will include the corpus containing some of those untriaged digests in the search query so the user will see results (instead of getting directed to a corpus with no results).

func (*Handlers) ChangelistSummaryHandler

func (wh *Handlers) ChangelistSummaryHandler(w http.ResponseWriter, r *http.Request)

ChangelistSummaryHandler returns a summary of the new and untriaged digests produced by this CL across all Patchsets.

func (*Handlers) ChangelistsHandler

func (wh *Handlers) ChangelistsHandler(w http.ResponseWriter, r *http.Request)

ChangelistsHandler returns the list of code_review.Changelists that have uploaded results to Gold (via TryJobs).

func (*Handlers) ClusterDiffHandler

func (wh *Handlers) ClusterDiffHandler(w http.ResponseWriter, r *http.Request)

ClusterDiffHandler computes the diffs between all digests that match the filters and returns them in a way that is convenient for rendering via d3.js

func (*Handlers) CommitsHandler

func (wh *Handlers) CommitsHandler(w http.ResponseWriter, r *http.Request)

CommitsHandler returns the last n commits with data that make up the sliding window.

func (*Handlers) DeleteIgnoreRule

func (wh *Handlers) DeleteIgnoreRule(w http.ResponseWriter, r *http.Request)

DeleteIgnoreRule deletes an existing ignores rule.

func (*Handlers) DetailsHandler

func (wh *Handlers) DetailsHandler(w http.ResponseWriter, r *http.Request)

DetailsHandler returns the details about a single digest.

func (*Handlers) DiffHandler

func (wh *Handlers) DiffHandler(w http.ResponseWriter, r *http.Request)

DiffHandler compares two digests and returns that information along with triage data.

func (*Handlers) DigestListHandler

func (wh *Handlers) DigestListHandler(w http.ResponseWriter, r *http.Request)

DigestListHandler returns a list of digests for a given test. This is used by goldctl's local diff tech.

func (*Handlers) GroupingsHandler

func (wh *Handlers) GroupingsHandler(w http.ResponseWriter, r *http.Request)

GroupingsHandler returns a map from corpus name to the list of keys that comprise the corpus grouping.

This method returns the union between the following two sets of corpus/grouping pairs:

• Those defined in the Gold instance's JSON5 configuration.
• A set constructed by getting a list of corpora from the status cache, and by assigning them the default (source_type, name) grouping (that is, corpus name and test name).

If a corpus appears on both sets, the grouping from the JSON5 configuration takes precedence. This gives us flexibility in case we want to support groupings other than (source_type, name) in the future.

For large Gold instances (e.g. Skia, Chrome) it is important to provide a dictionary of grouping param keys by corpus in its JSON5 config because:

• The status cache is periodically populated by a goroutine that runs a slow SQL query (~13 minutes in the case of the Skia instance).
• Upon launching an instance, the status cache remains empty for several minutes until said goroutine finishes running the aforementioned slow SQL query for the first time.
• During that time, this RPC (/json/v1/groupings) returns an empty dictionary if the JSON5 config does not include a dictionary of grouping param keys by corpus.
• The "goldctl imgtest add" command hits this RPC to validate that the test being added includes all the params required by its corpus' grouping.
• If the RPC returns an empty map, goldctl reports "grouping params for corpus X are unknown", which causes spurious test failures in the associated CI system.

Some possible alternatives:

• Write a fast SQL query specifically for /json/v1/groupings, but that's probably hard with the current schema. It might require factoring the corpora out into their own table.
• Delay starting the webserver until the status cache is populated, but that would be at the expense of a much longer startup time for large instances.

func (*Handlers) ImageHandler

func (wh *Handlers) ImageHandler(w http.ResponseWriter, r *http.Request)

ImageHandler returns either a single image or a diff between two images identified by their respective digests.

func (*Handlers) KnownHashesHandler

func (wh *Handlers) KnownHashesHandler(w http.ResponseWriter, r *http.Request)

KnownHashesHandler returns known hashes that have been written to GCS in the background Each line contains a single digest for an image. Bots will then only upload images which have a hash not found on this list, avoiding significant amounts of unnecessary uploads.

func (*Handlers) LatestPositiveDigestHandler

func (wh *Handlers) LatestPositiveDigestHandler(w http.ResponseWriter, r *http.Request)

LatestPositiveDigestHandler returns the most recent positive digest for the given trace. Starting at the tip of tree, it will skip over any missing data, untriaged digests or digests triaged negative until it finds a positive digest.

func (*Handlers) ListIgnoreRules2

func (wh *Handlers) ListIgnoreRules2(w http.ResponseWriter, r *http.Request)

ListIgnoreRules2 returns the current ignore rules in JSON format and the counts of how many traces they affect.

func (*Handlers) ListTestsHandler

func (wh *Handlers) ListTestsHandler(w http.ResponseWriter, r *http.Request)

ListTestsHandler returns all the tests in the given corpus and a count of how many digests have been seen for that.

func (*Handlers) ParamsHandler

func (wh *Handlers) ParamsHandler(w http.ResponseWriter, r *http.Request)

ParamsHandler returns all Params that could be searched over. It uses the SQL Backend and returns *only* the keys, not the options.

func (*Handlers) PatchsetsAndTryjobsForCL2

func (wh *Handlers) PatchsetsAndTryjobsForCL2(w http.ResponseWriter, r *http.Request)

PatchsetsAndTryjobsForCL2 returns a summary of the data we have collected for a given Changelist, specifically any TryJobs that have uploaded data to Gold belonging to various patchsets in it.

func (*Handlers) PositiveDigestsByGroupingIDHandler

func (wh *Handlers) PositiveDigestsByGroupingIDHandler(w http.ResponseWriter, r *http.Request)

PositiveDigestsByGroupingIDHandler returns all positively triaged digests seen in the sliding window for a given grouping, split up by trace. Used by https://source.chromium.org/chromium/chromium/src/+/main:content/test/gpu/gold_inexact_matching/base_parameter_optimizer.py

func (*Handlers) SearchHandler

func (wh *Handlers) SearchHandler(w http.ResponseWriter, r *http.Request)

SearchHandler searches the data in the new SQL backend. It times out after 3 minutes, to prevent outstanding requests from growing unbounded.

func (*Handlers) StartCacheWarming

func (wh *Handlers) StartCacheWarming(ctx context.Context)

StartCacheWarming starts warming the caches for data we want to serve quickly. It starts goroutines that will run in the background (until the provided context is cancelled).

func (*Handlers) StartKnownHashesCacheProcess

func (wh *Handlers) StartKnownHashesCacheProcess(ctx context.Context)

StartKnownHashesCacheProcess will fetch the known hashes on a timer and save it to the cache.

func (*Handlers) StatusHandler

func (wh *Handlers) StatusHandler(w http.ResponseWriter, r *http.Request)

StatusHandler returns information about the most recently ingested data and the triage status of the various corpora.

func (*Handlers) TriageHandlerV2

func (wh *Handlers) TriageHandlerV2(w http.ResponseWriter, r *http.Request)

TriageHandlerV2 handles a request to change the triage status of one or more digests of one test.

It accepts a POST'd JSON serialization of TriageRequest and updates the expectations. TODO(kjlubick) In V3, this should take groupings, not test names. Additionally, to avoid race

conditions where users triage the same thing at the same time, the request should include
before and after. Finally, to avoid confusion on CLs, we should fail to apply changes
on closed CLs (skbug.com/12122)

func (*Handlers) TriageHandlerV3

func (wh *Handlers) TriageHandlerV3(w http.ResponseWriter, r *http.Request)

TriageHandlerV3 handles a request to change the triage status of one or more digests.

func (*Handlers) TriageLogHandler

func (wh *Handlers) TriageLogHandler(w http.ResponseWriter, r *http.Request)

TriageLogHandler returns what has been triaged recently.

func (*Handlers) TriageUndoHandler

func (wh *Handlers) TriageUndoHandler(w http.ResponseWriter, r *http.Request)

TriageUndoHandler performs an "undo" for a given id. This id corresponds to the record id of the set of changes in the DB. If successful it returns the same result as a call to TriageLogHandler to reflect the changes.

func (*Handlers) UpdateIgnoreRule

func (wh *Handlers) UpdateIgnoreRule(w http.ResponseWriter, r *http.Request)

UpdateIgnoreRule updates an existing ignores rule.

func (*Handlers) Whoami

func (wh *Handlers) Whoami(w http.ResponseWriter, r *http.Request)

Whoami returns the email address of the user or service account used to authenticate the request. For debugging purposes only.

type HandlersConfig

type HandlersConfig struct {
	DB                        *pgxpool.Pool
	GCSClient                 storage.GCSClient
	IgnoreStore               ignore.Store
	ReviewSystems             []clstore.ReviewSystem
	Search2API                search.API
	WindowSize                int
	GroupingParamKeysByCorpus map[string][]string
}

HandlersConfig holds the environment needed by the various http handler functions.