aggregator

Documentation

Index

• Constants
• Variables
• type Aggregator
  • func StartAggregator(ctx context.Context, s *storage.Client, im *issues.IssuesManager, ...) (*Aggregator, error)
  • func (agg *Aggregator) ClearUploadedFuzzNames()
  • func (agg *Aggregator) ForceAnalysis(path, category string)
  • func (agg *Aggregator) RestartAnalysis(ctx context.Context) error
  • func (agg *Aggregator) ShutDown()
  • func (agg *Aggregator) UploadedFuzzNames() (bad, grey, duplicate []string)
  • func (agg *Aggregator) WaitForEmptyQueues()
• type SortedStringSlice
  • func (s *SortedStringSlice) Append(strs []string)
  • func (s *SortedStringSlice) Contains(str string) bool

Constants

const (
	BAD_FUZZ               = "bad"
	GREY_FUZZ              = "grey"
	EMPTY_THRESHOLD        = 5
	RETRIES_FOR_BAD_FUZZES = 3
)

Variables

var (
	CLANG_DEBUG   = common.TEST_HARNESS_NAME + "_clang_debug"
	CLANG_RELEASE = common.TEST_HARNESS_NAME + "_clang_release"
	ASAN_DEBUG    = common.TEST_HARNESS_NAME + "_asan_debug"
	ASAN_RELEASE  = common.TEST_HARNESS_NAME + "_asan_release"

	ANALYSIS_EXECUTABLE_LIST = []string{ASAN_RELEASE, ASAN_DEBUG, CLANG_RELEASE, CLANG_DEBUG}
)

Types

type Aggregator

type Aggregator struct {
	// If we are watching for regressions, all fuzzes passed in should be "grey".  If they are not
	// don't deduplicate them.
	WatchForRegressions bool
	// Should be set to true if a bug should be created for every bad fuzz found.
	// Example: detecting a bad fuzz in a "stable" fuzzer
	// TODO(kjlubick): consider making this a function that clients supply so they can decide.
	MakeBugOnBadFuzz bool
	// Should be set if we want to upload grey fuzzes.  This should only be true
	// if we are changing versions.
	UploadGreyFuzzes bool
	// contains filtered or unexported fields
}

Aggregator is a key part of the fuzzing operation (see https://skia.googlesource.com/buildbot/+/master/fuzzer/DESIGN.md). It will find new bad fuzzes generated by afl-fuzz and create the metadata required for them. It does this by searching in the specified AflOutputPath for new crashes and moves them to a temporary holding folder (specified by FuzzPath) for parsing, before sending them through the "aggregation pipeline". This pipeline has three steps, Analysis, Upload and Bug Reporting. Analysis runs the fuzz against a debug and release version of Skia which produces stacktraces and error output. Upload uploads these pieces to Google Storage (GCS). Bug Reporting is used to either create or update a bug related to the given fuzz.

func StartAggregator

func StartAggregator(ctx context.Context, s *storage.Client, im *issues.IssuesManager, startingReports map[string]<-chan data.FuzzReport) (*Aggregator, error)

StartAggregator creates and starts a Aggregator based on the Skia Revision in config.Common.SkiaVersion.Hash. If there is a problem starting up, an error is returned. Other errors will be logged.

func (*Aggregator) ClearUploadedFuzzNames

func (agg *Aggregator) ClearUploadedFuzzNames()

func (*Aggregator) ForceAnalysis

func (agg *Aggregator) ForceAnalysis(path, category string)

ForceAnalysis directly adds the given path to the analysis queue, where it will be analyzed, uploaded and possibly bug reported.

func (*Aggregator) RestartAnalysis

func (agg *Aggregator) RestartAnalysis(ctx context.Context) error

RestartAnalysis restarts the shut down aggregator, assuming that config.Common.SkiaVersion.Hash is updated to the desired revision of Skia that should be analyzied. Anything that is in the scanning directory should be cleared out, lest it be rescanned/analyzed.

func (*Aggregator) ShutDown

func (agg *Aggregator) ShutDown()

Shutdown gracefully shuts down the aggregator. Anything that was being processed will finish prior to the shutdown.

func (*Aggregator) UploadedFuzzNames

func (agg *Aggregator) UploadedFuzzNames() (bad, grey, duplicate []string)

func (*Aggregator) WaitForEmptyQueues

func (agg *Aggregator) WaitForEmptyQueues()

WaitForEmptyQueues will return once there is nothing more in the analysis-upload-report pipeline, waiting in increments of config.Aggregator.StatusPeriod until it is done.

type SortedStringSlice

type SortedStringSlice struct {
	// contains filtered or unexported fields
}

A SortedStringSlice has a sortable string slice which is always kept sorted. This allows for an implementation of Contains that runs in O(log n)

func (*SortedStringSlice) Append

func (s *SortedStringSlice) Append(strs []string)

Append adds all of the strings to the underlying slice and sorts it

func (*SortedStringSlice) Contains

func (s *SortedStringSlice) Contains(str string) bool

Contains returns true if the passed in string is in the underlying slice