common

Documentation

Overview

Package common has helper / utility functions used by all levels of flaky test monitor.

Index

• func AssertNoError(err error, panicMessage string)
• func AssertTestSummariesEqual(t *testing.T, expectedTestSummary, actualTestSummary TestResultSummary)
• func ConvertToNDecimalPlaces(n int, numerator, denominator int) float32
• func ConvertToNDecimalPlaces2(n int, numerator float32, denominator int) float32
• func DirExists(path string) bool
• func IsDirEmpty(name string) bool
• func SaveToFile(fileName string, testSummary interface{})
• type Config
  • func ReadProperties(directory string) Config
• type PackageResult
• type RawTestStep
• type TestResult
• type TestResultSummary
• type TestRun
• type TestSummary2
• type TestSummary3

Functions

func AssertNoError

func AssertNoError(err error, panicMessage string)

AssertNoError checks that the passed in error is nil and panics with the supplied message if that's not the case. Useful helper to eliminate the need to keep checking for errors.

func AssertTestSummariesEqual

func AssertTestSummariesEqual(t *testing.T, expectedTestSummary, actualTestSummary TestResultSummary)

AssertTestSummariesEqual checks that 2 TestResultSummary structs are equal, doing a deep comparison. This is needed for multiple test files (level 2 and 3) so it's extracted as a helper function here.

func ConvertToNDecimalPlaces

func ConvertToNDecimalPlaces(n int, numerator, denominator int) float32

ConvertToNDecimalPlaces converts the supplied numerator and denominator fraction into a decimal with n decimal places.

func ConvertToNDecimalPlaces2

func ConvertToNDecimalPlaces2(n int, numerator float32, denominator int) float32

ConvertToNDecimalPlaces2 converts the supplied numerator and denominator fraction into a decimal with n decimal places. Works the same way as ConvertToNDecimalPlaces() but has a float for the numerator.

func DirExists

func DirExists(path string) bool

DirExists checks if directory exists and return true if it does, false otherwise.

func IsDirEmpty

func IsDirEmpty(name string) bool

IsDirEmpty checks if directory is empty (has no files) and return true if it's empty, false otherwise. Useful for determining whether to delete the failures / no-results directories for cases when there were no failures / no-results. From https://stackoverflow.com/a/30708914/5719544.

func SaveToFile

func SaveToFile(fileName string, testSummary interface{})

SaveToFile save test run/summary to local JSON file.

Types

type Config

type Config struct {
	FailureThresholdPercent float32 `json:"failures_threshold_percent"`
	FailuresSliceMax        int     `json:"failures_slice_max"`

	DurationThresholdSeconds float32 `json:"duration_threshold_seconds"`
	DurationSliceMax         int     `json:"duration_slice_max"`
}

func ReadProperties

func ReadProperties(directory string) Config

type PackageResult

type PackageResult struct {
	Package string                  `json:"package"`
	Result  string                  `json:"result"`
	Elapsed float32                 `json:"elapsed"`
	Output  []string                `json:"output"`
	Tests   []TestResult            `json:"tests"`
	TestMap map[string][]TestResult `json:"-"`
}

PackageResult models test result of an entire package which can have multiple tests

type RawTestStep

type RawTestStep struct {
	Time    time.Time `json:"Time"`
	Action  string    `json:"Action"`
	Package string    `json:"Package"`
	Test    string    `json:"Test"`
	Output  string    `json:"Output"`
	Elapsed float32   `json:"Elapsed"`
}

RawTestStep models single line from "go test -json" output.

type TestResult

type TestResult struct {
	Test    string   `json:"test"`
	Package string   `json:"package"`
	Output  []string `json:"output"`
	Result  string   `json:"result"`
	Elapsed float32  `json:"elapsed"`
}

TestResult models result of a single test that's part of a larger package result

type TestResultSummary

type TestResultSummary struct {
	Test            string    `json:"test"`
	Package         string    `json:"package"`
	Runs            int       `json:"runs"`
	Passed          int       `json:"passed"`
	Failed          int       `json:"failed"`
	Skipped         int       `json:"skipped"`
	NoResult        int       `json:"no_result"`
	FailureRate     float32   `json:"failure_rate"`
	AverageDuration float32   `json:"average_duration"`
	Durations       []float32 `json:"durations"`
}

TestResultSummary models all results from a specific test over many (level 1) test runs

type TestRun

type TestRun struct {
	CommitSha      string          `json:"commit_sha"`
	CommitDate     time.Time       `json:"commit_date"`
	JobRunDate     time.Time       `json:"job_run_date"`
	PackageResults []PackageResult `json:"results"`
}

TestRun models full level 1 summary of a test run from "go test -json".

type TestSummary2

type TestSummary2 struct {
	TestResults map[string]*TestResultSummary `json:"tests"`
}

TestSummary2 models full level 2 summary of a test run from 1 or more level 1 test runs

type TestSummary3

type TestSummary3 struct {
	NoResults []TestResultSummary `json:"no_results"`

	// ordered list of tests (from level 2 summary) that:
	// a) met minimum failure threshold as specified in property file property `failures_threshold_percent`
	// b) is up to maximum slice size as specified in `failures_slice_max` in property file
	MostFailures []TestResultSummary `json:"most_failures"`

	// total # of tests (from level 2 summary) that:
	// a) met minimum failure threshold as specified in property file property `failures_threshold_percent`
	MostFailuresTotal int `json:"most_failures_total"`

	// ordered list of tests (from level 2 summary) that:
	// a) met minimum duration threshold as specified in property file property `duration_threshold_seconds`
	// b) is up to maximum slice size as specified in `duration_slice_max` in property file
	LongestRunning []TestResultSummary `json:"longest_running"`

	// total # of tests (from level 2 summary) that:
	// a) met minimum duration threshold as specified in property file property `duration_threshold_seconds`
	LongestRunningTotal int `json:"longest_running_total"`
}

models full level 3 summary of a test run from 1 (single) level 2 test run