README
¶
Suspicion
Suspicion is a property-based testing library for Go.
Property testing?
Property-based testing uses random generation of data to find edge cases that violate some property.
An example is a simple sort function
func sortFloats(f []float64) {
sort.Slice(f, func(i, j int) bool {
return f[i] < f[j]
})
}
After this function has been called and our implementation is correct, you can assume the property f[i] <= f[i+1]
With Suspicion, you can write a short test that verifies this property
func TestSort(t *testing.T) {
s := suss.NewTest(t)
s.Run(func() {
var f []float64
s.Slice().Gen(func() {
n := s.Float64()
f = append(f, n)
})
sortFloats(f)
for i := 0; i < len(f)-1; i++ {
if f[i] > f[i+1] {
s.Fatalf("invalid sort len=%v, %v\n", len(f), f)
}
}
})
Suspicion not only finds a counter example, but also tries to find a minimal example.
invalid sort len=13, [NaN 0 0 0 0 4.394777251413336e+230 0 0 0 0 0 0 0]
--- FAIL: TestSort (0.02s)
In this case, we find that a 13 length slice with a NaN and a non-zero value results in a invalid sort. Note that this condition doesn't fail with a 12 length array. Go uses a different sort algorithm for smaller slices and because of the float comparisons involved, it sorts NaNs at the end of the slice.
Suspicion is heavily influenced by the python library Hypothesis. Their website has a lot of useful information on what property-based testing is and how to use it effectively.
Documentation
¶
Overview ¶
Package suss (Full name Suspicion) is a property-based testing library
Property-based testing uses random generation of data to find edge cases that violate some property. Suspicion implements a state-of-the-art shrinker to find minimal examples of these edge cases.
Suspicion was heavily-inspired by the python project Hypothesis and its internal component, conjecture. Users curious about internal workings can find more info at http://hypothesis.works/
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Invalid ¶
func Invalid()
Invalid signals to the test runner that the current data is invalid and should no longer be considered. This is useful for setting up assumptions like, "This float cannot be a NaN" or "This string must be at least 5 bytes long"
Invalid calls panic internally. The test function should be aware of the non-local control-flow and use defers for cleanup.
Types ¶
type Data ¶
type Data interface {
// Draw is the main way to get random data
// for a Suspicion test. It takes a number of bytes
// to draw and a Sample function. The Sample function
// should return a valid byte sequence for the value.
//
// Since return values from Sample is used as examples
// during execution, it is adventageous to return
// interesting values. Floating point Not-a-Number
// and Infinity values is a good example of such a
// value.
//
// During test execution, Draw might return the value
// from Sample or a random one. Callers must handle
// any random byte sequence, either by reinterpreting it
// or calling Invalid.
Draw(n int, smp Sample) []byte
// StartExample and EndExample are used to specify boundaries for
// draws. The shrinking algorithm uses these boundaries to make
// decisions about how to shrink.
//
// Most users will not need to
// explicitly call these functions, since Draw inserts StartExample
// and EndExample calls around calls to Fill.
//
// Calls to these functions can be nested.
StartExample()
EndExample()
}
Data is an interface passed to Fill methods on types implementing the Generator interface. It is the main method for getting random data from the Suspicion runner.
type Float64Gen ¶
type Float64Gen float64
Float64Gen implements a generator for float64 values.
func (*Float64Gen) Fill ¶
func (f *Float64Gen) Fill(d Data)
type Generator ¶
type Generator interface {
Fill(d Data)
}
Generator generates data to be used in a test. Fill should draw bytes from the Data interface and change its own value based off those bytes.
Generators can be passed to the Runner.Draw function to be supplied with a Data to draw bytes from. This is the main way to get varying data that might cause tests to fail.
type Int63nGen ¶
Int63nGen generates a int64 between 0 and N, following the pattern of the math/rand function. After Fill the value can be read from Value
type Runner ¶
type Runner struct {
// contains filtered or unexported fields
}
Runner is the main entry point to a Suspicion test.
func (*Runner) Draw ¶
Draw takes a generator and fills it with data. This is used to get the data that might cause a failing example.
func (*Runner) Fatalf ¶
Fatalf signals to the test runner that this test has failed. It takes a fmt.Printf format string that is printed when a minimal failing example has been found.
func (*Runner) Float64 ¶
Float64 is a convenience function that returns a float64 value from the Runner.
func (*Runner) Run ¶
func (r *Runner) Run(f func())
Run is the main entry point to a suspicion test. To run a suspicion test, give it a function that verifies some property and calls Runner.Fatalf if it's violated. The function is executed multiple times with different data to find a failing test.
If data is found that causes the test to fail, then we will attempt to "shrink" the data. Shrinking involves making changes to the data, executing the test again and seeing if the test still fails.
The function given should be a self contained function that can be called multiple times. This can be done by either making the function side-effect free or making the function implement setup and teardown logic. Since Suspicion uses panics as control flow, teardown should be done using defers.
type Sample ¶
The Sample type is a function, used to return sample values during the draw process. This is used to guide shrinking towards values which are meaningful and interesting.
Meaningful means "can be interpreted to become a value". A good example of this is UTF-8 strings, where some byte sequences aren't valid values.
Interesting means, "may cause a failure". A good example of an interesting value is floating point NaNs, which are known to cause failure in many bits of code.
type SliceGen ¶
func Slice ¶
func Slice(f func()) *SliceGen
Slice returns a generator for a slice value. It will repeatedly call the given function during fill. It is the functions responsibility to build the slice wanted.
An example of the proper use of Slice:
var f []float64
s := suss.Slice(func() {
f = append(f, runner.Float64())
})
runner.Draw(s)
Source Files
Directories