rapid

package module
v0.3.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 3, 2020 License: MPL-2.0 Imports: 27 Imported by: 31

README

Rapid GoDoc CI

Rapid is a Go library for property-based testing.

Rapid checks that properties you define hold for a large number of automatically generated test cases. If a failure is found, rapid automatically minimizes the failing test case before presenting it.

Property-based testing emphasizes thinking about high level properties the program should satisfy rather than coming up with a list of individual examples of desired behavior (test cases). This results in concise and powerful tests that are a pleasure to write.

Design and implementation of rapid are heavily inspired by Hypothesis, which is itself a descendant of QuickCheck.

Features

  • Idiomatic Go API
  • Automatic minimization of failing test cases
  • No dependencies outside of the Go standard library
Planned features
  • Automatic persistence of failing test cases

Examples

Here is what a trivial test using rapid looks like:

package rapid_test

import (
	"net"
	"testing"

	"pgregory.net/rapid"
)

func TestParseValidIPv4(t *testing.T) {
	const ipv4re = `(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])` +
		`\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])` +
		`\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])` +
		`\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])`

	rapid.Check(t, func(t *rapid.T) {
		addr := rapid.StringMatching(ipv4re).Draw(t, "addr").(string)
		ip := net.ParseIP(addr)
		if ip == nil || ip.String() != addr {
			t.Fatalf("parsed %q into %v", addr, ip)
		}
	})
}

You can play around with the IPv4 regexp to see what happens when it is generating invalid addresses (or try to pass the test with your own ParseIP implementation). More complete function (source code, playground) and state machine (source code, playground) example tests are provided. They both fail. Making them pass is a good way to get first real experience of working with rapid.

Usage

Just run go test as usual, it will pick up also all rapid tests.

There are a number of optional flags to influence rapid behavior, run go test -args -h and look at the flags with the -rapid. prefix. You can then pass such flags as usual. For example:

go test -rapid.checks=1000

Comparison

Rapid aims to bring to Go the power and convenience Hypothesis brings to Python.

Compared to gopter, rapid:

  • has a much simpler API (queue test in rapid vs gopter)
  • does not require any user code to minimize failing test cases
  • generates biased data to explore "small" values and edge cases more thoroughly (inspired by SmallCheck)
  • enables interactive tests by allowing data generation and test code to arbitrarily intermix

Compared to testing/quick, rapid:

  • provides much more control over test case generation
  • supports state machine ("stateful" or "model-based") testing
  • automatically minimizes any failing test case
  • has to settle for rapid.Check being the main exported function instead of much more stylish quick.Check

Status

Rapid is alpha software. Important pieces of functionality are missing; API breakage and bugs should be expected.

If rapid fails to find a bug you believe it should, or the failing test case that rapid reports does not look like a minimal one, please open an issue.

License

Rapid is licensed under the Mozilla Public License version 2.0.

Documentation

Overview

Package rapid implements utilities for property-based testing.

Rapid checks that properties you define hold for a large number of automatically generated test cases. If a failure is found, rapid fails the current test and presents an automatically minimized version of the failing test case.

Here is what a trivial test using rapid looks like: 

package rapid_test

import (
	"net"
	"testing"

	"pgregory.net/rapid"
)

func TestParseValidIPv4(t *testing.T) {
	const ipv4re = `(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])` +
		`\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])` +
		`\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])` +
		`\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])`

	rapid.Check(t, func(t *rapid.T) {
		addr := rapid.StringMatching(ipv4re).Draw(t, "addr").(string)
		ip := net.ParseIP(addr)
		if ip == nil || ip.String() != addr {
			t.Fatalf("parsed %q into %v", addr, ip)
		}
	})
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Check 

func Check(t *testing.T, prop func(*T))

Check fails the current test if rapid can find a test case which falsifies prop.

Property is falsified in case of a panic or a call to (*T).Fatalf, (*T).Fatal, (*T).Errorf, (*T).Error, (*T).FailNow or (*T).Fail.

Example (ParseDate)

Rename to TestParseDate(t *testing.T) to make an actual (failing) test. 

package main

import (
	"fmt"
	"strconv"
	"testing"

	"pgregory.net/rapid"
)

type goExampleBugWorkaround int // work around whole file example detection bug introduced in Go 1.14

// ParseDate parses dates in the YYYY-MM-DD format.
func ParseDate(s string) (int, int, int, error) {
	if len(s) != 10 {
		return 0, 0, 0, fmt.Errorf("%q has wrong length: %v instead of 10", s, len(s))
	}

	if s[4] != '-' || s[7] != '-' {
		return 0, 0, 0, fmt.Errorf("'-' separators expected in %q", s)
	}

	y, err := strconv.Atoi(s[0:4])
	if err != nil {
		return 0, 0, 0, fmt.Errorf("failed to parse year: %v", err)
	}

	m, err := strconv.Atoi(s[6:7])
	if err != nil {
		return 0, 0, 0, fmt.Errorf("failed to parse month: %v", err)
	}

	d, err := strconv.Atoi(s[8:10])
	if err != nil {
		return 0, 0, 0, fmt.Errorf("failed to parse day: %v", err)
	}

	return y, m, d, nil
}

func testParseDate(t *rapid.T) {
	y := rapid.IntRange(0, 9999).Draw(t, "y").(int)
	m := rapid.IntRange(1, 12).Draw(t, "m").(int)
	d := rapid.IntRange(1, 31).Draw(t, "d").(int)

	s := fmt.Sprintf("%04d-%02d-%02d", y, m, d)

	y_, m_, d_, err := ParseDate(s)
	if err != nil {
		t.Fatalf("failed to parse date %q: %v", s, err)
	}

	if y_ != y || m_ != m || d_ != d {
		t.Fatalf("got back wrong date: (%d, %d, %d)", y_, m_, d_)
	}
}

// Rename to TestParseDate(t *testing.T) to make an actual (failing) test.
func main() {
	var t *testing.T
	rapid.Check(t, testParseDate)
}

Output:

func MakeCheck 

func MakeCheck(prop func(*T)) func(*testing.T)

MakeCheck is a convenience function for defining subtests suitable for (*testing.T).Run. It allows you to write this: 

t.Run("subtest name", rapid.MakeCheck(func(t *rapid.T) {
    // test code
}))

instead of this: 

t.Run("subtest name", func(t *testing.T) {
    rapid.Check(t, func(t *rapid.T) {
        // test code
    })
})

func Run added in v0.3.1 

func Run(m StateMachine) func(*T)

Run is a convenience function for defining "state machine" tests, to be run by Check or MakeCheck.

State machine test is a pattern for testing stateful systems that looks like this: 

m := new(StateMachineType)
m.RandomInitAction(t)  // optional
defer m.Cleanup()      // optional
m.Check(t)
for {
    m.RandomAction(t)
    m.Check(t)
}

Run synthesizes such test from the type of m, which must be a pointer. Note that for each test case, new state machine instance is created via reflection; any data inside m is ignored.

Example (Queue)

Rename to TestQueue(t *testing.T) to make an actual (failing) test. 

package main

import (
	"testing"

	"pgregory.net/rapid"
)

// Queue implements integer queue with a fixed maximum size.
type Queue struct {
	buf []int
	in  int
	out int
}

func NewQueue(n int) *Queue {
	return &Queue{
		buf: make([]int, n+1),
	}
}

// Precondition: Size() > 0.
func (q *Queue) Get() int {
	i := q.buf[q.out]
	q.out = (q.out + 1) % len(q.buf)
	return i
}

// Precondition: Size() < n.
func (q *Queue) Put(i int) {
	q.buf[q.in] = i
	q.in = (q.in + 1) % len(q.buf)
}

func (q *Queue) Size() int {
	return (q.in - q.out) % len(q.buf)
}

// queueMachine is a description of a rapid state machine for testing Queue
type queueMachine struct {
	q     *Queue // queue being tested
	n     int    // maximum queue size
	state []int  // model of the queue
}

// Init is an action for initializing  a queueMachine instance.
func (m *queueMachine) Init(t *rapid.T) {
	n := rapid.IntRange(1, 1000).Draw(t, "n").(int)
	m.q = NewQueue(n)
	m.n = n
}

// Get is a conditional action which removes an item from the queue.
func (m *queueMachine) Get(t *rapid.T) {
	if m.q.Size() == 0 {
		t.Skip("queue empty")
	}

	i := m.q.Get()
	if i != m.state[0] {
		t.Fatalf("got invalid value: %v vs expected %v", i, m.state[0])
	}
	m.state = m.state[1:]
}

// Put is a conditional action which adds an items to the queue.
func (m *queueMachine) Put(t *rapid.T) {
	if m.q.Size() == m.n {
		t.Skip("queue full")
	}

	i := rapid.Int().Draw(t, "i").(int)
	m.q.Put(i)
	m.state = append(m.state, i)
}

// Check runs after every action and verifies that all required invariants hold.
func (m *queueMachine) Check(t *rapid.T) {
	if m.q.Size() != len(m.state) {
		t.Fatalf("queue size mismatch: %v vs expected %v", m.q.Size(), len(m.state))
	}
}

// Rename to TestQueue(t *testing.T) to make an actual (failing) test.
func main() {
	var t *testing.T
	rapid.Check(t, rapid.Run(&queueMachine{}))
}

Output:

Types

type Generator 

type Generator struct {
	// contains filtered or unexported fields
}

func ArrayOf 

func ArrayOf(count int, elem *Generator) *Generator

func Boolean 

func Boolean() *Generator

func Byte 

func Byte() *Generator

func ByteMax 

func ByteMax(max byte) *Generator

func ByteMin 

func ByteMin(min byte) *Generator

func ByteRange 

func ByteRange(min byte, max byte) *Generator

func Custom 

func Custom(fn interface{}) *Generator
Example 
package main

import (
	"fmt"

	"pgregory.net/rapid"
)

func main() {
	type point struct {
		x int
		y int
	}

	gen := rapid.Custom(func(t *rapid.T) point {
		return point{
			x: rapid.Int().Draw(t, "x").(int),
			y: rapid.Int().Draw(t, "y").(int),
		}
	})

	for i := uint64(0); i < 5; i++ {
		fmt.Println(gen.Example(i))
	}
}

Output:

{-3 1303}
{-186981 -59881619}
{4 441488606}
{-2 -5863986}
{43 -3513}

func Float32 

func Float32() *Generator

func Float32Max 

func Float32Max(max float32) *Generator

func Float32Min 

func Float32Min(min float32) *Generator

func Float32Range 

func Float32Range(min float32, max float32) *Generator

func Float64 

func Float64() *Generator

func Float64Max 

func Float64Max(max float64) *Generator

func Float64Min 

func Float64Min(min float64) *Generator

func Float64Range 

func Float64Range(min float64, max float64) *Generator

func Int 

func Int() *Generator

func Int16 

func Int16() *Generator

func Int16Max 

func Int16Max(max int16) *Generator

func Int16Min 

func Int16Min(min int16) *Generator

func Int16Range 

func Int16Range(min int16, max int16) *Generator

func Int32 

func Int32() *Generator

func Int32Max 

func Int32Max(max int32) *Generator

func Int32Min 

func Int32Min(min int32) *Generator

func Int32Range 

func Int32Range(min int32, max int32) *Generator

func Int64 

func Int64() *Generator

func Int64Max 

func Int64Max(max int64) *Generator

func Int64Min 

func Int64Min(min int64) *Generator

func Int64Range 

func Int64Range(min int64, max int64) *Generator

func Int8 

func Int8() *Generator

func Int8Max 

func Int8Max(max int8) *Generator

func Int8Min 

func Int8Min(min int8) *Generator

func Int8Range 

func Int8Range(min int8, max int8) *Generator

func IntMax 

func IntMax(max int) *Generator

func IntMin 

func IntMin(min int) *Generator

func IntRange 

func IntRange(min int, max int) *Generator

func Just 

func Just(val interface{}) *Generator
Example 
package main

import (
	"fmt"

	"pgregory.net/rapid"
)

func main() {
	gen := rapid.Just(42)

	for i := uint64(0); i < 5; i++ {
		fmt.Println(gen.Example(i))
	}
}

Output:

42
42
42
42
42

func MapOf 

func MapOf(key *Generator, val *Generator) *Generator

func MapOfN 

func MapOfN(key *Generator, val *Generator, minLen int, maxLen int) *Generator

func MapOfNValues 

func MapOfNValues(val *Generator, minLen int, maxLen int, keyFn interface{}) *Generator

func MapOfValues 

func MapOfValues(val *Generator, keyFn interface{}) *Generator

func OneOf 

func OneOf(gens ...*Generator) *Generator
Example 
package main

import (
	"fmt"

	"pgregory.net/rapid"
)

func main() {
	gen := rapid.OneOf(rapid.Int32Range(1, 10), rapid.Float32Range(100, 1000))

	for i := uint64(0); i < 5; i++ {
		fmt.Println(gen.Example(i))
	}
}

Output:

997.0737
10
475.3125
2
9

func Ptr 

func Ptr(elem *Generator, allowNil bool) *Generator
Example 
package main

import (
	"fmt"

	"pgregory.net/rapid"
)

func main() {
	gen := rapid.Ptr(rapid.Int(), true)

	for i := uint64(0); i < 5; i++ {
		v := gen.Example(i).(*int)
		if v == nil {
			fmt.Println("<nil>")
		} else {
			fmt.Println("(*int)", *v)
		}
	}
}

Output:

(*int) 1
(*int) -3
<nil>
(*int) 590
<nil>

func Rune 

func Rune() *Generator

func RuneFrom 

func RuneFrom(runes []rune, tables ...*unicode.RangeTable) *Generator

func SampledFrom 

func SampledFrom(slice interface{}) *Generator
Example 
package main

import (
	"fmt"

	"pgregory.net/rapid"
)

func main() {
	gen := rapid.SampledFrom([]int{1, 2, 3})

	for i := uint64(0); i < 5; i++ {
		fmt.Println(gen.Example(i))
	}
}

Output:

2
3
2
3
1

func SliceOf 

func SliceOf(elem *Generator) *Generator

func SliceOfBytesMatching 

func SliceOfBytesMatching(expr string) *Generator

func SliceOfDistinct 

func SliceOfDistinct(elem *Generator, keyFn interface{}) *Generator

func SliceOfN 

func SliceOfN(elem *Generator, minLen int, maxLen int) *Generator

func SliceOfNDistinct 

func SliceOfNDistinct(elem *Generator, minLen int, maxLen int, keyFn interface{}) *Generator

func String 

func String() *Generator

func StringMatching 

func StringMatching(expr string) *Generator

func StringN 

func StringN(minRunes int, maxRunes int, maxLen int) *Generator

func StringOf 

func StringOf(elem *Generator) *Generator

func StringOfN 

func StringOfN(elem *Generator, minElems int, maxElems int, maxLen int) *Generator

func Uint 

func Uint() *Generator

func Uint16 

func Uint16() *Generator

func Uint16Max 

func Uint16Max(max uint16) *Generator

func Uint16Min 

func Uint16Min(min uint16) *Generator

func Uint16Range 

func Uint16Range(min uint16, max uint16) *Generator

func Uint32 

func Uint32() *Generator

func Uint32Max 

func Uint32Max(max uint32) *Generator

func Uint32Min 

func Uint32Min(min uint32) *Generator

func Uint32Range 

func Uint32Range(min uint32, max uint32) *Generator

func Uint64 

func Uint64() *Generator

func Uint64Max 

func Uint64Max(max uint64) *Generator

func Uint64Min 

func Uint64Min(min uint64) *Generator

func Uint64Range 

func Uint64Range(min uint64, max uint64) *Generator

func Uint8 

func Uint8() *Generator

func Uint8Max 

func Uint8Max(max uint8) *Generator

func Uint8Min 

func Uint8Min(min uint8) *Generator

func Uint8Range 

func Uint8Range(min uint8, max uint8) *Generator

func UintMax 

func UintMax(max uint) *Generator

func UintMin 

func UintMin(min uint) *Generator

func UintRange 

func UintRange(min uint, max uint) *Generator

func Uintptr 

func Uintptr() *Generator

func UintptrMax 

func UintptrMax(max uintptr) *Generator

func UintptrMin 

func UintptrMin(min uintptr) *Generator

func UintptrRange 

func UintptrRange(min uintptr, max uintptr) *Generator

func (*Generator) Draw 

func (g *Generator) Draw(t *T, label string) interface{}

func (*Generator) Example 

func (g *Generator) Example(seed ...uint64) interface{}

func (*Generator) Filter 

func (g *Generator) Filter(fn interface{}) *Generator

func (*Generator) Map 

func (g *Generator) Map(fn interface{}) *Generator

func (*Generator) String 

func (g *Generator) String() string

type StateMachine 

type StateMachine interface {
	// Check is ran after every action and should contain invariant checks.
	//
	// Other public methods are treated as follows:
	// - Init(t *rapid.T), InitAnySuffixHere(t *rapid.T), if present,
	//   are used as "initializer" actions; exactly one is ran at the beginning
	//   of each test case;
	// - Cleanup(), if present, is called at the end of each test case;
	// - All other public methods should have a form ActionName(t *rapid.T)
	//   and are used as possible actions. At least one action has to be specified.
	Check(*T)
}

type T 

type T struct {
	// contains filtered or unexported fields
}

func (*T) Error 

func (t *T) Error(args ...interface{})

func (*T) Errorf 

func (t *T) Errorf(format string, args ...interface{})

func (*T) Fail 

func (t *T) Fail()

func (*T) FailNow 

func (t *T) FailNow()

func (*T) Failed 

func (t *T) Failed() bool

func (*T) Fatal 

func (t *T) Fatal(args ...interface{})

func (*T) Fatalf 

func (t *T) Fatalf(format string, args ...interface{})

func (*T) Log 

func (t *T) Log(args ...interface{})

func (*T) Logf 

func (t *T) Logf(format string, args ...interface{})

func (*T) Skip 

func (t *T) Skip(args ...interface{})

func (*T) SkipNow 

func (t *T) SkipNow()

SkipNow marks the current test case as invalid. If too many test cases are skipped, rapid will mark the test as failing due to inability to generate enough valid test cases.

Prefer Filter to SkipNow, and prefer generators that always produce valid test cases to Filter.

func (*T) Skipf 

func (t *T) Skipf(format string, args ...interface{})

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.