metamorphic

package
v2.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 22, 2025 License: BSD-3-Clause Imports: 51 Imported by: 0

Documentation

Overview

Package metamorphic provides a testing framework for running randomized tests over multiple Pebble databases with varying configurations. Logically equivalent operations should result in equivalent output across all configurations.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// UseDisk configures RunAndCompare to use the physical filesystem for all
	// generated runs.
	UseDisk = closureOpt(func(ro *runAndCompareOptions) {
		ro.mutateTestOptions = append(ro.mutateTestOptions, func(to *TestOptions) { to.useDisk = true })
	})
	// UseInMemory configures RunAndCompare to use an in-memory virtual
	// filesystem for all generated runs.
	UseInMemory = closureOpt(func(ro *runAndCompareOptions) {
		ro.mutateTestOptions = append(ro.mutateTestOptions, func(to *TestOptions) { to.useDisk = false })
	})
)

Functions

func Compare 

func Compare(t TestingT, rootDir string, seed uint64, runDirs []string, rOpts ...RunOnceOption)

Compare runs the metamorphic tests in the provided runDirs and compares their histories.

func CompareHistories 

func CompareHistories(t TestingT, paths []string) (i int, diff string)

CompareHistories takes a slice of file paths containing history files. It performs a diff comparing the first path to all other paths. CompareHistories returns the index and diff for the first history that differs. If all the histories are identical, CompareHistories returns a zero index and an empty string.

func Execute 

func Execute(m *Test) error

Execute runs the provided test, writing the execution history into the Test's sink.

Example 
const seed = 1698702489658104000
rng := rand.New(rand.NewPCG(0, seed))

// Generate a random database by running the metamorphic test.
testOpts := metamorphic.RandomOptions(rng, nil /* custom opt parsers */)
ops := metamorphic.GenerateOps(rng, 10000, metamorphic.DefaultOpConfig())
test, err := metamorphic.New(ops, testOpts, "" /* dir */, io.Discard)
if err != nil {
	panic(err)
}
err = metamorphic.Execute(test)
fmt.Print(err)

Output:

<nil>

func RunAndCompare 

func RunAndCompare(t *testing.T, rootDir string, rOpts ...RunOption)

RunAndCompare runs the metamorphic tests, using the provided root directory to hold test data.

func RunOnce 

func RunOnce(t TestingT, runDir string, seed uint64, historyPath string, rOpts ...RunOnceOption)

RunOnce performs one run of the metamorphic tests. RunOnce expects the directory named by `runDir` to already exist and contain an `OPTIONS` file containing the test run's configuration. The history of the run is persisted to a file at the path `historyPath`.

The `seed` parameter is not functional; it's used for context in logging.

func TryToGenerateDiagram 

func TryToGenerateDiagram(opsData []byte) (string, error)

TryToGenerateDiagram attempts to generate a user-readable ASCII diagram of the keys involved in the operations.

If the diagram would be too large to be practical, returns the empty string (with no error).

func TryToSimplifyKeys 

func TryToSimplifyKeys(opsData []byte, retainSuffixes bool) []byte

TryToSimplifyKeys parses the operations data and tries to reassign keys to single lowercase characters. Note that the result is not necessarily semantically equivalent.

On success it returns the new operations data.

If there are too many distinct keys, returns nil.

Types

type CustomOption 

type CustomOption interface {
	// Name returns the name of the custom option. This is the key under which
	// the option appears in the OPTIONS file, within the [TestOptions] stanza.
	Name() string
	// Value returns the value of the custom option, serialized as it should
	// appear within the OPTIONS file.
	Value() string
	// Close is run after the test database has been closed at the end of the
	// test as well as during restart operations within the test sequence. It's
	// passed a copy of the *pebble.Options. If the custom options hold on to
	// any resources outside, Close should release them.
	Close(*pebble.Options) error
	// Open is run before the test runs and during a restart operation after the
	// test database has been closed and Close has been called. It's passed a
	// copy of the *pebble.Options. If the custom options must acquire any
	// resources before the test continues, it should reacquire them.
	Open(*pebble.Options) error
}

CustomOption defines a custom option that configures the behavior of an individual test run. Like all test options, custom options are serialized to the OPTIONS file even if they're not options ordinarily understood by Pebble.

type FailOnMatch 

type FailOnMatch struct {
	*regexp.Regexp
}

FailOnMatch configures the run to fail immediately if the history matches the provided regular expression.

type InjectErrorsRate 

type InjectErrorsRate float64

InjectErrorsRate configures the run to inject errors into read-only filesystem operations and retry injected errors.

type KeepData 

type KeepData struct{}

KeepData keeps the database directory, even on successful runs. If the test used an in-memory filesystem, the in-memory filesystem will be persisted to the run directory.

type MaxThreads 

type MaxThreads int

MaxThreads sets an upper bound on the number of parallel execution threads during replay.

type MultiInstance 

type MultiInstance int

MultiInstance configures the number of pebble instances to create.

type OpConfig 

type OpConfig struct {
	// contains filtered or unexported fields
}

OpConfig describes the distribution of operations and their attributes.

func DefaultOpConfig 

func DefaultOpConfig() OpConfig

DefaultOpConfig returns the default distribution of operations.

func ReadOpConfig 

func ReadOpConfig() OpConfig

ReadOpConfig builds an OpConfig that performs only read operations.

func WriteOpConfig 

func WriteOpConfig() OpConfig

WriteOpConfig builds an OpConfig suitable for generating a random test database. It generates Writer operations and some meta database operations like flushes and manual compactions, but it does not generate any reads.

func (OpConfig) WithNewPrefixProbability 

func (c OpConfig) WithNewPrefixProbability(p float64) OpConfig

WithNewPrefixProbability returns a modified op configuration with the probability of generating a new key prefix set to the provided value in [0,1.0].

func (OpConfig) WithOpWeight 

func (c OpConfig) WithOpWeight(op OpType, weight int) OpConfig

WithOpWeight returns a modified op configuration with the weight of the provided operation type overidden.

type OpTimeout 

type OpTimeout time.Duration

OpTimeout sets a timeout for each executed operation. A value of 0 means no timeout.

type OpType 

type OpType int

OpType is an enum of possible operation types. 

const (
	OpBatchAbort OpType = iota
	OpBatchCommit
	OpDBCheckpoint
	OpDBClose
	OpDBCompact
	OpDBDownload
	OpDBFlush
	OpDBRatchetFormatMajorVersion
	OpDBRestart
	OpIterClose
	OpIterFirst
	OpIterLast
	OpIterNext
	OpIterNextWithLimit
	OpIterNextPrefix
	OpIterCanSingleDelete
	OpIterPrev
	OpIterPrevWithLimit
	OpIterSeekGE
	OpIterSeekGEWithLimit
	OpIterSeekLT
	OpIterSeekLTWithLimit
	OpIterSeekPrefixGE
	OpIterSetBounds
	OpIterSetOptions
	OpNewBatch
	OpNewIndexedBatch
	OpNewIter
	OpNewIterUsingClone
	OpNewSnapshot
	OpNewExternalObj
	OpReaderGet
	OpReplicate
	OpSnapshotClose
	OpWriterApply
	OpWriterDelete
	OpWriterDeleteRange
	OpWriterIngest
	OpWriterIngestAndExcise
	OpWriterIngestExternalFiles
	OpWriterLogData
	OpWriterMerge
	OpWriterRangeKeyDelete
	OpWriterRangeKeySet
	OpWriterRangeKeyUnset
	OpWriterSet
	OpWriterSingleDelete
	NumOpTypes
)

These constants define the set of possible operation types performed by the metamorphic test.

type Ops 

type Ops []op

Ops holds a sequence of operations to be executed by the metamorphic tests.

func GenerateOps 

func GenerateOps(rng *rand.Rand, n uint64, cfg OpConfig) Ops

GenerateOps generates n random operations, drawing randomness from the provided pseudorandom generator and using cfg to determine the distribution of op types.

type RetryPolicy 

type RetryPolicy func(error) bool

A RetryPolicy determines what error values should trigger a retry of an operation. 

var (
	// NeverRetry implements a RetryPolicy that never retries.
	NeverRetry = func(error) bool { return false }
	// RetryInjected implements a RetryPolicy that retries whenever an
	// errorfs.ErrInjected error is returned.
	RetryInjected RetryPolicy = func(err error) bool {
		return errors.Is(err, errorfs.ErrInjected)
	}
)

type RunOnceOption 

type RunOnceOption interface {
	// contains filtered or unexported methods
}

A RunOnceOption configures the behavior of a single run of the metamorphic tests.

type RunOption 

type RunOption interface {
	// contains filtered or unexported methods
}

A RunOption configures the behavior of RunAndCompare.

func AddCustomRun 

func AddCustomRun(name string, serializedOptions string) RunOption

AddCustomRun adds an additional run of the metamorphic tests, using the provided OPTIONS file contents. The default options will be used, except those options that are overriden by the provided OPTIONS string.

func ExtendPreviousRun 

func ExtendPreviousRun(opsPath, initialStatePath, initialStateDesc string) RunOption

ExtendPreviousRun configures RunAndCompare to use the output of a previous metamorphic test run to seed the this run. It's used in the crossversion metamorphic tests, in which a data directory is upgraded through multiple versions of Pebble, exercising upgrade code paths and cross-version compatibility.

The opsPath should be the filesystem path to the ops file containing the operations run within the previous iteration of the metamorphic test. It's used to inform operation generation to prefer using keys used in the previous run, which are therefore more likely to be "interesting."

The initialStatePath argument should be the filesystem path to the data directory containing the database where the previous run of the metamorphic test left off.

The initialStateDesc argument is presentational and should hold a human-readable description of the initial state.

func InnerBinary 

func InnerBinary(path string) RunOption

InnerBinary configures the binary that is called for each run. If not specified, this binary (os.Args[0]) is called.

func OpCount 

func OpCount(rv randvar.Static) RunOption

OpCount configures the random variable for the number of operations to generate.

func ParseCustomTestOption 

func ParseCustomTestOption(name string, parseFn func(value string) (CustomOption, bool)) RunOption

ParseCustomTestOption adds support for parsing the provided CustomOption from OPTIONS files serialized by the metamorphic tests. This RunOption alone does not cause the metamorphic tests to run with any variant of the provided CustomOption set.

func RuntimeTrace 

func RuntimeTrace(name string) RunOption

RuntimeTrace configures each test run to collect a runtime trace and output it with the provided filename.

type Seed 

type Seed uint64

Seed configures generation to use the provided seed. Seed may be used to deterministically reproduce the same run.

type Test 

type Test struct {
	// contains filtered or unexported fields
}

A Test configures an individual test run consisting of a set of operations, TestOptions configuring the target database to which the operations should be applied, and a sink for outputting test history.

func New 

func New(ops Ops, opts *TestOptions, dir string, w io.Writer) (*Test, error)

New constructs a new metamorphic test that runs the provided operations against a database using the provided TestOptions and outputs the history of events to an io.Writer.

dir specifies the path within opts.Opts.FS to open the database.

func (*Test) Step 

func (t *Test) Step() (more bool, operationOutput string, err error)

Step runs one single operation, returning: whether there are additional operations remaining; the operation's output; and an error if any occurred while running the operation.

Step may be used instead of Execute to advance a test one operation at a time.

type TestOptions 

type TestOptions struct {
	// Opts holds the *pebble.Options for the test.
	Opts *pebble.Options
	// Threads configures the parallelism of the test. Each thread will run in
	// an independent goroutine and be responsible for executing operations
	// against an independent set of objects. The outcome of any individual
	// operation will still be deterministic, with the metamorphic test
	// inserting synchronization where necessary.
	Threads int
	// RetryPolicy configures which errors should be retried.
	RetryPolicy RetryPolicy
	// CustomOptions holds custom test options that are defined outside of this
	// package.
	CustomOpts []CustomOption
	// contains filtered or unexported fields
}

TestOptions describes the options configuring an individual run of the metamorphic tests.

func RandomOptions 

func RandomOptions(
	rng *rand.Rand, customOptionParsers map[string]func(string) (CustomOption, bool),
) *TestOptions

RandomOptions generates a random set of operations, drawing randomness from rng.

func (*TestOptions) InitRemoteStorageFactory 

func (testOpts *TestOptions) InitRemoteStorageFactory()

InitRemoteStorageFactory initializes Opts.Experimental.RemoteStorage.

type TestingT 

type TestingT interface {
	require.TestingT
	Failed() bool
}

TestingT is an interface wrapper around *testing.T

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.