regression

Documentation

Overview

Package regression facilitates the regression testing of emulation code. By adding test results to a database, the tests can be rerun automatically and checked for consistancy.

Currently, two main types of test are supported. First the video test. This test runs a ROM for a set number of frames. A hash of the final video output is created a stored for future comparison.

The second test is the Playback test. This is a slightly more complex test that replays user input from a previously recorded session. Recorded sessions take video hashes on every input trigger and so will succeed or fail if something has changed. The regression test automates the process.

The third test is the Log test. This takes a hash of the log after a set number of frames. Test failure for the Log test means that something (anything) in the log output has changed.

In addition to its basic function, the video test also supports recording of machine state. Four machine states are supported at the moment - TV state, RIOT/Ports state, RIOT/Timer and CPU. Aprt from the TV state this doesn't fit well with the idea of the video digest and may be separated into a completely separate test in the future.

Playback scripts and state scripts are stored in the "regressionScripts" directory of the emulator's configuration directory. See the gopher2600 paths package for details about the configuration directory.

To keep things simple regression runs will be performed in relation to the VCS hardware in its default state, in particular no randomisation. The state of the VCS in relation to playback regression entries is governed by the recorder package.

Index

• func RegressAdd(output io.Writer, reg Regressor) error
• func RegressCleanup(output io.Writer, confirmation io.Reader) error
• func RegressDelete(output io.Writer, confirmation io.Reader, key string) error
• func RegressList(output io.Writer) error
• func RegressRedux(output io.Writer, confirmation io.Reader) error
• func RegressRun(output io.Writer, verbose bool, filterKeys []string) error
• type LogRegression
  • func (reg LogRegression) CleanUp() error
  • func (reg LogRegression) EntryType() string
  • func (reg *LogRegression) Serialise() (database.SerialisedEntry, error)
  • func (reg LogRegression) String() string
• type PlaybackRegression
  • func (reg PlaybackRegression) CleanUp() error
  • func (reg PlaybackRegression) EntryType() string
  • func (reg *PlaybackRegression) Serialise() (database.SerialisedEntry, error)
  • func (reg PlaybackRegression) String() string
• type Regressor
• type StateType
  • func NewStateType(state string) (StateType, error)
  • func (t StateType) String() string
• type VideoRegression
  • func (reg VideoRegression) CleanUp() error
  • func (reg VideoRegression) EntryType() string
  • func (reg *VideoRegression) Serialise() (database.SerialisedEntry, error)
  • func (reg VideoRegression) String() string

Functions

func RegressAdd

func RegressAdd(output io.Writer, reg Regressor) error

RegressAdd adds a new regression handler to the database.

func RegressCleanup

func RegressCleanup(output io.Writer, confirmation io.Reader) error

CleanupScript removes orphaned script files from disk. An orphaned file is one that exists on disk but has no reference in the regression database file.

func RegressDelete

func RegressDelete(output io.Writer, confirmation io.Reader, key string) error

RegressDelete removes a cartridge from the regression db.

func RegressList

func RegressList(output io.Writer) error

RegressList displays all entries in the database.

func RegressRedux

func RegressRedux(output io.Writer, confirmation io.Reader) error

RegressRedux removes and adds an entry using the same parameters.

func RegressRun

func RegressRun(output io.Writer, verbose bool, filterKeys []string) error

RegressRun runs all the tests in the regression database. filterKeys list specified which entries to test. an empty keys list means that every entry should be tested.

Types

type LogRegression

type LogRegression struct {
	CartLoad  cartridgeloader.Loader
	TVtype    string
	NumFrames int
	Notes     string
	// contains filtered or unexported fields
}

LogRegression runs for N frames and takes a digest of the log at the end of the run. Regression passes if the subsequent runs produce the same log/digest.

func (LogRegression) CleanUp

func (reg LogRegression) CleanUp() error

CleanUp implements the database.Entry interface.

func (LogRegression) EntryType

func (reg LogRegression) EntryType() string

EntryType implements the database.Entry interface.

func (*LogRegression) Serialise

func (reg *LogRegression) Serialise() (database.SerialisedEntry, error)

Serialise implements the database.Entry interface.

func (LogRegression) String

func (reg LogRegression) String() string

String implements the regressions.Regressor interface

type PlaybackRegression

type PlaybackRegression struct {
	Script string
	Notes  string
}

PlaybackRegression represents a regression type that processes a VCS recording. playback regressions can take a while to run because by their nature they extend over many frames - many more than is typical with the FrameRegression type.

func (PlaybackRegression) CleanUp

func (reg PlaybackRegression) CleanUp() error

CleanUp implements the database.Entry interface.

func (PlaybackRegression) EntryType

func (reg PlaybackRegression) EntryType() string

EntryType implements the database.Entry interface.

func (*PlaybackRegression) Serialise

func (reg *PlaybackRegression) Serialise() (database.SerialisedEntry, error)

Serialise implements the database.Entry interface.

func (PlaybackRegression) String

func (reg PlaybackRegression) String() string

String implements the regression.Regressor interface.

type Regressor

type Regressor interface {
	database.Entry

	// String should return information about the entry in a human readable
	// format. by contrast, machine readable representation is returned by the
	// Serialise function
	String() string
	// contains filtered or unexported methods
}

Regressor is the generic entry type in the regressionDB.

type StateType

type StateType string

Indicates the State recording method to use.

const (
	StateNone  StateType = ""
	StateTV    StateType = "TV"
	StatePorts StateType = "PORTS"
	StateTimer StateType = "TIMER"
	StateCPU   StateType = "CPU"
)

List of valid StateType values.

func NewStateType

func NewStateType(state string) (StateType, error)

NewStateType parses a string and returns a new StateType or an error. Use this rather than casting a string to the StateType.

func (StateType) String

func (t StateType) String() string

type VideoRegression

type VideoRegression struct {
	CartLoad  cartridgeloader.Loader
	TVtype    string
	NumFrames int
	State     StateType

	Notes string
	// contains filtered or unexported fields
}

VideoRegression is the simplest regression type. it works by running the emulation for N frames and the video recorded at that point. Regression passes if subsequenct runs produce the same video value.

func (VideoRegression) CleanUp

func (reg VideoRegression) CleanUp() error

CleanUp implements the database.Entry interface.

func (VideoRegression) EntryType

func (reg VideoRegression) EntryType() string

EntryType implements the database.Entry interface.

func (*VideoRegression) Serialise

func (reg *VideoRegression) Serialise() (database.SerialisedEntry, error)

Serialise implements the database.Entry interface.

func (VideoRegression) String

func (reg VideoRegression) String() string

String implements the regression.Regressor interface