copyist

README

copyist

Mocking your SQL database in Go tests has never been easier. The copyist library automatically records low-level SQL calls made during your tests. It then generates recording files that can be used to play back those calls without connecting to the real SQL database. Run your tests again. This time, they'll run much faster, because now they do not require a database connection.

Best of all, your tests will run as if your test database was reset to a clean, well-known state between every test case. Gone are the frustrating problems where a test runs fine in isolation, but fails when run in concert with other tests that modify the database. In fact, during playback you can run different test packages in parallel, since they will not conflict with one another at the database level.

copyist imposes no overhead on production code, and it requires almost no changes to your application or testing code, as long as that code directly or indirectly uses Go's sql package (e.g. Go ORM's and the widely used sqlx package). This is because copyist runs at the driver level of Go's sql package.

What problems does copyist solve?

Imagine you have some application code that opens a connection to a Postgres database and queries some customer data:

func QueryName(db *sql.DB) string {
	rows, _ := db.Query("SELECT name FROM customers WHERE id=$1", 100)
	defer rows.Close()

	for rows.Next() {
		var name string
		rows.Scan(&name)
		return name
	}
	return ""
}

The customary way to test this code would be to create a test database and populate it with test customer data. However, what if application code modifies rows in the database, like removing customers? If the above code runs on a modified database, it may not return the expected customer. Therefore, it's important to reset the state of the database between test cases so that tests behave predictably. But connecting to a database is slow. Running queries is slow. And resetting the state of an entire database between every test is really slow.

Various mocking libraries are another alternative to using a test database. These libraries intercept calls at some layer of the application or data access stack, and return canned responses without needing to touch the database. The problem with many of these libraries is that they require the developer to manually construct the canned responses, which is time-consuming and fragile when application changes occur.

How does copyist solve these problems?

copyist includes a Go sql package driver that records the low-level SQL calls made by application and test code. When a Go test using copyist is invoked with the "-record" command-line flag, then the copyist driver will record all SQL calls. When the test completes, copyist will generate a custom text file that contains the recorded SQL calls. The Go test can then be run again without the "-record" flag. This time the copyist driver will play back the recorded calls, without needing to access the database. The Go test is none the wiser, and runs as if it was using the database.

How do I use copyist?

Below is the recommended test pattern for using copyist. The example shows how to unit test the QueryName function shown above.

func init() {
	copyist.Register("postgres")
}

func TestQueryName(t *testing.T) {
	defer copyist.Open(t).Close()

	db, _ := sql.Open("copyist_postgres", "postgresql://root@localhost")
	defer db.Close()

	name := QueryName(db)
	if name != "Andy" {
		t.Error("failed test")
	}
}

In your init or TestMain function (or any other place that gets called before any of the tests), call the copyist.Register function. This function registers a new driver with Go's sql package with the name copyist_<driverName>. In any tests you'd like to record, add a defer copyist.Open(t).Close() statement. This statement begins a new recording session, and then generates a playback file when Close is called at the end of the test.

copyist does need to know whether to run in "recording" mode or "playback" mode. To make copyist run in "recording" mode, invoke the test with the record flag:

go test -run TestQueryName -record

This will generate a new recording file in a testdata subdirectory, with the same name as the test file, but with a .copyist extension. For example, if the test file is called app_test.go, then copyist will generate a testdata/app_test.copyist file containing the recording for the TestQueryName test. Now try running the test again without the record flag:

go test -run TestQueryName

It should now run significantly faster. You can also define the COPYIST_RECORD environment variable (to any value) to make copyist run in recording mode:

COPYIST_RECORD=1 go test ./...

This is useful when running many test packages, some of which may not link to the copyist library, and therefore do not define the record flag.

How do I reset the database between tests?

You can call SetSessionInit to register a function that will clean your database:

func init() {
    copyist.Register("postgres")
    copyist.SetSessionInit(resetDB)
}

The resetDB function will be called by copyist each time you call copyist.Open in your tests, as long as copyist is running in "recording" mode. The session initialization function can do anything it likes, but usually it will run a SQL script against the database in order to reset it to a clean state, by dropping/creating tables, deleting data from tables, and/or inserting "fixture" data into tables that makes testing more convenient.

Troubleshooting

I'm seeing "unexpected call" panics telling me to "regenerate recording"

This just means that you need to re-run your tests with the "-record" command line flag, in order to generate new recordings. Most likely, you changed either your application or your test code so that they call the database differently, using a different sequence or content of calls.

However, there are rarer cases where you've regenerated recordings, have made no test or application changes, and yet are still seeing this error when you run your tests in different orders. This is caused by non-determinism in either your application or in the ORM you're using.

As an example of non-determinism, some ORMs send a setup query to the database when the first connection is opened in order to determine the database version. So whichever test happens to run first records an extra Query call. If you run a different test first, you'll see the "unexpected call" error, since other tests aren't expecting the extra call.

The solution to these problems is to eliminate the non-determinism. For example, in the case of an ORM sending a setup query, you might initialize it from your TestMain method:

func TestMain(m *testing.M) {
	flag.Parse()
	copyist.Register("postgres")
	copyist.SetSessionInit(resetDB)
	closer := copyist.OpenNamed("test.copyist", "OpenCopyist")
	pop.Connect("copyist-test")
	closer.Close()
	os.Exit(m.Run())
}

This triggers the first query in TestMain, which is always run before tests.

The generated copyist recording files are too big

The size of the recording files is directly related to the number of accesses your tests make to the database, as well as the amount of data that they request. While copyist takes pains to generate efficient recording files that eliminate as much redundancy as possible, there's only so much it can do. Try to write tests that operate over smaller amounts of interesting data. For tests that require large numbers of database calls, or large amounts of data, use a different form of verification. One nice thing about copyist is that you can pick and choose which tests will use it. The right tool for the right job, and all that.

Limitations

• Because of the way copyist works, it cannot be used with test and application code that accesses the database concurrently on multiple threads. This includes tests running with the "-parallel" testing flag, which enables tests in the same package to run in parallel. Multiple threads are problematic because the copyist driver code has no way to know which threads are associated with which tests. However, this limitation does not apply to running different test packages in parallel; in playback mode, this is both possible and highly encouraged! However, in recording mode, there may be problems if your tests conflict with one another at the database layer (i.e. by reading/modifying the same rows). The recommended pattern is to run test packages serially in recording mode, and then in parallel in playback mode.

• copyist currently supports only the Postgres pq and pgx stdlib drivers. If you'd like to extend copyist to support other drivers, like MySql or SQLite, you're invited to submit a pull request.

• copyist does not implement every sql package driver interface and method. This may mean that copyist may not fully work with some drivers with more advanced features. Contributions in this area are welcome.

Documentation

Index

• Constants
• Variables
• func IsConnQueryDisabled() bool
• func IsOpen() bool
• func IsRecording() bool
• func Open(t testingT) io.Closer
• func OpenNamed(t testingT, pathName, recordingName string) io.Closer
• func OpenSource(t testingT, source Source, recordingName string) io.Closer
• func Register(driverName string)
• func SetSessionInit(callback SessionInitCallback)
• type SessionInitCallback
• type Source

Constants

const (
	DriverOpen recordType
	ConnExec
	ConnPrepare
	ConnQuery
	ConnBegin
	StmtNumInput
	StmtExec
	StmtQuery
	TxCommit
	TxRollback
	ResultLastInsertId
	ResultRowsAffected
	RowsColumns
	RowsNext
)

This is a list of the event types, which correspond 1:1 with SQL driver methods.

Variables

var MaxRecordingSize = 1024 * 1024

MaxRecordingSize is the maximum size, in bytes, of a single recording in its text format.

Functions

func IsConnQueryDisabled

func IsConnQueryDisabled() bool

IsConnQueryDisabled returns true if usage of ConnQuery is disabled.

func IsOpen

func IsOpen() bool

IsOpen is true if a recording or playback session is currently in progress. That is, Open or OpenNamed has been called, but Close has not yet been called. This is useful when some tests use copyist and some don't, and testing utility code wants to automatically determine whether to open a connection using the copyist driver or the "real" driver.

func IsRecording

func IsRecording() bool

IsRecording returns true if copyist is currently in recording mode.

func Open

func Open(t testingT) io.Closer

Open begins a recording or playback session, depending on the value of the "record" command-line flag. If recording, then all calls to registered drivers will be recorded and then saved in a copyist recording file that sits alongside the calling test file. If playing back, then the recording will be fetched from that recording file. Here is a typical calling pattern:

func init() {
  copyist.Register("postgres")
}

func TestMyStuff(t *testing.T) {
  defer copyist.Open(t).Close()
  ...
}

The call to Open will initiate a new recording session. The deferred call to Close will complete the recording session and write the recording to a file in the testdata/ directory, like:

mystuff_test.go
testdata/
  mystuff_test.copyist

Each test or sub-test that needs to be executed independently needs to record its own session.

func OpenNamed

func OpenNamed(t testingT, pathName, recordingName string) io.Closer

OpenNamed is a variant of Open which accepts a caller-specified pathName and recordingName rather than deriving default values for them. The given pathName will be used as the name of the output file containing the recordings rather than the default "_test.copyist" file in the testdata directory. The given recordingName will be used as the recording name in that file rather than using the testing.T.Name() value.

func OpenSource

func OpenSource(t testingT, source Source, recordingName string) io.Closer

OpenSource is a variant of Open which accepts a caller-specified source and recordingName rather than deriving default values for them. The given source will be used to persist and load recordings rather than the default "_test.copyist" file in the testdata directory. The given recordingName will be used as the recording name in that file rather than using the testing.T.Name() value.

func Register

func Register(driverName string)

Register constructs a proxy driver that wraps the "real" driver of the given name. Depending on the value of the "record" command-line flag, the constructed proxy will either record calls to the wrapped driver, or else play back calls that were previously recorded. Register must be called before copyist.Open can be called, typically in an init() method. Note that the wrapped driver is lazily fetched from the `sql` package, so if a driver of that name does not exist, an error will not be raised until a connection is opened for the first time.

The Register method takes the name of the SQL driver to be wrapped (e.g. "postgres"). Below is an example of how copyist.Register should be invoked.

copyist.Register("postgres")

Note that Register can only be called once for a given driver; subsequent attempts will fail with an error. In addition, the same copyist driver must be used with playback as was was used during recording.

func SetSessionInit

func SetSessionInit(callback SessionInitCallback)

SetSessionInit sets the callback function that will be invoked at the beginning of each copyist session. This can be used to initialize the test database to a clean, well-known state.

NOTE: The callback is only invoked in "recording" mode. There is no need to call it in "playback" mode, as the database is not actually accessed at that time.

Types

type SessionInitCallback

type SessionInitCallback func()

SessionInitCallback types a function that is invoked once per session for each driver, when in recording mode, in order to initialize the database to a clean, well-known state.

type Source

type Source interface {
	// ReadAll reads the underlying resource this Source represents and returns
	// the contents.
	ReadAll() ([]byte, error)

	// WriteAll persists the given data to the underlying resource this Source
	// represents.
	WriteAll([]byte) error
}

Source represents a persistent copyist recording source, generally a file on disk.