dockertest

package module
v2.2.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 8, 2016 License: Apache-2.0 Imports: 18 Imported by: 0

README

ory.am/dockertest

Build Status Coverage Status

Use Docker to run your Go language integration tests against third party services on Microsoft Windows, Mac OSX and Linux! Dockertest uses docker-machine (aka Docker Toolbox) to spin up images on Windows and Mac OSX as well. Dockertest is based on docker.go from camlistore.

This fork detects automatically, if Docker Toolbox is installed. If it is, Docker integration on Windows and Mac OSX can be used without any additional work. To avoid port collisions when using docker-machine, Dockertest chooses a random port to bind the requested image.

Dockertest ships with support for these backends:

  • PostgreSQL
  • MySQL
  • MongoDB
  • NSQ
  • Redis
  • Elastic Search
  • RethinkDB
  • RabbitMQ
  • Mockserver
  • ActiveMQ
  • ZooKeeper
  • Cassandra

Table of Contents

Why should I use Dockertest?

When developing applications, it is often necessary to use services that talk to a database system. Unit Testing these services can be cumbersome because mocking database/DBAL is strenuous. Making slight changes to the schema implies rewriting at least some, if not all of the mocks. The same goes for API changes in the DBAL.
To avoid this, it is smarter to test these specific services against a real database that is destroyed after testing. Docker is the perfect system for running unit tests as you can spin up containers in a few seconds and kill them when the test completes. The Dockertest library provides easy to use commands for spinning up Docker containers and using them for your tests.

Installing and using Dockertest

Using Dockertest is straightforward and simple. Check the releases tab for available releases.

To install dockertest, run

go get gopkg.in/ory-am/dockertest.vX

where X is your desired version. For example:

go get gopkg.in/ory-am/dockertest.v2

Note:
When using the Docker Toolbox (Windows / OSX), make sure that the VM is started by running docker-machine start default.

Start a container
package main

import (
	"gopkg.in/ory-am/dockertest.v2"
	"gopkg.in/mgo.v2"
	"time"
)

func main() {
	var db *mgo.Session
	c, err := dockertest.ConnectToMongoDB(15, time.Millisecond*500, func(url string) bool {
	    // This callback function checks if the image's process is responsive.
	    // Sometimes, docker images are booted but the process (in this case MongoDB) is still doing maintenance
	    // before being fully responsive which might cause issues like "TCP Connection reset by peer".	
		var err error
		db, err = mgo.Dial(url)
		if err != nil {
			return false
		}
		
		// Sometimes, dialing the database is not enough because the port is already open but the process is not responsive.
		// Most database conenctors implement a ping function which can be used to test if the process is responsive.
		// Alternatively, you could execute a query to see if an error occurs or not.
		return db.Ping() == nil
	})
	
	if err != nil {
	    log.Fatalf("Could not connect to database: %s", err)
	}
	
	// Close db connection and kill the container when we leave this function body.
    defer db.Close()
	defer c.KillRemove()
	
	// The image is now responsive.
}

You can start PostgreSQL and MySQL in a similar fashion.

It is also possible to start a custom container (in this example, a RabbitMQ container):

	c, ip, port, err := dockertest.SetupCustomContainer("rabbitmq", 5672, 10*time.Second)
	if err != nil {
		log.Fatalf("Could not setup container: %s", err
	}
	defer c.KillRemove()

	err = dockertest.ConnectToCustomContainer(fmt.Sprintf("%v:%v", ip, port), 15, time.Millisecond*500, func(url string) bool {
		amqp, err := amqp.Dial(fmt.Sprintf("amqp://%v", url))
		if err != nil {
			return false
		}
		defer amqp.Close()
		return true
	})

	...

Write awesome tests

It is a good idea to start up the container only once when running tests.


import (
	"fmt"
	"testing"
    "log"
	"os"

	"database/sql"
	_ "github.com/lib/pq"
	"gopkg.in/ory-am/dockertest.v2"
)

var db *sql.DB

func TestMain(m *testing.M) {
	c, err := dockertest.ConnectToPostgreSQL(15, time.Second, func(url string) bool {
	    // Check if postgres is responsive...
		var err error
		db, err = sql.Open("postgres", url)
		if err != nil {
			return false
		}
		return db.Ping() == nil
	})
	if err != nil {
		log.Fatalf("Could not connect to database: %s", err)
	}
	
	// Execute tasks like setting up schemata.
	
	// Run tests
	result := m.Run()
	
	// Close database connection.
	db.Close()
	
	// Clean up image.
	c.KillRemove()
	
	// Exit tests.
	os.Exit(result)
}

func TestFunction(t *testing.T) {
    // db.Exec(...
}
Setting up Travis-CI

You can run the Docker integration on Travis easily:

# Sudo is required for docker
sudo: required

# Enable docker
services:
  - docker

# In Travis, we need to bind to 127.0.0.1 in order to get a working connection. This environment variable
# tells dockertest to do that.
env:
  - DOCKERTEST_BIND_LOCALHOST=true

Troubleshoot & FAQ

I need to use a specific container version for XYZ

You can specify a container version by setting environment variables or globals. For more information, check vars.go.

My build is broken!

With v2, we removed all Open* methods to reduce duplicate code, unnecessary dependencies and make maintenance easier. If you relied on these, run go get gopkg.in/ory-am/dockertest.v1 and replace import "github.com/ory-am/dockertest" with import "gopkg.in/ory-am/dockertest.v1".

Out of disk space

Try cleaning up the images with docker-cleanup-volumes.

I am using docker machine (OSX / Linux)

First of all, consider upgrading! If that's not an option, there are some steps you need to take:

  • Set dockertest.UseDockerMachine = "1" or set the environment variable DOCKERTEST_LEGACY_DOCKER_MACHINE=1
  • Set docker.BindDockerToLocalhost = "" or alternatively DOCKER_BIND_LOCALHOST=
Removing old containers

Sometimes container clean up fails. Check out this stackoverflow question on how to fix this.

Customized database

I am using postgres (or mysql) driver, how do I use customized database instead of default one? You can alleviate this helper function to do that, see testcase or example below:


func TestMain(m *testing.M) {
	if c, err := dockertest.ConnectToPostgreSQL(15, time.Second, func(url string) bool {
        customizedDB := "cherry" // here I am connecting cherry database
        newURL, err := SetUpPostgreDatabase(customizedDB, url)

        // or use SetUpMysqlDatabase for mysql driver

        if err != nil {
                log.Fatal(err)
        }
        db, err := sql.Open("postgres", newURL)
        if err != nil {
            return false
        }
        return db.Ping() == nil
    }); err != nil {
        log.Fatal(err)
    }

Thanks to our sponsors: Ory GmbH & Imarum GmbH

Documentation

Overview

Package dockertest contains helper functions for setting up and tearing down docker containers to aid in testing. dockertest supports spinning up MySQL, PostgreSQL and MongoDB out of the box.

Dockertest provides two environment variables

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ConsulDatacenter must be defined when starting a Consul datacenter; this
	// value will be used for both the datacenter and the ACL datacenter
	ConsulDatacenter = "test"

	// ConsulACLDefaultPolicy defines the default policy to use with Consul ACLs
	ConsulACLDefaultPolicy = "deny"

	// ConsulACLMasterToken defines the master ACL token
	ConsulACLMasterToken = "test"

	// A function with no arguments that outputs a valid JSON string to be used
	// as the value of the environment variable CONSUL_LOCAL_CONFIG.
	ConsulLocalConfigGen = DefaultConsulLocalConfig
)
View Source 
var (
	// Debug if set, prevents any container from being removed.
	Debug bool

	// DockerMachineAvailable if true, uses docker-machine to run docker commands (for running tests on Windows and Mac OS)
	DockerMachineAvailable bool

	// DockerMachineName is the machine's name. You might want to use a dedicated machine for running your tests.
	// You can set this variable either directly or by defining a DOCKERTEST_IMAGE_NAME env variable.
	DockerMachineName = env.Getenv("DOCKERTEST_IMAGE_NAME", "default")

	// BindDockerToLocalhost if set, forces docker to bind the image to localhost. This for example is required when running tests on travis-ci.
	// You can set this variable either directly or by defining a DOCKERTEST_BIND_LOCALHOST env variable.
	// FIXME DOCKER_BIND_LOCALHOST remove legacy support
	BindDockerToLocalhost = env.Getenv("DOCKERTEST_BIND_LOCALHOST", env.Getenv("DOCKER_BIND_LOCALHOST", "1"))

	// UseDockerMachine if set, forces docker to use the legacy docker-machine on OSX/Windows.
	UseDockerMachine = env.Getenv("DOCKERTEST_LEGACY_DOCKER_MACHINE", "")

	// ContainerPrefix will be prepended to all containers started by dockertest to make identification of these "test images" hassle-free.
	ContainerPrefix = env.Getenv("DOCKERTEST_CONTAINER_PREFIX", "dockertest-")
)

Dockertest configuration

View Source 
var (
	// MongoDBImageName is the MongoDB image name on dockerhub.
	MongoDBImageName = env.Getenv("DOCKERTEST_MONGODB_IMAGE_NAME", "mongo")

	// MySQLImageName is the MySQL image name on dockerhub.
	MySQLImageName = env.Getenv("DOCKERTEST_MYSQL_IMAGE_NAME", "mysql")

	// PostgresImageName is the PostgreSQL image name on dockerhub.
	PostgresImageName = env.Getenv("DOCKERTEST_POSTGRES_IMAGE_NAME", "postgres")

	// ElasticSearchImageName is the ElasticSearch image name on dockerhub.
	ElasticSearchImageName = env.Getenv("DOCKERTEST_ELASTICSEARCH_IMAGE_NAME", "elasticsearch")

	// RedisImageName is the Redis image name on dockerhub.
	RedisImageName = env.Getenv("DOCKERTEST_REDIS_IMAGE_NAME", "redis")

	// NSQImageName is the NSQ image name on dockerhub.
	NSQImageName = env.Getenv("DOCKERTEST_NSQ_IMAGE_NAME", "nsqio/nsq")

	// RethinkDBImageName is the RethinkDB image name on dockerhub.
	RethinkDBImageName = env.Getenv("DOCKERTEST_RETHINKDB_IMAGE_NAME", "rethinkdb")

	// RabbitMQImage name is the RabbitMQ image name on dockerhub.
	RabbitMQImageName = env.Getenv("DOCKERTEST_RABBITMQ_IMAGE_NAME", "rabbitmq")

	// ActiveMQImage name is the ActiveMQ image name on dockerhub.
	ActiveMQImageName = env.Getenv("DOCKERTEST_ACTIVEMQ_IMAGE_NAME", "webcenter/activemq")

	// MockserverImageName name is the Mockserver image name on dockerhub.
	MockserverImageName = env.Getenv("DOCKERTEST_MOCKSERVER_IMAGE_NAME", "jamesdbloom/mockserver")

	// ConsulImageName is the Consul image name on dockerhub.
	ConsulImageName = env.Getenv("DOCKERTEST_CONSUL_IMAGE_NAME", "consul")

	// ZooKeeperImageName is the ZooKeeper image name on dockerhub.
	ZooKeeperImageName = env.Getenv("DOCKERTEST_ZOOKEEPER_IMAGE_NAME", "jplock/zookeeper")

	// CassandraImageName is the Cassandra image name on dockerhub.
	CassandraImageName = env.Getenv("DOCKERTEST_CASSANDRA_IMAGE_NAME", "cassandra")
)

Image configuration

View Source 
var (
	// MySQLUsername must be passed as username when connecting to mysql
	MySQLUsername = "root"

	// MySQLPassword must be passed as password when connecting to mysql
	MySQLPassword = "root"

	// PostgresUsername must be passed as username when connecting to postgres
	PostgresUsername = "postgres"

	// PostgresPassword must be passed as password when connecting to postgres
	PostgresPassword = "docker"
)

Username and password configuration

Functions

func AwaitReachable 

func AwaitReachable(addr string, maxWait time.Duration) error

AwaitReachable tries to make a TCP connection to addr regularly. It returns an error if it's unable to make a connection before maxWait.

func ConnectToCustomContainer 

func ConnectToCustomContainer(url string, tries int, delay time.Duration, connector func(url string) bool) error

ConnectToCustomContainer attempts to connect to a custom container until successful or the maximum number of tries is reached.

func DefaultConsulLocalConfig 

func DefaultConsulLocalConfig() (string, error)

func GenerateContainerID 

func GenerateContainerID() string

GenerateContainerID generated a random container id.

func HaveImage 

func HaveImage(name string) (bool, error)

HaveImage reports if docker have image 'name'.

func IP 

func IP(containerID string) (string, error)

IP returns the IP address of the container.

func KillContainer 

func KillContainer(container string) error

KillContainer runs docker kill on a container.

func Pull 

func Pull(image string) error

Pull retrieves the docker image with 'docker pull'.

func RandomPort 

func RandomPort() int

RandomPort returns a random non-priviledged port.

func SetUpMySQLDatabase 

func SetUpMySQLDatabase(databaseName, connectURL string) (url string, err error)

SetUpMySQLDatabase connects mysql container with given $connectURL and also creates a new database named $databaseName A modified url used to connect the created database will be returned

func SetUpPostgreDatabase 

func SetUpPostgreDatabase(databaseName, connectURL string) (modifiedURL string, err error)

SetUpPostgreDatabase connects postgre container with given $connectURL and also creates a new database named $databaseName A modified url used to connect the created database will be returned

Types

type ContainerID 

type ContainerID string

ContainerID represents a container and offers methods like Kill or IP.

func ConnectToActiveMQ 

func ConnectToActiveMQ(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToActiveMQ starts a ActiveMQ image and passes the amqp url to the connector callback. The url will match the ip:port pattern (e.g. 123.123.123.123:4241)

func ConnectToCassandra 

func ConnectToCassandra(versionTag string, tries int, delay time.Duration, connector func(url string) bool, optionalParams ...string) (c ContainerID, err error)

ConnectToCassandra starts a Cassandra image and passes the nodes connection string to the connector callback function. The connection string will match the ip:port pattern, where port is the mapped CQL port.

func ConnectToConsul 

func ConnectToConsul(tries int, delay time.Duration, connector func(address string) bool) (c ContainerID, err error)

ConnectToConsul starts a Consul image and passes the address to the connector callback function.

func ConnectToElasticSearch 

func ConnectToElasticSearch(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToElasticSearch starts an ElasticSearch image and passes the database url to the connector callback function. The url will match the ip:port pattern (e.g. 123.123.123.123:4241)

func ConnectToMockserver 

func ConnectToMockserver(tries int, delay time.Duration, mockConnector func(url string) bool, proxyConnector func(url string) bool) (c ContainerID, err error)

ConnectToMockserver starts a Mockserver image and passes the mock and proxy urls to the connector callback functions. The urls will match the http://ip:port pattern (e.g. http://123.123.123.123:4241)

func ConnectToMongoDB 

func ConnectToMongoDB(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToMongoDB starts a MongoDB image and passes the database url to the connector callback. The url will match the ip:port pattern (e.g. 123.123.123.123:4241)

func ConnectToMySQL 

func ConnectToMySQL(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToMySQL starts a MySQL image and passes the database url to the connector callback function. The url will match the username:password@tcp(ip:port) pattern (e.g. `root:root@tcp(123.123.123.123:3131)`)

func ConnectToNSQLookupd 

func ConnectToNSQLookupd(tries int, delay time.Duration, connector func(ip string, httpPort int, tcpPort int) bool) (c ContainerID, err error)

ConnectToNSQLookupd starts a NSQ image with `/nsqlookupd` running and passes the IP, HTTP port, and TCP port to the connector callback function. The url will match the ip pattern (e.g. 123.123.123.123).

func ConnectToNSQd 

func ConnectToNSQd(tries int, delay time.Duration, connector func(ip string, httpPort int, tcpPort int) bool) (c ContainerID, err error)

ConnectToNSQd starts a NSQ image with `/nsqd` running and passes the IP, HTTP port, and TCP port to the connector callback function. The url will match the ip pattern (e.g. 123.123.123.123).

func ConnectToPostgreSQL 

func ConnectToPostgreSQL(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToPostgreSQL starts a PostgreSQL image and passes the database url to the connector callback.

func ConnectToRabbitMQ 

func ConnectToRabbitMQ(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToRabbitMQ starts a RabbitMQ image and passes the amqp url to the connector callback. The url will match the ip:port pattern (e.g. 123.123.123.123:4241)

func ConnectToRedis 

func ConnectToRedis(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToRedis starts a Redis image and passes the database url to the connector callback function. The url will match the ip:port pattern (e.g. 123.123.123.123:6379)

func ConnectToRethinkDB 

func ConnectToRethinkDB(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToRethinkDB starts a RethinkDB image and passes the database url to the connector callback. The url will match the ip:port pattern (e.g. 123.123.123.123:4241)

func ConnectToZooKeeper 

func ConnectToZooKeeper(tries int, delay time.Duration, connector func(url string) bool) (c ContainerID, err error)

ConnectToZooKeeper starts a ZooKeeper image and passes the nodes connection string to the connector callback function. The connection string will match the ip:port pattern.

func SetupActiveMQContainer 

func SetupActiveMQContainer() (c ContainerID, ip string, port int, err error)

SetupActiveMQContainer sets up a real ActiveMQ instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupCassandraContainer 

func SetupCassandraContainer(versionTag string, optionalParams ...string) (c ContainerID, ip string, port int, err error)

SetupCassandraContainer sets up a real Cassandra node for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupConsulContainer 

func SetupConsulContainer() (c ContainerID, ip string, port int, err error)

SetupConsulContainer sets up a real Consul instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupContainer 

func SetupContainer(image string, port int, timeout time.Duration, start func() (string, error)) (c ContainerID, ip string, err error)

SetupContainer sets up a container, using the start function to run the given image. It also looks up the IP address of the container, and tests this address with the given port and timeout. It returns the container ID and its IP address, or makes the test fail on error.

func SetupCustomContainer 

func SetupCustomContainer(imageName string, exposedPort int, timeOut time.Duration, extraDockerArgs ...string) (c ContainerID, ip string, localPort int, err error)

SetupCustomContainer sets up a real an instance of the given image for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupElasticSearchContainer 

func SetupElasticSearchContainer() (c ContainerID, ip string, port int, err error)

SetupElasticSearchContainer sets up a real ElasticSearch instance for testing purposes using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupMockserverContainer 

func SetupMockserverContainer() (c ContainerID, ip string, mockPort, proxyPort int, err error)

SetupMockserverContainer sets up a real Mockserver instance for testing purposes using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupMongoContainer 

func SetupMongoContainer() (c ContainerID, ip string, port int, err error)

SetupMongoContainer sets up a real MongoDB instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupMultiportContainer 

func SetupMultiportContainer(image string, ports []int, timeout time.Duration, start func() (string, error)) (c ContainerID, ip string, err error)

SetupMultiportContainer sets up a container, using the start function to run the given image. It also looks up the IP address of the container, and tests this address with the given ports and timeout. It returns the container ID and its IP address, or makes the test fail on error.

func SetupMySQLContainer 

func SetupMySQLContainer() (c ContainerID, ip string, port int, err error)

SetupMySQLContainer sets up a real MySQL instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupNSQLookupdContainer 

func SetupNSQLookupdContainer() (c ContainerID, ip string, tcpPort int, httpPort int, err error)

SetupNSQLookupdContainer sets up a real NSQ instance for testing purposes using a Docker container and executing `/nsqlookupd`. It returns the container ID and its IP address, or makes the test fail on error.

func SetupNSQdContainer 

func SetupNSQdContainer() (c ContainerID, ip string, tcpPort int, httpPort int, err error)

SetupNSQdContainer sets up a real NSQ instance for testing purposes using a Docker container and executing `/nsqd`. It returns the container ID and its IP address, or makes the test fail on error.

func SetupPostgreSQLContainer 

func SetupPostgreSQLContainer() (c ContainerID, ip string, port int, err error)

SetupPostgreSQLContainer sets up a real PostgreSQL instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupRabbitMQContainer 

func SetupRabbitMQContainer() (c ContainerID, ip string, port int, err error)

SetupRabbitMQContainer sets up a real RabbitMQ instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupRedisContainer 

func SetupRedisContainer() (c ContainerID, ip string, port int, err error)

SetupRedisContainer sets up a real Redis instance for testing purposes using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupRethinkDBContainer 

func SetupRethinkDBContainer() (c ContainerID, ip string, port int, err error)

SetupRethinkDBContainer sets up a real RethinkDB instance for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func SetupZooKeeperContainer 

func SetupZooKeeperContainer() (c ContainerID, ip string, port int, err error)

SetupZooKeeperContainer sets up a real ZooKeeper node for testing purposes, using a Docker container. It returns the container ID and its IP address, or makes the test fail on error.

func (ContainerID) IP 

func (c ContainerID) IP() (string, error)

IP retrieves the container's IP address.

func (ContainerID) Kill 

func (c ContainerID) Kill() error

Kill runs "docker kill" on the container.

func (ContainerID) KillRemove 

func (c ContainerID) KillRemove() error

KillRemove calls Kill on the container, and then Remove if there was no error.

func (ContainerID) Remove 

func (c ContainerID) Remove() error

Remove runs "docker rm" on the container

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.