goose

README

goose

goose is a database migration tool.

You can manage your database's evolution by creating incremental SQL or Go scripts.

Install

$ go get bitbucket.org/liamstask/goose

This will install the goose binary to your $GOPATH/bin directory.

Usage

goose provides several commands to help manage your database schema.

create

Create a new migration script.

$ goose create AddSomeColumns
$ goose: created db/migrations/20130106093224_AddSomeColumns.go

Edit the newly created script to define the behavior of your migration.

up

Apply all available migrations.

$ goose up
$ goose: migrating db environment 'development', current version: 0, target: 3
$ OK    001_basics.sql
$ OK    002_next.sql
$ OK    003_and_again.go

down

Roll back a single migration from the current version.

$ goose down
$ goose: migrating db environment 'development', current version: 3, target: 2
$ OK    003_and_again.go

redo

Roll back the most recently applied migration, then run it again.

$ goose redo
$ goose: migrating db environment 'development', current version: 3, target: 2
$ OK    003_and_again.go
$ goose: migrating db environment 'development', current version: 2, target: 3
$ OK    003_and_again.go

status

Print the status of all migrations:

$ goose status
$ goose: status for environment 'development'
$   Applied At                  Migration
$   =======================================
$   Sun Jan  6 11:25:03 2013 -- 001_basics.sql
$   Sun Jan  6 11:25:03 2013 -- 002_next.sql
$   Pending                  -- 003_and_again.go

goose -h provides more detailed info on each command.

Migrations

goose supports migrations written in SQL or in Go.

SQL Migrations

A sample SQL migration looks like:

:::sql
-- +goose Up
CREATE TABLE post (
    id int NOT NULL,
    title text,
    body text,
    PRIMARY KEY(id)
);

-- +goose Down
DROP TABLE post;

Notice the annotations in the comments. Any statements following -- +goose Up will be executed as part of a forward migration, and any statements following -- +goose Down will be executed as part of a rollback.

Go Migrations

A sample Go migration looks like:

:::go
package main

import (
    "database/sql"
    "fmt"
)

func Up_20130106222315(txn *sql.Tx) {
    fmt.Println("Hello from migration 20130106222315 Up!")
}

func Down_20130106222315(txn *sql.Tx) {
    fmt.Println("Hello from migration 20130106222315 Down!")
}

Up_20130106222315() will be executed as part of a forward migration, and Down_20130106222315() will be executed as part of a rollback.

The numeric portion of the function name (20130106222315) must be the leading portion of migration's filename, such as 20130106222315_descriptive_name.go. goose create does this by default.

A transaction is provided, rather than the DB instance directly, since goose also needs to record the schema version within the same transaction. Each migration should run as a single transaction to ensure DB integrity, so it's good practice anyway.

Configuration

goose expects you to maintain a folder (typically called "db"), which contains the following:

• a dbconf.yml file that describes the database configurations you'd like to use
• a folder called "migrations" which contains .sql and/or .go scripts that implement your migrations

You may use the -path option to specify an alternate location for the folder containing your config and migrations.

A sample dbconf.yml looks like

development:
    driver: postgres
    open: user=liam dbname=tester sslmode=disable

Here, development specifies the name of the environment, and the driver and open elements are passed directly to database/sql to access the specified database.

You may include as many environments as you like, and you can use the -env command line option to specify which one to use. goose defaults to using an environment called development.

goose will expand environment variables in the open element. For an example, see the Heroku section below.

Other Drivers

goose knows about some common SQL drivers, but it can still be used to run Go-based migrations with any driver supported by database/sql. An import path and known dialect are required.

Currently, available dialects are: "postgres" or "mysql"

To run Go-based migrations with another driver, specify its import path and dialect, as shown below.

customdriver:
    driver: custom
    open: custom open string
    import: github.com/custom/driver
    dialect: mysql

NOTE: Because migrations written in SQL are executed directly by the goose binary, only drivers compiled into goose may be used for these migrations.

Using goose with Heroku

These instructions assume that you're using Keith Rarick's Heroku Go buildpack. First, add a file to your project called (e.g.) install_goose.go to trigger building of the goose executable during deployment, with these contents:

// use build constraints to work around http://code.google.com/p/go/issues/detail?id=4210
// +build heroku
package main

import _ "bitbucket.org/liamstask/goose"

Set up your Heroku database(s) as usual.

Then make use of environment variable expansion in your dbconf.yml:

production:
    driver: postgres
    open: $DATABASE_URL

To run goose in production, use heroku run:

heroku run goose -env production up

Contributors

Thank you!

• Josh Bleecher Snyder (josharian)
• Abigail Walthall (ghthor)
• Daniel Heath (danielrheath)
• Chris Baynes (chris_baynes)
• Michael Gerow (gerow)
• Vytautas Šaltenis (rtfb)

Documentation

Index

• Variables
• func CreateMigration(name, migrationType, dir string, t time.Time) (path string, err error)
• func EnsureDBVersion(conf *DBConf, db *sql.DB) (int64, error)
• func GetDBVersion(conf *DBConf) (version int64, err error)
• func GetMostRecentDBVersion(dirpath string) (version int64, err error)
• func GetPreviousDBVersion(dirpath string, version int64) (previous int64, err error)
• func NumericComponent(name string) (int64, error)
• func Run(arguments ...string)
• func RunMigrations(conf *DBConf, migrationsDir string, target int64) (err error)
• type Command
  • func (c *Command) Exec(args []string)
• type CreateDBDriver
• type CreateSqlDialect
• type DBConf
  • func NewDBConf(path, env, tag string) (*DBConf, error)
• type DBDriver
  • func NewDBDriver(name, open string) DBDriver
  • func (drv *DBDriver) IsValid() bool
• type MSSQLDialect
  • func (pg *MSSQLDialect) CreateVersionTableSql() string
  • func (pg *MSSQLDialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)
  • func (pg *MSSQLDialect) InsertVersionSql() string
• type Migration
  • func CollectMigrations(dirpath string, current, target int64) (m []*Migration, err error)
• type MigrationRecord
• type MySqlDialect
  • func (m *MySqlDialect) CreateVersionTableSql() string
  • func (m *MySqlDialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)
  • func (m *MySqlDialect) InsertVersionSql() string
• type PostgresDialect
  • func (pg *PostgresDialect) CreateVersionTableSql() string
  • func (pg *PostgresDialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)
  • func (pg *PostgresDialect) InsertVersionSql() string
• type SqlDialect
  • func DialectByName(d string) SqlDialect
• type Sqlite3Dialect
  • func (m *Sqlite3Dialect) CreateVersionTableSql() string
  • func (m *Sqlite3Dialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)
  • func (m *Sqlite3Dialect) InsertVersionSql() string
• type StatusData

Variables

var (
	ErrTableDoesNotExist = errors.New("table does not exist")
	ErrNoPreviousVersion = errors.New("no previous version found")
)

var Commands = []*Command{
	upCmd,
	downCmd,
	redoCmd,
	statusCmd,
	createCmd,
	dbVersionCmd,
	cleanCmd,
	resetCmd,
}

var (
	DBDrivers = map[string]CreateDBDriver{}
)

var GooseFlagSet = flag.NewFlagSet(os.Args[0], flag.ExitOnError)

global options. available to any subcommands.

var ReadDbConf func() (*DBConf, error)

var (
	SqlDialects = map[string]CreateSqlDialect{"postgres": func() SqlDialect { return &PostgresDialect{} },
		"mysql":  func() SqlDialect { return &MySqlDialect{} },
		"sqlite": func() SqlDialect { return &Sqlite3Dialect{} }}
)

Functions

func CreateMigration

func CreateMigration(name, migrationType, dir string, t time.Time) (path string, err error)

func EnsureDBVersion

func EnsureDBVersion(conf *DBConf, db *sql.DB) (int64, error)

retrieve the current version for this DB. Create and initialize the DB version table if it doesn't exist.

func GetDBVersion

func GetDBVersion(conf *DBConf) (version int64, err error)

wrapper for EnsureDBVersion for callers that don't already have their own DB instance

func GetMostRecentDBVersion

func GetMostRecentDBVersion(dirpath string) (version int64, err error)

helper to identify the most recent possible version within a folder of migration scripts

func GetPreviousDBVersion

func GetPreviousDBVersion(dirpath string, version int64) (previous int64, err error)

func NumericComponent

func NumericComponent(name string) (int64, error)

look for migration scripts with names in the form:

XXX_descriptivename.ext

where XXX specifies the version number and ext specifies the type of migration

func Run

func Run(arguments ...string)

func RunMigrations

func RunMigrations(conf *DBConf, migrationsDir string, target int64) (err error)

Types

type Command

type Command struct {
	Run  func(cmd *Command, args ...string)
	Flag flag.FlagSet

	Name  string
	Usage string

	Summary string
	Help    string
}

shamelessly snagged from the go tool each command gets its own set of args, defines its own entry point, and provides its own help

func (*Command) Exec

func (c *Command) Exec(args []string)

type CreateDBDriver

type CreateDBDriver func(name, open string) DBDriver

type CreateSqlDialect

type CreateSqlDialect func() SqlDialect

type DBConf

type DBConf struct {
	MigrationsDir string
	Env           string
	Driver        DBDriver
}

func NewDBConf

func NewDBConf(path, env, tag string) (*DBConf, error)

extract configuration details from the given file

type DBDriver

type DBDriver struct {
	Name    string
	OpenStr string
	Import  string
	Dialect SqlDialect
}

DBDriver encapsulates the info needed to work with a specific database driver

func NewDBDriver

func NewDBDriver(name, open string) DBDriver

Create a new DBDriver and populate driver specific fields for drivers that we know about. Further customization may be done in NewDBConf

func (*DBDriver) IsValid

func (drv *DBDriver) IsValid() bool

ensure we have enough info about this driver

type MSSQLDialect

type MSSQLDialect struct{}

func (*MSSQLDialect) CreateVersionTableSql

func (pg *MSSQLDialect) CreateVersionTableSql() string

func (*MSSQLDialect) DbVersionQuery

func (pg *MSSQLDialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)

func (*MSSQLDialect) InsertVersionSql

func (pg *MSSQLDialect) InsertVersionSql() string

type Migration

type Migration struct {
	Version  int64
	Next     int64  // next version, or -1 if none
	Previous int64  // previous version, -1 if none
	Source   string // path to .go or .sql script
}

func CollectMigrations

func CollectMigrations(dirpath string, current, target int64) (m []*Migration, err error)

collect all the valid looking migration scripts in the migrations folder, and key them by version

type MigrationRecord

type MigrationRecord struct {
	VersionId int64
	TStamp    time.Time
	IsApplied bool // was this a result of up() or down()
}

type MySqlDialect

type MySqlDialect struct{}

func (*MySqlDialect) CreateVersionTableSql

func (m *MySqlDialect) CreateVersionTableSql() string

func (*MySqlDialect) DbVersionQuery

func (m *MySqlDialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)

func (*MySqlDialect) InsertVersionSql

func (m *MySqlDialect) InsertVersionSql() string

type PostgresDialect

type PostgresDialect struct{}

func (*PostgresDialect) CreateVersionTableSql

func (pg *PostgresDialect) CreateVersionTableSql() string

func (*PostgresDialect) DbVersionQuery

func (pg *PostgresDialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)

func (*PostgresDialect) InsertVersionSql

func (pg *PostgresDialect) InsertVersionSql() string

type SqlDialect

type SqlDialect interface {
	CreateVersionTableSql() string // sql string to create the goose_db_version table
	InsertVersionSql() string      // sql string to insert the initial version table row
	DbVersionQuery(db *sql.DB) (*sql.Rows, error)
}

SqlDialect abstracts the details of specific SQL dialects for goose's few SQL specific statements

func DialectByName

func DialectByName(d string) SqlDialect

drivers that we don't know about can ask for a dialect by name

type Sqlite3Dialect

type Sqlite3Dialect struct{}

func (*Sqlite3Dialect) CreateVersionTableSql

func (m *Sqlite3Dialect) CreateVersionTableSql() string

func (*Sqlite3Dialect) DbVersionQuery

func (m *Sqlite3Dialect) DbVersionQuery(db *sql.DB) (*sql.Rows, error)

func (*Sqlite3Dialect) InsertVersionSql

func (m *Sqlite3Dialect) InsertVersionSql() string

type StatusData

type StatusData struct {
	Source string
	Status string
}