goose

package module
v3.20.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2023 License: MIT Imports: 23 Imported by: 0

README

goose Goose CI Go Reference

Goose is a database migration tool. Manage your database schema by creating incremental SQL changes or Go functions.

Starting with v3.0.0 this project adds Go module support, but maintains backwards compatibility with older v2.x.y tags.

Goose supports embedding SQL migrations, which means you'll need go1.16 and up. If using go1.15 or lower, then pin v3.0.1.

Goals of this fork

github.com/SergeiSkv/goose/v3 is a fork of bitbucket.org/liamstask/goose with the following changes:

  • No config files
  • Default goose binary can migrate SQL files only
  • Go migrations:
    • We don't go build Go migrations functions on-the-fly from within the goose binary
    • Instead, we let you create your own custom goose binary, register your Go migration functions explicitly and run complex migrations with your own pgx.Conn connection
    • Go migration functions let you run your code within an SQL transaction, if you use the *sql.Tx argument
  • The goose pkg is decoupled from the binary:
    • goose pkg doesn't register any SQL drivers anymore, thus no driver panic() conflict within your codebase!
    • goose pkg doesn't have any vendor dependencies anymore
  • We use timestamped migrations by default but recommend a hybrid approach of using timestamps in the development process and sequential versions in production.
  • Supports missing (out-of-order) migrations with the -allow-missing flag, or if using as a library supply the functional option goose.WithAllowMissing() to Up, UpTo or UpByOne.
  • Supports applying ad-hoc migrations without tracking them in the schema table. Useful for seeding a database after migrations have been applied. Use -no-versioning flag or the functional option goose.WithNoVersioning().

Install

$ go install github.com/SergeiSkv/goose/v3/cmd/goose@latest

This will install the goose binary to your $GOPATH/bin directory.

For a lite version of the binary without DB connection dependent commands, use the exclusive build tags:

$ go build -tags='no_postgres no_mysql no_sqlite3' -o goose ./cmd/goose

For macOS users goose is available as a Homebrew Formulae:

$ brew install goose

See the docs for more installation instructions.

Usage

Usage: goose [OPTIONS] DRIVER DBSTRING COMMAND

Drivers:
    postgres
    mysql
    sqlite3
    mssql
    redshift
    tidb
    clickhouse
    vertica

Examples:
    goose sqlite3 ./foo.db status
    goose sqlite3 ./foo.db create init sql
    goose sqlite3 ./foo.db create add_some_column sql
    goose sqlite3 ./foo.db create fetch_user_data go
    goose sqlite3 ./foo.db up

    goose postgres "user=postgres password=postgres dbname=postgres sslmode=disable" status
    goose mysql "user:password@/dbname?parseTime=true" status
    goose redshift "postgres://user:password@qwerty.us-east-1.redshift.amazonaws.com:5439/db" status
    goose tidb "user:password@/dbname?parseTime=true" status
    goose mssql "sqlserver://user:password@dbname:1433?database=master" status
    goose clickhouse "tcp://127.0.0.1:9000" status
    goose vertica "vertica://user:password@localhost:5433/dbname?connection_load_balance=1" status

Options:

  -allow-missing
    	applies missing (out-of-order) migrations
  -certfile string
    	file path to root CA's certificates in pem format (only supported on mysql)
  -dir string
    	directory with migration files (default ".")
  -h	print help
  -no-versioning
    	apply migration commands with no versioning, in file order, from directory pointed to
  -s	use sequential numbering for new migrations
  -ssl-cert string
    	file path to SSL certificates in pem format (only supported on mysql)
  -ssl-key string
    	file path to SSL key in pem format (only supported on mysql)
  -table string
    	migrations table name (default "goose_db_version")
  -v	enable verbose mode
  -version
    	print version

Commands:
    up                   Migrate the DB to the most recent version available
    up-by-one            Migrate the DB up by 1
    up-to VERSION        Migrate the DB to a specific VERSION
    down                 Roll back the version by 1
    down-to VERSION      Roll back to a specific VERSION
    redo                 Re-run the latest migration
    reset                Roll back all migrations
    status               Dump the migration status for the current DB
    version              Print the current version of the database
    create NAME [sql|go] Creates new migration file with the current timestamp
    fix                  Apply sequential ordering to migrations

create

Create a new SQL migration.

$ goose create add_some_column sql
$ Created new file: 20170506082420_add_some_column.sql

Edit the newly created file to define the behavior of your migration.

You can also create a Go migration, if you then invoke it with your own goose binary:

$ goose create fetch_user_data go
$ Created new file: 20170506082421_fetch_user_data.go

up

Apply all available migrations.

$ goose up
$ OK    001_basics.sql
$ OK    002_next.sql
$ OK    003_and_again.go

up-to

Migrate up to a specific version.

$ goose up-to 20170506082420
$ OK    20170506082420_create_table.sql

up-by-one

Migrate up a single migration from the current version

$ goose up-by-one
$ OK    20170614145246_change_type.sql

down

Roll back a single migration from the current version.

$ goose down
$ OK    003_and_again.go

down-to

Roll back migrations to a specific version.

$ goose down-to 20170506082527
$ OK    20170506082527_alter_column.sql

redo

Roll back the most recently applied migration, then run it again.

$ goose redo
$ OK    003_and_again.go
$ OK    003_and_again.go

status

Print the status of all migrations:

$ goose status
$   Applied At                  Migration
$   =======================================
$   Sun Jan  6 11:25:03 2013 -- 001_basics.sql
$   Sun Jan  6 11:25:03 2013 -- 002_next.sql
$   Pending                  -- 003_and_again.go

Note: for MySQL parseTime flag must be enabled.

Note: for MySQL multiStatements must be enabled. This is required when writing multiple queries separated by ';' characters in a single sql file.

version

Print the current version of the database:

$ goose version
$ goose: version 002

Migrations

goose supports migrations written in SQL or in Go.

SQL Migrations

A sample SQL migration looks like:

-- +goose Up
CREATE TABLE post (
    id int NOT NULL,
    title text,
    body text,
    PRIMARY KEY(id)
);

-- +goose Down
DROP TABLE post;

Each migration file must have exactly one -- +goose Up annotation. The -- +goose Down annotation is optional. If the file has both annotations, then the -- +goose Up annotation must come first.

Notice the annotations in the comments. Any statements following -- +goose Up will be executed as part of a forward migration, and any statements following -- +goose Down will be executed as part of a rollback.

By default, all migrations are run within a transaction. Some statements like CREATE DATABASE, however, cannot be run within a transaction. You may optionally add -- +goose NO TRANSACTION to the top of your migration file in order to skip transactions within that specific migration file. Both Up and Down migrations within this file will be run without transactions.

By default, SQL statements are delimited by semicolons - in fact, query statements must end with a semicolon to be properly recognized by goose.

More complex statements (PL/pgSQL) that have semicolons within them must be annotated with -- +goose StatementBegin and -- +goose StatementEnd to be properly recognized. For example:

-- +goose Up
-- +goose StatementBegin
CREATE OR REPLACE FUNCTION histories_partition_creation( DATE, DATE )
returns void AS $$
DECLARE
  create_query text;
BEGIN
  FOR create_query IN SELECT
      'CREATE TABLE IF NOT EXISTS histories_'
      || TO_CHAR( d, 'YYYY_MM' )
      || ' ( CHECK( created_at >= timestamp '''
      || TO_CHAR( d, 'YYYY-MM-DD 00:00:00' )
      || ''' AND created_at < timestamp '''
      || TO_CHAR( d + INTERVAL '1 month', 'YYYY-MM-DD 00:00:00' )
      || ''' ) ) inherits ( histories );'
    FROM generate_series( $1, $2, '1 month' ) AS d
  LOOP
    EXECUTE create_query;
  END LOOP;  -- LOOP END
END;         -- FUNCTION END
$$
language plpgsql;
-- +goose StatementEnd

Embedded sql migrations

Go 1.16 introduced new feature: compile-time embedding files into binary and corresponding filesystem abstraction.

This feature can be used only for applying existing migrations. Modifying operations such as fix and create will continue to operate on OS filesystem even if using embedded files. This is expected behaviour because io/fs interfaces allows read-only access.

Make sure to configure the correct SQL dialect, see dialect.go for supported SQL dialects.

Example usage, assuming that SQL migrations are placed in the migrations directory:

package main

import (
    "database/sql"
    "embed"

    "github.com/SergeiSkv/goose/v3/v3"
)

//go:embed migrations/*.sql
var embedMigrations embed.FS

func main() {
    var db*pgx.Conn
    // setup database

    goose.SetBaseFS(embedMigrations)

    if err := goose.SetDialect("postgres"); err != nil {
        panic(err)
    }

    if err := goose.Up(db, "migrations"); err != nil {
        panic(err)
    }

    // run app
}

Note that we pass "migrations" as directory argument in Up because embedding saves directory structure.

Go Migrations

  1. Create your own goose binary, see example
  2. Import github.com/SergeiSkv/goose/v3
  3. Register your migration functions
  4. Run goose command, ie. goose.Up(db*pgx.Conn, dir string)

A sample Go migration 00002_users_add_email.go file looks like:

package migrations

import (
	"database/sql"

	"github.com/SergeiSkv/goose/v3"
)

func init() {
	goose.AddMigration(Up, Down)
}

func Up(tx pgx.Tx) error {
	_, err := tx.Exec("UPDATE users SET username='admin' WHERE username='root';")
	if err != nil {
		return err
	}
	return nil
}

func Down(tx pgx.Tx) error {
	_, err := tx.Exec("UPDATE users SET username='root' WHERE username='admin';")
	if err != nil {
		return err
	}
	return nil
}

Note that Go migration files must begin with a numeric value, followed by an underscore, and must not end with *_test.go.

Development

This can be used to build local goose binaries without having the latest Go version installed locally.

DOCKER_BUILDKIT=1  docker build -f Dockerfile.local --output bin .

Hybrid Versioning

Please, read the versioning problem first.

By default, if you attempt to apply missing (out-of-order) migrations goose will raise an error. However, If you want to apply these missing migrations pass goose the -allow-missing flag, or if using as a library supply the functional option goose.WithAllowMissing() to Up, UpTo or UpByOne.

However, we strongly recommend adopting a hybrid versioning approach, using both timestamps and sequential numbers. Migrations created during the development process are timestamped and sequential versions are ran on production. We believe this method will prevent the problem of conflicting versions when writing software in a team environment.

To help you adopt this approach, create will use the current timestamp as the migration version. When you're ready to deploy your migrations in a production environment, we also provide a helpful fix command to convert your migrations into sequential order, while preserving the timestamp ordering. We recommend running fix in the CI pipeline, and only when the migrations are ready for production.

Credit

The gopher mascot was designed by Renée French / CC 3.0. For more info check out the Go Blog. Adapted by Ellen.

License

Licensed under MIT License

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoCurrentVersion when a current migration version is not found.
	ErrNoCurrentVersion = errors.New("no current version found")
	// ErrNoNextVersion when the next migration version is not found.
	ErrNoNextVersion = errors.New("no next version found")
	// MaxVersion is the maximum allowed version.
	MaxVersion int64 = math.MaxInt64
)

Functions

func AddMigration 

func AddMigration(up, down GoMigration)

AddMigration adds Go migrations.

func AddMigrationNoTx 

func AddMigrationNoTx(up, down GoMigrationNoTx)

AddMigrationNoTx adds Go migrations that will be run outside transaction.

func AddNamedMigration 

func AddNamedMigration(filename string, up, down GoMigration)

AddNamedMigration adds named Go migrations.

func AddNamedMigrationNoTx 

func AddNamedMigrationNoTx(filename string, up, down GoMigrationNoTx)

AddNamedMigrationNoTx adds named Go migrations that will be run outside transaction.

func Create 

func Create(db *pgx.Conn, dir, name, migrationType string) error

Create writes a new blank migration file.

func CreateWithTemplate 

func CreateWithTemplate(_ *pgx.Conn, dir string, tmpl *template.Template, name, migrationType string) error

Create writes a new blank migration file.

func Down 

func Down(db *pgx.Conn, dir string, opts ...OptionsFunc) error

Down rolls back a single migration from the current version.

func DownTo 

func DownTo(db *pgx.Conn, dir string, version int64, opts ...OptionsFunc) error

DownTo rolls back migrations to a specific version.

func EnsureDBVersion 

func EnsureDBVersion(db *pgx.Conn) (int64, error)

EnsureDBVersion retrieves the current version for this DB. Create and initialize the DB version table if it doesn't exist.

func Fix 

func Fix(dir string) error

func GetDBVersion 

func GetDBVersion(db *pgx.Conn) (int64, error)

GetDBVersion is an alias for EnsureDBVersion, but returns -1 in error.

func NumericComponent 

func NumericComponent(name string) (int64, error)

NumericComponent looks for migration scripts with names in the form: XXX_descriptivename.ext where XXX specifies the version number and ext specifies the type of migration

func OpenDBWithDriver 

func OpenDBWithDriver(driver string, dbstring string) (*pgx.Conn, error)

OpenDBWithDriver creates a connection to a database, and modifies goose internals to be compatible with the supplied driver by calling SetDialect.

func Redo 

func Redo(db *pgx.Conn, dir string, opts ...OptionsFunc) error

Redo rolls back the most recently applied migration, then runs it again.

func Reset 

func Reset(db *pgx.Conn, dir string, opts ...OptionsFunc) error

Reset rolls back all migrations

func Run 

func Run(command string, db *pgx.Conn, dir string, args ...string) error

Run runs a goose command.

func RunWithOptions 

func RunWithOptions(command string, db *pgx.Conn, dir string, args []string, options ...OptionsFunc) error

Run runs a goose command with options.

func SetBaseFS 

func SetBaseFS(fsys fs.FS)

SetBaseFS sets a base FS to discover migrations. It can be used with 'embed' package. Calling with 'nil' argument leads to default behaviour: discovering migrations from os filesystem. Note that modifying operations like Create will use os filesystem anyway.

func SetDialect 

func SetDialect(s string) error

SetDialect sets the dialect to use for the goose package.

func SetLogger 

func SetLogger(l Logger)

SetLogger sets the logger for package output

func SetSequential 

func SetSequential(s bool)

SetSequential set whether to use sequential versioning instead of timestamp based versioning

func SetTableName 

func SetTableName(n string)

SetTableName set goose db version table name

func SetVerbose 

func SetVerbose(v bool)

SetVerbose set the goose verbosity mode

func Status 

func Status(db *pgx.Conn, dir string, opts ...OptionsFunc) error

Status prints the status of all migrations.

func TableName 

func TableName() string

TableName returns goose db version table name

func Up 

func Up(db *pgx.Conn, dir string, opts ...OptionsFunc) error

Up applies all available migrations.

func UpByOne 

func UpByOne(db *pgx.Conn, dir string, opts ...OptionsFunc) error

UpByOne migrates up by a single version.

func UpTo 

func UpTo(db *pgx.Conn, dir string, version int64, opts ...OptionsFunc) error

UpTo migrates up to a specific version.

func Version 

func Version(db *pgx.Conn, dir string, opts ...OptionsFunc) error

Version prints the current version of the database.

Types

type GoMigration 

type GoMigration func(tx pgx.Tx) error

GoMigration is a Go migration func that is run within a transaction.

type GoMigrationNoTx 

type GoMigrationNoTx func(db *pgx.Conn) error

GoMigrationNoTx is a Go migration func that is run outside a transaction.

type Logger 

type Logger interface {
	Fatal(v ...interface{})
	Fatalf(format string, v ...interface{})
	Print(v ...interface{})
	Println(v ...interface{})
	Printf(format string, v ...interface{})
}

Logger is standard logger interface

func NopLogger 

func NopLogger() Logger

NopLogger returns a logger that discards all logged output.

type Migration 

type Migration struct {
	Version              int64
	Next                 int64  // next version, or -1 if none
	Previous             int64  // previous version, -1 if none
	Source               string // path to .sql script or go file
	Registered           bool
	UseTx                bool
	UpFn, DownFn         GoMigration
	UpFnNoTx, DownFnNoTx GoMigrationNoTx
	// contains filtered or unexported fields
}

Migration struct.

func (*Migration) Down 

func (m *Migration) Down(db *pgx.Conn) error

Down runs a down migration.

func (*Migration) String 

func (m *Migration) String() string

func (*Migration) Up 

func (m *Migration) Up(db *pgx.Conn) error

Up runs an up migration.

type MigrationRecord 

type MigrationRecord struct {
	VersionID int64
	TStamp    time.Time
	IsApplied bool // was this a result of up() or down()
}

MigrationRecord struct.

type Migrations 

type Migrations []*Migration

Migrations slice.

func CollectMigrations 

func CollectMigrations(dirpath string, current, target int64) (Migrations, error)

CollectMigrations returns all the valid looking migration scripts in the migrations folder and go func registry, and key them by version.

func (Migrations) Current 

func (ms Migrations) Current(current int64) (*Migration, error)

Current gets the current migration.

func (Migrations) Last 

func (ms Migrations) Last() (*Migration, error)

Last gets the last migration.

func (Migrations) Len 

func (ms Migrations) Len() int

helpers so we can use pkg sort

func (Migrations) Less 

func (ms Migrations) Less(i, j int) bool

func (Migrations) Next 

func (ms Migrations) Next(current int64) (*Migration, error)

Next gets the next migration.

func (Migrations) Previous 

func (ms Migrations) Previous(current int64) (*Migration, error)

Previous : Get the previous migration.

func (Migrations) String 

func (ms Migrations) String() string

func (Migrations) Swap 

func (ms Migrations) Swap(i, j int)

type OptionsFunc 

type OptionsFunc func(o *options)

func WithAllowMissing 

func WithAllowMissing() OptionsFunc

func WithNoColor 

func WithNoColor(b bool) OptionsFunc

func WithNoVersioning 

func WithNoVersioning() OptionsFunc

Source Files

View all Source files

Directories

Path Synopsis
cmd
goose
examples
go-migrations
internal
cfg
check
dialect
dialect/dialectquery
migrationstats
migrationstats/migrationstatsos
sqlparser
testdb

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.