burrow

README

burrow

Go database migration tool.

When gophers migrate, they burrow!

Design goals

• Simple to use.
• No downgrade scripts. — Making every migration bi-directional is an immense amount of work that I've literally never seen done properly. And what's worse, it's bad practice. Just make sure your upgrades are non-destructive, and you can avoid this problem entirely.
• Database agnostic — Should work with any database/sql driver.
• Allow reading from multiple migration directories. — Sometimes you want to interweave migrations in some environments (notably test environments).

Documentation

Overview

Package burrow allows for no-frills database migrations.

Index

• Constants
• func ValidateSources(ctx context.Context, source ...Source) error
• type MemorySource
  • func (s MemorySource) Index(context.Context) ([]uint, error)
  • func (s MemorySource) Migration(_ context.Context, id uint) (*Migration, error)
• type Migration
  • func (m *Migration) Display() string
• type Migrator
  • func New(db *sql.DB, options ...Option) (*Migrator, error)
  • func (m *Migrator) AddSource(s Source)
  • func (m *Migrator) Migrate(ctx context.Context) (int, error)
  • func (m *Migrator) SetMinVer(ver uint)
• type Option
  • func Exclude(pattern string) Option
  • func Executable(pattern string) Option
  • func WithLogger(logger *slog.Logger) Option
  • func WithStateTable(table string) Option
• type Source
  • func FSSource(fsys fs.FS, options ...Option) (Source, error)
  • func FileSource(dir string, options ...Option) (Source, error)

Constants

const (
	// DialctNone is the default, and indicates that burrow will attempt to
	// detect the dialect automatically.
	DialectNone dialect = iota
	// DialectMySQL enables the use of GET_LOCK() and RELEASE_LOCK() in addition
	// to row-level locking on the migrations table, since MySQL doesn't
	// properly support normal transactions for table creation and alternation
	// queries.
	DialectMySQL
	// DialectPostgres enables Postgres-specific features.
	DialectPostgres
	// DialectPgx enables Postgres-specific features using the pgx driver, with
	// COPY support.
	DialectPgx
	// DialectSQLite enables SQLite-specific features.
	DialectSQLite
)

Supported SQL Dialects. Pass one of these as an argument to New to enable dialect-specific features.

const DefaultStateTable = "migrations"

DefaultStateTable is the default name of the table where migration state is stored. It is created the first time Migrator.Migrate is called.

const ErrUnimplemented = customError("unimplemented")

ErrUnimplemented may be returned by [Migration.PreRun], [Migration.Run], or [Migration.GetBody] to indicate that the respective method is not implemented, and that the migration should fall back to other methods.

Functions

func ValidateSources

func ValidateSources(ctx context.Context, source ...Source) error

ValidateSources does minimal validation of the sources. Notably, it checks that the migration IDs are unique across all sources. There is normally no need to call this function, as Migrator.Migrate does the same checks, but it can be useful to do these checks separataely, such as in a build pipeline test.

Types

type MemorySource

type MemorySource map[uint]string

MemorySource is a Source implemented by an in-memory map

func (MemorySource) Index

func (s MemorySource) Index(context.Context) ([]uint, error)

Index returns the list of migration IDs in the map.

func (MemorySource) Migration

func (s MemorySource) Migration(_ context.Context, id uint) (*Migration, error)

Migration returns the migration with the given ID, or an error if not found.

type Migration

type Migration struct {
	Version uint
	// Name is a descriptive name, such as full path to the migration, used for
	// debugging/informational purposes.
	Name string
	// Body is expected to return the SQL to execute for the migration.
	Body io.Reader

	// PreRun is an optional function to run before executing [Body] or [Run].
	// It is primarily intended for migrations that need to download large data
	// sets (i.e. from a cloud storage bucket) or perform other expensive
	// operations before running the migration. By putting such work in PreRun,
	// it may be run concurrently with other migrations or other PreRun
	// functions. If PreRun returns an error, the migration itself will fail.
	PreRun func(context.Context) error

	// GetBody is an optional function to get the body of the migration. If set,
	// [Body] is ignored.
	GetBody func(context.Context) (io.Reader, error)

	// Run is an optional function to run instead of executing [Body] as SQL. If
	// set, [Body] is ignored.
	Run func(context.Context, *sql.Tx) error
}

func (*Migration) Display

func (m *Migration) Display() string

type Migrator

type Migrator struct {
	// contains filtered or unexported fields
}

Migrator is an instance that applies migrations to a database. A Migrator instance is stateful, and should only be used once after initialization. If you need to apply the same migration to multiple databases, or to the same database multiple times, create one Migrator instance for migration run.

func New

func New(db *sql.DB, options ...Option) (*Migrator, error)

New returns a new Migrator that will apply changes to db.

func (*Migrator) AddSource

func (m *Migrator) AddSource(s Source)

AddSource adds a source to m. You may provide multiple sources, but the migration IDs returned must be unique across all sources.

func (*Migrator) Migrate

func (m *Migrator) Migrate(ctx context.Context) (int, error)

Migrate performs the actual migration. It returns the number of successful migration scripts applied.

Migrate will create the state table if it does not exist. It will also lock the state table for each migration. If Migrate returns a non-nil error, it means that all migrations up to that point were applied successfully.

func (*Migrator) SetMinVer

func (m *Migrator) SetMinVer(ver uint)

SetMinVer sets the minimum version to apply. It is meant to be used when adding migrations to an existing system, to avoid applying migrations that were already applied manually.

type Option

type Option interface {
	// contains filtered or unexported methods
}

Option is a functional that can be used to configure a Burrow instance or source.

func Exclude

func Exclude(pattern string) Option

Exclude returns an Option that excludes migrations matching the given regular expression pattern.

func Executable

func Executable(pattern string) Option

Executable returns an Option that matches files which should be executed directly, rather than interpreted as SQL.

Example:

source, err := FileSource("migrations", Executable(`\.sh$`))

Binaries executed this way will have certain environment variables set, depending on the database driver in use.

For SQLite:

• SQLITE_FILE is set to the path of the database file. (empty for in-memory databases)

func WithLogger

func WithLogger(logger *slog.Logger) Option

WithLogger allows setting a custom logger for logs.

A single INFO-level log is emitted for each migration, indicating whether it was run, or skipped. A failed migration logs an error, as well as returning the error value. All other information is logged at DEBUG level.

func WithStateTable

func WithStateTable(table string) Option

WithStateTable allows specifying a custom state table name.

type Source

type Source interface {
	// Index returns a list of all available migration versions.
	Index(context.Context) ([]uint, error)
	// Migration is expected to return the numbered migration.
	Migration(_ context.Context, id uint) (*Migration, error)
}

Source is the interface for a migration source.

func FSSource

func FSSource(fsys fs.FS, options ...Option) (Source, error)

FSSource returns a Source implemented by a directory of files. See FileSource for full description of behavior.

func FileSource

func FileSource(dir string, options ...Option) (Source, error)

FileSource returns a Source implemented by a directory of files. By default, it only considers files that end in ".sql", and you may optionally exclude other filepatterns by passing Exclude options.