tidal

README

Tidal

Database schema migration management and code generation.

Tidal provides a mechanism to define and manage database schema migrations using SQL files that both specify the up (apply) and down (rollback) actions to ensure consistent changes in the database schema as application versions change. Tidal includes a CLI tool to generate these files into descriptors, directly adding them to your application source code so they can be compiled into the binary. The tidal package implements utilities for managing the state of the database with respect to the migrations, even across different binaries and application versions.

Documentation

Overview

Package tidal provides a mechanism to define and manage database schema migrations using SQL files that both specify the up (apply) and down (rollback) actions to ensure consistent changes in the database schema as application versions change. Tidal provides a CLI tool to generate these files into descriptors, directly adding them to your application source code so they can be compiled into the binary. The tidal package provides utilities for managing the state of the database with respect to the migrations, even across different binaries and application versions.

Index

• Constants
• func Create(migrationsDirectory, name, packageName string) (err error)
• func Generate(migrations, outpath, packageName string) (err error)
• func Register(m Migration) (err error)
• func RegisterDescriptor(data []byte) (err error)
• func Reset() (err error)
• func Version() string
• type ByRevision
  • func (a ByRevision) Len() int
  • func (a ByRevision) Less(i, j int) bool
  • func (a ByRevision) Swap(i, j int)
• type Descriptor
  • func NewDescriptor(src io.Reader, name string) (_ Descriptor, err error)
  • func (d Descriptor) Down() (sql string, err error)
  • func (d Descriptor) Info() (name string, modTime time.Time, err error)
  • func (d Descriptor) Package() (s string, err error)
  • func (d Descriptor) Repr() string
  • func (d Descriptor) Up() (sql string, err error)
• type Migration
  • func Open(path string) (m Migration, err error)
  • func (m *Migration) Down(conn *sql.DB) (err error)
  • func (m *Migration) DownSQL() (string, error)
  • func (m *Migration) Package() (string, error)
  • func (m *Migration) Predecessors() (n int, err error)
  • func (m *Migration) Successors() (n int, err error)
  • func (m *Migration) Synchronized() bool
  • func (m *Migration) Up(conn *sql.DB) (err error)
  • func (m *Migration) UpSQL() (string, error)

Constants

const (
	VersionMajor = 1
	VersionMinor = 0
	VersionPatch = 0
)

Semantic versioning components for easy package comparsion.

Functions

func Create

func Create(migrationsDirectory, name, packageName string) (err error)

Create a new SQL migration file for code generation. The migration file is an ANSI SQL file that uses TSQL comments to deliniate the up and down migrations. Using the code generation tool, these files are directly embedded into your application code base using compressed Descriptors, which are registered as migrations at runtime. This helper utility adds the next migration sql file revision (based on the latest registered revision and the maximum revision number from sibling files) and writes out an empty template to the migrations directory.

func Generate

func Generate(migrations, outpath, packageName string) (err error)

Generate code and descriptors to embed migrations into an application package. The generate command requires the path to the migrations directory and the location to write the generated code file out to. Optionally, a packageName can be supplied, otherwise any package directives in the migration files will be used or simply the basename of the specified outpath.

func Register

func Register(m Migration) (err error)

Register a migration to be managed by tidal. Note that although migrations can be directly applied using the Migration interface, they must be registered in order to preserve dependency order. It is highly recommended to register migrations and to use the tidal migration interface rather than managing migrations manually.

func RegisterDescriptor

func RegisterDescriptor(data []byte) (err error)

RegisterDescriptor creates a Migration from descriptor data and registers it.

func Reset

func Reset() (err error)

Reset removes all registered migrations. Primarily used for testing.

func Version

func Version() string

Version returns a string with the semvar version of the current build.

Types

type ByRevision

type ByRevision []Migration

ByRevision implements sort.Interface for []Migration based on the Revision field.

func (ByRevision) Len

func (a ByRevision) Len() int

func (ByRevision) Less

func (a ByRevision) Less(i, j int) bool

func (ByRevision) Swap

func (a ByRevision) Swap(i, j int)

type Descriptor

type Descriptor []byte

Descriptor is the compressed bytes of the encoded SQL file that contains migration data. Descriptors are generated by the tidal command and embedded into the source code of applications. In order to minimize memory usage and binary size, the data is always stored in a compressed format, decompressed as necessary to run migration commands. This slows down the migration process a bit, but as migrations are rare, is an acceptable trade-off.

func NewDescriptor

func NewDescriptor(src io.Reader, name string) (_ Descriptor, err error)

NewDescriptor reads the data from the source migration file and gzip compresses it for in-memory storage. The reader should not be compressed before hand. Note that the name is not optional, it is used to identify descriptors via the gzip header information -- autogenerated descriptors use this property to ensure that the migrations can be created from a raw descriptor with no other information.

func (Descriptor) Down

func (d Descriptor) Down() (sql string, err error)

Down reads and returns the down migration command, including all comments and statements following the -- migrate: down comment and before the -- migrate: up or --migrate: end comments (or EOF).

func (Descriptor) Info

func (d Descriptor) Info() (name string, modTime time.Time, err error)

Info returns header information from the compressed data, generated Descriptors will have the associated filename and modification time returned.

func (Descriptor) Package

func (d Descriptor) Package() (s string, err error)

Package looks for a package directive, e.g. -- package: foo and returns the name of the specified package, otherwise it returns an empty string.

func (Descriptor) Repr

func (d Descriptor) Repr() string

Repr returns a string representation of the bytes data for embedding into source code. This function is primarily used by the code generation tool.

func (Descriptor) Up

func (d Descriptor) Up() (sql string, err error)

Up reads and returns the up migration command, including all comments and statements following the -- migrate: up comment and before the -- migrate: down or --migrate: end comments (or EOF).

type Migration

type Migration struct {
	Revision int       // the unique id of the migration, prefix from the migration file
	Name     string    // the human readable name of the migration, suffix of the migration file
	Active   bool      // if the migration has been applied and is part of the active schema
	Applied  time.Time // the timestamp the migration was applied
	Created  time.Time // the timestamp the migration was added to the database
	// contains filtered or unexported fields
}

Migration defines how changes to the database are applied (up) or rolled back (down). Each migration is defined by two distinct pieces of SQL code, one for up and one for down, which are are parsed from a single SQL file, delimited by tidal-parseable comments. Migrations are generally compiled into a compressed descriptor format that can be included with application source code so that the migrations are compiled with the binary, rather than sourced from external files.

Each migration can also include status information from the database (collected at runtime). This information defines the database's knowledge of the migration, e.g. has the migration been applied or not. Status information is stored in the database inside of a migrations table that is applied with Revision 0 (an application's first revision is Revision 1). The table is updated with migrate and sync commands.

Migrations are identified by a unique revision number that specifies the sequence which migrations must be applied. For now that means that migrations can only be applied linearly (and not as a directed acyclic graph with multiple dependencies). Future work is required to create a migration DAG structure.

func Open

func Open(path string) (m Migration, err error)

Open a migration SQL file and parse it into a Migration object.

func (*Migration) Down

func (m *Migration) Down(conn *sql.DB) (err error)

Down rolls back the migration from the database. The migration creates a transaction that executes the SQL DOWN code as well as an update to the migrations table reflecting the change in state. Both of these SQL commands must be executed together without error, otherwise the entire transaction is rolled back.

func (*Migration) DownSQL

func (m *Migration) DownSQL() (string, error)

DownSQL returns the sql statement defined for rolling back the migration to a state before this specific revision. This requires parsing the underlying descriptor correctly.

func (*Migration) Package

func (m *Migration) Package() (string, error)

Package returns the parsed package directive from the descriptor if it has one.

func (*Migration) Predecessors

func (m *Migration) Predecessors() (n int, err error)

Predecessors returns the number of migrations before this migration.

func (*Migration) Successors

func (m *Migration) Successors() (n int, err error)

Successors returns the number of migrations after this migration.

func (*Migration) Synchronized

func (m *Migration) Synchronized() bool

Synchronized returns true if the migration state has been synchronized with the database.

func (*Migration) Up

func (m *Migration) Up(conn *sql.DB) (err error)

Up applies the migration to the database. The migration creates a transaction that executes the SQL UP code as well as an update to the migrations table reflecting the change in state. Both of these SQL commands must be executed together without error otherwise the entire transaction is rolled back.

func (*Migration) UpSQL

func (m *Migration) UpSQL() (string, error)

UpSQL returns the sql statement defined for applying the migration to the specific revision. This requires parsing the underlying descriptor correctly.