Documentation
¶
Overview ¶
Package core implements a simple, non-magic general purpose migration library that gets embedded into your application.
adapt can be easily extended through various interfaces like Driver, DatabaseDriver, DatabaseDriverCustomMigration, SqlStatementsDriver, Source, SqlStatementsSource, HookSource, FilesystemAdapter.
Index ¶
- Constants
- func Migrate(executor string, driver Driver, sources SourceCollection, options ...Option) error
- type AvailableMigration
- type DBTarget
- type DatabaseDriver
- type DatabaseDriverCustomMigration
- type DirEntry
- type Driver
- type FileDriverOption
- type FilesystemAdapter
- type Hook
- type HookSource
- type Migration
- type MySQLOption
- func MySQLDBCreateStatement(stmt string) MySQLOption
- func MySQLDBName(dbName string) MySQLOption
- func MySQLDisableDBClose() MySQLOption
- func MySQLDisableTx() MySQLOption
- func MySQLTableName(tableName string) MySQLOption
- func MySQLTxBeginOpts(factory func() (context.Context, *sql.TxOptions)) MySQLOption
- type Option
- type ParsedMigration
- type PostgresOption
- type SQLiteOption
- type Source
- type SourceCollection
- type SqlStatementsDriver
- type SqlStatementsSource
Constants ¶
const (
// Version is the package's version string used to store in meta tables
Version = "adapt@v0.1.0"
)
Variables ¶
This section is empty.
Functions ¶
func Migrate ¶
func Migrate(executor string, driver Driver, sources SourceCollection, options ...Option) error
Migrate migrates all available migrations in your SourceCollection against the Driver, when they weren't already run before.
Example:
var db *sql.DB = getDB() err := adapt.Migrate( "myService@v1.3.12", adapt.NewMySQLDriver(db, adapt.MySQLDriverOptionTableName("_auth_migrations"), adapt.MySQLDriverOptionDisableTx, ), adapt.SourceCollection{ adapt116.NewEmbedFSSource(nil, ""), adapt.NewMemoryFSSource(map[string]string{}), adapt.NewCodeSource("x", adapt.Hook{ MigrateUpTx: func(tx *sql.Tx) error { return nil }, }), }, )
Types ¶
type AvailableMigration ¶
type AvailableMigration struct { // ID is the unique identifier of this AvailableMigration ID string // Source is the origin of this AvailableMigration Source Source // ParsedUp is a ParsedMigration set by Enrich if the Source is a // SqlStatementsSource ParsedUp *ParsedMigration // Hash is the unique migration hash from ParsedMigration.Hash set by Enrich // if the Source is a SqlStatementsSource Hash *string }
AvailableMigration is a container for a locally found migration that could be applied to the database. In it's base-form it consists of a ID and a Source element. When calling Enrich the type of Source is checked and additional information added
type DBTarget ¶
type DBTarget interface { Exec(query string, args ...interface{}) (sql.Result, error) Query(query string, args ...interface{}) (*sql.Rows, error) }
DBTarget is a container for a sql execution target (either sql.DB or sql.Tx)
type DatabaseDriver ¶
type DatabaseDriver interface { Driver // DB should return the database connection that gets used to execute // sql statements DB() *sql.DB // SupportsTx reports whether the driver supports database transactions. // If SupportsTx is true and ParsedMigration wants transactions to be used // all migration statements will be executed within a single transaction. SupportsTx() bool // TxBeginOpts provides the transaction begin options that should be used // when adapt starts a new transaction. TxBeginOpts() (ctx context.Context, opts *sql.TxOptions) // DeleteMigration should delete the migration from the database. It is // important that the provided DBTarget is used, which is a container for // the underlying execution target (either sql.DB directly or an eventually // running sql.Tx). DeleteMigration(migrationID string, target DBTarget) error }
DatabaseDriver is a special extension of Driver. It is always needed when adapt should execute a migration from a SqlStatementsSource.
func FromSqlStatementsDriver ¶
func FromSqlStatementsDriver(driver SqlStatementsDriver) DatabaseDriver
FromSqlStatementsDriver converts a SqlStatementsDriver to a full DatabaseDriver by wrapping it in an internal adapter that handles all sql.DB operations according to the features specified by SqlStatementsDriver
func NewMySQLDriver ¶
func NewMySQLDriver(db *sql.DB, opts ...MySQLOption) DatabaseDriver
NewMySQLDriver returns a DatabaseDriver from a sql.DB and variadic MySQLOption that can interact with a MySQL database.
func NewPostgresDriver ¶
func NewPostgresDriver(db *sql.DB, opts ...PostgresOption) DatabaseDriver
NewPostgresDriver returns a DatabaseDriver from a sql.DB and variadic PostgresOption that can interact with a PostgreSQL database.
func NewSQLiteDriver ¶
func NewSQLiteDriver(db *sql.DB, opts ...SQLiteOption) DatabaseDriver
NewSQLiteDriver returns a DatabaseDriver from a sql.DB and variadic SQLiteOption that can interact with a SQLite database.
type DatabaseDriverCustomMigration ¶
type DatabaseDriverCustomMigration interface { DatabaseDriver // Migrate provides a callback for fine-grained manual migrations. It is // responsible for the full transaction-lifecycle and checking if all // components support transactions. As long as Migrate doesn't return an // error adapt assumes that the ParsedMigration was applied successfully // and continues with setting the finished time in it's meta store. If // Migrate internally starts a transaction is should call beforeFinish // before committing the transaction. In certain situations (for example // during Down-migrations) adapt will want to execute statements within // the same transaction as the migration itself. If Migrate doesn't start // it's own migration it should call beforeFinish before returning the // function. beforeFinish is allowed to be nil. Migrate(migration *ParsedMigration, beforeFinish func(target DBTarget) error) error }
DatabaseDriverCustomMigration extends DatabaseDriver by providing a custom migration callback. This can be used when the default execution strategy of a DatabaseDriver isn't sufficient and the Driver needs fine-grained control over every single executed statement. When using DatabaseDriverCustomMigration the Driver itself is fully responsible for starting/committing transactions and checking if ParsedMigrations can be executed within a transaction.
type DirEntry ¶
DirEntry is similar to the fs.DirEntry interface provided by Go 1.16+. It's used by adapt to remove the Go 1.16 requirement and allow implementations like NewMemoryFSSource to provide a minimal directory entry object.
type Driver ¶
type Driver interface { // Name reports the name of this Driver. It is mainly used for logging // output and can be an empty string. Name() string // Init initializes the internal state of a driver. It should be used to // apply options or return an error when the internal state is invalid. // For tasks like establishing connections and performing health checks // Healthy should be used. Init(log logger.Logger) error // Healthy should report if everything is ready and healthy to proceed // with running migrations. One example would be to ping the database // and check if the connection is intact. Also Healthy is responsible for // creating the structure of your meta-storage (in the context of a // DatabaseDriver this would be, that the database and meta-table are // created) Healthy() error // SupportsLocks reports whether the driver supports locking or not. This // influences if AcquireLock and ReleaseLock are called. SupportsLocks() bool // AcquireLock acquires a lock if SupportsLocks reports that this Driver // supports locking AcquireLock() error // ReleaseLock is called after running migrations and only if AcquireLock // successfully acquired a lock (e.g. didn't return an error). ReleaseLock // is always called when an lock was acquired, even when an error is encountered // somewhere or the library panics ReleaseLock() error // ListMigrations lists all already applied migrations ListMigrations() ([]*Migration, error) // AddMigration adds the meta-data of a new migration. After successfully // adding this migration to the driver the migration Source will be executed. AddMigration(migration *Migration) error // SetMigrationToFinished must set the finished field of a migration to // the current time, which indicates that this migration has finished // successfully. SetMigrationToFinished(migrationID string) error // Close should close all underlying connections opened during Healthy and // perform any necessary clean-up operations. Close is always called, even // when an error is encountered somewhere or the library panics Close() error }
Driver is the most basic backend against which migrations from a SourceCollection are executed. The special extension DatabaseDriver extends this Driver, but doesn't differ in usage. The main advantage in splitting Driver and DatabaseDriver is to facilitate usages that don't depend an SQL or databases. This property gives adapt a general purpose migration library character.
func NewFileDriver ¶
func NewFileDriver(filename string, opts ...FileDriverOption) Driver
NewFileDriver returns a Driver from a filename and variadic FileDriverOption that can interact with local JSON-file as storage for meta information.
type FileDriverOption ¶
type FileDriverOption func(*fileDriver) error
FileDriverOption provides configuration values for a Driver implemented using on-disk file storage for meta-data.
func FileDriverFilePermission ¶
func FileDriverFilePermission(perm os.FileMode) FileDriverOption
FileDriverFilePermission sets the file permission used when writing files to disk. By default 0600 ist used.
type FilesystemAdapter ¶
type FilesystemAdapter interface { ReadDir(name string) ([]DirEntry, error) Open(name string) (io.ReadCloser, error) }
FilesystemAdapter is a minimal interface for any filesystem. It can be used in combination with FromFilesystemAdapter to convert the FilesystemAdapter to a full SqlStatementsSource
type Hook ¶
type Hook struct { // MigrateUp must be used when the used Driver isn't a DatabaseDriver. The // returned error specifies whether the migration succeeded or not. MigrateUp func() error // MigrateUpDB can be used when the used Driver is a DatabaseDriver. The // returned error specifies whether the migration succeeded or not. MigrateUpDB func(db *sql.DB) error // MigrateUpTx can be used when the executing Driver is a DatabaseDriver. The // returned error specifies whether the migration succeeded or not. The // provided sql.Tx is fully managed. Therefore the Hook callback is NOT // allowed to call tx.Commit or tx.Rollback. MigrateUpTx func(tx *sql.Tx) error // MigrateDown provides a ParsedMigration that will get stored as the // Down-Element of the Hook's associated Migration inside the Driver's // meta-storage MigrateDown func() *ParsedMigration }
Hook provides callback functions for a HookSource migration. Either MigrateUp, MigrateUpDB or MigrateUpTx must be used. When the Driver executing the migrations isn't a DatabaseDriver only MigrateUp can be used, as adapt has no chance of providing a sql.DB or information to start a sql.Tx. MigrateDown can provide a ParsedMigration object that will get stored as the Down-Element of a stored Migration.
type HookSource ¶
type HookSource interface { Source // GetHook must return the Hook object for the passed ID. The ID is always an // element from the list returned from Source.ListMigrations. GetHook(id string) Hook }
HookSource provides migrations via a callback Hook object. Adapt will manage the migration meta-information and callback to the Hook when the migration needs to be executed. If the current Driver is an DatabaseDriver uses can even outsource the sql.Tx lifecycle management (Begin/Commit/Rollback) to adapt.
func NewCodePackageSource ¶
func NewCodePackageSource(pkg map[string]Hook) HookSource
NewCodePackageSource provides a new HookSource for the map of id-hook pairs passed to it.
func NewCodeSource ¶
func NewCodeSource(id string, hook Hook) HookSource
NewCodeSource provides a new HookSource for the single id-hook pair passed to it.
type Migration ¶
type Migration struct { // ID is the unique identifier of this Migration ID string // Executor is the name of the program that run this migration. Usually // this should be combination of name and version like "myService@v1.17.0" Executor string // Started is the time this Migration was started Started time.Time // Finished is the time this Migrations was finished. When nil the Migration // hasn't finished or errored Finished *time.Time // Hash contains the calculated hash identifier of this migration's content. // It is calculated if this Migration associated Source provides a Hash // function, like ParsedMigration does Hash *string // Adapt is the version string of adapt itself. The information is embedded // into this module with the public Version field. Adapt string // Deployment is a unique identifier that groups together multiple migrations // that have been executed within the same deployment cycle. Deployment string // DeploymentOrder is the order in which migrations within a Deployment group // were executed. DeploymentOrder int // Down can contain a json-marshaled ParsedMigration that can be used to // rollback this migration. Down *[]byte }
Migration is a object containing meta-information of an applied migration
type MySQLOption ¶
type MySQLOption func(*mysqlDriver) error
MySQLOption provides configuration values for a DatabaseDriver implementing the MySQL dialect.
func MySQLDBCreateStatement ¶
func MySQLDBCreateStatement(stmt string) MySQLOption
MySQLDBCreateStatement sets the statement used to create-if-not-exists the database used for adapts meta-table. The name must contain a single %s placeholder, which will be formatted with the set MySQLDBName or "_adapt" by default.
The default script used is:
CREATE DATABASE IF NOT EXISTS %s CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci
func MySQLDBName ¶
func MySQLDBName(dbName string) MySQLOption
MySQLDBName sets the database name in which adapts meta-table is stored. By default, this database is named "_adapt". However, you can also specify your own database. During starting adapt the database will be created/checked if exists using the MySQLDBCreateStatement
func MySQLDisableDBClose ¶ added in v0.1.2
func MySQLDisableDBClose() MySQLOption
MySQLDisableDBClose instructs the driver not to close the *sql.DB on the Driver.Close callback, but leave it open.
func MySQLDisableTx ¶
func MySQLDisableTx() MySQLOption
MySQLDisableTx disables transaction for this driver. When set adapt will never run a migration inside a transaction, even when the ParsedMigration reports using a transaction.
func MySQLTableName ¶
func MySQLTableName(tableName string) MySQLOption
MySQLTableName sets the table name for adapts meta-table. By default, this is "_migrations"
func MySQLTxBeginOpts ¶
func MySQLTxBeginOpts(factory func() (context.Context, *sql.TxOptions)) MySQLOption
MySQLTxBeginOpts provides a factory function for creating a context.Context and *sql.TxOptions. If this factory is provided it will be called when adapt needs to start a sql.Tx for running migrations. By default, the values from the Go standard library are use (context.Background() and nil for *sql.TxOptions)
type Option ¶
type Option func(*exec) error
Option can modify the behaviour of Migrate and/or provide additional configuration values, like custom contract.Logger
func CustomLogger ¶
CustomLogger provides a custom contract.Logger implementation to adapt. It will be used within the whole module and passed down to Driver and Source children.
func DisableDriverLocks ¶
func DisableDriverLocks() Option
DisableDriverLocks disables mutex acquiring/releasing of a Driver, even if the Driver itself reports to support locking.
func DisableHashIntegrityChecks ¶
func DisableHashIntegrityChecks() Option
DisableHashIntegrityChecks disables the hash integrity checks of SqlStatementsSource migrations against the already applied ones. By default adapt always performs these checks to protect against unwanted changes to SQL-Statements scripts after they have already been applied to your Driver. Disabling it should be done with caution!
type ParsedMigration ¶
type ParsedMigration struct { UseTx bool `json:"UseTransaction"` Stmts []string `json:"Statements"` }
ParsedMigration is a parsed migration
func Parse ¶
func Parse(r io.Reader) (*ParsedMigration, error)
Parse scans everything from an io.Reader into a ParsedMigration structure, while preserving SQL-specific structures like multi-line statements (procedures). It also checks for special "-- +adapt" options at the beginning of the file, like "NoTransaction".
The following example should give you an overview how Parse works. Given the following file-content:
-- +adapt NoTransaction CREATE DATABASE IF NOT EXISTS testdb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; CREATE TABLE testdb.accounts_old (id INT NOT NULL, PRIMARY KEY (id)); CREATE TABLE testdb.accounts_new (id INT NOT NULL, PRIMARY KEY (id)); -- +adapt BeginStatement CREATE TRIGGER `accounts_trigger` BEFORE UPDATE ON `testdb.accounts_old` FOR EACH ROW BEGIN INSERT INTO testdb.accounts_new (id) VALUES(OLD.id) END -- +adapt EndStatement INSERT INTO testdb.accounts_old (id) VALUES(1); INSERT INTO testdb.accounts_old (id) VALUES(2);
Parse would create the following ParsedMigration:
&ParsedMigration{ UseTx: false, Stmts: []string{ "CREATE DATABASE IF NOT EXISTS testdb\n CHARACTER SET utf8mb4\n COLLATE utf8mb4_unicode_ci;", "CREATE TABLE testdb.accounts_old (id INT NOT NULL, PRIMARY KEY (id));", "CREATE TABLE testdb.accounts_new (id INT NOT NULL, PRIMARY KEY (id));", "CREATE TRIGGER `accounts_trigger` BEFORE UPDATE ON `testdb.accounts_old` FOR EACH ROW BEGIN\n INSERT INTO testdb.accounts_new (id) VALUES(OLD.id)\nEND", "INSERT INTO testdb.accounts_old (id) VALUES(1);", "INSERT INTO testdb.accounts_old (id) VALUES(2);", }, }
func (*ParsedMigration) Hash ¶
func (m *ParsedMigration) Hash() *string
Hash calculates a unique hash for the ParsedMigration. It includes the UseTx field and every single statement from the Stmts field
type PostgresOption ¶
type PostgresOption func(*postgresDriver) error
PostgresOption provides configuration values for a DatabaseDriver implementing the PostgreSQL dialect.
type SQLiteOption ¶
type SQLiteOption func(*sqliteDriver) error
SQLiteOption provides configuration values for a DatabaseDriver implementing the SQLite dialect.
func SQLiteDisableTx ¶
func SQLiteDisableTx() SQLiteOption
SQLiteDisableTx disables transaction for this driver. When set adapt will never run a migration inside a transaction, even when the ParsedMigration reports to use a transaction.
func SQLiteTxBeginOpts ¶
func SQLiteTxBeginOpts(factory func() (context.Context, *sql.TxOptions)) SQLiteOption
SQLiteTxBeginOpts provides a factory function for creating a context.Context and *sql.TxOptions. If this factory is provided it will be called when adapt needs to start an sql.Tx for running migrations. By default the values from the Go standard library are use (context.Background() and nil for *sql.TxOptions)
type Source ¶
type Source interface { // Init should initialize everything inside the Source. If an error is returned // adapt will abort execution and stop with an Source failure. Init(log logger.Logger) error // ListMigrations should list unique migration-IDs for all available migrations. // If a particular migration supports Up and Down variants only a single-ID (with // the Up/Down differentiator removed) should be returned. The IDs don't need to // be in any particular order, as adapt will merge it will other Source providers // and sort the complete SourceCollection afterwards ListMigrations() ([]string, error) }
Source is the basis interface for every single migration-source. It provides information about the available migrations via ListMigrations. Every Source must additionally implement SqlStatementsSource or HookSource.
type SourceCollection ¶
type SourceCollection []Source
SourceCollection is a collection of Source elements
type SqlStatementsDriver ¶
type SqlStatementsDriver interface { // Name reports the name of this Driver. It is mainly used for logging // output and can be an empty string. Name() string // Init initializes the internal state of a driver. It should be used to // apply options or return an error which the internal state is invalid. // For tasks like establishing connections and performing health checks // Healthy should be used. Init(log logger.Logger) error // Healthy should report if everything is ready and healthy to proceed // with running migrations. One example would be to ping the database // and check if the connection is intact. Also Healthy is responsible for // creating the structure of your meta-storage, e.g. the database and // meta-table. Healthy() error // SupportsLocks reports whether the driver supports locking or not. This // influences if AcquireLock and ReleaseLock are called. SupportsLocks() bool // AcquireLock must return a database query that acquires an exclusive // lock. AcquireLock() (query string) // ReleaseLock must return a database query that released the previously // acquired lock. ReleaseLock() (query string) // ListMigrations must return a database query that selects all Migration // data in the following order: ID, Executor, Started, Finished, Hash, Adapt // Deployment, DeploymentOrder, Down. The field's types are the same as in the // Migration struct. ListMigrations() (query string) // AddMigration must return a database query and it's corresponding args // that insert the passed Migration into the meta-table. AddMigration(m *Migration) (query string, args []interface{}) // SetMigrationToFinished must return a database query and it's corresponding // args in order to set the migration with migrationID to finished. SetMigrationToFinished(migrationID string) (query string, args []interface{}) // Close should close all underlying connections opened during Healthy and // perform any necessary clean-up operations. Close is always called, even // when an error is encountered somewhere or the library panics Close() error // DB should return the database connection that gets used to execute // sql statements DB() *sql.DB // SupportsTx reports whether the driver supports database transactions. // If SupportsTx is true and ParsedMigration wants transactions to be used // all migration statements will be executed within a single transaction. SupportsTx() bool // TxBeginOpts provides the transaction begin options that should be used // when adapt starts a new transaction. TxBeginOpts() (ctx context.Context, opts *sql.TxOptions) // UseGlobalTx instructs the adapter to start a single global transaction // for all database queries/executes. When used the transaction is started // during Init and committed/rollbacked during Close. UseGlobalTx() bool // DeleteMigration must return a database query and it's corresponding args // in order to delete the specified migration. DeleteMigration(migrationID string) (query string, args []interface{}) }
SqlStatementsDriver is a special interface for defining a DatabaseDriver that only wants to use there database specific dialect. A SqlStatementsDriver can be converted into a full DatabaseDriver using a provided adapter, that can be accessed using FromSqlStatementsDriver. Basically a SqlStatementsDriver only reports it's features and (query, args) pairs for the different needed operations that depend on the underlying SQL dialect.
The big advantage of SqlStatementsDriver is that it reduces boilerplate and error checking drastically for all primitive DatabaseDriver that support sql.DB.
type SqlStatementsSource ¶
type SqlStatementsSource interface { Source // GetParsedUpMigration must return a ParsedMigration for the passed ID. The ID // is always an element from the list returned from Source.ListMigrations. When // no custom parser is needed one can use Parse which implements a general // purpose parser for most SQL styles. GetParsedUpMigration(id string) (*ParsedMigration, error) // GetParsedDownMigration can return a ParsedMigration or nil (if no Down migration // is available) for the passed ID. The ID is always an element from the list // returned from Source.ListMigrations. When no custom parser is needed one can use // Parse which implements a general purpose parser for most SQL styles. GetParsedDownMigration(id string) (*ParsedMigration, error) }
SqlStatementsSource is a Source interface that provides parsed SQL-Statements via the ParsedMigration struct. It can provide Up (GetParsedUpMigration) or Down (GetParsedDownMigration) migrations for the same migration id.
func FromFilesystemAdapter ¶
func FromFilesystemAdapter(adapter FilesystemAdapter, directory string) SqlStatementsSource
FromFilesystemAdapter converts an FilesystemAdapter implementation to a full-fledged SqlStatementsSource. It unifies the code across most filesystem and the in-memory statements sources.
func NewFilesystemSource ¶
func NewFilesystemSource(directory string) SqlStatementsSource
NewFilesystemSource provides a new SqlStatementsSource that uses the SQL-files within the passed directory as migrations.
func NewMemoryFSSource ¶
func NewMemoryFSSource(fs map[string]string) SqlStatementsSource
NewMemoryFSSource provides a SqlStatementsSource for an in-memory filesystem represented by a Name->FileContent map
Source Files
¶
- adapt.go
- dialect_mysql.go
- dialect_postgres.go
- dialect_sqlite.go
- doc.go
- driver.go
- driver_db.go
- driver_file.go
- driver_sql.go
- exec.go
- exec_close.go
- exec_healthcheck.go
- exec_init.go
- exec_migrate.go
- exec_migrate_hook.go
- exec_migrate_sqlstatements.go
- exec_mutex.go
- exec_preparelocal.go
- exec_prepareremote.go
- exec_rollback.go
- exec_start.go
- hook.go
- logfield.go
- migration.go
- option.go
- parser.go
- source.go
- source_code.go
- source_filesystem.go
- source_filesystem_adapter.go
- source_memory.go