README
¶
Migrations
Database migrations are handled by this package. All migrations should be created in separate files, following a starndard naming convetion
The migrations.go file defines an array of migrate functions that are called by the gormigrate helper. Each migration function should perform a specific migration.
Creating a new migration
Create a migration ID based on the time using the YYYYMMDDHHMM format. Example: August 21 2018 at 2:54pm would be 201808211454.
Your migration's name should be used in the file name and in the function name and should adequately represent the actions your migration is taking. If your migration is doing too much to fit in a name, you should consider creating multiple migrations.
Create a separate file in pkg/db/ following the naming schema in place: <migration_id>_<migration_name>.go. In the file, you'll create a function that returns a gormmigrate.Migration object with gormigrate.Migrate and gormigrate.Rollback functions defined.
Add the function you created in the separate file to the migrations list in pkg/db/migrations.go.
If necessary, write a test to verify the migration. See test/integration/migrations_test.go for examples.
Migration Rules
Migration IDs
Each migration has an ID that defines the order in which the migration is run.
IDs are numerical timestamps that must sort ascending. Use YYYYMMDDHHMM w/ 24 hour time for format.
Example: August 21 2018 at 2:54pm would be 201808211454.
Migration IDs must be descending. If you create a migration, submit an MR, and another MR is merged before yours is able to be merged, you must update the ID to represent a date later than any previous migration.
Models in Migrations
Represent modesl inline with migrations to represent the evolution of the object over time.
For example, it is necessary to add a boolean field "hidden" to the "Account" model. This is how you would represent the account model in that migration:
type Account struct {
Model
Username string
FirstName string
LastName string
Hidden boolean
}
return tx.AutoMigrate(&Account{})
...
DO NOT IMPORT THE API PKG. When a migration imports the api pkg and uses models defined in it, the migration may work the first time it is run. The models in pkg/api are bound to change as the project grows. Eventually, the models could change so that the migration breaks, causing any new deployments to fail on your old shitty migration.
Record Deletions
If it is necessary to delete a record in a migration, be aware of a couple caveats around deleting wth gorm:
-
You must pass a record with an ID to
gorm.Delete(&record), otherwise ALL RECORDS WILL BE DELETED -
Gorm soft deletes by default. This means it sets the
deleted_atfield to a non-null value and any subsequent gorm calls will ignore it. If you are deleting a record that needs to be permanently deleted (like permissions), usegorm.Unscoped().Delete.
See the gorm documentation around deletions for more information
Migration tests
In most cases, it shouldn't be necessary to create a test for a migration. However, if the migration is manipulating records and poses a significant risk of completely borking up important data, a test should be written.
Tests are difficult to write for migrations and are likely to fail one day long after the migration has already run in production. After a migration is run in production, it is safe to delete the test from the integration test suite.
The test helper has a couple helpful functions for testing migrations. You can use h.CleanDB() to completely wipe the database clean, then h.MigrateTo(<migration_id>) to migrate to a specific migration ID. You should then be able to create whatever records in the database you need to test against and finally run h.MigrateDB() to run your created migration and all subsequent migrations.
Documentation
¶
Index ¶
- Variables
- func AddPostCommitAction(ctx context.Context, f func()) error
- func Begin(ctx context.Context) error
- func CreateMigrationFromActions(id string, actions ...MigrationAction) *gormigrate.Migration
- func FromContext(ctx context.Context) (*sql.Tx, error)
- func MarkForRollback(ctx context.Context, err error)
- func Resolve(ctx context.Context) error
- func TransactionMiddleware(db *ConnectionFactory) func(next http.Handler) http.Handler
- type ConnectionFactory
- type DatabaseConfig
- type Migration
- type MigrationAction
- func AddTableColumnAction(table interface{}, columnName string) MigrationAction
- func AddTableColumnsAction(table interface{}) MigrationAction
- func CreateTableAction(table interface{}) MigrationAction
- func DropTableAction(table interface{}) MigrationAction
- func DropTableColumnAction(table interface{}, columnName string) MigrationAction
- func DropTableColumnsAction(table interface{}, tableName ...string) MigrationAction
- func ExecAction(applySql string, unapplySql string) MigrationAction
- func FuncAction(applyFunc func(*gorm.DB) error, unapplyFunc func(*gorm.DB) error) MigrationAction
- func RenameTableColumnAction(table interface{}, oldFieldName string, newFieldName string) MigrationAction
- type Model
Constants ¶
This section is empty.
Variables ¶
var KafkaAdditionalLeasesExpireTime = time.Now().Add(1 * time.Minute)
Set new additional leases expire time to a minute later from now so that the old "kafka" leases finishes its execution before the new jobs kicks in.
Functions ¶
func AddPostCommitAction ¶
func CreateMigrationFromActions ¶
func CreateMigrationFromActions(id string, actions ...MigrationAction) *gormigrate.Migration
func FromContext ¶
FromContext Retrieves the transaction from the context.
func MarkForRollback ¶
MarkForRollback flags the transaction stored in the context for rollback and logs whatever error caused the rollback
func TransactionMiddleware ¶
func TransactionMiddleware(db *ConnectionFactory) func(next http.Handler) http.Handler
TransactionMiddleware creates a new HTTP middleware that begins a database transaction and stores it in the request context.
Types ¶
type ConnectionFactory ¶
type ConnectionFactory struct {
Config *DatabaseConfig
DB *gorm.DB
}
func NewConnectionFactory ¶
func NewConnectionFactory(config *DatabaseConfig) (*ConnectionFactory, func())
NewConnectionFactory will initialize a singleton ConnectionFactory as needed and return the same instance. Go includes database connection pooling in the platform. Gorm uses the same and provides a method to clone a connection via New(), which is safe for use by concurrent Goroutines.
func NewMockConnectionFactory ¶
func NewMockConnectionFactory(dbConfig *DatabaseConfig) *ConnectionFactory
NewMockConnectionFactory should only be used for defining mock database drivers This uses mocket under the hood, use the global mocket.Catcher to change how the database should respond to SQL queries
func (*ConnectionFactory) CheckConnection ¶
func (f *ConnectionFactory) CheckConnection() error
Checks to ensure a connection is present
func (*ConnectionFactory) New ¶
func (f *ConnectionFactory) New() *gorm.DB
New returns a new database connection
func (*ConnectionFactory) NewContext ¶
NewContext returns a new context with transaction stored in it. Upon error, the original context is still returned along with an error
type DatabaseConfig ¶
type DatabaseConfig struct {
Dialect string `json:"dialect"`
SSLMode string `json:"sslmode"`
Debug bool `json:"debug"`
MaxOpenConnections int `json:"max_connections"`
Host string `json:"host"`
Port int `json:"port"`
Name string `json:"name"`
Username string `json:"username"`
Password string `json:"password"`
DatabaseCaCertFile string `json:"db_ca_cert_file"`
HostFile string `json:"host_file"`
PortFile string `json:"port_file"`
NameFile string `json:"name_file"`
UsernameFile string `json:"username_file"`
PasswordFile string `json:"password_file"`
}
func NewDatabaseConfig ¶
func NewDatabaseConfig() *DatabaseConfig
func (*DatabaseConfig) AddFlags ¶
func (c *DatabaseConfig) AddFlags(fs *pflag.FlagSet)
func (*DatabaseConfig) ConnectionString ¶
func (c *DatabaseConfig) ConnectionString() string
func (*DatabaseConfig) LogSafeConnectionString ¶
func (c *DatabaseConfig) LogSafeConnectionString() string
func (*DatabaseConfig) ReadFiles ¶
func (c *DatabaseConfig) ReadFiles() error
type Migration ¶
type Migration struct {
DbFactory *ConnectionFactory
Gormigrate *gormigrate.Gormigrate
GormOptions *gormigrate.Options
}
func NewMigration ¶
func NewMigration(dbConfig *DatabaseConfig, gormOptions *gormigrate.Options, migrations []*gormigrate.Migration) (*Migration, func(), error)
func (*Migration) CountMigrationsApplied ¶
func (*Migration) MigrateTo ¶
Migrating to a specific migration will not seed the database, seeds are up to date with the latest schema based on the most recent migration This should be for testing purposes mainly
func (*Migration) RollbackLast ¶
func (m *Migration) RollbackLast()
func (*Migration) RollbackTo ¶
type MigrationAction ¶
func AddTableColumnAction ¶
func AddTableColumnAction(table interface{}, columnName string) MigrationAction
func AddTableColumnsAction ¶
func AddTableColumnsAction(table interface{}) MigrationAction
func CreateTableAction ¶
func CreateTableAction(table interface{}) MigrationAction
func DropTableAction ¶
func DropTableAction(table interface{}) MigrationAction
func DropTableColumnAction ¶
func DropTableColumnAction(table interface{}, columnName string) MigrationAction
func DropTableColumnsAction ¶
func DropTableColumnsAction(table interface{}, tableName ...string) MigrationAction
func ExecAction ¶
func ExecAction(applySql string, unapplySql string) MigrationAction
func FuncAction ¶
func RenameTableColumnAction ¶
func RenameTableColumnAction(table interface{}, oldFieldName string, newFieldName string) MigrationAction
Source Files