Documentation
¶
Overview ¶
Package migrationuc provides the database migration use cases. It exposes two main use cases, namely InitDBUseCase for initializing of database schema with initial sample data (for development or production environment) and MigrateDBUseCase for migrating from a source database to a destination database while converting the schema format from one version to another version (upwards or downwards). This package also exposes the Settings and SchemaSettings interfaces which represent the version-independent expectations from any configuration file representation type, so all configuration types with different formats may be taken similarly in the use cases layer for sake of seamless migration.
Index ¶
- func ForeignSchemaName(major, minor uint) string
- func ForeignServerName(major, minor uint) string
- func HasTheSameConnectionInfo(s1, s2 SchemaSettings) (bool, error)
- func MigrationSchemaName(major uint) string
- func SchemaName(major uint) string
- type ConfigFileLoader
- type InitDBUseCase
- type MigrateDBUseCase
- type SchemaSettings
- type Settings
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ForeignSchemaName ¶
ForeignSchemaName returns the name of a schema which should be created in the destination database in order to be filled with the foreign tables which are imported from a foreign server which represents the source database with the given major and minor versions. When a foreign server with the name computed by the ForeignServerName was created and its corresponding schema was imported into a local schema with the name computed by the ForeignSchemaName, source database contents can be read from the destination database and used in the migration process (without having to pass them through this Golang process memory).
func ForeignServerName ¶
ForeignServerName returns the name of a foreign server which should be created in order to represent the source database with the given major and minor version numbers within the destination database.
func HasTheSameConnectionInfo ¶
func HasTheSameConnectionInfo(s1, s2 SchemaSettings) (bool, error)
HasTheSameConnectionInfo returns true if and only if both of the `s1` and `s2` schema settings contain the connection information for a common database. That is, their host, port, and database name do match. If they described the same database, they must also have the same database schema semantic major version and the minor version of `s1` must be equal to or greater than the `s2` minor version. Otherwise, a non-nil error will be returned too.
This method is useful for finding out if we have two distinct databases, hence, one can be used as a read-only source database while writing to the other database as a migration destination.
func MigrationSchemaName ¶
MigrationSchemaName returns the intermediate schema name which is used for storage of a database schema with given major version during a migration operation. After loading the source database schema in a local schema which its name is computed by ForeignSchemaName function, it will be migrated upwards to its latest supported minor version (even if it is at the latest minor version already), filling MigrationSchemaName schema for the source major version. Thereafter, its schema will be used in order to fill MigrationSchemaName for one major version upper or downer based on the migration direction until it reaches to the destination schema major version. Finally, the last intermediate migration schema will be used for creation of tables in the destination schema which its name is computed by SchemaName function. The intermediate schema will be dropped at the end.
func SchemaName ¶
SchemaName returns the target database schema name for the given major version. It should return cawebN for version N.
Types ¶
type ConfigFileLoader ¶
ConfigFileLoader is a version-independent function type which accepts a configuration file path, reads it entirely, parses it based on its detected version, and finally adapts it to the Settings interface.
type InitDBUseCase ¶
type InitDBUseCase struct {
// contains filtered or unexported fields
}
InitDBUseCase represents the database initialization use case. It may be used to initialize database with development or production suitable data as asked by the InitDev and InitProd methods.
func NewInitDB ¶
func NewInitDB(ss SchemaSettings) *InitDBUseCase
NewInitDB creates an InitDBUseCase instance, using the `ss` schema settings in order to find the target database connection information and also create its schema initializer based on the expected database semantic version. The `repo.Schema` repo will be taken from the `ss` in order to be used for dropping and (re)creating an empty schema, creating normal role, granting it privileges on the empty schema, and renewing the passwords of admin and normal roles.
func (*InitDBUseCase) InitDev ¶
func (iduc *InitDBUseCase) InitDev(ctx context.Context) error
InitDev drops cawebN schema (if N is the relevant major version) and (re)creates it, assuming that it is an empty schema, using the admin role. It also creates the normal role (if it does not exist), grants privileges on the created schema to normal role so it can create tables, and renews passwords of both admin and normal roles. These operations will be performed using the admin role in a single transaction and coordinated with password files so they can be repeated in case of an abrupt failure as elaborated in docs of the pkg/core/usecase/migrationuc.SchemaSettings.RenewPasswords method. Thereafter, it connects to the target database using the normal role and completes its operation (in a second transaction) by creating all relevant tables and filling them with the development suitable data.
func (*InitDBUseCase) InitProd ¶
func (iduc *InitDBUseCase) InitProd(ctx context.Context) error
InitProd drops cawebN schema (if N is the relevant major version) and (re)creates it, assuming that it is an empty schema, using the admin role. It also creates the normal role (if it does not exist), grants privileges on the created schema to normal role so it can create tables, and renews passwords of both admin and normal roles. These operations will be performed using the admin role in a single transaction and coordinated with password files so they can be repeated in case of an abrupt failure as elaborated in docs of the pkg/core/usecase/migrationuc.SchemaSettings.RenewPasswords method. Thereafter, it connects to the target database using the normal role and completes its operation (in a second transaction) by creating all relevant tables and filling them with the production suitable data.
type MigrateDBUseCase ¶
type MigrateDBUseCase struct {
// contains filtered or unexported fields
}
MigrateDBUseCase represents the configuration and database schema migration use cases. The source and destination versions are determined by Settings instances which also provide the database connection information for the source and destination databases. For details, see the NewMigrateDB function.
func NewMigrateDB ¶
func NewMigrateDB( mig repo.Migrator[Settings], dstSettings Settings, targetCfgPath string, loader ConfigFileLoader, ) *MigrateDBUseCase
NewMigrateDB creates a new MigrateDBUseCase instance which uses the `mig` migrator in order to learn about the source configuration file settings. Using `mig`, source settings can be migrated upwards or downwards until they match with the format version which is used by the `dstSettings` settings. Thereafter, those items which are missing due to the version changes (such as fields which are recently introduced and had no older counterpart) and those items which must change unconditionally (such as the database connection information which must be taken from the destination configuration settings) will be overwritten from `dstSettings` into the settings instance which was obtained from the `mig` settings migrator. Similarly, the source and destination database schema version and connection information are taken from `mig` and `dstSettings` in order to build relevant schema migrator and migrate the source schema upwards or downwards until it matches with the expected destination schema version (only if the source and destination databases are different and the destination database is empty). The `targetCfgPath` indicates the final path which should be overwritten atomically when the target settings are prepared.
The `repo.Schema` repository is taken from the `dstSettings` in order to create roles and schema and drop temporary/intermediate schema. See repo.Schema for more details.
Target settings are first written into `targetCfgPath + ".migrated"` file, then destination database contents are committed, and finally extra schema are dropped and that migrated settings file is moved in order to atomically write the `targetCfgPath` file. The loader argument is used during an incomplete migration operation resumption, as it allows the target migrated configuration file to be read and compared with the `dstSettings`. If the destination database is not empty, but there is a `.migrated` file which its contents match with the expected destination database settings, we can conclude that database is filled by a previous migration attempt and we can resume it from the commitment step.
NewMigrateDB only creates the migrator and performs no actual operation, hence, it may not return an error.
func (*MigrateDBUseCase) Migrate ¶
func (mduc *MigrateDBUseCase) Migrate(ctx context.Context) error
Migrate runs the configuration settings and database schema migration determining the migration direction by the settings versions and schema versions as recorded in the source and destination configs.
type SchemaSettings ¶
type SchemaSettings interface {
// ConnectionPool creates a database connection pool using the
// connection information which are kept in this SchemaSettings
// instance. The `r` argument specifies the role name for the
// created connection pool.
//
// Password values are kept in files in a specific password dir
// and creation of a connection pool depends on identification of
// a valid password for the given role and the database host, port,
// and name which are taken from this SchemaSettings instance.
// Each non-empty and non-commented line of the passwords file
// should conform with this format:
//
// host:port:dbname:role:password
//
// For sake of atomic passwords updating operations (during a DB
// migration), a second temporary passwords file may be created
// in order to hold the new values of passwords. Therefore, even in
// case of a failed migration operation, either old or new passwords
// from the main or temporary passwords file may be used to connect
// to the database. If such a temporary passwords file was used for
// establishment of a connection pool, it will be moved to the main
// passwords file before returning (so the temporary file may be
// overwritten safely by the subsequent migration operations).
ConnectionPool(ctx context.Context, r repo.Role) (repo.Pool, error)
// ConnectionInfo returns the database name, host, and port of the
// connection information which are kept in this SchemaSettings
// instance. The ConnectionPool method can be used to employ these
// information and connect to a database.
ConnectionInfo() (dbName, host string, port int)
// NewSchemaRepo instantiates a fresh Schema repository.
// Role names may be optionally suffixed based on the settings and
// in that case, repo.Role role names which are passed to the
// ConnectionPool method or RenewPasswords will be suffixed
// automatically. Since the Schema repository has methods for
// creation of roles or asking to grant specific privileges to
// them, it needs to obtain the same role name suffix (as stored
// in the current SchemaSettings instance).
NewSchemaRepo() repo.Schema
// SchemaMigrator creates a repo.Migrator[repo.SchemaSettler]
// instance which wraps the given transaction argument and can be
// used for (1) loading the source database schema information
// with this assumption that tx belongs to the destination database
// and this SchemaSettings contains the source database connection
// information, so it can modify the destination database within a
// given transaction and fill a schema with tables which represent
// the source database contents (not moving data items necessarily,
// but may create them as a foreign data wrapper, aka FDW),
// (2) creating upwards or downwards migrator objects in order to
// transform the loaded data into their upper/lower schema versions
// (again with minimal data transfer and using views instead of
// tables as far as possible, while creating tables or even loading
// data into this Golang process if it is necessary), and at last
// (3) obtaining a repo.SchemaSettler instance for the target schema
// major version, so it can persist the target schema version by
// creating tables and filling them with contents of corresponding
// views.
SchemaMigrator(tx repo.Tx) (
repo.Migrator[repo.SchemaSettler], error,
)
// SchemaInitializer creates a repo.SchemaInitializer instance
// which wraps the given transaction argument and can be used to
// initialize the database with development or production suitable
// data. The format of the created tables and their initial data
// rows are chosen based on the database schema version, as
// indicated by SchemaVersion method. All table creation and data
// insertion operations will be performed in the given transaction
// and will be persisted only if the `tx` could commit successfully.
SchemaInitializer(tx repo.Tx) (repo.SchemaInitializer, error)
// RenewPasswords generates new secure passwords for the given roles
// and after recording them in a temporary file, will use the change
// function in order to update the passwords of those roles in the
// database too. The change function argument should perform the
// update operation in a transaction which may or may not be
// committed when RenewPasswords returns. In case of a successful
// commitment, the temporary passwords file should be moved over
// the main passwords file, as known in the current SchemaSettings
// instance (so it may be used for the future calls to the
// ConnectionPool method). This final file movement can be performed
// using the returned finalizer function.
RenewPasswords(
ctx context.Context,
change func(
ctx context.Context,
roles []repo.Role,
passwords []string,
) error,
roles ...repo.Role,
) (finalizer func() error, err error)
// SchemaVersion returns the semantic version of the database schema
// which its connection information are kept by this SchemaSettings.
SchemaVersion() model.SemVer
// SetSchemaVersion updates the semantic version of the database
// schema as recorded in this schema settings and reported by the
// SchemaVersion method.
SetSchemaVersion(sv model.SemVer)
}
SchemaSettings represents the database-related settings which should be provided by a configuration file. It allows a database connection pool to be established for an asked role using the ConnectionPool method, reports the database schema version which is required for querying the stored tables, may be used as a factory for the repo.Migrator (in order to migrate the database to an upper or downer version), for changing passwords of a set of database roles and storing new passwords in relevant files (with the atomic updating considerations), or as a factory for repo.SchemaInitializer (in order to initialize an empty database with development or production suitable data).
type Settings ¶
type Settings interface {
// Marshaler interface customizes the YAML serialization of a
// configuration file contents, so it can replace specific settings
// such as a slices of numbers in a vers.Config with alternative
// data types and have control on the final serilization result.
//
// See the Marshal function of any Config struct for the reification
// details and how marshaling logic can be distributed among nested
// Config structs.
yaml.Marshaler
// SchemaSettings represents the database-related parts of Settings.
SchemaSettings
// Clone creates a deep copy of this Settings instance.
Clone() Settings
// MergeSettings expects to receive a Settings instance which has
// the same settings version as the current instance. When migrating
// settings to newer or older versions, some of the target version
// settings may be left uninitialized. This method fills those items
// with their correspoding values from the given `s` argument.
// Some settings, such as the database connection information, are
// unconditionally taken from the `s` argument because they need to
// describe the destination settings values.
MergeSettings(s Settings) error
// Version returns the semantic version of this Settings format.
Version() model.SemVer
}
Settings interface represents the expectations of the migration use cases from the configuration files contents. Each Config struct version has to be adapted in order to provide this interface before being passed to the migration use cases.
The Config structs and their adapter-layer migrators may use the concrete struct types for sake of type-safety. For example, migrating a cfg1.Config instance up leads to creation of cfg2.Config struct. However, these adapter-layer structs differences among a series of versions are masked out (by their Adapter implementations), so they can be managed uniformly in the use cases layer.
Source Files