schema

package
v0.12.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 9, 2023 License: MPL-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package schema is used to apply sql migrations to modify the state of a database instance. It also provides functionality to report on the state of a database instance and how it compares with the migration editions in the running binary.

This package requires that the migration editions are first registered, prior to creating a schema.Manager. This is generally done in an init function of another package, i.e.: 

//go:embed postgres
var postgres embed.FS

func init() {
    schema.RegisterEdition("oss", schema.Postgres, postgres, 0)
}

Then a manager can be created and used to apply the migrations: 

m := schema.NewManager(ctx, schema.Postgres, db)
m.ApplyMigrations()

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func MigrateStore 

func MigrateStore(ctx context.Context, dialect Dialect, url string, opt ...Option) (bool, error)

MigrateStore executes the migrations needed to initialize the store. It returns true if migrations actually ran; false if the database is already current or if there was an error. Supports the WithEditions(...) option.

func RegisterEdition added in v0.7.0 

func RegisterEdition(name string, dialect Dialect, fs embed.FS, priority int, opt ...edition.Option)

RegisterEdition registers an edition for use by the Manager. It will panic if: - An unsupported dialect is provided. - The same (dialect, name) is registered. - The same (dialect, priority) is registered.

func TestCreatePartialEditions added in v0.7.0 

func TestCreatePartialEditions(dialect Dialect, p PartialEditions) edition.Editions

TestCreatePartialEditions is used by tests to create a subset of the Edition migrations.

Types

type DatabaseState added in v0.7.0 

type DatabaseState int

DatabaseState defines the state of the Database schema as compared to the latest version for the binary. 

const (
	Behind DatabaseState = iota // Database schema version is older then latest for the binary, so it needs migrations applied.
	Ahead                       // Database schema version is newer then latest for the binary, so binary needs to be updated.
	Equal                       // Database schema version matches latest version for the binary.
)

Valid states.

type Dialect added in v0.7.0 

type Dialect = edition.Dialect

Dialect is same as edition.Dialect 

const (
	Postgres Dialect = "postgres"
)

Supported dialects.

type EditionState added in v0.7.0 

type EditionState struct {
	// Name is the identifier of the Edition.
	Name string

	// DatabaseSchemaVersion is the schema version that is currently running in the database.
	DatabaseSchemaVersion int
	// BinarySchemaVersion is the schema version which this boundary binary supports.
	BinarySchemaVersion int

	DatabaseSchemaState DatabaseState
}

EditionState is the current state of a schema Edition.

type LogEntry added in v0.2.0 

type LogEntry struct {
	Id               int
	MigrationVersion int
	MigrationEdition string
	CreateTime       time.Time
	Entry            string
}

LogEntry represents a log entry generated during migrations.

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager provides a way to run operations and retrieve information regarding the underlying boundary database schema. Manager is not thread safe.

Example (Hooks) 
package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"
	"strings"

	"github.com/hashicorp/boundary/internal/db/common"
	"github.com/hashicorp/boundary/internal/db/schema"
	"github.com/hashicorp/boundary/internal/db/schema/internal/edition"
	"github.com/hashicorp/boundary/internal/db/schema/migration"
	"github.com/hashicorp/boundary/testing/dbtest"
)

func main() {
	ctx := context.Background()
	dialect := dbtest.Postgres

	c, u, _, err := dbtest.StartUsingTemplate(dialect, dbtest.WithTemplate(dbtest.Template1))
	defer c()

	d, err := common.SqlOpen(dialect, u)
	if err != nil {
		log.Fatalf(err.Error())
	}
	editions := edition.Editions{
		{
			Name:    "hooks_example",
			Dialect: schema.Postgres,
			Migrations: migration.Migrations{
				1: migration.Migration{
					Edition: "hooks_example",
					Version: 1,
					Statements: []byte(`
					create table foo (
						id bigint generated always as identity primary key,
						public_id text,
						name text
					);

					-- Not a normal thing to have in a migration
					-- but this is done to put "invalid" data
					-- into a table, that will then have a constraint added
					-- in a future migration.
					insert into foo
						(public_id, name)
					values
						(null, 'Alice'),
						(null, 'Bob'),
						('foo_cathy', 'Cathy');
					`),
				},
				2: migration.Migration{
					Edition: "hooks_example",
					Version: 2,
					Statements: []byte(`
					-- this would fail if data is not updated first
					alter table foo
						alter column public_id
							set not null;
					`),
					PreHook: &migration.Hook{
						CheckFunc: func(ctx context.Context, tx *sql.Tx) (migration.Problems, error) {
							rows, err := tx.QueryContext(
								ctx,
								`select
									id, name
								from foo
								where
									public_id is null`,
							)
							if err != nil {
								return nil, err
							}

							invalid := make([]string, 0)
							for rows.Next() {
								var id int
								var name string
								if err := rows.Scan(&id, &name); err != nil {
									return nil, err
								}
								invalid = append(invalid, fmt.Sprintf("%d:%s", id, name))
							}

							if len(invalid) > 0 {
								return append([]string{"invalid foos:"}, invalid...), nil
							}
							return nil, nil
						},
						RepairFunc: func(ctx context.Context, tx *sql.Tx) (migration.Repairs, error) {
							rows, err := tx.QueryContext(
								ctx,
								`delete
								from foo
								where
									public_id is null
								returning
									id, name;
								`,
							)
							if err != nil {
								return nil, err
							}
							invalid := make([]string, 0)
							for rows.Next() {
								var id int
								var name string
								if err := rows.Scan(&id, &name); err != nil {
									return nil, err
								}
								invalid = append(invalid, fmt.Sprintf("%d:%s", id, name))
							}

							if len(invalid) > 0 {
								return append([]string{"deleted foos:"}, invalid...), nil
							}
							return nil, nil
						},
						RepairDescription: "will delete any foo that has a null public_id",
					},
				},
			},
			Priority: 0,
		},
	}

	// Run manager with marking any migrations for repair.
	// The check function in the hook should detect a problem and
	// fail the migration.
	m, err := schema.NewManager(
		ctx,
		schema.Dialect(dialect),
		d,
		schema.WithEditions(editions),
	)
	if err != nil {
		log.Fatalf(err.Error())
	}
	_, err = m.ApplyMigrations(ctx)
	checkErr, _ := err.(schema.MigrationCheckError)
	fmt.Println(checkErr.Error())
	fmt.Println(strings.Join(checkErr.Problems, "\n"))
	fmt.Printf("repair: %s\n", checkErr.RepairDescription)

	// Now run with the migration marked for repair.
	// The repair function should run, delete data, and the migration
	// will succeed.
	m, err = schema.NewManager(
		ctx,
		schema.Dialect(dialect),
		d,
		schema.WithEditions(editions),
		schema.WithRepairMigrations(schema.RepairMigrations{
			"hooks_example": map[int]bool{
				2: true,
			},
		}),
	)

	logs, err := m.ApplyMigrations(ctx)
	if err != nil {
		log.Fatalf(err.Error())
	}
	for _, log := range logs {
		fmt.Printf("%s:%d:\n", log.Edition, log.Version)
		fmt.Println(strings.Join(log.Entry, "\n"))
	}

}

Output:

check failed for hooks_example:2
invalid foos:
1:Alice
2:Bob
repair: will delete any foo that has a null public_id
hooks_example:2:
deleted foos:
1:Alice
2:Bob

func NewManager 

func NewManager(ctx context.Context, dialect Dialect, db *sql.DB, opt ...Option) (*Manager, error)

NewManager creates a new schema manager. An error is returned if the provided dialect is unrecognized or if the passed in db is unreachable.

func (*Manager) ApplyMigrations added in v0.7.0 

func (b *Manager) ApplyMigrations(ctx context.Context) ([]RepairLog, error)

ApplyMigrations updates the database schema to match the latest version known by the boundary binary. An error is not returned if the database is already at the most recent version.

func (*Manager) Close added in v0.10.4 

func (b *Manager) Close(ctx context.Context) error

Close unlocks the database shared lock and closes the underlying database connection afterwards.

func (*Manager) CurrentState 

func (b *Manager) CurrentState(ctx context.Context) (*State, error)

CurrentState provides the state of the boundary schema contained in the backing database.

func (*Manager) ExclusiveLock 

func (b *Manager) ExclusiveLock(ctx context.Context) error

ExclusiveLock attempts to obtain an exclusive lock on the database. An error is returned if a lock was unable to be obtained.

func (*Manager) ExclusiveUnlock 

func (b *Manager) ExclusiveUnlock(ctx context.Context) error

ExclusiveUnlock releases a shared lock on the database. If this fails for whatever reason an error is returned. Unlocking a lock that is not held is not an error.

func (*Manager) GetMigrationLog added in v0.7.0 

func (b *Manager) GetMigrationLog(ctx context.Context, opt ...Option) ([]LogEntry, error)

GetMigrationLog will retrieve the migration logs from the db for the last migration. Once it's read the entries, it will delete them from the database. The WithDeleteLog option is supported and will remove all log entries when provided.

func (*Manager) SharedLock 

func (b *Manager) SharedLock(ctx context.Context) error

SharedLock attempts to obtain a shared lock on the database. This can fail if an exclusive lock is already held. If the lock can't be obtained an error is returned.

func (*Manager) SharedUnlock 

func (b *Manager) SharedUnlock(ctx context.Context) error

SharedUnlock releases a shared lock on the database. If this fails for whatever reason an error is returned. Unlocking a lock that is not held is not an error.

type MigrationCheckError added in v0.10.2 

type MigrationCheckError struct {
	Version           int
	Edition           string
	Problems          migration.Problems
	RepairDescription string
}

MigrationCheckError is an error returned when a migration hook check function reports an error.

func (MigrationCheckError) Error added in v0.10.2 

func (e MigrationCheckError) Error() string

type Option added in v0.2.0 

type Option func(*options)

Option - how Options are passed as arguments.

func WithDeleteLog added in v0.2.0 

func WithDeleteLog(del bool) Option

WithDeleteLog provides an option to specify the deletion of log entries.

func WithEditions added in v0.7.0 

func WithEditions(editions edition.Editions) Option

WithEditions provides an optional migration states.

func WithRepairMigrations added in v0.10.2 

func WithRepairMigrations(r RepairMigrations) Option

WithRepairMigrations provides an option to specify the set of migrations that should run their repair functions if there is a failure on a prehook check.

type PartialEditions added in v0.7.0 

type PartialEditions map[string]int

PartialEditions is used by TestCreatePartialEditions. It is a map of edition names to the max version that should be included.

type RepairLog added in v0.10.2 

type RepairLog struct {
	Edition string
	Version int
	Entry   migration.Repairs
}

RepairLog represents a log entry generated by a repair function.

type RepairMigrations added in v0.10.2 

type RepairMigrations map[string]map[int]bool

RepairMigrations is a set of migration versions grouped by edition that should have their coresponding repair functions run if the check function reports an error.

func (RepairMigrations) Add added in v0.10.2 

func (r RepairMigrations) Add(edition string, version int)

Add adds the edition and version to the set.

func (RepairMigrations) IsSet added in v0.10.2 

func (r RepairMigrations) IsSet(edition string, version int) bool

IsSet checks for the existence of the given edition and version.

type State 

type State struct {
	// Initialized indicates if the current database has been previously initialized.
	Initialized bool
	Editions    []EditionState
}

State contains information regarding the current state of a boundary database's schema.

func (State) MigrationsApplied added in v0.7.0 

func (s State) MigrationsApplied() bool

MigrationsApplied checks to see that all Editions are in the Equal SchemaState.

Source Files

View all Source files

Directories

Path Synopsis
internal
edition
Package edition provides internal structs for the schema package for defining and organizing database migration editions.
 Package edition provides internal structs for the schema package for defining and organizing database migration editions.
log
Package log provides internal structs and options for the schema package for tracking logs generated when applying migrations.
 Package log provides internal structs and options for the schema package for tracking logs generated when applying migrations.
postgres
Package postgres provides an implementation of the schema.driver interface for a PostgreSQL database.
 Package postgres provides an implementation of the schema.driver interface for a PostgreSQL database.
provider
Package provider provides an iterator for iterating over all of the migration statements that need to be applied.
 Package provider provides an iterator for iterating over all of the migration statements that need to be applied.
Package migrations contains the base sql statements needed to bootstrap the migration process.
 Package migrations contains the base sql statements needed to bootstrap the migration process.
oss
Package oss is used to embed the sql statements for the oss edition and registering the edition for the schema.Manager.
 Package oss is used to embed the sql statements for the oss edition and registering the edition for the schema.Manager.
oss/internal/hook46001
Package hook46001 implements the hook CheckFunc & RepairFunc definitions for CVE-2022-36130.
 Package hook46001 implements the hook CheckFunc & RepairFunc definitions for CVE-2022-36130.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.