migration

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 15, 2019 License: GPL-3.0 Imports: 11 Imported by: 0

README

Migrations

The migration package provides the tools to detect and manage the changes made to application models, store them in version control and apply them to the database schema.

  1. Quick start
  2. Managing changes
  3. Supported operations

Quick start

Example usage:

package main

import (
    "fmt"
    _ "github.com/gwenn/gosqlite"  // Imports SQLite driver.
    "github.com/moiseshiraldo/gomodel"
    "github.com/moiseshiraldo/gomodel/migration"
    "os"
)

var User = gomodel.New(
    "User",
    gomodel.Fields{
        "email":   gomodel.CharField{MaxLength: 100, Index: true},
        "active":  gomodel.BooleanField{DefaultFalse: true},
        "created": gomodel.DateTimeField{AutoNowAdd: true},
    },
    gomodel.Options{},
)

var app = gomodel.NewApp("main", "", User.Model)

func setup() {
    gomodel.Register(app)
    gomodel.Start(map[string]gomodel.Database{
        "default": {
            Driver: "sqlite3",
            Name:   ":memory:",
        },
    })
}

func main() {
    setup()
    // Detects changes and writes migration files.
    if _, err := migration.Make("main", migration.MakeOptions{}); err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    // Runs all pending migrations.
    if err := migration.Run(migration.RunOptions{}); err != nil {
        fmt.Println("something went wrong:", err)
        os.Exit(1)
    }
    fmt.Println("Changes applied to database schema!")
}

Check the MakeOptions and RunOptions documentation for more details.

If the database schema is not going to change, or you don't need to keep track of the changes (e.g. in-memory SQLite database), you can detect and apply all the changes directly using the MakeAndRun function:

func main() {
    setup()
    if err := migration.MakeAndRun("default"); err != nil {
        fmt.Println("something went wrong:", err)
    }
}

Managing changes

When your project contains a considerable number of models that change over time, it's probably a good idea to keep track of the database schema changes along with the code ones.

Migration files

GoModel will read and write the schema changes to the directory specified when you create and register an application:

var app = gomodel.NewApp("main", "/home/dev/project/main/migrations", User.Model)

If you specify a relative path, the full one will be constructed from $GOPATH/src. Each application must have a unique migrations path, that only contains JSON files following the naming convention {number}_{name}.json, where number is a four digit number representing the order of the node in the graph of changes and name can be any arbitrary name (e.g. 0001_initial.json).

You can create a new empty migration using the Make function:

_, err := migration.Make("main", migration.MakeOptions{Empty: true})
if err != nil {
    fmt.Println("something went wrong:", err)
}

The migration file will look something like this:

{
  "App": "main",
  "Dependencies": [
    [
      "main",
      "0001_initial"
    ]
  ],
  "Operations": []
}

The App attribute is the name of the application. Dependencies is the list of migrations (application and full name) that should be applied before the described one. And Operations is the list of changes to be applied to the database schema (see supported operations).

Automatic detection

The Make function can be used to automatically detect any changes on the application models and write them to migration files:

if _, err := migration.Make("main", migration.MakeOptions{}); err != nil {
    fmt.Println(err)
}

The function will load and process the existing migration files, compare the resulted state with the model definitions and write any changes to new migration files.

Applying migrations

The Run function can be used to apply changes from migration files to the database schema. For example:

options := migration.RunOptions{
    App: "main",
    Node: "0002",
    Database: "default",
}
if _, err := migration.Run(options); err != nil {
    fmt.Println(err)
}

The code above would apply the changes up to and including the second migration. If any migration greater than the second one was already applied, it would be reverted.

Supported operations

CreateModel

{
  "CreateModel": {
    "Name": "User",
    "Table": "users",
    "Fields": {
      "id": {
        "IntegerField": {
          "PrimaryKey": true,
          "Auto": true
        }
      },
      "email": {
        "CharField": {
          "MaxLength": 100,
          "Index": true
        }
      }
    }
  }
}

DeleteModel

{
  "DeleteModel": {
    "Name": "User",
  }
}

AddFields

{
  "AddFields": {
    "Model": "User",
    "Fields": {
      "active": {
        "BooleanField": {
          "DefaultFalse": true
        }
      },
      "created": {
        "DateTimeField": {
          "AutoNowAdd": true
        }
      }
    }
  }
}

RemoveFields

{
  "RemoveFields": {
    "Model": "User",
    "Fields": [
      "active",
      "created"
    ]
  }
}

AddIndex

{
  "AddIndex": {
    "Model": "User",
    "Name": "users_user_email_auto_idx",
    "Fields": [
      "email"
    ]
  }
}

RemoveIndex

{
  "RemoveIndex": {
    "Model": "User",
    "Name": "users_user_email_auto_idx"
  }
}

Documentation

Overview

Package migration provides the tools to detect and manage the changes made to application models, store them in version control and apply them to the database schema.

The Make function detects and returns all the changes made to the models in the specified application, and optionally writes them to the project source in JSON format.

The Run function loads the changes from the project source and apply them to the database schema.

Alternatively, the MakeAndRun function can be used to directly create all the models on the database schema without storing changes in the project source.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var FileNameRegex = regexp.MustCompile(`^([0-9]{4})_\w+\.json$`)

FileNameRegex is a regular expression to validate migration file names (e.g. 0001_initial.json).

View Source 
var Migration = gomodel.New(
	"Migration",
	gomodel.Fields{
		"app":     gomodel.CharField{MaxLength: 50},
		"number":  gomodel.IntegerField{},
		"name":    gomodel.CharField{MaxLength: 100},
		"applied": gomodel.DateTimeField{AutoNowAdd: true},
	},
	gomodel.Options{},
)

Migration holds the model definition to store applied nodes in the database.

View Source 
var NodeNameRegex = regexp.MustCompile(`^([0-9]{4})_\w+$`)

NodeNameRegex is a regular expression to validate node names (e.g. 0001_initial).

Functions

func MakeAndRun 

func MakeAndRun(database string) error

MakeAndRun propagates all the application models to the database schema.

func RegisterOperation 

func RegisterOperation(op Operation) error

RegisterOperation registers a custom operation. Returns an error if the operation name returned by the OpName method already exists.

func Run 

func Run(options RunOptions) error

Run applies the migrations specified by RunOptions to the database schema.

Example 
User := gomodel.New(
	"User",
	gomodel.Fields{
		"email":   gomodel.CharField{Unique: true, MaxLength: 100},
		"active":  gomodel.BooleanField{DefaultFalse: true},
		"created": gomodel.DateTimeField{AutoNowAdd: true},
	},
	gomodel.Options{},
)
app := gomodel.NewApp("users", "myproject/users/migrations", User.Model)
gomodel.Register(app)
gomodel.Start(map[string]gomodel.Database{
	"default": {
		Driver:   "postgres",
		Name:     "gomodeltest",
		User:     "local",
		Password: "1234",
	},
})
if err := Run(RunOptions{App: "users", Node: "0001_initial"}); err != nil {
	fmt.Println(err)
}

Output:

Types

type AddFields 

type AddFields struct {
	Model  string
	Fields gomodel.Fields
}

AddFields implements the Operation interface to add new fields.

func (AddFields) Backwards 

func (op AddFields) Backwards(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Backwards removes the columns from the table on the database.

func (AddFields) OpName 

func (op AddFields) OpName() string

OpName returns the operation name.

func (AddFields) Run 

func (op AddFields) Run(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Run adds the new columns to the table on the database.

func (AddFields) SetState 

func (op AddFields) SetState(state *AppState) error

SetState adds the new fields to the model in the given the application state.

type AddIndex 

type AddIndex struct {
	Model  string
	Name   string
	Fields []string
}

AddIndex implements the Operation interface to add an index.

func (AddIndex) Backwards 

func (op AddIndex) Backwards(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Backwards drops the index from the database.

func (AddIndex) OpName 

func (op AddIndex) OpName() string

OpName returns the operation name.

func (AddIndex) Run 

func (op AddIndex) Run(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Run creates the index on the database.

func (AddIndex) SetState 

func (op AddIndex) SetState(state *AppState) error

SetState adds the index to the model in the given application state.

type AppNotFoundError 

type AppNotFoundError struct {
	Name  string // Application name.
	Trace ErrorTrace
}

AppNotFoundError is raised when the named application is not registered.

func (*AppNotFoundError) Error 

func (e *AppNotFoundError) Error() string

Error implements the error interface.

type AppState 

type AppState struct {

	// Models holds the model definitions for this application state.
	Models map[string]*gomodel.Model
	// contains filtered or unexported fields
}

AppState holds the application state for a certain node of the changes graph.

func Make 

func Make(appName string, options MakeOptions) (*AppState, error)

Make detects the changes between the gomodel.Model definitions and the migration files for the application named by appName.

It returns the *AppState containing the migrations and writes them to files depending on MakeOptions.

Example 
User := gomodel.New(
	"User",
	gomodel.Fields{
		"email":   gomodel.CharField{Unique: true, MaxLength: 100},
		"active":  gomodel.BooleanField{DefaultFalse: true},
		"created": gomodel.DateTimeField{AutoNowAdd: true},
	},
	gomodel.Options{},
)
app := gomodel.NewApp("users", "myproject/users/migrations", User.Model)
gomodel.Register(app)
gomodel.Start(map[string]gomodel.Database{
	"default": {
		Driver:   "postgres",
		Name:     "gomodeltest",
		User:     "local",
		Password: "1234",
	},
})
if _, err := Make("users", MakeOptions{}); err != nil {
	fmt.Println(err)
}

Output:

func (AppState) Fake 

func (state AppState) Fake(database string, nodeName string) error

Fake fakes the changes up to and including the node named by nodeName, for the db schema named by database.

func (*AppState) MakeMigrations 

func (state *AppState) MakeMigrations() ([]*Node, error)

MakeMigrations returns a list of nodes containing the changes between the gomodel.Model definitions and the migrations files for this application.

func (AppState) Migrate 

func (state AppState) Migrate(database string, nodeName string) error

Migrate applies the changes up to and including the node named by nodeName, using the db schema named by database.

It will migrate all the nodes if nodeName is blank.

It will run backwards if the given nodeName precedws the current applied node.

type CircularDependencyError 

type CircularDependencyError struct {
	Trace ErrorTrace
}

CircularDependencyError is raised when the migration files contain circular dependencies.

func (*CircularDependencyError) Error 

func (e *CircularDependencyError) Error() string

Error implements the error interface.

type CreateModel 

type CreateModel struct {
	Name string
	// Table is used to set a custom name for the database table. If blank,
	// the table will be created as {app_name}_{model_name} all lowercase.
	Table  string `json:",omitempty"`
	Fields gomodel.Fields
}

CreateModel implements the Operation interface to create a new model.

func (CreateModel) Backwards 

func (op CreateModel) Backwards(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Backwards drops the table from the database.

func (CreateModel) OpName 

func (op CreateModel) OpName() string

OpName returns the operation name.

func (CreateModel) Run 

func (op CreateModel) Run(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Run creates the table on the database.

func (CreateModel) SetState 

func (op CreateModel) SetState(state *AppState) error

SetState adds the new model to the given application state.

type DeleteModel 

type DeleteModel struct {
	Name string
}

DeleteModel implements the operation to delete a model.

func (DeleteModel) Backwards 

func (op DeleteModel) Backwards(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Backwards creates the table on the database.

func (DeleteModel) OpName 

func (op DeleteModel) OpName() string

OpName returns the operation name.

func (DeleteModel) Run 

func (op DeleteModel) Run(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Run drops the table from the database.

func (DeleteModel) SetState 

func (op DeleteModel) SetState(state *AppState) error

SetState removes the model from the given application state.

type DuplicateNumberError 

type DuplicateNumberError struct {
	Trace ErrorTrace
}

DuplicateNumberError is raised if an application contains any migration file with a duplicate number.

func (*DuplicateNumberError) Error 

func (e *DuplicateNumberError) Error() string

Error implements the error interface.

type ErrorTrace 

type ErrorTrace struct {
	Node      *Node
	Operation Operation
	Err       error
}

ErrorTrace holds the context information of a migration error.

func (ErrorTrace) String 

func (e ErrorTrace) String() string

String implements the fmt.Stringer interface.

type InvalidDependencyError 

type InvalidDependencyError struct {
	Trace ErrorTrace
}

InvalidDependencyError is raised when a migration file contains invalid dependencies.

func (*InvalidDependencyError) Error 

func (e *InvalidDependencyError) Error() string

Error implements the error interface.

type LoadError 

type LoadError struct {
	Trace ErrorTrace
}

LoadError is raised when a node fails to read from file.

func (*LoadError) Error 

func (e *LoadError) Error() string

Error implements the error interface.

type MakeOptions 

type MakeOptions struct {
	Empty     bool // Empty option is used to create an empty migration file.
	OmitWrite bool // OmitWrite options is used to skip writing the file.
}

MakeOptions holds the options for the Make function.

type NameError 

type NameError struct {
	Name  string // Node name.
	Trace ErrorTrace
}

NameError is raised if an invalid migration node name is provided.

func (*NameError) Error 

func (e *NameError) Error() string

Error implements the error interface.

type NoAppMigrationsError 

type NoAppMigrationsError struct {
	Name  string // Node name.
	Trace ErrorTrace
}

NoAppMigrationsError is raised when trying to migrate an application with no migrations.

func (*NoAppMigrationsError) Error 

func (e *NoAppMigrationsError) Error() string

Error implements the error interface.

type Node 

type Node struct {
	// App is the application name.
	App string
	// Dependencies is the list of dependencies for this node. Each dependency
	// is a list of two elements, application name and migration name
	// (e.g. ["users", "0001_initial"])
	Dependencies [][]string
	// Operations is the list of operations describing the changes.
	Operations OperationList
	// contains filtered or unexported fields
}

A Node represents a point in the application changes graph.

func (*Node) Backwards 

func (n *Node) Backwards(db gomodel.Database) error

Backwards reverses the node changes (and the applied nodes depending on it) on the database schema given by db. All the queries will be run inside a transaction if supported by the database.

func (*Node) Fake 

func (n *Node) Fake(db gomodel.Database) error

Fake marks the node as applied without making any changes to the database schema.

func (*Node) FakeBackwards 

func (n *Node) FakeBackwards(db gomodel.Database) error

FakeBackwards marks the node as unapplied without reversing any change on the database schema.

func (*Node) Load 

func (n *Node) Load() error

Load reads the node details from the corresponding file in the migrations folder.

func (Node) Name 

func (n Node) Name() string

Name returns the node name (e.g. 0001_initial)

func (*Node) Run 

func (n *Node) Run(db gomodel.Database) error

Run applies the node changes (and its unapplied dependencies) to the database schema given by db. All the queries will be run inside a transaction if supported by the database.

func (Node) Save 

func (n Node) Save() error

Save writes the the JSON representation of the node to the migrations folder.

type Operation 

type Operation interface {
	// OpName returns the operation name, which must be unique.
	OpName() string
	// SetState applies the operation changes to the given application state.
	SetState(state *AppState) error
	// Run applies the operation changes to the database schema. The method
	// is automatically called when a Node is migrating forwards.
	//
	// The engine is the database (or transaction if supported) Engine.
	//
	// The state is the application state resulted from the operation changes.
	//
	// The prevState is the application state previous to the operation changes.
	Run(engine gomodel.Engine, state *AppState, prevState *AppState) error
	// Backwards reverse the operation changes on the database schema. The
	// method is automatically called when a Node is migrating backwards.
	//
	// The engine is the database (or transaction if supported) Engine.
	//
	// The state is the application state resulted from the operation changes.
	//
	// The prevState is the application state previous to the operation changes.
	Backwards(engine gomodel.Engine, state *AppState, prevState *AppState) error
}

The Operation interface represents a change in the application state (models, fields, indexes...) and how to propagate it to the database schema.

Custom operations can be registered using the RegisterOperation function.

type OperationList 

type OperationList []Operation

OperationList represents the list of operations of a migration Node.

func (OperationList) MarshalJSON 

func (opList OperationList) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for the OperationList type. The type is serialized into a list of JSON objects, where the key is the name of the Operation returned by the OpName method and the value is the serliazed type implementing the Operation interface.

func (*OperationList) UnmarshalJSON 

func (opList *OperationList) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface. An error is returned if the name of the operation is not registered.

type OperationRunError 

type OperationRunError struct {
	Trace ErrorTrace
}

OperationRunError is raised if an operation fails to apply to the database schema.

func (*OperationRunError) Error 

func (e *OperationRunError) Error() string

Error implements the error interface.

type OperationStateError 

type OperationStateError struct {
	Trace ErrorTrace
}

OperationStateError is raised if a migration file contains invalid operations.

func (*OperationStateError) Error 

func (e *OperationStateError) Error() string

Error implements the error interface.

type PathError 

type PathError struct {
	App   string // Application name.
	Trace ErrorTrace
}

PathError is raised if reaading from the applicatoin path or writing to it is not possible.

func (*PathError) Error 

func (e *PathError) Error() string

Error implements the error interface.

type RemoveFields 

type RemoveFields struct {
	Model  string
	Fields []string
}

RemoveFields implements the Operation interface to remove fields.

func (RemoveFields) Backwards 

func (op RemoveFields) Backwards(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Backwards adds the columns to the table on the database.

func (RemoveFields) OpName 

func (op RemoveFields) OpName() string

OpName returns the operation name.

func (RemoveFields) Run 

func (op RemoveFields) Run(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Run removes the columns from the table on the database.

func (RemoveFields) SetState 

func (op RemoveFields) SetState(state *AppState) error

SetState removes the fields from the model in the given the application state.

type RemoveIndex 

type RemoveIndex struct {
	Model string
	Name  string
}

RemoveIndex implements the Operation interface to remove an index.

func (RemoveIndex) Backwards 

func (op RemoveIndex) Backwards(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Backwards creates the index on the database.

func (RemoveIndex) OpName 

func (op RemoveIndex) OpName() string

OpName returns the operation name.

func (RemoveIndex) Run 

func (op RemoveIndex) Run(
	engine gomodel.Engine,
	state *AppState,
	prevState *AppState,
) error

Run drops the index from the database.

func (RemoveIndex) SetState 

func (op RemoveIndex) SetState(state *AppState) error

SetState removes the index from the model in the given application state.

type RunOptions 

type RunOptions struct {
	App      string // Application name or blank for all applications.
	Node     string // Node name (e.g. 0001_initial or just 0001).
	Fake     bool   // Fake is used to apply the migration without DB changes.
	Database string // Database name or blank for default.
}

RunOptions holds the options for the Run function.

type SaveError 

type SaveError struct {
	Trace ErrorTrace
}

SaveError is raised when a node fails to write to file.

func (*SaveError) Error 

func (e *SaveError) Error() string

Error implements the error interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.