migration

package
v1.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 12, 2020 License: Apache-2.0 Imports: 12 Imported by: 13

Documentation

Overview

Package migration provides tooling necessary for working with schema versioned entities. Functionality provided here can be applied both to messages and models.

Global preparation.

1. update application genesis to provide "migration" configuration. You can find documented configuration declaration in the protobuf declaration file,

2. register migration message handlers using `RegisterRouters` function

3. register migration bucket query using `RegisterQuery` function

Extension integration.

1.update all protobuf message declarations that are to be schema versioned. First attribute must be metadata. For example: 

import "github.com/iov-one/weave/codec.proto";
message MyMessage {
  weave.Metadata metadata = 1;
  ...
}

Make sure that whenever you create a new entity, metadata attribute is provided as `nil` metadata value is not valid.

2. register your migrations functions in package `init`. Schema version is declared per package not per entity so each upgrade must provide migration function for all entities. Use `migration.NoModification` for those entities that require no change. For example: 

func init() {
    func init() {
        migration.MustRegister(1, &MyModel{}, migration.NoModification)
        migration.MustRegister(1, &MyMessage{}, migration.NoModification)
    }
}

3. change your bucket implementation to embed `migration.Bucket` instead of `orm.Bucket`

4. wrap your handler with `migration.SchemaMigratingHandler` to ensure all messages are always migrated to the latest schema before being passed to the handler,

5. make sure `.Metadata.Schema` attribute of newly created messages is set. This is not necessary for models as it will default to the current schema version.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrInvalidLengthCodec = fmt.Errorf("proto: negative length found during unmarshaling")
	ErrIntOverflowCodec   = fmt.Errorf("proto: integer overflow")
)

Functions

func Apply added in v0.17.0 

func Apply(db weave.ReadOnlyKVStore, m Migratable, migrateTo uint32) error

Apply updates the object by applying all missing data migrations. Even a no modification migration is updating the metadata to point to the latest data format version.

Because changes are applied directly on the passed object (in place), even if this function fails some of the data migrations might be applied.

A valid object metadata must contain a schema version greater than zero. Not migrated object (initial state) is always having a metadata schema value set to 1.

Validation method is called only on the final version of the object.

func CurrentAdmin added in v0.22.0 

func CurrentAdmin(db weave.ReadOnlyKVStore) (weave.Address, error)

CurrentAdmin returns migration extension admin address as currently configured.

This function is useful for the `gconf` package users to provide a one time authentication address during configuration initialization. See `gconf.NewUpdateConfigurationHandler` for more details.

func Migrate added in v0.17.0 

func Migrate(
	db weave.ReadOnlyKVStore,
	packageName string,
	value interface{},
) error

Migrate will query the current schema of the named package and attempt to Migrate the passed value up to the current value.

Returns an error if the passed value is not Migratable, not registered with migrations, missing Metadata, has a Schema higher than currentSchema, if the final migrated value is invalid, or other such conditions.

If this returns no error, you can safely use the contents of value in code working with the currentSchema.

func MustInitPkg 

func MustInitPkg(db weave.KVStore, packageNames ...string)

MustInitPkg initialize schema versioning for given package names. This registers a version one schema. This function panics if not successful. It is safe to call this function many times as duplicate registrations are ignored.

func MustRegister 

func MustRegister(migrationTo uint32, msgOrModel Migratable, fn Migrator)

MustRegister registers a migration function for a given message or model. Migration function will be called when migrating data from a version one less than migrationTo value. Minimal allowed migrationTo version is 1. Version upgrades for each type must be registered in sequential order.

func NoModification 

func NoModification(weave.ReadOnlyKVStore, Migratable) error

NoModification is a migration function that migrates data that requires no change. It should be used to register migrations that do not require any modifications.

func RefuseMigration 

func RefuseMigration(weave.ReadOnlyKVStore, Migratable) error

RefuseMigration is a migration function that always fails. Its use is expected when there is no migration path to given version. This is accepted migration callback function for messages but should be avoided for models.

func RegisterQuery 

func RegisterQuery(qr weave.QueryRouter)

RegisterQuery registers schema bucket for querying.

func RegisterRoutes 

func RegisterRoutes(r weave.Registry, auth x.Authenticator)

RegisterRoutes registers handlers for feedlist message processing.

func SchemaMigratingHandler 

func SchemaMigratingHandler(packageName string, h weave.Handler) weave.Handler

SchemaMigratingHandler returns a weave handler that will ensure incoming messages are in the current schema version format. If a message in older schema is handled then it is first being migrated. Messages that cannot be migrated to current schema version are returning migration error. This functionality is executed before the decorated handler and it is completely transparent to the wrapped handler.

func SchemaMigratingRegistry added in v0.16.0 

func SchemaMigratingRegistry(packageName string, r weave.Registry) weave.Registry

SchemaMigratingRegistry decorates given registry to always migrate schema of an incoming message, before passing it down to a registered handler. Decorating a registry with this function is equivalent to using a raw registry and wrapping each handler with SchemaMigratingHandler before registering. This function force all registration to use schema migration and must not be used together with SchemaRoutingHandler functionality.

func SchemaRoutingHandler 

func SchemaRoutingHandler(handlers []weave.Handler) weave.Handler

SchemaRoutingHandler clubs together message handlers for a single type message but different schema formats. Each handler is registered together with the lowest schema version that it supports. For example 

handler := SchemaRoutingHandler([]weave.Handler{
  1: &MyHandlerVersionAlpha{},
  7: &MyHandlerVersionBeta{},
})

In the above setup, messages with schema version 1 to 6 will be handled by the alpha handler. Messages with schema version 7 and above are passed to the beta handler.

It is not allowed to use an empty schemaRoutingHandler instance. It is not allowed to register a handler for schema version zero. This function panics if any of those requirements is not met.

All messages processed by this handler must implement Migratable interface.

Types

type Bucket 

type Bucket struct {
	orm.Bucket
	// contains filtered or unexported fields
}

Bucket is a storage engine that supports and requires schema versioning. I enforce every model to contain schema version information and where needed migrates objects on the fly, before returning to the user.

This bucket does not migrate on the fly the data returned by the queries. Both Register and Query methods are using orm.Bucket implementation to return data as stored in the database. This is important for the proof to work. Query returned data must never be altered.

func NewBucket 

func NewBucket(packageName string, bucketName string, model orm.Model) Bucket

NewBucket returns a new instance of a schema aware bucket implementation. Package name is used to track schema version. Bucket name is the namespace for the stored entity. Model is the type of the entity this bucket is maintaining.

func (Bucket) Get 

func (svb Bucket) Get(db weave.ReadOnlyKVStore, key []byte) (orm.Object, error)

func (Bucket) Save 

func (svb Bucket) Save(db weave.KVStore, obj orm.Object) error

func (Bucket) WithIndex 

func (svb Bucket) WithIndex(name string, indexer orm.Indexer, unique bool) orm.Bucket

func (Bucket) WithMultiKeyIndex 

func (svb Bucket) WithMultiKeyIndex(name string, indexer orm.MultiKeyIndexer, unique bool) orm.Bucket

type Configuration 

type Configuration struct {
	// Admin holds the address of the administrator allowed to upgrade schema
	// versions. If you wish to permit more than one entity to be an admin, use
	// multisig.
	Admin github_com_iov_one_weave.Address `protobuf:"bytes,2,opt,name=admin,proto3,casttype=github.com/iov-one/weave.Address" json:"admin,omitempty"`
}

func (*Configuration) Descriptor 

func (*Configuration) Descriptor() ([]byte, []int)

func (*Configuration) GetAdmin 

func (m *Configuration) GetAdmin() github_com_iov_one_weave.Address

func (*Configuration) Marshal 

func (m *Configuration) Marshal() (dAtA []byte, err error)

func (*Configuration) MarshalTo 

func (m *Configuration) MarshalTo(dAtA []byte) (int, error)

func (*Configuration) ProtoMessage 

func (*Configuration) ProtoMessage()

func (*Configuration) Reset 

func (m *Configuration) Reset()

func (*Configuration) Size 

func (m *Configuration) Size() (n int)

func (*Configuration) String 

func (m *Configuration) String() string

func (*Configuration) Unmarshal 

func (m *Configuration) Unmarshal(dAtA []byte) error

func (*Configuration) Validate 

func (c *Configuration) Validate() error

func (*Configuration) XXX_DiscardUnknown 

func (m *Configuration) XXX_DiscardUnknown()

func (*Configuration) XXX_Marshal 

func (m *Configuration) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Configuration) XXX_Merge 

func (m *Configuration) XXX_Merge(src proto.Message)

func (*Configuration) XXX_Size 

func (m *Configuration) XXX_Size() int

func (*Configuration) XXX_Unmarshal 

func (m *Configuration) XXX_Unmarshal(b []byte) error

type Initializer 

type Initializer struct{}

Initializer fulfils the InitStater interface to load data from the genesis file

func (Initializer) FromGenesis 

func (Initializer) FromGenesis(opts weave.Options, params weave.GenesisParams, kv weave.KVStore) error

FromGenesis will parse initial account info from genesis and save it to the database

type Migratable 

type Migratable interface {
	// GetMetadata returns a metadata information about given entity. This
	// method is generated by the protobuf compiler for every message that
	// has a metadata field.
	GetMetadata() *weave.Metadata

	// Validate returns an error instance state is not valid. This method
	// is implemented by all models and messages.
	Validate() error
}

type Migrator 

type Migrator func(weave.ReadOnlyKVStore, Migratable) error

Migrator is a function that migrates in place an entity of a single type.

type ModelBucket added in v0.16.0 

type ModelBucket struct {
	// contains filtered or unexported fields
}

ModelBucket implements the orm.ModelBucket interface and provides the same functionality with additional model schema migration.

func NewModelBucket added in v0.16.0 

func NewModelBucket(packageName string, b orm.ModelBucket) *ModelBucket

func (*ModelBucket) ByIndex added in v0.16.0 

func (m *ModelBucket) ByIndex(db weave.ReadOnlyKVStore, indexName string, key []byte, dest orm.ModelSlicePtr) ([][]byte, error)

func (*ModelBucket) Delete added in v0.16.0 

func (m *ModelBucket) Delete(db weave.KVStore, key []byte) error

func (*ModelBucket) Has added in v0.16.0 

func (m *ModelBucket) Has(db weave.KVStore, key []byte) error

func (*ModelBucket) Index added in v0.24.0 

func (m *ModelBucket) Index(name string) (orm.Index, error)

func (*ModelBucket) One added in v0.16.0 

func (m *ModelBucket) One(db weave.ReadOnlyKVStore, key []byte, dest orm.Model) error

func (*ModelBucket) Put added in v0.16.0 

func (m *ModelBucket) Put(db weave.KVStore, key []byte, model orm.Model) ([]byte, error)

func (*ModelBucket) Register added in v0.16.0 

func (m *ModelBucket) Register(name string, r weave.QueryRouter)

type Schema 

type Schema struct {
	Metadata *weave.Metadata `protobuf:"bytes,1,opt,name=metadata,proto3" json:"metadata,omitempty"`
	// Pkg holds the name of the package that this migration is declared for.
	// For example, for extension `x/myext` package value is `myext`
	Pkg string `protobuf:"bytes,2,opt,name=pkg,proto3" json:"pkg,omitempty"`
	// Version holds the highest supported schema version.
	Version uint32 `protobuf:"varint,3,opt,name=version,proto3" json:"version,omitempty"`
}

Schema declares the maxiumum supported schema version for a package.

func (*Schema) Copy 

func (s *Schema) Copy() orm.CloneableData

func (*Schema) Descriptor 

func (*Schema) Descriptor() ([]byte, []int)

func (*Schema) GetMetadata 

func (m *Schema) GetMetadata() *weave.Metadata

func (*Schema) GetPkg 

func (m *Schema) GetPkg() string

func (*Schema) GetVersion 

func (m *Schema) GetVersion() uint32

func (*Schema) Marshal 

func (m *Schema) Marshal() (dAtA []byte, err error)

func (*Schema) MarshalTo 

func (m *Schema) MarshalTo(dAtA []byte) (int, error)

func (*Schema) ProtoMessage 

func (*Schema) ProtoMessage()

func (*Schema) Reset 

func (m *Schema) Reset()

func (*Schema) Size 

func (m *Schema) Size() (n int)

func (*Schema) String 

func (m *Schema) String() string

func (*Schema) Unmarshal 

func (m *Schema) Unmarshal(dAtA []byte) error

func (*Schema) Validate 

func (s *Schema) Validate() error

func (*Schema) XXX_DiscardUnknown 

func (m *Schema) XXX_DiscardUnknown()

func (*Schema) XXX_Marshal 

func (m *Schema) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Schema) XXX_Merge 

func (m *Schema) XXX_Merge(src proto.Message)

func (*Schema) XXX_Size 

func (m *Schema) XXX_Size() int

func (*Schema) XXX_Unmarshal 

func (m *Schema) XXX_Unmarshal(b []byte) error

type SchemaBucket 

type SchemaBucket struct {
	orm.Bucket
}

func NewSchemaBucket 

func NewSchemaBucket() *SchemaBucket

func (*SchemaBucket) Create 

func (b *SchemaBucket) Create(db weave.KVStore, s *Schema) (orm.Object, error)

Create adds given schema instance to the store and returns the ID of the newly inserted entity.

func (*SchemaBucket) CurrentSchema 

func (b *SchemaBucket) CurrentSchema(db weave.ReadOnlyKVStore, packageName string) (uint32, error)

CurrentSchema returns the current version of the schema for a given package. It returns ErrNotFound if no schema version was registered for this package. Minimum schema version is 1.

func (*SchemaBucket) Get 

func (b *SchemaBucket) Get(db weave.KVStore, key []byte) error

func (*SchemaBucket) Save 

func (b *SchemaBucket) Save(db weave.KVStore, obj orm.Object) error

Save persists the state of a given schema entity.

type SerialModelBucket added in v0.22.0 

type SerialModelBucket struct {
	// contains filtered or unexported fields
}

SerialModelBucket implements the orm.SerialModelBucket interface and provides the same functionality with additional model schema migration.

func NewSerialModelBucket added in v0.22.0 

func NewSerialModelBucket(packageName string, model orm.SerialModel, bucket orm.SerialModelBucket) *SerialModelBucket

func (*SerialModelBucket) ByID added in v0.22.0 

func (smb *SerialModelBucket) ByID(db weave.ReadOnlyKVStore, key []byte, dest orm.SerialModel) error

func (*SerialModelBucket) ByIndex added in v0.22.0 

func (smb *SerialModelBucket) ByIndex(db weave.ReadOnlyKVStore, indexName string, key []byte, dest orm.SerialModelSlicePtr) error

func (*SerialModelBucket) Delete added in v0.22.0 

func (smb *SerialModelBucket) Delete(db weave.KVStore, key []byte) error

func (*SerialModelBucket) Has added in v0.22.0 

func (smb *SerialModelBucket) Has(db weave.KVStore, key []byte) error

func (*SerialModelBucket) IndexScan added in v0.22.0 

func (smb *SerialModelBucket) IndexScan(db weave.ReadOnlyKVStore, indexName string, prefix []byte, reverse bool) (orm.SerialModelIterator, error)

func (*SerialModelBucket) PrefixScan added in v0.22.0 

func (smb *SerialModelBucket) PrefixScan(db weave.ReadOnlyKVStore, prefix []byte, reverse bool) (orm.SerialModelIterator, error)

func (*SerialModelBucket) Register added in v0.22.0 

func (smb *SerialModelBucket) Register(name string, r weave.QueryRouter)

func (*SerialModelBucket) Save added in v0.22.0 

func (smb *SerialModelBucket) Save(db weave.KVStore, model orm.SerialModel) error

type UpgradeSchemaMsg 

type UpgradeSchemaMsg struct {
	Metadata *weave.Metadata `protobuf:"bytes,1,opt,name=metadata,proto3" json:"metadata,omitempty"`
	// Name of the package that schema version upgrade is made for.
	Pkg string `protobuf:"bytes,2,opt,name=pkg,proto3" json:"pkg,omitempty"`
	// To version defines to which version upgrade this schema. It always must be
	// a value one higher than the currently registered schema or 1 if this is
	// the initial upgrade.
	// This value must be provided to avoid consecutive upgrades submitted by a
	// mistake. This ensures at most one delivery as upgrades to an invalid
	// version are rejected.
	ToVersion uint32 `protobuf:"varint,3,opt,name=to_version,json=toVersion,proto3" json:"to_version,omitempty"`
}

UpgradeSchemaMsg is a request to upgrade schema version of a given package by one version.

func (*UpgradeSchemaMsg) Descriptor 

func (*UpgradeSchemaMsg) Descriptor() ([]byte, []int)

func (*UpgradeSchemaMsg) GetMetadata 

func (m *UpgradeSchemaMsg) GetMetadata() *weave.Metadata

func (*UpgradeSchemaMsg) GetPkg 

func (m *UpgradeSchemaMsg) GetPkg() string

func (*UpgradeSchemaMsg) GetToVersion added in v0.22.0 

func (m *UpgradeSchemaMsg) GetToVersion() uint32

func (*UpgradeSchemaMsg) Marshal 

func (m *UpgradeSchemaMsg) Marshal() (dAtA []byte, err error)

func (*UpgradeSchemaMsg) MarshalTo 

func (m *UpgradeSchemaMsg) MarshalTo(dAtA []byte) (int, error)

func (UpgradeSchemaMsg) Path 

func (UpgradeSchemaMsg) Path() string

func (*UpgradeSchemaMsg) ProtoMessage 

func (*UpgradeSchemaMsg) ProtoMessage()

func (*UpgradeSchemaMsg) Reset 

func (m *UpgradeSchemaMsg) Reset()

func (*UpgradeSchemaMsg) Size 

func (m *UpgradeSchemaMsg) Size() (n int)

func (*UpgradeSchemaMsg) String 

func (m *UpgradeSchemaMsg) String() string

func (*UpgradeSchemaMsg) Unmarshal 

func (m *UpgradeSchemaMsg) Unmarshal(dAtA []byte) error

func (*UpgradeSchemaMsg) Validate 

func (msg *UpgradeSchemaMsg) Validate() error

func (*UpgradeSchemaMsg) XXX_DiscardUnknown 

func (m *UpgradeSchemaMsg) XXX_DiscardUnknown()

func (*UpgradeSchemaMsg) XXX_Marshal 

func (m *UpgradeSchemaMsg) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*UpgradeSchemaMsg) XXX_Merge 

func (m *UpgradeSchemaMsg) XXX_Merge(src proto.Message)

func (*UpgradeSchemaMsg) XXX_Size 

func (m *UpgradeSchemaMsg) XXX_Size() int

func (*UpgradeSchemaMsg) XXX_Unmarshal 

func (m *UpgradeSchemaMsg) XXX_Unmarshal(b []byte) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.