migration

README

Migration Manager

The migration package contains the migration.Manager, which is responsible for migrating data from store/v1 to store/v2. To ensure a smooth transition, the process is designed to lazily migrate data in the background without blocking root.Store operations.

Overview

The migration process involves several steps:

1. Create a snapshot of the current state while Commit operations continue to function with store/v1.
2. Restore the snapshot into the new StateStorage (SS) and StateCommitment (SC).
3. Sync recent state changes from store/v1 to the new SS and SC.
4. After syncing, the Commit operation will be switched to the new store/v2.

Taking a snapshot is a lightweight operation. The snapshot is not stored on disk but consumed by the Restore process, which replays state changes to the new SS and SC.

Note: After migration, store/v2 does not support historical queries. If historical data access is required, a full state migration to store/v2 is necessary.

Usage

You can create a new migration.Manager by calling the following function:

func NewManager(
    db corestore.KVStoreWithBatch, 
    sm *snapshots.Manager, 
    ss *storage.StorageStore, 
    sc *commitment.CommitStore, 
    logger log.Logger
) *Manager

• sc (Commitment Store) can be nil. In that case, the Manager will migrate only the state storage.
• The migration process is lazy, meaning data is migrated in the background while root.Store remains fully operational.

To initiate the migration process, call the Start method:

func (m *Manager) Start(ctx context.Context) error

Note: It should be called by the RootStore, running in the background.

Migration Flow

sequenceDiagram
    autonumber

    participant A as RootStore
    participant B as MigrationManager
    participant C as SnapshotsManager
    participant D as StateCommitment
    participant E as StateStorage

    A->>B: Start
    loop Old Data Migration
        B->>C: Create Snapshot
        C->>B: Stream Snapshot
        B->>D: State Sync (Restore)
        B->>E: Write Changeset (Restore)
    end

    loop New Commit Data Sync
        A->>B: Commit(Changeset)
        B->>B: Store Changeset
        B->>D: Commit Changeset
        B->>E: Write Changeset
    end

    B->>A: Switch to new store/v2

Key Considerations

Laziness and Background Operation

The migration is performed lazily, meaning it occurs in the background without interrupting the current operations on root.Store. This allows the chain to continue running while data is gradually migrated to store/v2. State synchronization ensures that any new state changes during the migration are also applied to store/v2.

However, note that there may be a performance impact depending on the size of the data being migrated, and it’s essential to monitor the migration process in production environments.

Handling Failures and Rollbacks

It is important to consider how the migration manager handles errors or system failures during the migration process:

• If the migration fails, there is no impact on the existing store/v1 operations, but need to restart the migration process from the scratch.
• In the event of a critical failure after migration, a rollback may not be possible, and it is needed to keep the store/v1 backup for a certain period.

Impact on Historical Queries

After the migration, the new store/v2 does not support historical queries. This limitation should be clearly understood before starting the migration process, especially if the node relies on historical data for any operations.

If historical queries are required, users must fully migrate all historical data to store/v2. Alternatively, keeping store/v1 accessible for historical queries could be an option.

Documentation

Index

• type Manager
  • func NewManager(db corestore.KVStoreWithBatch, sm *snapshots.Manager, ...) *Manager
  • func (m *Manager) Close() error
  • func (m *Manager) GetMigratedVersion() uint64
  • func (m *Manager) GetStateCommitment() *commitment.CommitStore
  • func (m *Manager) Migrate(height uint64) error
  • func (m *Manager) Start(version uint64, chChangeset <-chan *VersionedChangeset, chDone <-chan struct{}) error
  • func (m *Manager) Sync() error
• type MigrationStream
  • func NewMigrationStream(chBufferSize int) *MigrationStream
  • func (ms *MigrationStream) Close() error
  • func (ms *MigrationStream) CloseWithError(err error)
  • func (ms *MigrationStream) ReadMsg(msg proto.Message) error
  • func (ms *MigrationStream) WriteMsg(msg proto.Message) error
• type VersionedChangeset

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the migration of the whole state from store/v1 to store/v2.

func NewManager

func NewManager(db corestore.KVStoreWithBatch, sm *snapshots.Manager, sc *commitment.CommitStore, logger log.Logger) *Manager

NewManager returns a new Manager.

NOTE: `sc` can be `nil` if don't want to migrate the commitment.

func (*Manager) Close

func (m *Manager) Close() error

Close closes the manager. It should be called after the migration is done. It will close the db and notify the snapshotsManager that the migration is done.

func (*Manager) GetMigratedVersion

func (m *Manager) GetMigratedVersion() uint64

GetMigratedVersion returns the migrated version. It is used to check the migrated version in the RootStore.

func (*Manager) GetStateCommitment

func (m *Manager) GetStateCommitment() *commitment.CommitStore

GetStateCommitment returns the state commitment.

func (*Manager) Migrate

func (m *Manager) Migrate(height uint64) error

Migrate migrates the whole state at the given height to the new store/v2.

func (*Manager) Start

func (m *Manager) Start(version uint64, chChangeset <-chan *VersionedChangeset, chDone <-chan struct{}) error

Start starts the whole migration process. It migrates the whole state at the given version to the new store/v2 (both SC and SS). It also catches up the Changesets which are committed while the migration is in progress. `chChangeset` is the channel to receive the committed Changesets from the RootStore. `chDone` is the channel to receive the done signal from the RootStore. NOTE: It should be called by the RootStore, running in the background.

func (*Manager) Sync

func (m *Manager) Sync() error

Sync catches up the Changesets which are committed while the migration is in progress. It should be called after the migration is done.

type MigrationStream

type MigrationStream struct {
	// contains filtered or unexported fields
}

MigrationStream is a stream for migrating the whole IAVL state as a snapshot. It's used to sync the whole state from the store/v1 to store/v2. The main idea is to use the same snapshotter interface without writing to disk.

func NewMigrationStream

func NewMigrationStream(chBufferSize int) *MigrationStream

NewMigrationStream returns a new MigrationStream.

func (*MigrationStream) Close

func (ms *MigrationStream) Close() error

Close implements io.Closer interface.

func (*MigrationStream) CloseWithError

func (ms *MigrationStream) CloseWithError(err error)

CloseWithError implements snapshots.WriteCloser interface.

func (*MigrationStream) ReadMsg

func (ms *MigrationStream) ReadMsg(msg proto.Message) error

ReadMsg implements the protoio.Read interface.

NOTE: It we follow the pattern of snapshot.Restore, however, the migration is done in memory. It doesn't require any deserialization -- just passing the pointer to the <msg>.

func (*MigrationStream) WriteMsg

func (ms *MigrationStream) WriteMsg(msg proto.Message) error

WriteMsg implements protoio.Write interface.

type VersionedChangeset

type VersionedChangeset struct {
	Version   uint64
	Changeset *corestore.Changeset
}

VersionedChangeset is a pair of version and Changeset.