mg11

Documentation

Overview

package mg11 contains the code to perform 11-12 repository migration in go-ipfs. This performs a switch to raw multihashes for all keys in the go-ipfs datastore (https://github.com/ipfs/go-ipfs/issues/6815).

Index

• Variables
• func IsBasicFlatFSBlockstore(dstore ds.Datastore) (dsPath string, v1 *flatfs.ShardIdV1, err error)
• type CidSwapper
  • func (cswap *CidSwapper) Prepare() (uint64, error)
  • func (cswap *CidSwapper) Revert(unswapCh <-chan Swap) (uint64, error)
  • func (cswap *CidSwapper) Run(swapCh <-chan Swap) (uint64, error)
• type Migration
  • func (m *Migration) Apply(opts migrate.Options) error
  • func (m *Migration) Reversible() bool
  • func (m *Migration) Revert(opts migrate.Options) error
  • func (m *Migration) Versions() string
• type Swap

Variables

var EnableFlatFSFastPath bool = true

var NWorkers int = 1

NWorkers sets the number of swapping threads to run when applying a migration.

var SyncSize uint64 = 100 * 1024 * 1024 // 100MiB

SyncSize specifies how much we batch data before committing and syncing.

Functions

func IsBasicFlatFSBlockstore

func IsBasicFlatFSBlockstore(dstore ds.Datastore) (dsPath string, v1 *flatfs.ShardIdV1, err error)

Types

type CidSwapper

type CidSwapper struct {
	Prefix ds.Key      // A prefix/namespace to limit the query.
	Store  ds.Batching // the datastore to migrate.
	SwapCh chan Swap   // a channel that gets notified for every swap
}

CidSwapper reads all the keys in a datastore and replaces them with their raw multihash.

func (*CidSwapper) Prepare

func (cswap *CidSwapper) Prepare() (uint64, error)

Prepare performs a dry run without copying anything but notifying SwapCh as it runs.

Retruns the total number of keys swapped.

func (*CidSwapper) Revert

func (cswap *CidSwapper) Revert(unswapCh <-chan Swap) (uint64, error)

Revert allows to undo any operations made by Run(). The given channel should receive Swap objects as they were sent by Run. It returns the number of swap operations performed.

func (*CidSwapper) Run

func (cswap *CidSwapper) Run(swapCh <-chan Swap) (uint64, error)

Run performs a migration reading the Swaps that need to be performed from the given Swap channel. The swaps can be obtained with Prepare().

Run returns the total number of keys swapped.

type Migration

type Migration struct {
	// contains filtered or unexported fields
}

Migration implements the migration described above.

func (*Migration) Apply

func (m *Migration) Apply(opts migrate.Options) error

Apply runs the migration and writes a log file that can be used by Revert. Steps:

• Open a raw datastore using go-ipfs settings
• Simulate the migration and save a backup log
• Run the migration by storing all CIDv1 addressed logs as raw-multihash addressed.

func (*Migration) Reversible

func (m *Migration) Reversible() bool

Reversible returns true.

func (*Migration) Revert

func (m *Migration) Revert(opts migrate.Options) error

Revert attempts to undo the migration using the log file written by Apply. Steps: - Read the backup log and write all entries as a CIDv1-addressed block - Do the same with the MFS root - Do the same with all the CidV1 blocks recursively referenced in the pinset

Revert does not delete blocks that are reverted so cover some corner cases.

func (*Migration) Versions

func (m *Migration) Versions() string

Versions returns the current version string for this migration.

type Swap

type Swap struct {
	Old ds.Key
	New ds.Key
}

Swap holds the datastore keys for the original CID and for the destination Multihash.