state_surgery

README

op-chain-ops

This package performs state surgery. It takes the following input:

1. A v0 database
2. A partial genesis.json
3. A list of addresses that transacted on the network prior to this past regenesis.
4. A list of addresses that performed approvals on prior versions of the OVM ETH contract.

It creates an initialized Bedrock Geth database as output. It does this by performing the following steps:

1. Iterates over the old state.
2. For each account in the old state, add that account and its storage to the new state after copying its balance from the OVM_ETH contract.
3. Iterates over the pre-allocated accounts in the genesis file and adds them to the new state.
4. Imports any accounts that have OVM ETH balances but aren't in state.
5. Configures a genesis block in the new state using genesis.json.

It performs the following integrity checks:

1. OVM ETH storage slots must be completely accounted for.
2. The total supply of OVM ETH migrated must match the total supply of the OVM ETH contract.

This process takes about two hours on mainnet.

Unlike previous iterations of our state surgery scripts, this one does not write results to a genesis.json file. This is for the following reasons:

1. Performance. It's much faster to write binary to LevelDB than it is to write strings to a JSON file.
2. State Size. There are nearly 1MM accounts on mainnet, which would create a genesis file several gigabytes in size. This is impossible for Geth to import without a large amount of memory, since the entire JSON gets buffered into memory. Importing the entire state database will be much faster, and can be done with fewer resources.

Data Files

The following data files are used for mainnet:

1. mainnet-ovm-4-addresses.csv: Contains all addresses that used OVM ETH during regenesis 4. Calculated by parsing Mint, Burn, and Transfer events from that network's OVM ETH contract.
2. mainnet-ovm-4-allowances.csv: Contains all addresses that performed an approval on OVM ETH during regenesis 4 and who they approved. Calculated by parsing Approve events on that network's OVM ETH contract.

These files are used to build the list of OVM ETH storage slots.

Compilation

Run make surgery.

Usage

NAME:
   surgery - migrates data from v0 to Bedrock

USAGE:
   surgery [global options] command [command options] [arguments...]

COMMANDS:
   dump-addresses  dumps addresses from OVM ETH
   migrate         migrates state in OVM ETH
   help, h         Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --data-dir value, -d value  data directory to read
   --help, -h                  show help (default: false)

Documentation

Index

• Variables
• func CalcAllowanceStorageKey(owner common.Address, spender common.Address) common.Hash
• func CalcOVMETHStorageKey(addr common.Address) common.Hash
• func CalcStorageKey(a, b BytesBacked) common.Hash
• func DumpAddresses(dataDir string, outFile string) error
• func IterateAddrList(r io.Reader, cb AddressCB) error
• func IterateAllowanceList(r io.Reader, cb AllowanceCB) error
• func IterateDBAddresses(inDB ethdb.Database, cb AddressCB) error
• func IterateMintEvents(inDB ethdb.Database, headNum uint64, cb AddressCB) error
• func Migrate(dataDir, outDir string, genesis *core.Genesis, ...) error
• func MustOpenDB(dataDir string) ethdb.Database
• func MustOpenDBWithCacheOpts(dataDir string, cacheSize, handles int) ethdb.Database
• func ProgressLogger(n int, msg string) func()
• func ReadGenesis(r io.Reader) (*core.Genesis, error)
• func ReadGenesisFromFile(path string) (*core.Genesis, error)
• type AddressCB
• type AllowanceCB
• type BytesBacked
• type Params

Variables

var (
	// AddressPreimagePrefix is the byte prefix of address preimages
	// in Geth's database.
	AddressPreimagePrefix = []byte("addr-preimage-")

	// ErrStopIteration will stop iterators early when returned from the
	// iterator's callback.
	ErrStopIteration = errors.New("iteration stopped")

	// MintTopic is the topic for mint events on OVM ETH.
	MintTopic = common.HexToHash("0x0f6798a560793a54c3bcfe86a93cde1e73087d944c0ea20544137d4121396885")
)

var (
	// OVMETHAddress is the address of the OVM ETH predeploy.
	OVMETHAddress = common.HexToAddress("0xDeadDeAddeAddEAddeadDEaDDEAdDeaDDeAD0000")
)

var ParamsByChainID = map[int]*Params{
	1: {

		map[common.Hash]bool{
			common.HexToHash("0x8632b3478ce27e6c2251f16f71bf134373ff9d23cff5b8d5f95475fa6e52fe22"): true,
			common.HexToHash("0x47c25b07402d92e0d7f0cd9e347329fa0d86d16717cf933f836732313929fc1f"): true,
			common.HexToHash("0x2acc0ec5cc86ffda9ceba005a317bcf0e86863e11be3981e923d5b103990055d"): true,
		},

		new(big.Int).SetUint64(1637102600003999992),
	},
}

Functions

func CalcAllowanceStorageKey

func CalcAllowanceStorageKey(owner common.Address, spender common.Address) common.Hash

CalcAllowanceStorageKey calculates the storage key of an allowance in OVM ETH.

func CalcOVMETHStorageKey

func CalcOVMETHStorageKey(addr common.Address) common.Hash

CalcOVMETHStorageKey calculates the storage key of an OVM ETH balance.

func CalcStorageKey

func CalcStorageKey(a, b BytesBacked) common.Hash

CalcStorageKey is a helper method to calculate storage keys.

func DumpAddresses

func DumpAddresses(dataDir string, outFile string) error

DumpAddresses dumps address preimages in Geth's database to disk.

func IterateAddrList

func IterateAddrList(r io.Reader, cb AddressCB) error

IterateAddrList iterates over each address in an address list, calling the callback with the address.

func IterateAllowanceList

func IterateAllowanceList(r io.Reader, cb AllowanceCB) error

IterateAllowanceList iterates over each address in an allowance list, calling the callback with the owner and the spender.

func IterateDBAddresses

func IterateDBAddresses(inDB ethdb.Database, cb AddressCB) error

IterateDBAddresses iterates over each address in Geth's address preimage database, calling the callback with the address.

func IterateMintEvents

func IterateMintEvents(inDB ethdb.Database, headNum uint64, cb AddressCB) error

IterateMintEvents iterates over each mint event in the database starting from head and stopping at genesis.

func Migrate

func Migrate(dataDir, outDir string, genesis *core.Genesis, addrLists, allowanceLists []string, chainID, levelDBCacheSize, levelDBHandles int) error

Migrate performs the actual state migration. It does quite a lot:

1. It uses address lists, allowance lists, Mint events, and address preimages in the input state database to create a comprehensive list of storage slots in the OVM ETH contract.
2. It iterates over the slots in OVM ETH, and compares then against the list in (1). If the list doesn't match, or the total supply of OVM ETH doesn't match the sum of all balance storage slots, it panics.
3. It performs the actual migration by copying the input state DB into a new state DB.
4. It imports the provided genesis into the new state DB like Geth would during geth init.

It takes the following arguments:

• dataDir: A Geth data dir.
• outDir: A directory to output the migrated database to.
• genesis: The new chain's genesis configuration.
• addrLists: A list of address list file paths. These address lists are used to populate balances from previous regenesis events.
• allowanceLists: A list of allowance list file paths. These allowance lists are used to calculate allowance storage slots from previous regenesis events.
• chainID: The chain ID of the chain being migrated.

func MustOpenDB

func MustOpenDB(dataDir string) ethdb.Database

MustOpenDB opens a Geth database, or panics. Note that the database must be opened with a freezer in order to properly read historical data.

func MustOpenDBWithCacheOpts

func MustOpenDBWithCacheOpts(dataDir string, cacheSize, handles int) ethdb.Database

MustOpenDBWithCacheOpts opens a Geth database or panics. Allows the caller to pass in LevelDB cache parameters.

func ProgressLogger

func ProgressLogger(n int, msg string) func()

func ReadGenesis

func ReadGenesis(r io.Reader) (*core.Genesis, error)

ReadGenesis reads a genesis object from an io.Reader.

func ReadGenesisFromFile

func ReadGenesisFromFile(path string) (*core.Genesis, error)

ReadGenesisFromFile reads a genesis object from a file.

Types

type AddressCB

type AddressCB func(address common.Address) error

type AllowanceCB

type AllowanceCB func(owner, spender common.Address) error

type BytesBacked

type BytesBacked interface {
	Bytes() []byte
}

BytesBacked is a re-export of the same interface in Geth, which is unfortunately private.

type Params

type Params struct {
	// KnownMissingKeys is a set of known OVM ETH storage keys that are unaccounted for.
	KnownMissingKeys map[common.Hash]bool

	// ExpectedSupplyDelta is the expected delta between the total supply of OVM ETH,
	// and ETH we were able to migrate. This is used to account for supply bugs in
	//previous regenesis events.
	ExpectedSupplyDelta *big.Int
}

Params contains the configuration parameters used for verifying the integrity of the migration.