rawdb

package
v1.0.24 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2023 License: ISC Imports: 28 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ChainFreezerBlockTable = "blocks"

	ChainFreezerDAGBlockTable = "dagblocks"
)

The list of table names of chain freezer.

View Source 
const PebbleEnabled = true

Pebble is unsuported on 32bit architecture

Variables

View Source 
var (

	// SnapshotRootKey tracks the hash of the last snapshot.
	SnapshotRootKey = []byte("SnapshotRoot")

	// snapshot
	SnapshotBlockOrderPrefix  = []byte("o") // SnapshotBlockOrderPrefix + block order -> block id
	SnapshotBlockStatusPrefix = []byte("s") // SnapshotBlockStatusPrefix + block id -> block status
)

The fields below define the low level database schema prefixing.

Functions

func DeleteBlock 

func DeleteBlock(db ethdb.KeyValueWriter, hash *hash.Hash)

func DeleteBlockID 

func DeleteBlockID(db ethdb.KeyValueWriter, hash *hash.Hash)

func DeleteBlockOrderSnapshot 

func DeleteBlockOrderSnapshot(db ethdb.KeyValueWriter, order uint64) error

func DeleteDAGBlock 

func DeleteDAGBlock(db ethdb.KeyValueWriter, id uint64)

func DeleteMainChainTip 

func DeleteMainChainTip(db ethdb.KeyValueWriter) error

func DeleteSnapshotDisabled 

func DeleteSnapshotDisabled(db ethdb.KeyValueWriter)

DeleteSnapshotDisabled deletes the flag keeping the snapshot maintenance disabled.

func DeleteSnapshotGenerator 

func DeleteSnapshotGenerator(db ethdb.KeyValueWriter)

DeleteSnapshotGenerator deletes the serialized snapshot generator saved at the last shutdown

func DeleteSnapshotJournal 

func DeleteSnapshotJournal(db ethdb.KeyValueWriter)

DeleteSnapshotJournal deletes the serialized in-memory diff layers saved at the last shutdown

func DeleteSnapshotRecoveryNumber 

func DeleteSnapshotRecoveryNumber(db ethdb.KeyValueWriter)

DeleteSnapshotRecoveryNumber deletes the block number of the last persisted snapshot layer.

func DeleteSnapshotRoot 

func DeleteSnapshotRoot(db ethdb.KeyValueWriter)

DeleteSnapshotRoot deletes the hash of the block whose state is contained in the persisted snapshot. Since snapshots are not immutable, this method can be used during updates, so a crash or failure will mark the entire snapshot invalid.

func DeleteTxLookupEntry 

func DeleteTxLookupEntry(db ethdb.KeyValueWriter, hash *hash.Hash) error

func HasBlock 

func HasBlock(db ethdb.Reader, hash *hash.Hash) bool

func InspectFreezerTable 

func InspectFreezerTable(ancient string, freezerName string, tableName string, start, end int64) error

InspectFreezerTable dumps out the index of a specific freezer table. The passed ancient indicates the path of root ancient directory where the chain freezer can be opened. Start and end specify the range for dumping out indexes. Note this function can only be used for debugging purposes.

func NewDatabase 

func NewDatabase(db ethdb.KeyValueStore) ethdb.Database

NewDatabase creates a high level database on top of a given key-value data store without a freezer moving immutable chain segments into cold storage.

func NewDatabaseWithFreezer 

func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace string, readonly bool) (ethdb.Database, error)

NewDatabaseWithFreezer creates a high level database on top of a given key- value data store with a freezer moving immutable chain segments into cold storage. The passed ancient indicates the path of root ancient directory where the chain freezer can be opened.

func NewKeyLengthIterator 

func NewKeyLengthIterator(it ethdb.Iterator, keyLen int) ethdb.Iterator

NewKeyLengthIterator returns a wrapped version of the iterator that will only return key-value pairs where keys with a specific key length will be returned.

func NewLevelDBDatabase 

func NewLevelDBDatabase(file string, cache int, handles int, namespace string, readonly bool) (ethdb.Database, error)

NewLevelDBDatabase creates a persistent key-value database without a freezer moving immutable chain segments into cold storage.

func NewMemoryDatabase 

func NewMemoryDatabase() ethdb.Database

NewMemoryDatabase creates an ephemeral in-memory key-value database without a freezer moving immutable chain segments into cold storage.

func NewMemoryDatabaseWithCap 

func NewMemoryDatabaseWithCap(size int) ethdb.Database

NewMemoryDatabaseWithCap creates an ephemeral in-memory key-value database with an initial starting capacity, but without a freezer moving immutable chain segments into cold storage.

func NewPebbleDBDatabase 

func NewPebbleDBDatabase(file string, cache int, handles int, namespace string, readonly bool) (ethdb.Database, error)

NewPebbleDBDatabase creates a persistent key-value database without a freezer moving immutable chain segments into cold storage.

func NewTable 

func NewTable(db ethdb.Database, prefix string) ethdb.Database

NewTable returns a database object that prefixes all keys with a given string.

func Open 

func Open(o OpenOptions) (ethdb.Database, error)

Open opens both a disk-based key-value database such as leveldb or pebble, but also integrates it with a freezer database -- if the AncientDir option has been set on the provided OpenOptions. The passed o.AncientDir indicates the path of root ancient directory where the chain freezer can be opened.

func PopUncleanShutdownMarker 

func PopUncleanShutdownMarker(db ethdb.KeyValueStore)

PopUncleanShutdownMarker removes the last unclean shutdown marker

func PushUncleanShutdownMarker 

func PushUncleanShutdownMarker(db ethdb.KeyValueStore) ([]uint64, uint64, error)

PushUncleanShutdownMarker appends a new unclean shutdown marker and returns the previous data - a list of timestamps - a count of how many old unclean-shutdowns have been discarded

func ReadBlock 

func ReadBlock(db ethdb.Reader, hash *hash.Hash) *types.SerializedBlock

func ReadBlockBaw 

func ReadBlockBaw(db ethdb.Reader, hash *hash.Hash) []byte

func ReadBlockID 

func ReadBlockID(db ethdb.Reader, hash *hash.Hash) *uint64

func ReadBlockOrderSnapshot 

func ReadBlockOrderSnapshot(db ethdb.KeyValueReader, order uint64) *uint64

func ReadChainMetadata 

func ReadChainMetadata(db ethdb.KeyValueStore) [][]string

ReadChainMetadata returns a set of key/value pairs that contains informatin about the database chain status. This can be used for diagnostic purposes when investigating the state of the node.

func ReadDAGBlock 

func ReadDAGBlock(db ethdb.Reader, id uint64) meerdag.IBlock

func ReadDAGBlockBaw 

func ReadDAGBlockBaw(db ethdb.Reader, id uint64) []byte

func ReadDatabaseVersion 

func ReadDatabaseVersion(db ethdb.KeyValueReader) *uint64

func ReadMainChainTip 

func ReadMainChainTip(db ethdb.Reader) *uint64

func ReadSnapshotDisabled 

func ReadSnapshotDisabled(db ethdb.KeyValueReader) bool

ReadSnapshotDisabled retrieves if the snapshot maintenance is disabled.

func ReadSnapshotGenerator 

func ReadSnapshotGenerator(db ethdb.KeyValueReader) []byte

ReadSnapshotGenerator retrieves the serialized snapshot generator saved at the last shutdown.

func ReadSnapshotJournal 

func ReadSnapshotJournal(db ethdb.KeyValueReader) []byte

func ReadSnapshotRecoveryNumber 

func ReadSnapshotRecoveryNumber(db ethdb.KeyValueReader) *uint64

ReadSnapshotRecoveryNumber retrieves the block number of the last persisted snapshot layer.

func ReadSnapshotRoot 

func ReadSnapshotRoot(db ethdb.KeyValueReader) *hash.Hash

ReadSnapshotRoot retrieves the root of the block whose state is contained in the persisted snapshot.

func ReadSnapshotSyncStatus 

func ReadSnapshotSyncStatus(db ethdb.KeyValueReader) []byte

ReadSnapshotSyncStatus retrieves the serialized sync status saved at shutdown.

func ReadTxLookupEntry 

func ReadTxLookupEntry(db ethdb.Reader, hash *hash.Hash) ([]byte, error)

func UpdateUncleanShutdownMarker 

func UpdateUncleanShutdownMarker(db ethdb.KeyValueStore)

UpdateUncleanShutdownMarker updates the last marker's timestamp to now.

func UseLogger 

func UseLogger(logger l.Logger)

UseLogger uses a specified Logger to output package logging info.

func WriteAncientBlocks 

func WriteAncientBlocks(db ethdb.AncientWriter, blocks []*types.SerializedBlock, dagblocks []meerdag.IBlock) (int64, error)

func WriteBlock 

func WriteBlock(db ethdb.KeyValueWriter, block *types.SerializedBlock) error

func WriteBlockID 

func WriteBlockID(db ethdb.KeyValueWriter, hash *hash.Hash, id uint64)

func WriteBlockOrderSnapshot 

func WriteBlockOrderSnapshot(db ethdb.KeyValueWriter, order uint64, id uint64) error

func WriteDAGBlock 

func WriteDAGBlock(db ethdb.KeyValueWriter, block meerdag.IBlock) error

func WriteDatabaseVersion 

func WriteDatabaseVersion(db ethdb.KeyValueWriter, version uint64) error

func WriteMainChainTip 

func WriteMainChainTip(db ethdb.KeyValueWriter, mainchaintip uint64) error

func WriteSnapshotDisabled 

func WriteSnapshotDisabled(db ethdb.KeyValueWriter)

WriteSnapshotDisabled stores the snapshot pause flag.

func WriteSnapshotGenerator 

func WriteSnapshotGenerator(db ethdb.KeyValueWriter, generator []byte)

WriteSnapshotGenerator stores the serialized snapshot generator to save at shutdown.

func WriteSnapshotJournal 

func WriteSnapshotJournal(db ethdb.KeyValueWriter, journal []byte)

func WriteSnapshotRecoveryNumber 

func WriteSnapshotRecoveryNumber(db ethdb.KeyValueWriter, number uint64)

WriteSnapshotRecoveryNumber stores the block number of the last persisted snapshot layer.

func WriteSnapshotRoot 

func WriteSnapshotRoot(db ethdb.KeyValueWriter, root *hash.Hash)

WriteSnapshotRoot stores the root of the block whose state is contained in the persisted snapshot.

func WriteSnapshotSyncStatus 

func WriteSnapshotSyncStatus(db ethdb.KeyValueWriter, status []byte)

WriteSnapshotSyncStatus stores the serialized sync status to save at shutdown.

func WriteTxLookupEntry 

func WriteTxLookupEntry(db ethdb.KeyValueWriter, hash *hash.Hash, bytes []byte) error

Types

type Freezer 

type Freezer struct {
	// contains filtered or unexported fields
}

Freezer is a memory mapped append-only database to store immutable ordered data into flat files:

  • The append-only nature ensures that disk writes are minimized.
  • The memory mapping ensures we can max out system memory for caching without reserving it for go-ethereum. This would also reduce the memory requirements of Geth, and thus also GC overhead.

func NewChainFreezer 

func NewChainFreezer(datadir string, namespace string, readonly bool) (*Freezer, error)

NewChainFreezer is a small utility method around NewFreezer that sets the default parameters for the chain storage.

func NewFreezer 

func NewFreezer(datadir string, namespace string, readonly bool, maxTableSize uint32, tables map[string]bool) (*Freezer, error)

NewFreezer creates a freezer instance for maintaining immutable ordered data according to the given parameters.

The 'tables' argument defines the data tables. If the value of a map entry is true, snappy compression is disabled for the table.

func (*Freezer) Ancient 

func (f *Freezer) Ancient(kind string, number uint64) ([]byte, error)

Ancient retrieves an ancient binary blob from the append-only immutable files.

func (*Freezer) AncientRange 

func (f *Freezer) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error)

AncientRange retrieves multiple items in sequence, starting from the index 'start'. It will return

  • at most 'max' items,
  • at least 1 item (even if exceeding the maxByteSize), but will otherwise return as many items as fit into maxByteSize.

func (*Freezer) AncientSize 

func (f *Freezer) AncientSize(kind string) (uint64, error)

AncientSize returns the ancient size of the specified category.

func (*Freezer) Ancients 

func (f *Freezer) Ancients() (uint64, error)

Ancients returns the length of the frozen items.

func (*Freezer) Close 

func (f *Freezer) Close() error

Close terminates the chain freezer, unmapping all the data files.

func (*Freezer) HasAncient 

func (f *Freezer) HasAncient(kind string, number uint64) (bool, error)

HasAncient returns an indicator whether the specified ancient data exists in the freezer.

func (*Freezer) MigrateTable 

func (f *Freezer) MigrateTable(kind string, convert convertLegacyFn) error

MigrateTable processes the entries in a given table in sequence converting them to a new format if they're of an old format.

func (*Freezer) ModifyAncients 

func (f *Freezer) ModifyAncients(fn func(ethdb.AncientWriteOp) error) (writeSize int64, err error)

ModifyAncients runs the given write operation.

func (*Freezer) ReadAncients 

func (f *Freezer) ReadAncients(fn func(ethdb.AncientReaderOp) error) (err error)

ReadAncients runs the given read operation while ensuring that no writes take place on the underlying freezer.

func (*Freezer) Sync 

func (f *Freezer) Sync() error

Sync flushes all data tables to disk.

func (*Freezer) Tail 

func (f *Freezer) Tail() (uint64, error)

Tail returns the number of first stored item in the freezer.

func (*Freezer) TruncateHead 

func (f *Freezer) TruncateHead(items uint64) error

TruncateHead discards any recent data above the provided threshold number.

func (*Freezer) TruncateTail 

func (f *Freezer) TruncateTail(tail uint64) error

TruncateTail discards any recent data below the provided threshold number.

type KeyLengthIterator 

type KeyLengthIterator struct {
	ethdb.Iterator
	// contains filtered or unexported fields
}

KeyLengthIterator is a wrapper for a database iterator that ensures only key-value pairs with a specific key length will be returned.

func (*KeyLengthIterator) Next 

func (it *KeyLengthIterator) Next() bool

type OpenOptions 

type OpenOptions struct {
	Type              string // "leveldb" | "pebble"
	Directory         string // the datadir
	AncientsDirectory string // the ancients-dir
	Namespace         string // the namespace for database relevant metrics
	Cache             int    // the capacity(in megabytes) of the data caching
	Handles           int    // number of files to be open simultaneously
	ReadOnly          bool
}

OpenOptions contains the options to apply when opening a database. OBS: If AncientsDirectory is empty, it indicates that no freezer is to be used.

type ResettableFreezer 

type ResettableFreezer struct {
	// contains filtered or unexported fields
}

ResettableFreezer is a wrapper of the freezer which makes the freezer resettable.

func NewResettableFreezer 

func NewResettableFreezer(datadir string, namespace string, readonly bool, maxTableSize uint32, tables map[string]bool) (*ResettableFreezer, error)

NewResettableFreezer creates a resettable freezer, note freezer is only resettable if the passed file directory is exclusively occupied by the freezer. And also the user-configurable ancient root directory is **not** supported for reset since it might be a mount and rename will cause a copy of hundreds of gigabyte into local directory. It needs some other file based solutions.

The reset function will delete directory atomically and re-create the freezer from scratch.

func (*ResettableFreezer) Ancient 

func (f *ResettableFreezer) Ancient(kind string, number uint64) ([]byte, error)

Ancient retrieves an ancient binary blob from the append-only immutable files.

func (*ResettableFreezer) AncientRange 

func (f *ResettableFreezer) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error)

AncientRange retrieves multiple items in sequence, starting from the index 'start'. It will return

  • at most 'max' items,
  • at least 1 item (even if exceeding the maxByteSize), but will otherwise return as many items as fit into maxByteSize

func (*ResettableFreezer) AncientSize 

func (f *ResettableFreezer) AncientSize(kind string) (uint64, error)

AncientSize returns the ancient size of the specified category.

func (*ResettableFreezer) Ancients 

func (f *ResettableFreezer) Ancients() (uint64, error)

Ancients returns the length of the frozen items.

func (*ResettableFreezer) Close 

func (f *ResettableFreezer) Close() error

Close terminates the chain freezer, unmapping all the data files.

func (*ResettableFreezer) HasAncient 

func (f *ResettableFreezer) HasAncient(kind string, number uint64) (bool, error)

HasAncient returns an indicator whether the specified ancient data exists in the freezer

func (*ResettableFreezer) MigrateTable 

func (f *ResettableFreezer) MigrateTable(kind string, convert convertLegacyFn) error

MigrateTable processes the entries in a given table in sequence converting them to a new format if they're of an old format.

func (*ResettableFreezer) ModifyAncients 

func (f *ResettableFreezer) ModifyAncients(fn func(ethdb.AncientWriteOp) error) (writeSize int64, err error)

ModifyAncients runs the given write operation.

func (*ResettableFreezer) ReadAncients 

func (f *ResettableFreezer) ReadAncients(fn func(ethdb.AncientReaderOp) error) (err error)

ReadAncients runs the given read operation while ensuring that no writes take place on the underlying freezer.

func (*ResettableFreezer) Reset 

func (f *ResettableFreezer) Reset() error

Reset deletes the file directory exclusively occupied by the freezer and recreate the freezer from scratch. The atomicity of directory deletion is guaranteed by the rename operation, the leftover directory will be cleaned up in next startup in case crash happens after rename.

func (*ResettableFreezer) Sync 

func (f *ResettableFreezer) Sync() error

Sync flushes all data tables to disk.

func (*ResettableFreezer) Tail 

func (f *ResettableFreezer) Tail() (uint64, error)

Tail returns the number of first stored item in the freezer.

func (*ResettableFreezer) TruncateHead 

func (f *ResettableFreezer) TruncateHead(items uint64) error

TruncateHead discards any recent data above the provided threshold number.

func (*ResettableFreezer) TruncateTail 

func (f *ResettableFreezer) TruncateTail(tail uint64) error

TruncateTail discards any recent data below the provided threshold number.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.