overlay

README

Overlay DB Notes

Overlay database interfaces are generally useful. In EtherCattle, we used them to have an NVME overlay volume with a snapshotted underlay volume, meaning we could sync from Kafka and launch our servers much more quickly than we could with a "cold" underlay volume. We also used them so that we could test syncing a Geth server to verify the integrity of the underlay without altering its state.

With Cardinal, I imagine using Overlays with a local overlay volume and a remote underlay (such as DynamoDB or BigTable).

When a node starts up, it would get the streams resumption token from the underlay, and sync all updates into the overlay. The underlay can then continue to move forward, and synchronization between overlay and underlay isn't a concern because the overlay will have all the changes that have occurred since the resumption token, and will only go back to the underlay for data that has not changed since the resumption token. The underlay need not even be synced continuously - it could have periodic processes that come online and sync it up from the stream, and shutdown for a while.

Documentation

Index

• type Database
  • func (db *Database) Update(fn func(dbpkg.Transaction) error) error
  • func (db *Database) View(fn func(dbpkg.Transaction) error) error

Types

type Database

type Database struct {
	// contains filtered or unexported fields
}

Database: The overlay database will return values from the overlay if present, the underlay if not. If `cache` is true, it will cache values retrieved from the underlay into the overlay (caching should only be used if the overlay is expected to have significantly better performance than the underlay).

func (*Database) Update

func (db *Database) Update(fn func(dbpkg.Transaction) error) error

Update invokes a closure, providing a read/write transaction

func (*Database) View

func (db *Database) View(fn func(dbpkg.Transaction) error) error

View invokes a closure, providing a read-only transaction.