Documentation ¶
Overview ¶
Package localstore provides disk storage layer for Aurora Chunk persistence. It uses boson/shed abstractions.
The main type is DB which manages the storage by providing methods to access and add Chunks and to manage their status.
Modes are abstractions that do specific changes to Chunks. There are three mode types:
- ModeGet, for Chunk access
- ModePut, for adding Chunks to the database
- ModeSet, for changing Chunk statuses
Every mode type has a corresponding type (Getter, Putter and Setter) that provides adequate method to perform the opperation and that type should be injected into localstore consumers instead the whole DB. This provides more clear insight which operations consumer is performing on the database.
Getters, Putters and Setters accept different get, put and set modes to perform different actions. For example, ModeGet has two different variables ModeGetRequest and ModeGetSync and two different Getters can be constructed with them that are used when the chunk is requested or when the chunk is synced as this two events are differently changing the database.
Subscription methods are implemented for a specific purpose of continuous iterations over Chunks that should be provided to Push and Pull syncing.
DB implements an internal garbage collector that removes only synced Chunks from the database based on their most recent access time.
Internally, DB stores Chunk data and any required information, such as store and access timestamps in different shed indexes that can be iterated on by garbage collector or subscriptions.
Index ¶
- Constants
- Variables
- type DB
- func (db *DB) Close() (err error)
- func (db *DB) DebugIndices() (indexInfo map[string]int, err error)
- func (db *DB) Export(w io.Writer) (count int64, err error)
- func (db *DB) Get(ctx context.Context, mode storage.ModeGet, addr boson.Address) (ch boson.Chunk, err error)
- func (db *DB) GetMulti(ctx context.Context, mode storage.ModeGet, addrs ...boson.Address) (chunks []boson.Chunk, err error)
- func (db *DB) Has(ctx context.Context, mode storage.ModeHas, addr boson.Address) (bool, error)
- func (db *DB) HasMulti(ctx context.Context, hasMode storage.ModeHas, addrs ...boson.Address) ([]bool, error)
- func (db *DB) Import(ctx context.Context, r io.Reader) (count int64, err error)
- func (db *DB) Metrics() []prometheus.Collector
- func (db *DB) Put(ctx context.Context, mode storage.ModePut, chs ...boson.Chunk) (exist []bool, err error)
- func (db *DB) Set(ctx context.Context, mode storage.ModeSet, addrs ...boson.Address) (err error)
- func (db *DB) SetChunkInfo(chunkInfo chunkinfo.Interface)
- type Options
Constants ¶
const DbSchemaCode = "code"
DbSchemaCode is the first aurorafs schema identifier
const DbSchemaNone = ""
There was a time when we had no schema at all.
Variables ¶
var DbSchemaCurrent = DbSchemaCode
The DB schema we want to use. The actual/current DB schema might differ until migrations are run.
var ( // ErrInvalidMode is retuned when an unknown Mode // is provided to the function. ErrInvalidMode = errors.New("invalid mode") )
Functions ¶
This section is empty.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is the local store implementation and holds database related objects.
func New ¶
New returns a new DB. All fields and indexes are initialized and possible conflicts with schema from existing database is checked. One goroutine for writing batches is created.
func (*DB) DebugIndices ¶
DebugIndices returns the index sizes for all indexes in localstore the returned map keys are the index name, values are the number of elements in the index
func (*DB) Export ¶
Export writes a tar structured data to the writer of all chunks in the retrieval data index. It returns the number of chunks exported.
func (*DB) Get ¶
func (db *DB) Get(ctx context.Context, mode storage.ModeGet, addr boson.Address) (ch boson.Chunk, err error)
Get returns a chunk from the database. If the chunk is not found storage.ErrNotFound will be returned. All required indexes will be updated required by the Getter Mode. Get is required to implement chunk.Store interface.
func (*DB) GetMulti ¶
func (db *DB) GetMulti(ctx context.Context, mode storage.ModeGet, addrs ...boson.Address) (chunks []boson.Chunk, err error)
GetMulti returns chunks from the database. If one of the chunks is not found storage.ErrNotFound will be returned. All required indexes will be updated required by the Getter Mode. GetMulti is required to implement chunk.Store interface.
func (*DB) HasMulti ¶
func (db *DB) HasMulti(ctx context.Context, hasMode storage.ModeHas, addrs ...boson.Address) ([]bool, error)
HasMulti returns a slice of booleans which represent if the provided chunks are stored in database.
func (*DB) Import ¶
Import reads a tar structured data from the reader and stores chunks in the database. It returns the number of chunks imported.
func (*DB) Metrics ¶
func (db *DB) Metrics() []prometheus.Collector
func (*DB) Put ¶
func (db *DB) Put(ctx context.Context, mode storage.ModePut, chs ...boson.Chunk) (exist []bool, err error)
Put stores Chunks to database and depending on the Putter mode, it updates required indexes. Put is required to implement storage.Store interface.
func (*DB) Set ¶
Set updates database indexes for chunks represented by provided addresses. Set is required to implement chunk.Store interface.
func (*DB) SetChunkInfo ¶ added in v1.3.0
type Options ¶
type Options struct { // Driver support: leveldb/wiredtiger Driver string // Capacity is a limit that triggers garbage collection when // number of items in gcIndex equals or exceeds it. Capacity uint64 // MetricsPrefix defines a prefix for metrics names. MetricsPrefix string }
Options struct holds optional parameters for configuring DB.