Documentation
¶
Overview ¶
Package sdb offers a simple key-value database that can be utilized with the go-shelve project.
It should be suitable for a wide range of applications, but the driver directory (go-shelve/driver) provides additional options for configuring the Shelf with other supported databases from the Go ecosystem.
DB Records ¶
In sdb, each database record is represented by a distinct file stored in a bucket, which is a corresponding filesystem directory. The number of documents stored in each bucket is unlimited, and modern filesystems should be able to handle large buckets without significantly affecting performance.
Each file record's name is "base32hex" encoding of the key, which preserves lexical sort order [1]. Keys are limited to 128 characters. The record file is stored as binary data. With this design, Users do not need to worry about hitting the maximum filename length or storing keys with forbidden characters.
Cache ¶
The sdb database uses a memory-based cache to speed up operations. By default, the cache size is unlimited, but it can be configured to a fixed size or disabled altogether.
The cache's design, albeit simple, can enhance the performance of "DB.Get" and "DB.Items" to more than 1 million reads per second on standard hardware.
Atomicity ¶
New records are written atomically to the key-value store. With a file-per-record design, sdb achieves this by using atomic file writes, which consist of creating a temporary file and then renaming it [2].
This ensures that the database's methods are always performed with one atomic operation, significantly simplifying the recovery process.
Currently, the only data that can become inconsistent is the count of stored records, but if this happens, it is detected and corrected at the DB initialization.
As an optimization, records might be written directly without needing a temporary file if the data fits in a single sector since a single-sector write can be assumed to be atomic on some systems [3] [4].
Durability ¶
By default, sdb leverages the filesystem cache to speed up the database writes. This is generally suitable for most applications for which sdb is intended, as modern hardware can offer sufficient protection against crashes and ensure durability.
For the highest level of durability, the WithSynchronousWrites option makes the database synchronize data to persistent storage on each write.
Notes ¶
[1] https://datatracker.ietf.org/doc/html/rfc4648#section-7
[2] On Windows, additional configuration is involved.
[3] https://stackoverflow.com/questions/2009063/are-disk-sector-writes-atomic
[4] https://web.cs.ucla.edu/classes/spring07/cs111-2/scribe/lecture14.html
Example ¶
package main
import (
"bytes"
"encoding/gob"
"log"
"os"
"path/filepath"
"github.com/lucmq/go-shelve/sdb"
)
func main() {
path := filepath.Join(os.TempDir(), "sdb")
// Open the database
db, err := sdb.Open(path)
if err != nil {
log.Printf("open: %s", err)
return
}
defer db.Close()
item := struct {
Name string
Price float64
Quantity int
Active bool
}{
Name: "Apple",
Price: 1.99,
Quantity: 10,
Active: true,
}
// Encode data as gob
data := new(bytes.Buffer)
enc := gob.NewEncoder(data)
if err = enc.Encode(item); err != nil {
log.Printf("encode gob: %s", err)
return
}
// Save the data
err = db.Put([]byte("apple"), data.Bytes())
if err != nil {
log.Printf("put: %s", err)
return
}
}
Output:
Index ¶
- Constants
- Variables
- type DB
- func (db *DB) Close() error
- func (db *DB) Delete(key []byte) error
- func (db *DB) Get(key []byte) ([]byte, error)
- func (db *DB) Has(key []byte) (bool, error)
- func (db *DB) Items(start []byte, order int, fn Yield) error
- func (db *DB) Len() int64
- func (db *DB) Put(key, value []byte) error
- func (db *DB) Sync() error
- type Option
- type Yield
Examples ¶
Constants ¶
const ( // DefaultCacheSize is the default size of the cache used to speed up the // database operations. A value of -1 represents an unlimited cache. DefaultCacheSize = -1 // MaxKeyLength is the maximum size of a key. MaxKeyLength = 128 )
Variables ¶
var ( // ErrKeyTooLarge is returned when a key exceeds the maximum length. ErrKeyTooLarge = errors.New("key exceeds maximum length") )
Functions ¶
This section is empty.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB represents a database, which is created with the Open function.
Client applications must call DB.Close() when done with the database.
A DB is safe for concurrent use by multiple goroutines.
func Open ¶
Open opens the database at the given path. If the path does not exist, it is created.
Client applications must call DB.Close() when done with the database.
func (*DB) Get ¶
Get retrieves the value associated with a key from the database. If the key is not found, it returns nil.
func (*DB) Items ¶
Items iterates over key-value pairs in the database, invoking fn(k, v) for each pair. Iteration stops early if fn returns false.
The start and order parameters are present for compatibility with the shelve.DB interface but are currently unused.
This operation acquires a read lock each time a database record is read and holds it for the duration of the fn callback. Implementations that require faster lock release should copy the key-value pair and return from the callback as quickly as possible.
The user-provided fn(k, v) must not modify the database within the same goroutine as the iteration, as this would cause a deadlock.
func (*DB) Len ¶
Len returns the number of items in the database. If an error occurs, it returns -1.
func (*DB) Put ¶
Put adds a key-value pair to the database. If the key already exists, it overwrites the existing value.
It returns an error if the key is greater than MaxKeyLength.
type Option ¶
type Option func(*DB)
Option is passed to the Open function to create a customized DB.
func WithCacheSize ¶
WithCacheSize sets the size of the cache used by the database. A value of -1 represents an unlimited cache and a value of 0 disables the cache. The default cache size is -1.
func WithSynchronousWrites ¶
WithSynchronousWrites enables synchronous writes to the database. By default, synchronous writes are disabled.
Source Files
Directories