README
¶
pinion
Package pinion provides a fast and simple set of routines to manage the storage and retrieval of structured records.
Overview
Pinion automates the task of managing record storage and multiple retrieval indexes. Its simple programming interface, comprising methods like Put() and Get(), operate on types that implement the pinion.Record interface. This interface isolates key-building and value-encoding to one place in your application. When used with the piniondb/store package, a fast implementation of the interface can be made simply that does not require reflection. Any practical number of record types that satisfy the pinion.Record interface can be managed by a pinion database.
Currently, pinion does not support joined records. This is obviated to some degree with its support for structures that may include maps and slices.
The pinion package depends on boltdb. All tests pass on Linux, Mac and Windows platforms.
Example
This example manages records of type personType. This type implements the pinion.Record interface; see the heavily commented file person_test.go for more details. Note that no type assertions or explicit encoding or decoding needs to be done to populate and retrieve records.
var db *pinion.DB
var err error
var person personType
db, err = pinion.Create("example/standalobe.db", 0600, pinion.Options{})
if err == nil {
wdb := db.Wrap()
list := []nameType{
{last: "Smith", middle: "J", first: "Carol"},
{last: "Jones", middle: "W", first: "Robert"},
}
wdb.Add(&person, func() bool {
if len(list) > 0 {
person = personType{id: 0, name: list[0]}
list = list[1:]
return true
}
return false
})
for idx := uint8(0); idx < idxPersonCount; idx++ {
fmt.Printf("%-12s", personIndexNames[idx])
person = personType{} // Start search at beginning with zeroed record
wdb.Get(&person, idx, func() bool {
fmt.Printf(" [%s]", person)
return true
})
fmt.Println("")
}
}
if err != nil {
fmt.Println(err)
}
Running this example produces the following output:
Output:
ID [Carol J Smith / 1] [Robert W Jones / 2]
Last name [Robert W Jones / 2] [Carol J Smith / 1]
First name [Carol J Smith / 1] [Robert W Jones / 2]
Installation
To install the package on your system, run
go get github.com/piniondb/pinion
Errors
The methods of a *pinion.DB instance return an error if the operation fails. Since database activity often involves a lot of steps, you may find it useful to locally wrap the database instance with Wrap() in order to defer error handling to a single place.
Keys
In addition to the required primary index, up to 255 secondary indexes can be defined for the record type you want to manage. Only keys in the primary index (index 0) need to be unique. Keys must be sortable when inserted into the underlying database as byte slices. The piniondb/store package provides support for fixed-length key segments. Alternatively, you can use fmt.Sprintf() to format fixed-length fields.
Best practices
- Implement the pinion.Record interface in the same location at which the structure itself is defined.
- When working with multiple records, single calls to Add(), Put() and Get() will be faster than individual calls to AddRec(), PutRec() and GetRec().
Contributing Changes
pinion is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. Your pull request should
- be compatible with the MIT License
- be properly documented
- include an example in one of the test files (for example, pinion_test.go) if appropriate
Use the Go Report Card to assure that no compliance issues have been introduced.
License
pinion is released under the MIT License.
Documentation
¶
Overview ¶
Package pinion provides a fast and simple set of routines to manage the storage and retrieval of structured records.
Overview ¶
Pinion automates the task of managing record storage and multiple retrieval indexes. Its simple programming interface, comprising methods like Put() and Get(), operate on types that implement the pinion.Record interface. This interface isolates key-building and value-encoding to one place in your application. When used with the piniondb/store package, a fast implementation of the interface can be made simply that does not require reflection. Any practical number of record types that satisfy the pinion.Record interface can be managed by a pinion database.
Currently, pinion does not support joined records. This is obviated to some degree with its support for structures that may include maps and slices.
The pinion package depends on boltdb. All tests pass on Linux, Mac and Windows platforms.
Example ¶
This example manages records of type personType. This type implements the pinion.Record interface; see the heavily commented file person_test.go for more details. Note that no type assertions or explicit encoding or decoding needs to be done to populate and retrieve records.
var db *pinion.DB
var err error
var person personType
db, err = pinion.Create("example/standalobe.db", 0600, pinion.Options{})
if err == nil {
wdb := db.Wrap()
list := []nameType{
{last: "Smith", middle: "J", first: "Carol"},
{last: "Jones", middle: "W", first: "Robert"},
}
wdb.Add(&person, func() bool {
if len(list) > 0 {
person = personType{id: 0, name: list[0]}
list = list[1:]
return true
}
return false
})
for idx := uint8(0); idx < idxPersonCount; idx++ {
fmt.Printf("%-12s", personIndexNames[idx])
person = personType{} // Start search at beginning with zeroed record
wdb.Get(&person, idx, func() bool {
fmt.Printf(" [%s]", person)
return true
})
fmt.Println("")
}
}
if err != nil {
fmt.Println(err)
}
Running this example produces the following output:
Output: ID [Carol J Smith / 1] [Robert W Jones / 2] Last name [Robert W Jones / 2] [Carol J Smith / 1] First name [Carol J Smith / 1] [Robert W Jones / 2]
Installation ¶
To install the package on your system, run
go get github.com/piniondb/pinion
Errors ¶
The methods of a *pinion.DB instance return an error if the operation fails. Since database activity often involves a lot of steps, you may find it useful to locally wrap the database instance with Wrap() in order to defer error handling to a single place.
Keys ¶
In addition to the required primary index, up to 255 secondary indexes can be defined for the record type you want to manage. Only keys in the primary index (index 0) need to be unique. Keys must be sortable when inserted into the underlying database as byte slices. The piniondb/store package provides support for fixed-length key segments. Alternatively, you can use fmt.Sprintf() to format fixed-length fields.
Best practices ¶
• Implement the pinion.Record interface in the same location at which the structure itself is defined
• When working with multiple records, single calls to Add(), Put() and Get() will be faster than individual calls to AddRec(), PutRec() and GetRec().
Contributing Changes ¶
pinion is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. Your pull request should
• be compatible with the MIT License
• be properly documented
• include an example in one of the test files (for example, pinion_test.go) if appropriate
Use https://goreportcard.com/report/github.com/piniondb/pinion to assure that no compliance issues have been introduced.
License ¶
pinion is released under the MIT License.
Example ¶
Example demonstrates record management with pinion.
var db *pinion.DB
var err error
db, err = pinion.Create("example/person.db", 0600, pinion.Options{})
if err == nil {
wdb := db.Wrap()
populate(wdb)
change(wdb)
db.Close()
err = wdb.Error()
}
if err != nil {
fmt.Println(err)
}
Output: --- Initial --- ID [Carol J Smith / 1] [Robert W Jones / 2] Last name [Robert W Jones / 2] [Carol J Smith / 1] First name [Carol J Smith / 1] [Robert W Jones / 2] --- Robert changed to Bob --- ID [Carol J Smith / 1] [Bob W Jones / 2] Last name [Bob W Jones / 2] [Carol J Smith / 1] First name [Bob W Jones / 2] [Carol J Smith / 1]
Example (Standalone) ¶
var db *pinion.DB
var err error
var person personType
db, err = pinion.Create("example/standalobe.db", 0600, pinion.Options{})
if err == nil {
wdb := db.Wrap()
list := []nameType{
{last: "Smith", middle: "J", first: "Carol"},
{last: "Jones", middle: "W", first: "Robert"},
}
wdb.Add(&person, func() bool {
if len(list) > 0 {
person = personType{id: 0, name: list[0]}
list = list[1:]
return true
}
return false
})
for idx := uint8(0); idx < idxPersonCount; idx++ {
fmt.Printf("%-12s", personIndexNames[idx])
person = personType{} // Start search at beginning with zeroed record
wdb.Get(&person, idx, func() bool {
fmt.Printf(" [%s]", person)
return true
})
fmt.Println("")
}
}
if err != nil {
fmt.Println(err)
}
Output: ID [Carol J Smith / 1] [Robert W Jones / 2] Last name [Robert W Jones / 2] [Carol J Smith / 1] First name [Carol J Smith / 1] [Robert W Jones / 2]
Index ¶
- Constants
- Variables
- type DB
- func (db *DB) Add(recPtr Record, f func() bool) (putErr error)
- func (db *DB) AddRec(recPtr Record) (err error)
- func (db *DB) Close() (err error)
- func (db *DB) Delete(recPtr Record, f func() bool) (delErr error)
- func (db *DB) DeleteRec(recPtr Record) (err error)
- func (db *DB) Get(recPtr Record, idx uint8, f func() bool) (getErr error)
- func (db *DB) GetRec(recPtr Record, idx uint8) (err error)
- func (db *DB) HexDump(wr io.Writer)
- func (db *DB) Put(recPtr Record, f func() bool) (putErr error)
- func (db *DB) PutRec(recPtr Record) (err error)
- func (db *DB) Wrap() (wdb *WrapDB)
- type Options
- type Record
- type WrapDB
- func (wdb *WrapDB) Add(recPtr Record, f func() bool)
- func (wdb *WrapDB) AddRec(recPtr Record)
- func (wdb *WrapDB) Delete(recPtr Record, f func() bool)
- func (wdb *WrapDB) DeleteRec(recPtr Record)
- func (wdb *WrapDB) Error() error
- func (wdb *WrapDB) ErrorClear() (err error)
- func (wdb *WrapDB) ErrorSet(err error)
- func (wdb *WrapDB) Get(recPtr Record, idx uint8, f func() bool)
- func (wdb *WrapDB) GetRec(recPtr Record, idx uint8)
- func (wdb *WrapDB) HexDump(wr io.Writer)
- func (wdb *WrapDB) Put(recPtr Record, f func() bool)
- func (wdb *WrapDB) PutRec(recPtr Record)
Examples ¶
Constants ¶
const (
// Version identifies the database compatibility level
Version = 1
)
Variables ¶
var ( // ErrMissingIndex is reported when data is to be accessed and the application // record indicates that no indexes are present ErrMissingIndex = errors.New("at least one index must be defined") // ErrMissingRecord indicates a corrupt database in which data is not // associated with a valid primary key ErrMissingRecord = errors.New("missing record for valid index") // ErrNotOpen is reported when a database operation is attempted on a closed // database instance ErrNotOpen = errors.New("database is not open") // ErrRecNotFound is reported when no match is found for the requested record ErrRecNotFound = errors.New("record not found") )
Functions ¶
This section is empty.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
The DB type manages data access with an underlying bbolt database. It is safe for concurrent goroutine use. Only one instance of this type should be active at a time.
Example (Quantity) ¶
ExampleDB_quantity demonstrates various operations with multiple indexes.
var db *pinion.DB
var wdb *pinion.WrapDB
var err error
db, err = quantityDB("example/quantity.db", 0, 256)
if err == nil {
wdb = db.Wrap()
var q quantityType
q.id = 99
fmt.Println("--- ID sequence ---")
wdb.Get(&q, idxQuantityID, func() bool {
if q.id < 104 {
fmt.Println(q)
return true
}
return false
})
q.val, _ = str.QuantityEncode(72)
count := 5
fmt.Println("--- Word sequence ---")
wdb.Get(&q, idxQuantityVal, func() bool {
if count > 0 {
fmt.Println(q)
count--
return true
}
return false
})
db.Close()
err = wdb.Error()
}
if err != nil {
fmt.Println(err)
}
Output: --- ID sequence --- [ 99 : ninety nine] [ 100 : one hundred] [ 101 : one hundred one] [ 102 : one hundred two] [ 103 : one hundred three] --- Word sequence --- [ 72 : seventy two] [ 6 : six] [ 16 : sixteen] [ 60 : sixty] [ 68 : sixty eight]
func (*DB) Add ¶
Add inserts one or more records in the database. It functions like Put() except that Add() will pass an autoincremented ID by means of the NextID interface method. If the application manages its own unique primary keys, it is more efficient to call Put() instead of Add(). It is crucial that all keys of each record processed by this method be properly assigned. This assures that modified keys are properly replaced.
func (*DB) AddRec ¶
AddRec inserts one record in the database. recPtr is a pointer to a variable that fully assigned. The requirements documented for Add() apply.
func (*DB) Close ¶
Close shuts down the database and releases all associated resources. Any subsequent calls to methods of DB will result in an error.
func (*DB) Delete ¶
Delete removes records and their associated keys from the database. recPtr is a pointer to a variable that will, each time f() returns true, be populated with a successive value to be delete. The iteration is stopped when f() returns false. Only the field or fields needed to generate the primary key (index 0) need be assigned.
func (*DB) DeleteRec ¶
DeleteRec deletes one record from the database. recPtr is a pointer to a variable that has at least the field or fields that make up the primary key (index 0) assigned.
func (*DB) Get ¶
Get returns zero or more records. It calls f iteratively until f() returns false or no more records are found. For each call of f, the record variable pointed to be recPtr will be populated with a successive value from the database. The record order is determined by the index specified by idx. The first record returned is the first one that matches the initial value of the record pointed to by recPtr. Only the field or fields that make up the key associated with index idx need to be assigned initially.
func (*DB) GetRec ¶
GetRec returns zero or one record from the database. The first record that matches the key field or fields associated with index idx will be put in the variable pointed to be recPtr. In this case, an error value of nil is returned. If no match is found, ErrRecNotFound is returned.
func (*DB) HexDump ¶
HexDump is a diagnostic routine to help with viewing the records and keys in a database.
func (*DB) Put ¶
Put inserts or replaces zero or more records in the database. recPtr is a pointer to a variable that will, each time f() returns true, be populated with a successive value to be stored. The iteration is stopped when f() returns false. If the primary key (that is, Key(0)) of a record already exists, the record will overwrite its previous value. If the primary key is unique, the record will be inserted; however, unlike Add(), the inserted record's NextID() method will not be called. It is crucial that all keys of each record processed by this method be properly assigned. This assures that modified keys are properly replaced.
type Record ¶
type Record interface {
// Convert the record identified by the method receiver to a byte sequence.
encoding.BinaryMarshaler
// Convert the specified byte sequence to the record identified by the method
// receiver.
encoding.BinaryUnmarshaler
// Return the number of indexes associated with the record receiver. This
// value must remain constant.
IndexCount() uint8
// Name of record for database table (invariant).
Name() string
// Construct a key for the index specified by idx.
Key(idx uint8) (key []byte, err error)
// Provide a temporary buffer for internal use. The returned record should be
// of the same type as the method receiver.
New() Record
// Receive an autoincremented ID prior to inserting a record. This is called
// only when the application calls Add(), not Put(). It is called before calls
// are made to Key().
NextID(uint64)
}
The Record interface specifies methods that allow pinion to manage multiply indexed records. The Name() and IndexCount() methods return constants; they return information about the type of the method receiver rather than the receiver instance itself. They are included here to allow all the information needed for pinion to manage database operations to be specified in one location.
It is very important that the implementation of this interface not change after data has been stored. The implementation is in essence a contract between the application and the stored data.
type WrapDB ¶
type WrapDB struct {
// contains filtered or unexported fields
}
WrapDB is a wrapper around DB that maintains error state internally. Its methods are the same as those of DB except that they do not return an error value. Instead, if an error is detected, it is stored for later examination. In this case, subsequent calls will be bypassed. The use of this wrapper may simplify code paths by deferring error handling until a series of database operations have completed.
Unlike a DB instance, a WrapDB instance is not safe for concurrent use. It is intended to be used locally for a relatively small sequence of method calls and then, after examining the error value returned by Error(), allowed to fall out of scope. Multiple goroutines may wrap a single *pinion.DB instance concurrently.
Example ¶
This code exemplifies the use of various WrapDB methods. These are like corresponding DB methods except that error values are not returned. Instead, they retain the error value internally. In this example, the DB instance is local. In a typical application, the database instance is global so that it can be shared among goroutines. Unlike the DB instance, a WrapDB instance should always be local so that errors can be managed locally and do not spill over to other goroutines.
var (
q quantityType
db *pinion.DB
wdb *pinion.WrapDB
fl *os.File
err error
list []uint32
)
show := func(str string) {
fmt.Printf("--- %s ---\n", str)
q = quantityType{} // Start at beginning with zeroed record
wdb.Get(&q, idxQuantityID, func() bool {
fmt.Println(q)
return true
})
}
db, err = quantityDB("example/dump.db", 1234, 1236)
if err == nil {
wdb = db.Wrap()
list = []uint32{42, 0}
wdb.Put(&q, func() bool {
if len(list) > 0 {
q = quantityRec(list[0])
list = list[1:]
return true
}
return false
})
fl, err = os.Create("example/dump.txt")
if err == nil {
wdb.HexDump(fl)
show("Full")
list = []uint32{1235, 0}
wdb.Delete(&q, func() bool {
if len(list) > 0 {
q.id = list[0]
list = list[1:]
return true
}
return false
})
show("Deleted ID 0 and 1235")
q = quantityRec(1232)
wdb.AddRec(&q)
show("Added 1232")
q.id = 42
wdb.DeleteRec(&q)
show("Deleted ID 42")
fl.Close()
if wdb.Error() == nil {
wdb.ErrorSet(pinion.ErrRecNotFound)
if wdb.Error() != nil {
wdb.ErrorClear()
}
}
err = wdb.Error()
}
db.Close()
}
if err != nil {
fmt.Println(err)
}
Output: --- Full --- [ 0 : zero] [ 42 : forty two] [ 1,234 : one thousand two hundred thirty four] [ 1,235 : one thousand two hundred thirty five] [ 1,236 : one thousand two hundred thirty six] --- Deleted ID 0 and 1235 --- [ 42 : forty two] [ 1,234 : one thousand two hundred thirty four] [ 1,236 : one thousand two hundred thirty six] --- Added 1232 --- [ 42 : forty two] [ 1,232 : one thousand two hundred thirty two] [ 1,234 : one thousand two hundred thirty four] [ 1,236 : one thousand two hundred thirty six] --- Deleted ID 42 --- [ 1,232 : one thousand two hundred thirty two] [ 1,234 : one thousand two hundred thirty four] [ 1,236 : one thousand two hundred thirty six]
func (*WrapDB) Error ¶
Error returns the internal error value. It does not change the internal value.
func (*WrapDB) ErrorClear ¶
ErrorClear clears the internal error value. The current value before being cleared is returned.
func (*WrapDB) ErrorSet ¶
ErrorSet allows the application to transfer its own error to the wrapped database instance. This may simplify code paths in the application because it allows the response to an error to be handled in one place. WrapDB cannot already be in an error state. If err is nil, it is ignored and will not overwrite the internal error value.
Source Files