e2db
e2db is an experimental abstraction layer built on top of etcd providing an ORM-like interface. It is heavily influenced by the design of storm.
Table of Contents
Getting Started
Open a database
import (
"log"
"github.com/criticalstack/e2d/pkg/e2db"
)
func main() {
db, err := e2db.New(&e2db.Config{
ClientAddr: ":2379",
})
if err != nil {
log.Fatal(err)
}
defer db.Close()
}
Since e2db relies on the etcd clientv3, the connection must call the Close()
method when finished.
Configuration
Name |
Description |
ClientAddr |
The address for the etcd client server. This should not specify the URL parts like scheme as that will be built automatically. |
Namespace |
A namespace can be provided to transparently prefix all keys and isolate them from other non-e2db keys that may be in the database. |
CertFile |
Client cert |
KeyFile |
Client key |
CAFile |
Trusted CA cert |
To connect to an etcd server that has mTLS client authentication, all of the following values must be provided: CertFile
, KeyFile
, and CAFile
. This will also ensure that the appropriate scheme of https is used when generating the ClientURL
from the provided ClientAddr
.
Error handling
e2db uses the package github.com/pkg/errors for handling errors. For example, a query that does not return rows will returned the wrapped error
type ErrNoRows
, so the function errors.Cause must be called to get the underlying type for comparison:
if errors.Cause(err) == e2db.ErrNoRows {
// handle ErrNoRows error
}
Usage
Define a table
Table schema is defined by defining structs:
type User struct {
ID int `e2db:"increment"`
Name string `e2db:"index"`
Email string `e2db:"unique"`
Role string `e2db:"index,required"`
Enabled bool `e2db:"index"`
Created time.Time
}
Struct tags provide flexible ways of defining indexes or constraints:
Tag |
Description |
id |
Defines a field as the primary key |
increment |
Defines a field as the primary key and automatically increments the value starting from 1 |
index |
Creates an index for the field value |
unique |
Creates an index for the field value along with a unique constraint |
required |
Field must have a value provided |
Table metadata is stored the first time data is added for a table to ensure that other operations will not violate the table schema that has been established. Other important table-specific metadata includes table-level locks and auto-incrementing field information.
Index metadata is stored along with the table also and is modified in the same operation as the data (i.e. the cost of building the index is amortized with the operation).
So internally the table starts look like this:
Key |
Value Description |
/<namespace>/User/_table |
gob-encoded table metadata |
/<namespace>/User/_table/ID/last |
last increment value |
/<namespace>/User/_table/lock |
N/A |
/<namespace>/User/_index/Name/<value>/<pk> |
full key for the indexed item |
/<namespace>/User/_index/Email/<value> |
full key for the indexed item |
/<namespace>/User/_index/Role/<value>/<pk> |
full key for the indexed item |
/<namespace>/User/_index/Created/<value>/<pk> |
full key for the indexed item |
where an index key/value exists for every item that is indexed. In other words, for a table with schema like User
, 5 rows will result in 20 key/value pairs being stored given the above configuration for User
to satisfy building all the defined indexes.
Create a table object
Creating a table object can be achieved by passing in a concrete type for the defined table:
users := db.Table(new(User))
This can now be used as a reference to refer to that table. Under the hood, e2db is using this to lazily store and check any subsequent operations to match an existing schema (stored in the table metadata) with the one passed in. Checking this schema ensures that a table schema other than one already defined for a table will result in an error.
Insert a new object
user := User{
Name: "Smoot Wellington",
Email: "smoot.wellington@hotmail.com",
Role: "user",
Enabled: true,
Created: time.Now()
}
err := users.Insert(&user)
In this case there is an auto-incrementing field for ID
so after the call to Insert
the value for user.ID
will be set (before it will be the zero value).
Fetch one object
Using the tag id
or increment
designates a field to be the tables primary key:
var u User
err := users.Find("ID", 1, &u)
Getting a single object back by index is accomplished the same way:
err := users.Find("Name", "Smoot Wellington", &u)
Fetch multiple objects
var u []User
err := users.Find("Role", "user", &u)
Or simply fetch all objects in a table:
err := users.All(&u)
Fetch multiple objects sorted by index
To sort by index in ascending order:
var u []User
err := users.OrderBy("Name").Find("Role", "user", &u)
For descending, simply call Reverse()
:
err := users.OrderBy("Name").Reverse().Find("Role", "user", &u)
Update an object
user.Role = "admin"
err := users.Update(&user)
Delete one object
err := users.Delete("ID", 1)
Delete multiple objects
err := users.Delete("Role", "user")
Drop a table
Table metadata is stored in the database to ensure that the types match before an operation is performed. If a table has changed or no longer needed it might need to be dropped so a new table can replace it:
err := users.Drop()
This can be used to help migrate from one schema version to another.
Advanced Usage
Transactions
Transactions can be used to reduce the amount of table locking that is occurring. This is helpful when doing bulk insert/update/delete operations:
err := users.Tx(func(tx *Tx) error {
for _, row := range rows {
if err := tx.Insert(row); err != nil {
return err
}
}
return nil
})
In this case, only one lock will be acquired for the duration of the transaction.
Query filtering
err := users.Filter(q.Eq("Enabled", false)).Find("Role", "user", &u)
err := users.Filter(
q.And(
q.Eq("Enabled", false),
q.Not("Name", "superadmin")
)
).Find("Role", "user", &u)
err := users.Limit(5).Find("Role", "user", &u)
Distributed locks
Distributed locking is a powerful feature made possible by etcd. Arbitrary locks can be established based upon the key string passed to db.Lock()
, which allows for any node using e2db to synchronize.
func syncSomething() error {
unlock, err := db.Lock("sync/something", 30 * time.Second)
if err != nil {
return err
}
defer unlock()
// do stuff
return nil
}