storm

package module
v0.0.0-...-d8dddb2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2016 License: MIT Imports: 10 Imported by: 0

README

Storm

Build Status GoDoc Go Report Card Coverage

Storm is simple and powerful ORM for BoltDB. The goal of this project is to provide a simple way to save any object in BoltDB and to easily retrieve it.

Getting Started

go get -u github.com/asdine/storm

Import Storm

import "github.com/asdine/storm"

Open a database

Quick way of opening a database

db, err := storm.Open("my.db")

defer db.Close()

Open can receive multiple options to customize the way it behaves. See Options below

Simple ORM

Declare your structures
type User struct {
  ID int // primary key
  Group string `storm:"index"` // this field will be indexed
  Email string `storm:"unique"` // this field will be indexed with a unique constraint
  Name string // this field will not be indexed
  Age int `storm:"index"`
}

The primary key can be of any type as long as it is not a zero value. Storm will search for the tag id, if not present Storm will search for a field named ID.

type User struct {
  ThePrimaryKey string `storm:"id"`// primary key
  Group string `storm:"index"` // this field will be indexed
  Email string `storm:"unique"` // this field will be indexed with a unique constraint
  Name string // this field will not be indexed
}

Storm handles tags in nested structures with the inline tag

type Base struct {
  Ident bson.ObjectId `storm:"id"`
}

type User struct {
	Base      `storm:"inline"`
	Group     string `storm:"index"`
	Email     string `storm:"unique"`
	Name      string
	CreatedAt time.Time `storm:"index"`
}
Save your object
user := User{
  ID: 10,
  Group: "staff",
  Email: "john@provider.com",
  Name: "John",
  Age: 21,
  CreatedAt: time.Now(),
}

err := db.Save(&user)
// err == nil

user.ID++
err = db.Save(&user)
// err == "already exists"

That's it.

Save creates or updates all the required indexes and buckets, checks the unique constraints and saves the object to the store.

Fetch your object

Only indexed fields can be used to find a record

var user User
err := db.One("Email", "john@provider.com", &user)
// err == nil

err = db.One("Name", "John", &user)
// err == "not found"
Fetch multiple objects
var users []User
err := db.Find("Group", "staff", &users)
Fetch all objects
var users []User
err := db.All(&users)
Fetch all objects sorted by index
var users []User
err := db.AllByIndex("CreatedAt", &users)
Fetch a range of objects
var users []User
err := db.Range("Age", 10, 21, &users)
Skip and Limit
var users []User
err := db.Find("Group", "staff", &users, storm.Skip(10))
err = db.Find("Group", "staff", &users, storm.Limit(10))
err = db.Find("Group", "staff", &users, storm.Limit(10), storm.Skip(10))

err = db.All(&users, storm.Limit(10), storm.Skip(10))
err = db.AllByIndex("CreatedAt", &users, storm.Limit(10), storm.Skip(10))
err = db.Range("Age", 10, 21, &users, storm.Limit(10), storm.Skip(10))
Remove an object
err := db.Remove(&user)
Initialize buckets and indexes before saving an object
err := db.Init(&User{})

Useful when starting your application

Transactions
tx, err := db.Begin(true)

accountA.Amount -= 100
accountB.Amount += 100

err = tx.Save(accountA)
if err != nil {
  tx.Rollback()
  return err
}

err = tx.Save(accountB)
if err != nil {
  tx.Rollback()
  return err
}

tx.Commit()
Options

Storm options are functions that can be passed when constructing you Storm instance. You can pass it any number of options.

BoltOptions

By default, Storm opens a database with the mode 0600 and a timeout of one second. You can change this behavior by using BoltOptions

db, err := storm.Open("my.db", storm.BoltOptions(0600, &bolt.Options{Timeout: 1 * time.Second}))
EncodeDecoder

To store the data in BoltDB, Storm encodes it in GOB by default. If you wish to change this behavior you can pass a codec that implements codec.EncodeDecoder via the storm.Codec option:

db := storm.Open("my.db", storm.Codec(myCodec))
Provided Codecs

You can easily implement your own EncodeDecoder, but Storm comes with built-in support for GOB (default), JSON, and Sereal.

These can be used by importing the relevant package and use that codec to configure Storm. The example below shows all three (without proper error handling):

import (
	"github.com/asdine/storm"
	"github.com/asdine/storm/codec/gob"
	"github.com/asdine/storm/codec/json"
	"github.com/asdine/storm/codec/sereal"
)

var gobDb, _ = storm.Open("gob.db", storm.Codec(gob.Codec))
var jsonDb, _ = storm.Open("json.db", storm.Codec(json.Codec))
var serealDb, _ = storm.Open("sereal.db", storm.Codec(sereal.Codec))
Auto Increment

Storm can auto increment integer IDs so you don't have to worry about that when saving your objects.

db := storm.Open("my.db", storm.AutoIncrement())

Nodes and nested buckets

Storm takes advantage of BoltDB nested buckets feature by using storm.Node. A storm.Node is the underlying object used by storm.DB to manipulate a bucket. To create a nested bucket and use the same API as storm.DB, you can use the DB.From method.

repo := db.From("repo")

err := repo.Save(&Issue{
  Title: "I want more features",
  Author: user.ID,
})

err = repo.Save(newRelease("0.10"))

var issues []Issue
err = repo.Find("Author", user.ID, &issues)

var release Release
err = repo.One("Tag", "0.10", &release)

You can also chain the nodes to create a hierarchy

chars := db.From("characters")
heroes := chars.From("heroes")
enemies := chars.From("enemies")

items := db.From("items")
potions := items.From("consumables").From("medicine").From("potions")

You can even pass the entire hierarchy as arguments to From:

privateNotes := db.From("notes", "private")
workNotes :=  db.From("notes", "work")

Simple Key/Value store

Storm can be used as a simple, robust, key/value store that can store anything. The key and the value can be of any type as long as the key is not a zero value.

Saving data :

db.Set("logs", time.Now(), "I'm eating my breakfast man")
db.Set("sessions", bson.NewObjectId(), &someUser)
db.Set("weird storage", "754-3010", map[string]interface{}{
  "hair": "blonde",
  "likes": []string{"cheese", "star wars"},
})

Fetching data :

user := User{}
db.Get("sessions", someObjectId, &user)

var details map[string]interface{}
db.Get("weird storage", "754-3010", &details)

db.Get("sessions", someObjectId, &details)

Deleting data :

db.Delete("sessions", someObjectId)
db.Delete("weird storage", "754-3010")

BoltDB

BoltDB is still easily accessible and can be used as usual

db.Bolt.View(func(tx *bolt.Tx) error {
  bucket := tx.Bucket([]byte("my bucket"))
  val := bucket.Get([]byte("any id"))
  fmt.Println(string(val))
  return nil
})

A transaction can be also be passed to Storm

db.Bolt.Update(func(tx *bolt.Tx) error {
  ...
  dbx := db.WithTransaction(tx)
  err = dbx.Save(&user)
  ...
  return nil
})

TODO

  • Search
  • Reverse order
  • More indexes

License

MIT

Author

Asdine El Hrychy

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoID is returned when no ID field or id tag is found in the struct.
	ErrNoID = errors.New("missing struct tag id or ID field")

	// ErrZeroID is returned when the ID field is a zero value.
	ErrZeroID = errors.New("id field must not be a zero value")

	// ErrBadType is returned when a method receives an unexpected value type.
	ErrBadType = errors.New("provided data must be a struct or a pointer to struct")

	// ErrAlreadyExists is returned uses when trying to set an existing value on a field that has a unique index.
	ErrAlreadyExists = errors.New("already exists")

	// ErrNilParam is returned when the specified param is expected to be not nil.
	ErrNilParam = errors.New("param must not be nil")

	// ErrUnknownTag is returned when an unexpected tag is specified.
	ErrUnknownTag = errors.New("unknown tag")

	// ErrIdxNotFound is returned when the specified index is not found.
	ErrIdxNotFound = errors.New("index not found")

	// ErrSlicePtrNeeded is returned when an unexpected value is given, instead of a pointer to slice.
	ErrSlicePtrNeeded = errors.New("provided target must be a pointer to slice")

	// ErrSlicePtrNeeded is returned when an unexpected value is given, instead of a pointer to struct.
	ErrStructPtrNeeded = errors.New("provided target must be a pointer to struct")

	// ErrSlicePtrNeeded is returned when an unexpected value is given, instead of a pointer.
	ErrPtrNeeded = errors.New("provided target must be a pointer to a valid variable")

	// ErrNoName is returned when the specified struct has no name.
	ErrNoName = errors.New("provided target must have a name")

	// ErrNotFound is returned when the specified record is not saved in the bucket.
	ErrNotFound = errors.New("not found")

	// ErrNotInTransaction is returned when trying to rollback or commit when not in transaction.
	ErrNotInTransaction = errors.New("not in transaction")
)

Errors

Functions

func AutoIncrement 

func AutoIncrement() func(*DB) error

AutoIncrement used to enable bolt.NextSequence on empty integer ids.

func BoltOptions 

func BoltOptions(mode os.FileMode, options *bolt.Options) func(*DB) error

BoltOptions used to pass options to BoltDB.

func Codec 

func Codec(c codec.EncodeDecoder) func(*DB) error

Codec used to set a custom encoder and decoder. The default is GOB.

func Limit 

func Limit(limit int) func(q *queryOptions)

Limit sets the maximum number of records to return

func Root 

func Root(root ...string) func(*DB) error

Root used to set the root bucket. See also the From method.

func Skip 

func Skip(offset int) func(q *queryOptions)

Skip sets the number of records to skip

Types

type DB 

type DB struct {
	// Path of the database file
	Path string

	// Handles encoding and decoding of objects
	Codec codec.EncodeDecoder

	// Bolt is still easily accessible
	Bolt *bolt.DB
	// contains filtered or unexported fields
}

DB is the wrapper around BoltDB. It contains an instance of BoltDB and uses it to perform all the needed operations

func Open 

func Open(path string, stormOptions ...func(*DB) error) (*DB, error)

Open opens a database at the given path with optional Storm options.

func OpenWithOptions 

func OpenWithOptions(path string, mode os.FileMode, boltOptions *bolt.Options, stormOptions ...func(*DB)) (*DB, error)

OpenWithOptions opens a database with the given boltDB options and optional Storm options. Deprecated: Use storm.Open with storm.BoltOptions instead.

func (*DB) All 

func (s *DB) All(to interface{}, options ...func(*queryOptions)) error

All get all the records of a bucket

func (*DB) AllByIndex 

func (s *DB) AllByIndex(fieldName string, to interface{}, options ...func(*queryOptions)) error

AllByIndex gets all the records of a bucket that are indexed in the specified index

func (*DB) Begin 

func (s *DB) Begin(writable bool) (*Node, error)

Begin starts a new transaction.

func (*DB) Close 

func (s *DB) Close() error

Close the database

func (*DB) Commit 

func (s *DB) Commit() error

Commit writes all changes to disk.

func (*DB) Delete 

func (s *DB) Delete(bucketName string, key interface{}) error

Delete deletes a key from a bucket

func (*DB) Drop 

func (s *DB) Drop(bucketName string) error

Drop a bucket

func (*DB) Find 

func (s *DB) Find(fieldName string, value interface{}, to interface{}, options ...func(q *queryOptions)) error

Find returns one or more records by the specified index

func (*DB) From 

func (s *DB) From(root ...string) *Node

From returns a new Storm node with a new bucket root. All DB operations on the new node will be executed relative to the given bucket.

func (*DB) Get 

func (s *DB) Get(bucketName string, key interface{}, to interface{}) error

Get a value from a bucket

func (*DB) Init 

func (s *DB) Init(data interface{}) error

Init creates the indexes and buckets for a given structure

func (*DB) One 

func (s *DB) One(fieldName string, value interface{}, to interface{}) error

One returns one record by the specified index

func (*DB) Range 

func (s *DB) Range(fieldName string, min, max, to interface{}, options ...func(*queryOptions)) error

Range returns one or more records by the specified index within the specified range

func (*DB) Remove 

func (s *DB) Remove(data interface{}) error

Remove removes a structure from the associated bucket

func (*DB) Rollback 

func (s *DB) Rollback() error

Rollback closes the transaction and ignores all previous updates.

func (*DB) Save 

func (s *DB) Save(data interface{}) error

Save a structure

func (*DB) Set 

func (s *DB) Set(bucketName string, key interface{}, value interface{}) error

Set a key/value pair into a bucket

func (*DB) WithTransaction 

func (s *DB) WithTransaction(tx *bolt.Tx) *Node

WithTransaction returns a New Storm node that will use the given transaction.

type Index 

type Index interface {
	Add(value []byte, targetID []byte) error
	Remove(value []byte) error
	RemoveID(id []byte) error
	Get(value []byte) []byte
	All(value []byte, opts *queryOptions) ([][]byte, error)
	AllRecords(opts *queryOptions) ([][]byte, error)
	Range(min []byte, max []byte, opts *queryOptions) ([][]byte, error)
}

Index interface

type ListIndex 

type ListIndex struct {
	Parent      *bolt.Bucket
	IndexBucket *bolt.Bucket
	IDs         *UniqueIndex
}

ListIndex is an index that references values and the corresponding IDs.

func NewListIndex 

func NewListIndex(parent *bolt.Bucket, indexName []byte) (*ListIndex, error)

NewListIndex loads a ListIndex

func (*ListIndex) Add 

func (idx *ListIndex) Add(value []byte, targetID []byte) error

Add a value to the list index

func (*ListIndex) All 

func (idx *ListIndex) All(value []byte, opts *queryOptions) ([][]byte, error)

All the IDs corresponding to the given value

func (*ListIndex) AllRecords 

func (idx *ListIndex) AllRecords(opts *queryOptions) ([][]byte, error)

AllRecords returns all the IDs of this index

func (*ListIndex) Get 

func (idx *ListIndex) Get(value []byte) []byte

Get the first ID corresponding to the given value

func (*ListIndex) Range 

func (idx *ListIndex) Range(min []byte, max []byte, opts *queryOptions) ([][]byte, error)

Range returns the ids corresponding to the given range of values

func (*ListIndex) Remove 

func (idx *ListIndex) Remove(value []byte) error

Remove a value from the unique index

func (*ListIndex) RemoveID 

func (idx *ListIndex) RemoveID(targetID []byte) error

RemoveID removes an ID from the list index

type Node 

type Node struct {
	// contains filtered or unexported fields
}

A Node in Storm represents the API to a BoltDB bucket.

func (*Node) All 

func (n *Node) All(to interface{}, options ...func(*queryOptions)) error

All gets all the records of a bucket

func (*Node) AllByIndex 

func (n *Node) AllByIndex(fieldName string, to interface{}, options ...func(*queryOptions)) error

AllByIndex gets all the records of a bucket that are indexed in the specified index

func (Node) Begin 

func (n Node) Begin(writable bool) (*Node, error)

Begin starts a new transaction.

func (*Node) Commit 

func (n *Node) Commit() error

Commit writes all changes to disk.

func (*Node) CreateBucketIfNotExists 

func (n *Node) CreateBucketIfNotExists(tx *bolt.Tx, bucket string) (*bolt.Bucket, error)

CreateBucketIfNotExists creates the bucket below the current node if it doesn't already exist.

func (*Node) Delete 

func (n *Node) Delete(bucketName string, key interface{}) error

Delete deletes a key from a bucket

func (*Node) Drop 

func (n *Node) Drop(bucketName string) error

Drop a bucket

func (*Node) Find 

func (n *Node) Find(fieldName string, value interface{}, to interface{}, options ...func(q *queryOptions)) error

Find returns one or more records by the specified index

func (Node) From 

func (n Node) From(addend ...string) *Node

From returns a new Storm node with a new bucket root below the current. All DB operations on the new node will be executed relative to this bucket.

func (*Node) Get 

func (n *Node) Get(bucketName string, key interface{}, to interface{}) error

Get a value from a bucket

func (*Node) GetBucket 

func (n *Node) GetBucket(tx *bolt.Tx, children ...string) *bolt.Bucket

GetBucket returns the given bucket below the current node.

func (*Node) Init 

func (n *Node) Init(data interface{}) error

Init creates the indexes and buckets for a given structure

func (*Node) One 

func (n *Node) One(fieldName string, value interface{}, to interface{}) error

One returns one record by the specified index

func (*Node) Range 

func (n *Node) Range(fieldName string, min, max, to interface{}, options ...func(*queryOptions)) error

Range returns one or more records by the specified index within the specified range

func (*Node) Remove 

func (n *Node) Remove(data interface{}) error

Remove removes a structure from the associated bucket

func (*Node) Rollback 

func (n *Node) Rollback() error

Rollback closes the transaction and ignores all previous updates.

func (*Node) Save 

func (n *Node) Save(data interface{}) error

Save a structure

func (*Node) Set 

func (n *Node) Set(bucketName string, key interface{}, value interface{}) error

Set a key/value pair into a bucket

func (Node) WithTransaction 

func (n Node) WithTransaction(tx *bolt.Tx) *Node

WithTransaction returns a New Storm node that will use the given transaction.

type UniqueIndex 

type UniqueIndex struct {
	Parent      *bolt.Bucket
	IndexBucket *bolt.Bucket
}

UniqueIndex is an index that references unique values and the corresponding ID.

func NewUniqueIndex 

func NewUniqueIndex(parent *bolt.Bucket, indexName []byte) (*UniqueIndex, error)

NewUniqueIndex loads a UniqueIndex

func (*UniqueIndex) Add 

func (idx *UniqueIndex) Add(value []byte, targetID []byte) error

Add a value to the unique index

func (*UniqueIndex) All 

func (idx *UniqueIndex) All(value []byte, opts *queryOptions) ([][]byte, error)

All returns all the ids corresponding to the given value

func (*UniqueIndex) AllRecords 

func (idx *UniqueIndex) AllRecords(opts *queryOptions) ([][]byte, error)

AllRecords returns all the IDs of this index

func (*UniqueIndex) Get 

func (idx *UniqueIndex) Get(value []byte) []byte

Get the id corresponding to the given value

func (*UniqueIndex) Range 

func (idx *UniqueIndex) Range(min []byte, max []byte, opts *queryOptions) ([][]byte, error)

Range returns the ids corresponding to the given range of values

func (*UniqueIndex) Remove 

func (idx *UniqueIndex) Remove(value []byte) error

Remove a value from the unique index

func (*UniqueIndex) RemoveID 

func (idx *UniqueIndex) RemoveID(id []byte) error

RemoveID removes an ID from the unique index

Source Files

View all Source files

Directories

Path Synopsis
Package codec contains sub-packages with different codecs that can be used to encode and decode entities in Storm.
 Package codec contains sub-packages with different codecs that can be used to encode and decode entities in Storm.
gob
internal
json
sereal

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.