README
¶
Lightweight document-oriented NoSQL Database
🇬🇧 English | 🇨🇳 简体中文 | 🇪🇸 Spanish
CloverDB is a lightweight NoSQL database designed for being simple and easily maintainable, thanks to its small code base. It has been inspired by tinyDB.
Features
- Document oriented
- Written in pure Golang
- Simple and intuitive api
- Easily maintainable
Why CloverDB?
CloverDB has been written for being easily maintainable. As such, it trades performance with simplicity, and is not intended to be an alternative to more performant databases such as MongoDB or MySQL. However, there are projects where running a separate database server may result overkilled, and, for simple queries, network delay may be the major performance bottleneck. For such scenarios, CloverDB may be a more suitable alternative.
Database Layout
Previously, CloverDB relied on the Badger key-value store as a storage layer. However, Badger is not suitable for every scenario (for example, when the database size is a constraint). This is why, the storage layer of CloverDB has been abstracted through a set of interface types to work with any key-value store. At the moment, CloverDB can work with both Badger and Bolt (by default Bolt is used).
Installation
Make sure you have a working Go environment (Go 1.18 or higher is required).
GO111MODULE=on go get github.com/MRHGST/cloverdb/v2
Databases and Collections
CloverDB stores data records as JSON documents, which are grouped together in collections. A database is made up of one or more collections.
Database
To store documents inside collections, you have to open a Clover database using the Open() function.
import (
"log"
"github.com/dgraph-io/badger/v4"
c "github.com/ostafen/clover"
badgerstore "github.com/MRHGST/cloverdb/v2/store/badger"
)
...
// by default, Bolt will be used internally
db, _ := c.Open("clover-db")
// use OpenWithStore() if you want to select a different storage backend
store, _ := badgerstore.Open(badger.DefaultOptions("").WithInMemory(true)) // opens a badger in memory database
db, _ := c.OpenWithStore(store)
defer db.Close() // remember to close the db when you have done
Collections
CloverDB stores documents inside collections. Collections are the schemaless equivalent of tables in relational databases. A collection is created by calling the CreateCollection() function on a database instance. New documents can be inserted using the Insert() or InsertOne() methods. Each document is uniquely identified by a Version 4 UUID stored in the _id special field and generated during insertion.
db, _ := c.Open("clover-db")
db.CreateCollection("myCollection") // create a new collection named "myCollection"
// insert a new document inside the collection
doc := c.NewDocument()
doc.Set("hello", "clover!")
// InsertOne returns the id of the inserted document
docId, _ := db.InsertOne("myCollection", doc)
fmt.Println(docId)
Importing and Exporting Collections
CloverDB is capable of easily importing and exporting collections to JSON format regardless of the storage engine used.
// dump the content of the "todos" collection in a "todos.json" file
db.ExportCollection("todos", "todos.json")
...
// recover the todos collection from the exported json file
db.DropCollection("todos")
db.ImportCollection("todos", "todos.json")
docs, _ := db.FindAll(c.NewQuery("todos"))
for _, doc := range docs {
log.Println(doc)
}
Queries
CloverDB is equipped with a fluent and elegant API to query your data. A query is represented by the Query object, which allows to retrieve documents matching a given criterion. A query can be created by passing a valid collection name to the Query() method.
Select All Documents in a Collection
The FindAll() method is used to retrieve all documents satisfying a given query.
docs, _ := db.FindAll(c.NewQuery("myCollection"))
todo := &struct {
Completed bool `clover:"completed"`
Title string `clover:"title"`
UserId int `clover:"userId"`
}{}
for _, doc := range docs {
doc.Unmarshal(todo)
log.Println(todo)
}
Filter Documents with Criteria
In order to filter the documents returned by FindAll(), you have to specify a query Criteria using the Where() method. A Criteria object simply represents a predicate on a document, evaluating to true only if the document satisfies all the query conditions.
The following example shows how to build a simple Criteria, matching all the documents having the completed field equal to true.
db.FindAll(c.NewQuery("todos").Where(c.Field("completed").Eq(true)))
// or equivalently
db.FindAll(c.NewQuery("todos").Where(c.Field("completed").IsTrue()))
In order to build very complex queries, we chain multiple Criteria objects by using the And() and Or() methods, each returning a new Criteria obtained by applying the corresponding logical operator.
// find all completed todos belonging to users with id 5 and 8
db.FindAll(c.NewQuery("todos").Where(c.Field("completed").Eq(true).And(c.Field("userId").In(5, 8))))
Naturally, you can also create Criteria involving multiple fields. CloverDB provides you with two equivalent ways to accomplish this:
db.FindAll(c.NewQuery("myCollection").Where(c.Field("myField1").Gt(c.Field("myField2"))))
// or, if you prefer
db.FindAll(c.NewQuery("myCollection").Where(c.Field("myField1").Gt("$myField2")))
Sorting Documents
To sort documents in CloverDB, you need to use Sort(). It is a variadic function which accepts a sequence of SortOption, each allowing to specify a field and a sorting direction.
A sorting direction can be one of 1 or -1, respectively corresponding to ascending and descending order. If no SortOption is provided, Sort() uses the _id field by default.
// Find any todo belonging to the most recent inserted user
db.FindFirst(c.NewQuery("todos").Sort(c.SortOption{"userId", -1}))
Skip/Limit Documents
Sometimes, it can be useful to discard some documents from the output, or simply set a limit on the maximum number of results returned by a query. For this purpose, CloverDB provides the Skip() and Limit() functions, both accepting an integer $n$ as parameter.
// discard the first 10 documents from the output,
// also limiting the maximum number of query results to 100
db.FindAll(c.NewQuery("todos").Skip(10).Limit(100))
Update/Delete Documents
The Update() method is used to modify specific fields of documents in a collection. The Delete() method is used to delete documents. Both methods belong to the Query object, so that it is easy to update and delete documents matching a particular query.
// mark all todos belonging to user with id 1 as completed
updates := make(map[string]interface{})
updates["completed"] = true
db.Update(c.NewQuery("todos").Where(c.Field("userId").Eq(1)), updates)
// delete all todos belonging to users with id 5 and 8
db.Delete(c.NewQuery("todos").Where(c.Field("userId").In(5,8)))
To update or delete a single document using a specific document id, use UpdateById() or DeleteById(), respectively:
docId := "1dbce353-d3c6-43b3-b5a8-80d8d876389b"
// update the document with the specified id
db.UpdateById("todos", docId, map[string]interface{}{"completed": true})
// or delete it
db.DeleteById("todos", docId)
Indexes
In CloverDB, indexes support the efficient execution of queries. Without indexes, a collection must be fully scanned to select those documents matching a given query. An index is a special data structure storing the values of a specific document field (or set of fields), sorted by the value of the field itself. This means that they can be exploited to supports efficient equality matches and range-based queries. Moreover, when documents are iterated through an index, results can be returned in sorted order without performing any additional sorting step. Note however that using indexes is not completely for free. Apart from increasing disk space, indexes require additional cpu-time during each insert and update/delete operation. Moreover, when accessing a document through an index, two disk reads must be performed, since indexes only store a reference (the document id) to the actual document. As a consequence, the speed-up is sensitive only when the specified criteria is used to access a restricted set of documents.
Creating an index
Currently, CloverDB only support single-field indexes. An index can be created simply by calling the CreateIndex() method, which takes both the names of the collection and the field to be indexed.
db.CreateIndex("myCollection", "myField")
Assume you have the following query:
criteria := c.Field("myField").Gt(a).And(c.Field("myField").Lt(b))
db.FindAll(c.NewQuery("myCollection").Where(criteria).Sort(c.SortOption{"myField", -1}))
where a and b are values of your choice. CloverDB will use the created index both to perform the range query and to return results in sorted order.
Data Types
Internally, CloverDB supports the following primitive data types: int64, uint64, float64, string, bool and time.Time. When possible, values having different types are silently converted to one of the internal types: signed integer values get converted to int64, while unsigned ones to uint64. Float32 values are extended to float64.
For example, consider the following snippet, which sets an uint8 value on a given document field:
doc := c.NewDocument()
doc.Set("myField", uint8(10)) // "myField" is automatically promoted to uint64
fmt.Println(doc.Get("myField").(uint64))
Pointer values are dereferenced until either nil or a non-pointer value is found:
var x int = 10
var ptr *int = &x
var ptr1 **int = &ptr
doc.Set("ptr", ptr)
doc.Set("ptr1", ptr1)
fmt.Println(doc.Get("ptr").(int64) == 10)
fmt.Println(doc.Get("ptr1").(int64) == 10)
ptr = nil
doc.Set("ptr1", ptr1)
// ptr1 is not nil, but it points to the nil "ptr" pointer, so the field is set to nil
fmt.Println(doc.Get("ptr1") == nil)
Invalid types leaves the document untouched:
doc := c.NewDocument()
doc.Set("myField", make(chan struct{}))
log.Println(doc.Has("myField")) // will output false
Contributing
CloverDB is actively developed. Any contribution, in the form of a suggestion, bug report or pull request, is well accepted 😊
Contributions and suggestions have been gratefully received from the following users:
Made with contrib.rocks.
Documentation
¶
Index ¶
- Variables
- func NewObjectId() string
- type CriteriaNormalizeVisitor
- type DB
- func (db *DB) Close() error
- func (db *DB) Count(q *query.Query) (int, error)
- func (db *DB) CreateCollection(name string) error
- func (db *DB) CreateCollectionByQuery(name string, q *query.Query) error
- func (db *DB) CreateIndex(collection, field string) error
- func (db *DB) Delete(q *query.Query) error
- func (db *DB) DeleteById(collection string, id string) error
- func (db *DB) DropCollection(name string) error
- func (db *DB) DropIndex(collection, field string) error
- func (db *DB) Exists(q *query.Query) (bool, error)
- func (db *DB) ExportCollection(collectionName string, exportPath string) error
- func (db *DB) FindAll(q *query.Query) ([]*d.Document, error)
- func (db *DB) FindById(collection string, id string) (*d.Document, error)
- func (db *DB) FindFirst(q *query.Query) (*d.Document, error)
- func (db *DB) ForEach(q *query.Query, consumer func(_ *d.Document) bool) error
- func (db *DB) HasCollection(name string) (bool, error)
- func (db *DB) HasIndex(collection, field string) (bool, error)
- func (db *DB) ImportCollection(collectionName string, importPath string) error
- func (db *DB) Insert(collectionName string, docs ...*d.Document) error
- func (db *DB) InsertOne(collectionName string, doc *d.Document) (string, error)
- func (db *DB) IterateDocs(q *query.Query, consumer docConsumer) error
- func (db *DB) ListCollections() ([]string, error)
- func (db *DB) ListIndexes(collection string) ([]index.Info, error)
- func (db *DB) ReplaceById(collection, docId string, doc *d.Document) error
- func (db *DB) Save(collectionName string, data interface{}) error
- func (db *DB) Update(q *query.Query, updateMap map[string]interface{}) error
- func (db *DB) UpdateById(collectionName string, docId string, updater func(doc *d.Document) *d.Document) error
- func (db *DB) UpdateFunc(q *query.Query, updateFunc func(doc *d.Document) *d.Document) error
- type FieldRangeVisitor
- type IndexSelectVisitor
- type NotFlattenVisitor
Constants ¶
This section is empty.
Variables ¶
var ( ErrCollectionExist = errors.New("collection already exist") ErrCollectionNotExist = errors.New("no such collection") ErrIndexExist = errors.New("index already exist") ErrIndexNotExist = errors.New("no such index") ErrDocumentNotExist = errors.New("no such document") ErrDuplicateKey = errors.New("duplicate key") )
Collection creation errors
Functions ¶
func NewObjectId ¶
func NewObjectId() string
Types ¶
type CriteriaNormalizeVisitor ¶
type CriteriaNormalizeVisitor struct {
// contains filtered or unexported fields
}
func (*CriteriaNormalizeVisitor) VisitBinaryCriteria ¶
func (v *CriteriaNormalizeVisitor) VisitBinaryCriteria(c *query.BinaryCriteria) interface{}
func (*CriteriaNormalizeVisitor) VisitNotCriteria ¶
func (v *CriteriaNormalizeVisitor) VisitNotCriteria(c *query.NotCriteria) interface{}
func (*CriteriaNormalizeVisitor) VisitUnaryCriteria ¶
func (v *CriteriaNormalizeVisitor) VisitUnaryCriteria(c *query.UnaryCriteria) interface{}
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB represents the entry point of each clover database.
func Open ¶
Open opens a new clover database on the supplied path. If such a folder doesn't exist, it is automatically created.
func OpenWithStore ¶
OpenWithStore opens a new clover database using the provided store.
func (*DB) Close ¶
Close releases all the resources and closes the database. After the call, the instance will no more be usable.
func (*DB) Count ¶
Count returns the number of documents which satisfy the query (i.e. len(q.FindAll()) == q.Count()).
func (*DB) CreateCollection ¶
CreateCollection creates a new empty collection with the given name.
func (*DB) CreateCollectionByQuery ¶
func (*DB) CreateIndex ¶
CreateIndex creates an index for the specified for the specified (index, collection) pair.
func (*DB) DeleteById ¶
DeleteById removes the document with the given id from the underlying collection, provided that such a document exists and satisfies the underlying query.
func (*DB) DropCollection ¶
DropCollection removes the collection with the given name, deleting any content on disk.
func (*DB) DropIndex ¶
DropIndex deletes the index, is such index exists for the specified (index, collection) pair.
func (*DB) ExportCollection ¶
ExportCollection exports an existing collection to a JSON file.
func (*DB) FindById ¶
FindById returns the document with the given id, if such a document exists and satisfies the underlying query, or null.
func (*DB) ForEach ¶
ForEach runs the consumer function for each document matching the provided query. If false is returned from the consumer function, then the iteration is stopped.
func (*DB) HasCollection ¶
HasCollection returns true if and only if the database contains a collection with the given name.
func (*DB) HasIndex ¶
HasIndex returns true if an index exists for the specified (index, collection) pair.
func (*DB) ImportCollection ¶
ImportCollection imports a collection from a JSON file.
func (*DB) InsertOne ¶
InsertOne inserts a single document to an existing collection. It returns the id of the inserted document.
func (*DB) ListCollections ¶
ListCollections returns a slice of strings containing the name of each collection stored in the db.
func (*DB) ListIndexes ¶
ListIndexes returns a list containing the names of all the indexes for the specified collection.
func (*DB) ReplaceById ¶
ReplaceById replaces the document with the specified id with the one provided. If no document exists, an ErrDocumentNotExist is returned.
func (*DB) Save ¶
Save or update a document, If you pass in a custom struct instead of a Document object, it is recommended to specify the _id field using struct tags.
func (*DB) Update ¶
Update updates all the document selected by q using the provided updateMap. Each update is specified by a mapping fieldName -> newValue.
func (*DB) UpdateById ¶
func (db *DB) UpdateById(collectionName string, docId string, updater func(doc *d.Document) *d.Document) error
UpdateById updates the document with the specified id using the supplied update map. If no document with the specified id exists, an ErrDocumentNotExist is returned.
type FieldRangeVisitor ¶
func NewFieldRangeVisitor ¶
func NewFieldRangeVisitor(fields []string) *FieldRangeVisitor
func (*FieldRangeVisitor) VisitBinaryCriteria ¶
func (v *FieldRangeVisitor) VisitBinaryCriteria(c *query.BinaryCriteria) interface{}
func (*FieldRangeVisitor) VisitNotCriteria ¶
func (v *FieldRangeVisitor) VisitNotCriteria(c *query.NotCriteria) interface{}
func (*FieldRangeVisitor) VisitUnaryCriteria ¶
func (v *FieldRangeVisitor) VisitUnaryCriteria(c *query.UnaryCriteria) interface{}
type IndexSelectVisitor ¶
func (*IndexSelectVisitor) VisitBinaryCriteria ¶
func (v *IndexSelectVisitor) VisitBinaryCriteria(c *query.BinaryCriteria) interface{}
func (*IndexSelectVisitor) VisitNotCriteria ¶
func (v *IndexSelectVisitor) VisitNotCriteria(c *query.NotCriteria) interface{}
func (*IndexSelectVisitor) VisitUnaryCriteria ¶
func (v *IndexSelectVisitor) VisitUnaryCriteria(c *query.UnaryCriteria) interface{}
type NotFlattenVisitor ¶
type NotFlattenVisitor struct {
}
func (*NotFlattenVisitor) VisitBinaryCriteria ¶
func (v *NotFlattenVisitor) VisitBinaryCriteria(c *query.BinaryCriteria) interface{}
func (*NotFlattenVisitor) VisitNotCriteria ¶
func (v *NotFlattenVisitor) VisitNotCriteria(c *query.NotCriteria) interface{}
func (*NotFlattenVisitor) VisitUnaryCriteria ¶
func (v *NotFlattenVisitor) VisitUnaryCriteria(c *query.UnaryCriteria) interface{}
Source Files
Directories