bolthold

package module
v0.0.0-...-30aac69 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 14, 2024 License: MIT Imports: 13 Imported by: 87

README

BoltHold

GoGoDoc Coverage Status Go Report Card

BoltHold is a simple querying and indexing layer on top of a Bolt DB instance. For a similar library built on Badger see BadgerHold.

The goal is to create a simple, higher level interface on top of Bolt DB that simplifies dealing with Go Types and finding data, but exposes the underlying Bolt DB for customizing as you wish. By default the encoding used is Gob, so feel free to use the GobEncoder/Decoder interface for faster serialization. Or, alternately, you can use any serialization you want by supplying encode / decode funcs to the Options struct on Open.

One Go Type will have one bucket, and multiple index buckets in a BoltDB file, so you can store multiple Go Types in the same database.

Why not just use Bolt DB directly?

I love BoltDB, and I've used it in several projects. However, I find myself writing the same code over and over again, for encoding and decoding objects and searching through data. I figure formalizing how I've been using BoltDB and including tests and benchmarks will, at a minimum, be useful to me. Maybe it'll be useful to others as well.

Indexes

Indexes allow you to skip checking any records that don't meet your index criteria. If you have 1000 records and only 10 of them are of the Division you want to deal with, then you don't need to check to see if the other 990 records match your query criteria if you create an index on the Division field. The downside of an index is added disk reads and writes on every write operation. For read heavy operations datasets, indexes can be very useful.

In every BoltHold store, there will be a reserved bucket _indexes which will be used to hold indexes that point back to another bucket's Key system. Indexes will be defined by setting the boltholdIndex struct tag on a field in a type.

type Person struct {
	Name string
	Division string `boltholdIndex:"Division"`
}

This means that there will be an index created for Division that will contain the set of unique divisions, and the main record keys they refer to. More information on how indexes work can be found here

Optionally, you can implement the Storer interface, to specify your own indexes, rather than using the boltholdIndex struct tag.

Slice Indexes

When you create an index on a slice of items, by default it may not do what you expect. Consider the following records:

ID Name Categories
1 John red, green, blue
2 Bill red, purple
3 Jane red, orange
4 Brian red, purple

You may expect your Categories index to look like the following:

Categories ID
red 1, 2, 3, 4
green 1
blue 1
purple 2, 4
orange 3

But they'll actually look like this:

Categories ID
red, green, blue 1
red, purple 2, 4
red, orange 3

So if you did a query like this:

bh.Where("Categories").Contains("red").Index("Categories")

It'll work, but you'll be reading more records than you'd expect. You'd only "save reads" on values where the list of categories exactly match.

If instead you want to index each individual item in the slice, you can use the struct tag boltholdSliceIndex. It will then individually index each item in the slice, and potentially give you performance benefits if you have a lot of overlap with individual items in your sliced fields.

Be sure to benchmark both a regular index and a sliced index to see which performs better for your specific dataset.

Queries

Queries are chain-able constructs that filters out any data that doesn't match it's criteria. An index will be used if the .Index() chain is called, otherwise bolthold won't use any index.

Queries will look like this:

s.Find(&result, bolthold.Where("FieldName").Eq(value).And("AnotherField").Lt(AnotherValue).Or(bolthold.Where("FieldName").Eq(anotherValue)))

Fields must be exported, and thus always need to start with an upper-case letter. Available operators include:

  • Equal - Where("field").Eq(value)
  • Not Equal - Where("field").Ne(value)
  • Greater Than - Where("field").Gt(value)
  • Less Than - Where("field").Lt(value)
  • Less than or Equal To - Where("field").Le(value)
  • Greater Than or Equal To - Where("field").Ge(value)
  • In - Where("field").In(val1, val2, val3)
  • IsNil - Where("field").IsNil()
  • Regular Expression - Where("field").RegExp(regexp.MustCompile("ea"))
  • Matches Function
    • Where("field").MatchFunc(func(ra *RecordAccess) (bool, error)) // see RecordAccess Type
    • Where("field").MatchFunc(func(m *MyType) (bool, error))
    • Where("field").MatchFunc(func(field string) (bool, error))
  • Skip - Where("field").Eq(value).Skip(10)
  • Limit - Where("field").Eq(value).Limit(10)
  • SortBy - Where("field").Eq(value).SortBy("field1", "field2")
  • Reverse - Where("field").Eq(value).SortBy("field").Reverse()
  • Index - Where("field").Eq(value).Index("indexName")
  • Not - Where("field").Not().In(val1, val2, val3)
  • Contains - Where("field").Contains(val1)
  • ContainsAll - Where("field").Contains(val1, val2, val3)
  • ContainsAny - Where("field").Contains(val1, val2, val3)
  • HasKey - Where("field").HasKey(val1) // to test if a Map value has a key

An empty / zero value query matches against all records, because it has no critiera. You can then use Skip and Limit to page through all records in your dataset:

q := &bolthold.Query{}

err := store.Find(&result, q.Skip(10).Limit(50))

If you want to run a query's criteria against the Key value, you can use the bolthold.Key constant:

store.Find(&result, bolthold.Where(bolthold.Key).Ne(value))

You can access nested structure fields in queries like this:

type Repo struct {
  Name string
  Contact ContactPerson
}

type ContactPerson struct {
  Name string
}

store.Find(&repo, bolthold.Where("Contact.Name").Eq("some-name")

Instead of passing in a specific value to compare against in a query, you can compare against another field in the same struct. Consider the following struct:

type Person struct {
	Name string
	Birth time.Time
	Death time.Time
}

If you wanted to find any invalid records where a Person's death was before their birth, you could do the following:

store.Find(&result, bolthold.Where("Death").Lt(bolthold.Field("Birth")))

Queries can be used in more than just selecting data. You can delete or update data that matches a query.

Using the example above, if you wanted to remove all of the invalid records where Death < Birth:

// you must pass in a sample type, so BoltHold knows which bucket to use and what indexes to update
store.DeleteMatching(&Person{}, bolthold.Where("Death").Lt(bolthold.Field("Birth")))

Or if you wanted to update all the invalid records to flip/flop the Birth and Death dates:

store.UpdateMatching(&Person{}, bolthold.Where("Death").Lt(bolthold.Field("Birth")), func(record interface{}) error {
	update, ok := record.(*Person) // record will always be a pointer
	if !ok {
		return fmt.Errorf("Record isn't the correct type!  Wanted Person, got %T", record)
	}

	update.Birth, update.Death = update.Death, update.Birth

	return nil
})

If you simply want to count the number of records returned by a query use the Count method:

// need to pass in empty datatype so bolthold knows what type to count
count, err := store.Count(&Person{}, bolthold.Where("Death").Lt(bolthold.Field("Birth")))
Keys in Structs

A common scenario is to store the bolthold Key in the same struct that is stored in the boltDB value. You can automatically populate a record's Key in a struct by using the boltholdKey struct tag when running Find queries.

Another common scenario is to insert data with an auto-incrementing key assigned by the database. When performing an Insert, if the type of the key matches the type of the boltholdKey tagged field, the data is passed in by reference, and the field's current value is the zero-value for that type, then it is set on the data before insertion.

type Employee struct {
	ID string `boltholdKey:"ID"`  // the tagName isn't required, but some linters will complain without it
	FirstName string
	LastName string
	Division string
	Hired time.Time
}

Bolthold assumes only one of such struct tags exists. If a value already exists in the key field, it will be overwritten.

If you want to insert an auto-incrementing Key you can pass the bolthold.NextSequence() func as the Key value.

err := store.Insert(bolthold.NextSequence(), data)

The key value will be a uint64.

If you want to know the value of the auto-incrementing Key that was generated using bolthold.NextSequence(), then make sure to pass your data by reference and that the boltholdKey tagged field is of type uint64.

err := store.Insert(bolthold.NextSequence(), &data)
Slices in Structs and Queries

When querying slice fields in structs you can use the Contains, ContainsAll and ContainsAny criterion.

val := struct {
    Set []string
}{
    Set: []string{"1", "2", "3"},
}
bh.Where("Set").Contains("1") // true
bh.Where("Set").ContainsAll("1", "3") // true
bh.Where("Set").ContainsAll("1", "3", "4") // false
bh.Where("Set").ContainsAny("1", "7", "4") // true

The In, ContainsAll and ContainsAny critierion accept a slice of interface{} values. This means you can build your queries by passing in your values as arguments:

where := bolthold.Where("Id").In("1", "2", "3")

However if you have an existing slice of values to test against, you can't pass in that slice because it is not of type []interface{}.

t := []string{"1", "2", "3", "4"}
where := bolthold.Where("Id").In(t...) // compile error

Instead you need to copy your slice into another slice of empty interfaces:

t := []string{"1", "2", "3", "4"}
s := make([]interface{}, len(t))
for i, v := range t {
    s[i] = v
}
where := bolthold.Where("Id").In(s...)

You can use the helper function bolthold.Slice which does exactly that.

t := []string{"1", "2", "3", "4"}
where := bolthold.Where("Id").In(bolthold.Slice(t)...)
Unique Constraints

You can create a unique constraint on a given field by using the boltholdUnique:"ConstraintName" struct tag:

type User struct {
  Name string
  Email string `boltholdUnique:"UniqueEmail"` // this field will be indexed with a unique constraint
}

The example above will only allow one record of type User to exist with a given Email field. Any insert, update or upsert that would violate that constraint will fail and return the bolthold.ErrUniqueExists error.

ForEach

When working with large datasets, you may not want to have to store the entire dataset in memory. It's be much more efficient to work with a single record at a time rather than grab all the records and loop through them, which is what cursors are used for in databases. In BoltHold you can accomplish the same thing by calling ForEach:

err := store.ForEach(boltholdWhere("Id").Gt(4), func(record *Item) error {
	// do stuff with record

	// if you return an error, then the query will stop iterating through records

	return nil
})
Aggregate Queries

Aggregate queries are queries that group results by a field. For example, lets say you had a collection of employees:

type Employee struct {
	FirstName string
	LastName string
	Division string
	Hired time.Time
}

And you wanted to find the most senior (first hired) employee in each division:

result, err := store.FindAggregate(&Employee{}, nil, "Division") //nil query matches against all records

This will return a slice of Aggregate Result from which you can extract your groups and find Min, Max, Avg, Count, etc.

for i := range result {
	var division string
	employee := &Employee{}

	result[i].Group(&division)
	result[i].Min("Hired", employee)

	fmt.Printf("The most senior employee in the %s division is %s.\n",
		division, employee.FirstName + " " + employee.LastName)
}

Aggregate queries become especially powerful when combined with the sub-querying capability of MatchFunc.

Many more examples of queries can be found in the find_test.go file in this repository.

Comparing

Just like with Go, types must be the same in order to be compared with each other. You cannot compare an int to a int32. The built-in Go comparable types (ints, floats, strings, etc) will work as expected. Other types from the standard library can also be compared such as time.Time, big.Rat, big.Int, and big.Float. If there are other standard library types that I missed, let me know.

You can compare any custom type either by using the MatchFunc criteria, or by satisfying the Comparer interface with your type by adding the Compare method: Compare(other interface{}) (int, error).

If a type doesn't have a predefined comparer, and doesn't satisfy the Comparer interface, then the types value is converted to a string and compared lexicographically.

Behavior Changes

Since BoltHold is a higher level interface than BoltDB, there are some added helpers. Instead of Put, you have the options of:

  • Insert - Fails if key already exists.
  • Update - Fails if key doesn't exist ErrNotFound.
  • Upsert - If key doesn't exist, it inserts the data, otherwise it updates the existing record.

When getting data instead of returning nil if a value doesn't exist, BoltHold returns bolthold.ErrNotFound, and similarly when deleting data, instead of silently continuing if a value isn't found to delete, BoltHold returns bolthold.ErrNotFound. The exception to this is when using query based functions such as Find (returns an empty slice), DeleteMatching and UpdateMatching where no error is returned.

When should I use BoltHold?

BoltHold will be useful in the same scenarios where BoltDB is useful, with the added benefit of being able to retire some of your data filtering code and possibly improved performance.

You can also use it instead of SQLite for many scenarios. BoltHold's main benefit over SQLite is its simplicity when working with Go Types. There is no need for an ORM layer to translate records to types, simply put types in, and get types out. You also don't have to deal with database initialization. Usually with SQLite you'll need several scripts to create the database, create the tables you expect, and create any indexes. With BoltHold you simply open a new file and put any type of data you want in it.

store, err := bolthold.Open(filename, 0666, nil)
if err != nil {
	//handle error
}
err = store.Insert("key", &Item{
	Name:    "Test Name",
	Created: time.Now(),
})

That's it!

Bolthold currently has over 80% coverage in unit tests, and it's backed by BoltDB which is a very solid and well built piece of software, so I encourage you to give it a try.

If you end up using BoltHold, I'd love to hear about it.

Documentation

Overview

Package bolthold is an indexing and querying layer on top of a Bolt DB. The goal is to allow easy, persistent storage and retrieval of Go types. BoltDB is an embedded key-value store, and bolthold servers a similar use case however with a higher level interface for common uses of BoltDB.

Go Types

BoltHold deals directly with Go Types. When inserting data, you pass in your structure directly. When querying data you pass in a pointer to a slice of the type you want to return. By default Gob encoding is used. You can put multiple different types into the same DB file and they (and their indexes) will be stored separately. 

err := store.Insert(1234, Item{
	Name:    "Test Name",
	Created: time.Now(),
})

var result []Item

err := store.Find(&result, query)

Indexes

BoltHold will automatically create an index for any struct fields tags with "boltholdIndex" 

type Item struct {
	ID       int
	Name     string
	Category string `boltholdIndex:"Category"`
	Created  time.Time
}

The first field specified in query will be used as the index (if one exists).

Queries are chained together criteria that applies to a set of fields: 

bolthold.Where("Name").Eq("John Doe").And("DOB").Lt(time.Now())
Example 
package main

import (
	"fmt"
	"log"
	"os"
	"time"

	"github.com/timshannon/bolthold"
	bolt "go.etcd.io/bbolt"
)

type Item struct {
	ID       int
	Category string `boltholdIndex:"Category"`
	Created  time.Time
}

func main() {
	data := []Item{
		{
			ID:       0,
			Category: "blue",
			Created:  time.Now().Add(-4 * time.Hour),
		},
		{
			ID:       1,
			Category: "red",
			Created:  time.Now().Add(-3 * time.Hour),
		},
		{
			ID:       2,
			Category: "blue",
			Created:  time.Now().Add(-2 * time.Hour),
		},
		{
			ID:       3,
			Category: "blue",
			Created:  time.Now().Add(-20 * time.Minute),
		},
	}

	filename := tempfile()
	store, err := bolthold.Open(filename, 0666, nil)
	defer store.Close()
	defer os.Remove(filename)

	if err != nil {
		// handle error
		log.Fatal(err)
	}

	// insert the data in one transaction

	err = store.Bolt().Update(func(tx *bolt.Tx) error {
		for i := range data {
			err := store.TxInsert(tx, data[i].ID, data[i])
			if err != nil {
				return err
			}
		}
		return nil
	})

	if err != nil {
		// handle error
		log.Fatal(err)
	}

	// Find all items in the blue category that have been created in the past hour
	var result []Item

	err = store.Find(&result, bolthold.Where("Category").Eq("blue").And("Created").Ge(time.Now().Add(-1*time.Hour)))

	if err != nil {
		// handle error
		log.Fatal(err)
	}

	fmt.Println(result[0].ID)
}

Output:

3

Index

Examples

Constants

View Source 
const BoltholdIndexTag = "boltholdIndex"

BoltholdIndexTag is the struct tag used to define a field as indexable for a bolthold

View Source 
const BoltholdKeyTag = "boltholdKey"

BoltholdKeyTag is the struct tag used to define an a field as a key for use in a Find query

View Source 
const BoltholdSliceIndexTag = "boltholdSliceIndex"

BoltholdSliceIndexTag is the struct tag used to define a slice field as indexable, where each item in the slice is indexed separately rather than as one index

View Source 
const BoltholdUniqueTag = "boltholdUnique"

BoltholdUniqueTag is the struct tag used to define a field as unique constraint

View Source 
const Key = ""

Key is shorthand for specifying a query to run again the Key in a bolthold, simply returns "" Where(bolthold.Key).Eq("testkey")

Variables

View Source 
var ErrKeyExists = errors.New("This Key already exists in this bolthold for this type")

ErrKeyExists is the error returned when data is being Inserted for a Key that already exists

View Source 
var ErrNotFound = errors.New("No data found for this key")

ErrNotFound is returned when no data is found for the given key

View Source 
var ErrUniqueExists = errors.New("This value cannot be written due to the unique constraint on the field")

ErrUniqueExists is the error thrown when data is being inserted for a unique constraint value that already exists

Functions

func DefaultDecode 

func DefaultDecode(data []byte, value interface{}) error

DefaultDecode is the default decoding func for bolthold (Gob)

func DefaultEncode 

func DefaultEncode(value interface{}) ([]byte, error)

DefaultEncode is the default encoding func for bolthold (Gob)

func NextSequence 

func NextSequence() interface{}

NextSequence is used to create a sequential key for inserts Inserts a uint64 as the key store.Insert(bolthold.NextSequence(), data)

func Slice 

func Slice(value interface{}) []interface{}

Slice turns a slice of any time into []interface{} by copying the slice values so it can be easily passed into queries that accept variadic parameters. Will panic if value is not a slice

Types

type AggregateResult 

type AggregateResult struct {
	// contains filtered or unexported fields
}

AggregateResult allows you to access the results of an aggregate query

func (*AggregateResult) Avg 

func (a *AggregateResult) Avg(field string) float64

Avg returns the average float value of the aggregate grouping panics if the field cannot be converted to an float64

func (*AggregateResult) Count 

func (a *AggregateResult) Count() int

Count returns the number of records in the aggregate grouping

func (*AggregateResult) Group 

func (a *AggregateResult) Group(result ...interface{})

Group returns the field grouped by in the query

func (*AggregateResult) Max 

func (a *AggregateResult) Max(field string, result interface{})

Max Returns the maxiumum value of the Aggregate Grouping, uses the Comparer interface

func (*AggregateResult) Min 

func (a *AggregateResult) Min(field string, result interface{})

Min returns the minimum value of the Aggregate Grouping, uses the Comparer interface

func (*AggregateResult) Reduction 

func (a *AggregateResult) Reduction(result interface{})

Reduction is the collection of records that are part of the AggregateResult Group

func (*AggregateResult) Sort 

func (a *AggregateResult) Sort(field string)

Sort sorts the aggregate reduction by the passed in field in ascending order Sort is called automatically by calls to Min / Max to get the min and max values

func (*AggregateResult) Sum 

func (a *AggregateResult) Sum(field string) float64

Sum returns the sum value of the aggregate grouping panics if the field cannot be converted to an float64

type BucketSource 

type BucketSource interface {
	Bucket(name []byte) *bolt.Bucket
	CreateBucketIfNotExists(name []byte) (*bolt.Bucket, error)
}

BucketSource is the source of a bucket for running a query or updating data Buckets and Transactions both implement BucketSource. This allows for choosing a specific bucket or transaction when running a query

type Comparer 

type Comparer interface {
	Compare(other interface{}) (int, error)
}

Comparer compares a type against the encoded value in the store. The result should be 0 if current==other, -1 if current < other, and +1 if current > other. If a field in a struct doesn't specify a comparer, then the default comparison is used (convert to string and compare) this interface is already handled for standard Go Types as well as more complex ones such as those in time and big an error is returned if the type cannot be compared The concrete type will always be passedin, not a pointer

type Criterion 

type Criterion struct {
	// contains filtered or unexported fields
}

Criterion is an operator and a value that a given field needs to match on

func Where 

func Where(field string) *Criterion

Where starts a query for specifying the criteria that an object in the bolthold needs to match to be returned in a Find result

Query API Example 

s.Find(bolthold.Where("FieldName").Eq(value).And("AnotherField").Lt(AnotherValue).Or(bolthold.Where("FieldName").Eq(anotherValue)

Since Gobs only encode exported fields, this will panic if you pass in a field with a lower case first letter

func (*Criterion) Contains 

func (c *Criterion) Contains(value interface{}) *Query

Contains tests if the current field is a slice that contains the passed in value

func (*Criterion) ContainsAll 

func (c *Criterion) ContainsAll(values ...interface{}) *Query

ContainsAll tests if the current field is a slice that contains all of the passed in values. If any of the values are NOT contained in the slice, then no match is made

func (*Criterion) ContainsAny 

func (c *Criterion) ContainsAny(values ...interface{}) *Query

ContainsAny tests if the current field is a slice that contains any of the passed in values. If any of the values are contained in the slice, then a match is made

func (*Criterion) Eq 

func (c *Criterion) Eq(value interface{}) *Query

Eq tests if the current field is Equal to the passed in value

func (*Criterion) Ge 

func (c *Criterion) Ge(value interface{}) *Query

Ge test if the current field is Greater Than or Equal To the passed in value

func (*Criterion) Gt 

func (c *Criterion) Gt(value interface{}) *Query

Gt test if the current field is Greater Than the passed in value

func (*Criterion) HasKey 

func (c *Criterion) HasKey(value interface{}) *Query

HasKey tests if the field has a map key matching the passed in value

func (*Criterion) In 

func (c *Criterion) In(values ...interface{}) *Query

In test if the current field is a member of the slice of values passed in

func (*Criterion) IsNil 

func (c *Criterion) IsNil() *Query

IsNil will test if a field is equal to nil

func (*Criterion) Le 

func (c *Criterion) Le(value interface{}) *Query

Le test if the current field is Less Than or Equal To the passed in value

func (*Criterion) Lt 

func (c *Criterion) Lt(value interface{}) *Query

Lt test if the current field is Less Than the passed in value

func (*Criterion) MatchFunc 

func (c *Criterion) MatchFunc(match interface{}) *Query

MatchFunc will test if a field matches the passed in function

func (*Criterion) Ne 

func (c *Criterion) Ne(value interface{}) *Query

Ne test if the current field is Not Equal to the passed in value

func (*Criterion) Not 

func (c *Criterion) Not() *Criterion

Not will negate the following critierion

func (*Criterion) RegExp 

func (c *Criterion) RegExp(expression *regexp.Regexp) *Query

RegExp will test if a field matches against the regular expression The Field Value will be converted to string (%s) before testing

func (*Criterion) String 

func (c *Criterion) String() string

type DecodeFunc 

type DecodeFunc func(data []byte, value interface{}) error

DecodeFunc is a function for decoding a value from bytes

type EncodeFunc 

type EncodeFunc func(value interface{}) ([]byte, error)

EncodeFunc is a function for encoding a value into bytes

type ErrTypeMismatch 

type ErrTypeMismatch struct {
	Value interface{}
	Other interface{}
}

ErrTypeMismatch is the error thrown when two types cannot be compared

func (*ErrTypeMismatch) Error 

func (e *ErrTypeMismatch) Error() string

type Field 

type Field string

Field allows for referencing a field in structure being compared

type Index 

type Index struct {
	IndexFunc func(name string, value interface{}) ([]byte, error)
	Unique    bool
}

Index is a function that returns the indexable, encoded bytes of the passed in value

type MatchFunc 

type MatchFunc func(ra interface{}) (bool, error)

MatchFunc is a function used to test an arbitrary matching value in a query

type Options 

type Options struct {
	Encoder EncodeFunc
	Decoder DecodeFunc
	*bolt.Options
}

Options allows you set different options from the defaults For example the encoding and decoding funcs which default to Gob

type Query 

type Query struct {
	// contains filtered or unexported fields
}

Query is a chained collection of criteria of which an object in the bolthold needs to match to be returned an empty query matches against all records

func (*Query) And 

func (q *Query) And(field string) *Criterion

And creates a nother set of criterion the needs to apply to a query

func (*Query) Index 

func (q *Query) Index(indexName string) *Query

Index specifies the index to use when running this query

func (*Query) IsEmpty 

func (q *Query) IsEmpty() bool

IsEmpty returns true if the query is an empty query an empty query matches against everything

func (*Query) Limit 

func (q *Query) Limit(amount int) *Query

Limit sets the maximum number of records that can be returned by a query Setting Limit multiple times, or to a negative value will panic

func (*Query) Or 

func (q *Query) Or(query *Query) *Query

Or creates another separate query that gets unioned with any other results in the query Or will panic if the query passed in contains a limit or skip value, as they are only allowed on top level queries

func (*Query) Reverse 

func (q *Query) Reverse() *Query

Reverse will reverse the current result set useful with SortBy

func (*Query) Skip 

func (q *Query) Skip(amount int) *Query

Skip skips the number of records that match all the rest of the query criteria, and does not return them in the result set. Setting skip multiple times, or to a negative value will panic

func (*Query) SortBy 

func (q *Query) SortBy(fields ...string) *Query

SortBy sorts the results by the given fields name Multiple fields can be used

func (*Query) String 

func (q *Query) String() string

type RecordAccess 

type RecordAccess struct {
	// contains filtered or unexported fields
}

RecordAccess allows access to the current record, field or allows running a subquery within a MatchFunc

func (*RecordAccess) Field 

func (r *RecordAccess) Field() interface{}

Field is the current field being queried

func (*RecordAccess) Record 

func (r *RecordAccess) Record() interface{}

Record is the complete record for a given row in bolthold

func (*RecordAccess) SubAggregateQuery 

func (r *RecordAccess) SubAggregateQuery(query *Query, groupBy ...string) ([]*AggregateResult, error)

SubAggregateQuery allows you to run another aggregate query in the same transaction for each record in a parent query

func (*RecordAccess) SubQuery 

func (r *RecordAccess) SubQuery(result interface{}, query *Query) error

SubQuery allows you to run another query in the same transaction for each record in a parent query

type SliceIndex 

type SliceIndex func(name string, value interface{}) ([][]byte, error)

SliceIndex is a function that returns all of the indexable values in a slice

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is a bolthold wrapper around a bolt DB

func Open 

func Open(filename string, mode os.FileMode, options *Options) (*Store, error)

Open opens or creates a bolthold file.

func (*Store) Bolt 

func (s *Store) Bolt() *bolt.DB

Bolt returns the underlying Bolt DB the bolthold is based on

func (*Store) Close 

func (s *Store) Close() error

Close closes the bolt db

func (*Store) Count 

func (s *Store) Count(dataType interface{}, query *Query) (int, error)

Count returns the current record count for the passed in datatype

func (*Store) CountInBucket 

func (s *Store) CountInBucket(parent *bolt.Bucket, dataType interface{}, query *Query) (int, error)

CountInBucket returns the current record count from within the given parent bucket

func (*Store) Delete 

func (s *Store) Delete(key, dataType interface{}) error

Delete deletes a record from the bolthold, datatype just needs to be an example of the type stored so that the proper bucket and indexes are updated

func (*Store) DeleteFromBucket 

func (s *Store) DeleteFromBucket(parent *bolt.Bucket, key, dataType interface{}) error

DeleteFromBucket allows you to specify the parent bucket to delete from

func (*Store) DeleteMatching 

func (s *Store) DeleteMatching(dataType interface{}, query *Query) error

DeleteMatching deletes all of the records that match the passed in query

func (*Store) DeleteMatchingFromBucket 

func (s *Store) DeleteMatchingFromBucket(parent *bolt.Bucket, dataType interface{}, query *Query) error

DeleteMatchingFromBucket does the same as DeleteMatching, but allows you to specify your own parent bucket

func (*Store) Find 

func (s *Store) Find(result interface{}, query *Query) error

Find retrieves a set of values from the bolthold that matches the passed in query result must be a pointer to a slice. The result of the query will be appended to the passed in result slice, rather than the passed in slice being emptied.

func (*Store) FindAggregate 

func (s *Store) FindAggregate(dataType interface{}, query *Query, groupBy ...string) ([]*AggregateResult, error)

FindAggregate returns an aggregate grouping for the passed in query groupBy is optional

func (*Store) FindInBucket 

func (s *Store) FindInBucket(parent *bolt.Bucket, result interface{}, query *Query) error

FindInBucket allows you to specify a parent bucke to search in

func (*Store) FindOne 

func (s *Store) FindOne(result interface{}, query *Query) error

FindOne returns a single record, and so result is NOT a slice, but an pointer to a struct, if no record is found that matches the query, then it returns ErrNotFound

func (*Store) FindOneInBucket 

func (s *Store) FindOneInBucket(parent *bolt.Bucket, result interface{}, query *Query) error

FindOneInBucket allows you to pass in your own bucket to retrieve a single record from the bolthold

func (*Store) ForEach 

func (s *Store) ForEach(query *Query, fn interface{}) error

ForEach runs the function fn against every record that matches the query Useful for when working with large sets of data that you don't want to hold the entire result set in memory, similar to database cursors Return an error from fn, will stop the cursor from iterating

func (*Store) ForEachInBucket 

func (s *Store) ForEachInBucket(parent *bolt.Bucket, query *Query, fn interface{}) error

ForEachInBucket is the same as ForEach but you get to specify your parent bucket

func (*Store) Get 

func (s *Store) Get(key, result interface{}) error

Get retrieves a value from bolthold and puts it into result. Result must be a pointer

func (*Store) GetFromBucket 

func (s *Store) GetFromBucket(parent *bolt.Bucket, key, result interface{}) error

GetFromBucket allows you to specify the parent bucket for retrieving records

func (*Store) IndexExists 

func (s *Store) IndexExists(source BucketSource, typeName, indexName string) bool

IndexExists tests if an index exists for the passed in field name

func (*Store) Insert 

func (s *Store) Insert(key, data interface{}) error

Insert inserts the passed in data into the the bolthold

If the the key already exists in the bolthold, then an ErrKeyExists is returned If the data struct has a field tagged as `boltholdKey` and it is the same type as the Insert key, AND the data struct is passed by reference, AND the key field is currently set to the zero-value for that type, then that field will be set to the value of the insert key.

To use this with bolthold.NextSequence() use a type of `uint64` for the key field.

func (*Store) InsertIntoBucket 

func (s *Store) InsertIntoBucket(parent *bolt.Bucket, key, data interface{}) error

InsertIntoBucket is the same as Insert except it allows you specify your own parent bucket

func (*Store) ReIndex 

func (s *Store) ReIndex(exampleType interface{}, bucketName []byte) error

ReIndex removes any existing indexes and adds all the indexes defined by the passed in datatype example This function allows you to index an already existing boltDB file, or refresh any missing indexes if bucketName is nil, then we'll assume a bucketName of storer.Type() if a bucketname is specified, then the data will be copied to the bolthold standard bucket of storer.Type()

func (*Store) RemoveIndex 

func (s *Store) RemoveIndex(dataType interface{}, indexName string) error

RemoveIndex removes an index from the store.

func (*Store) TxCount 

func (s *Store) TxCount(tx *bolt.Tx, dataType interface{}, query *Query) (int, error)

TxCount returns the current record count from within the given transaction for the passed in datatype

func (*Store) TxDelete 

func (s *Store) TxDelete(tx *bolt.Tx, key, dataType interface{}) error

TxDelete is the same as Delete except it allows you specify your own transaction

func (*Store) TxDeleteMatching 

func (s *Store) TxDeleteMatching(tx *bolt.Tx, dataType interface{}, query *Query) error

TxDeleteMatching does the same as DeleteMatching, but allows you to specify your own transaction

func (*Store) TxFind 

func (s *Store) TxFind(tx *bolt.Tx, result interface{}, query *Query) error

TxFind allows you to pass in your own bolt transaction to retrieve a set of values from the bolthold

func (*Store) TxFindAggregate 

func (s *Store) TxFindAggregate(tx *bolt.Tx, dataType interface{}, query *Query,
	groupBy ...string) ([]*AggregateResult, error)

TxFindAggregate is the same as FindAggregate, but you specify your own transaction groupBy is optional

func (*Store) TxFindOne 

func (s *Store) TxFindOne(tx *bolt.Tx, result interface{}, query *Query) error

TxFindOne allows you to pass in your own bolt transaction to retrieve a single record from the bolthold

func (*Store) TxForEach 

func (s *Store) TxForEach(tx *bolt.Tx, query *Query, fn interface{}) error

TxForEach is the same as ForEach but you get to specify your transaction

func (*Store) TxGet 

func (s *Store) TxGet(tx *bolt.Tx, key, result interface{}) error

TxGet allows you to pass in your own bolt transaction to retrieve a value from the bolthold and puts it into result

func (*Store) TxInsert 

func (s *Store) TxInsert(tx *bolt.Tx, key, data interface{}) error

TxInsert is the same as Insert except it allows you specify your own transaction

func (*Store) TxUpdate 

func (s *Store) TxUpdate(tx *bolt.Tx, key interface{}, data interface{}) error

TxUpdate is the same as Update except it allows you to specify your own transaction

func (*Store) TxUpdateMatching 

func (s *Store) TxUpdateMatching(tx *bolt.Tx, dataType interface{}, query *Query,
	update func(record interface{}) error) error

TxUpdateMatching does the same as UpdateMatching, but allows you to specify your own transaction

func (*Store) TxUpsert 

func (s *Store) TxUpsert(tx *bolt.Tx, key interface{}, data interface{}) error

TxUpsert is the same as Upsert except it allows you to specify your own transaction

func (*Store) Update 

func (s *Store) Update(key interface{}, data interface{}) error

Update updates an existing record in the bolthold if the Key doesn't already exist in the store, then it fails with ErrNotFound

func (*Store) UpdateBucket 

func (s *Store) UpdateBucket(parent *bolt.Bucket, key interface{}, data interface{}) error

UpdateBucket allows you to run an update against any parent bucket

func (*Store) UpdateMatching 

func (s *Store) UpdateMatching(dataType interface{}, query *Query, update func(record interface{}) error) error

UpdateMatching runs the update function for every record that match the passed in query Note that the type of record in the update func always has to be a pointer

func (*Store) UpdateMatchingInBucket 

func (s *Store) UpdateMatchingInBucket(parent *bolt.Bucket, dataType interface{}, query *Query,
	update func(record interface{}) error) error

UpdateMatchingInBucket does the same as UpdateMatching, but allows you to specify your own parent bucket

func (*Store) Upsert 

func (s *Store) Upsert(key interface{}, data interface{}) error

Upsert inserts the record into the bolthold if it doesn't exist. If it does already exist, then it updates the existing record

func (*Store) UpsertBucket 

func (s *Store) UpsertBucket(parent *bolt.Bucket, key interface{}, data interface{}) error

UpsertBucket allows you to run an upsert against any bucket parent

type Storer 

type Storer interface {
	Type() string                        // used as the boltdb bucket name
	Indexes() map[string]Index           // [indexname]indexFunc
	SliceIndexes() map[string]SliceIndex // [indexname]sliceIndexFunc
}

Storer is the Interface to implement to skip reflect calls on all data passed into the bolthold

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.