Documentation
¶
Overview ¶
Package lmdbscan provides a wrapper for lmdb.Cursor to simplify iteration. This package is experimental and it's API may change.
Example ¶
This (mildly contrived) example demonstrates how the features of lmdbscan combine to effectively query a database. In the example time series data is being filtered. The timestamp of each series entry is encoded in the database key, prefixed with the bytes "data:". This information is used more efficiently filter keys.
package main
import (
"bytes"
"encoding/binary"
"log"
"time"
"github.com/bmatsuo/lmdb-go/exp/lmdbscan"
"github.com/bmatsuo/lmdb-go/lmdb"
)
var env *lmdb.Env
func main() {
prefix := []byte("data:")
cutoff := time.Now().Add(-time.Minute)
err := env.View(func(txn *lmdb.Txn) (err error) {
dbroot, _ := txn.OpenRoot(0)
scanner := lmdbscan.New(txn, dbroot)
defer scanner.Close()
scanner.SetNext(prefix, nil, lmdb.SetRange, lmdb.Next)
hasPrefix := lmdbscan.While(func(k, v []byte) bool { return bytes.HasPrefix(k, prefix) })
isTimeSeries := lmdbscan.Select(func(k, v []byte) bool { return len(k)-len(prefix) == 8 })
notCutOff := lmdbscan.Select(func(k, v []byte) bool {
nsbytes := k[len(prefix):]
ns := binary.BigEndian.Uint64(nsbytes)
t := time.Unix(0, int64(ns))
return t.After(cutoff)
})
for scanner.Scan(hasPrefix, isTimeSeries, notCutOff) {
log.Print(scanner.Val())
// ... process the series entry
}
return scanner.Err()
})
if err != nil {
panic(err)
}
}
Output:
Index ¶
- Variables
- func Each(txn *lmdb.Txn, dbi lmdb.DBI, fn Func) error
- func EachReverse(txn *lmdb.Txn, dbi lmdb.DBI, fn Func) error
- type Func
- type Scanner
- func (s *Scanner) Close()
- func (s *Scanner) Del(flags uint) error
- func (s *Scanner) Err() error
- func (s *Scanner) Key() []byte
- func (s *Scanner) Scan(filter ...Func) bool
- func (s *Scanner) Set(k, v []byte, op uint)
- func (s *Scanner) SetNext(k, v []byte, opset, opnext uint)
- func (s *Scanner) Val() []byte
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var Skip = fmt.Errorf("skip")
Skip is returned by a Func to signal that Scanner.Scan should not yield the current (k, v) pair.
var Stop = fmt.Errorf("stop")
Stop is returned by a Func to signal that Scanner.Scan should terminate and return false.
Functions ¶
Types ¶
type Func ¶
A Func is used to control iteration of (k, v) pairs in a database.
func Select ¶
Select returns a Func that returns nil for all (k, v) pairs that fn(k,v) returns true and Stop otherwise.
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner is a low level construct for scanning databases inside a transaction.
Example ¶
This example demonstrates basic usage of a Scanner to scan the root database. It is important to always call scanner.Err() which will returned any unexpected error which interrupted scanner.Scan().
package main
import (
"log"
"github.com/bmatsuo/lmdb-go/exp/lmdbscan"
"github.com/bmatsuo/lmdb-go/lmdb"
)
var env *lmdb.Env
func main() {
err := env.View(func(txn *lmdb.Txn) (err error) {
dbroot, _ := txn.OpenRoot(0)
scanner := lmdbscan.New(txn, dbroot)
defer scanner.Close()
for scanner.Scan() {
log.Printf("k=%q v=%q", scanner.Key(), scanner.Val())
}
return scanner.Err()
})
if err != nil {
panic(err)
}
}
Output:
func (*Scanner) Close ¶
func (s *Scanner) Close()
Close clears internal structures. Close does not attempt to terminate the enclosing transaction.
Scan must not be called after Close.
func (*Scanner) Err ¶
Err returns a non-nil error if and only if the previous call to s.Scan() resulted in an error other than Stop or lmdb.ErrNotFound.
func (*Scanner) Scan ¶
Scan gets key-value successive pairs with the underlying cursor until one matches the supplied filters. If all filters return a nil error for the current pair, true is returned. Scan automatically gets the next pair if any filter returns Skip. Scan returns false if all key-value pairs where exhausted or another non-nil error was returned by a filter.
Example ¶
This simple example shows how to iterate over a database that indexes json document.
package main
import (
"log"
"github.com/bmatsuo/lmdb-go/exp/lmdbscan"
"github.com/bmatsuo/lmdb-go/lmdb"
)
var env *lmdb.Env
func main() {
err := env.View(func(txn *lmdb.Txn) (err error) {
dbroot, _ := txn.OpenRoot(0)
scanner := lmdbscan.New(txn, dbroot)
defer scanner.Close()
for scanner.Scan(nil) {
log.Printf("%q=%q", scanner.Key(), scanner.Val())
}
return scanner.Err()
})
if err != nil {
panic(err)
}
}
Output:
func (*Scanner) Set ¶
Set marks the starting position for iteration. On the next call to s.Scan() the underlying cursor will be moved as
c.Get(k, v, op)
Example ¶
This example demonstrates scanning a key range in the root database. Set is used to move the cursor's starting position to the desired prefix.
package main
import (
"bytes"
"log"
"github.com/bmatsuo/lmdb-go/exp/lmdbscan"
"github.com/bmatsuo/lmdb-go/lmdb"
)
var env *lmdb.Env
func main() {
keyprefix := []byte("users:")
err := env.View(func(txn *lmdb.Txn) (err error) {
dbroot, _ := txn.OpenRoot(0)
scanner := lmdbscan.New(txn, dbroot)
defer scanner.Close()
scanner.Set(keyprefix, nil, lmdb.SetRange)
for scanner.Scan(nil) {
if !bytes.HasPrefix(scanner.Key(), keyprefix) {
break
}
log.Printf("k=%q v=%q", scanner.Key(), scanner.Val())
}
return scanner.Err()
})
if err != nil {
panic(err)
}
}
Output:
func (*Scanner) SetNext ¶
Set determines the cursor behavior for subsequent calls to s.Scan(). The immediately following call to s.Scan() behaves as if s.Set(k,v,opset) was called. Subsequent calls move the cursor as
c.Get(nil, nil, opnext)
Example ¶
This example demonstrates scanning all values for a key in a root database with the lmdb.DupSort flag set. SetNext is used instead of Set to configure Cursor the to return ErrNotFound (EOF) after all duplicate keys have been iterated.
package main
import (
"log"
"github.com/bmatsuo/lmdb-go/exp/lmdbscan"
"github.com/bmatsuo/lmdb-go/lmdb"
)
var env *lmdb.Env
func main() {
key := []byte("userphone:123")
err := env.View(func(txn *lmdb.Txn) (err error) {
dbroot, _ := txn.OpenRoot(0)
scanner := lmdbscan.New(txn, dbroot)
defer scanner.Close()
scanner.SetNext(key, nil, lmdb.GetBothRange, lmdb.NextDup)
for scanner.Scan() {
log.Printf("k=%q v=%q", scanner.Key(), scanner.Val())
}
return scanner.Err()
})
if err != nil {
panic(err)
}
}
Output:
Source Files