btrdb

README

BTrDB

The Berkeley TRee DataBase is a high performance time series database designed to support high density data storage applications.

We are now doing binary and container releases with (mostly) standard semantic versioning. The only variation on this is that we will use odd-numbered minor version numbers to indicate and unstable/development release series. Therefore the meanings of the version numbers are:

• Major: an increase in major version number indicates that there is no backwards compatibility with existing databases. To upgrade, we recommend using the migration tool.
• Minor: minor versions are compatible on-disk, but may have an incompatible network API. Therefore while it is safe to upgrade to a new minor version number, you may need to upgrade other programs that connect to BTrDB too. Furthermore, odd-numbered minor version numbers should be considered unstable and for development use only, patch releases within an odd numbered minor version number may not be compatible with eachother.
• Patch: patch releases on an odd numbered minor version number are not necessarily compatible with eachother in any way. Patch releases on an even minor version number are guaranteed to be compatible both in the disk format and in network API.

While using odd-numbered versions to indicate development releases is a somewhat archaic practice, it allows us to use our production release system for development, which reduces the odds that there is a discrepancy between the well-tested development binaries/containers and the subsequently released production version. Note that we will flag all development releases as "pre-release" on github.

The main distribution of BTrDB v4 is smartgridstore which is basically BTrDB packaged for deployment on Kubernetes, along with some (optional) utilities for working with synchrophasors. You can follow the installation guide on https://docs.smartgrid.store

If you are interested in deploying BTrDB, please file an issue on this repository so we can get an idea for what type of deployments we should focus on supporting. Smartgrid.store is focused on very large city or country-scale data aggregation from smart grid devices, but BTrDB can support other use cases.

Documentation

Index

• Constants
• type ChangedRange
• type Quasar
  • func NewQuasar(cfg configprovider.Configuration) (*Quasar, error)
  • func (q *Quasar) CreateStream(ctx context.Context, uuid []byte, collection string, tags map[string]string, ...) bte.BTE
  • func (q *Quasar) DeleteRange(ctx context.Context, id uuid.UUID, start int64, end int64) bte.BTE
  • func (q *Quasar) Flush(ctx context.Context, id uuid.UUID) bte.BTE
  • func (q *Quasar) GetClusterConfiguration() configprovider.ClusterConfiguration
  • func (q *Quasar) GetStreamDescriptor(ctx context.Context, uuid []byte) (res *mprovider.LookupResult, err bte.BTE)
  • func (q *Quasar) GetStreamVersion(ctx context.Context, uuid []byte) (ver uint64, err bte.BTE)
  • func (q *Quasar) InitiateShutdown() chan struct{}
  • func (q *Quasar) InsertValues(ctx context.Context, id uuid.UUID, r []qtree.Record) bte.BTE
  • func (q *Quasar) ListCollections(ctx context.Context, prefix string, startingFrom string, limit uint64) ([]string, bte.BTE)
  • func (q *Quasar) LookupStreams(ctx context.Context, collection string, isCollectionPrefix bool, ...) (chan *mprovider.LookupResult, chan bte.BTE)
  • func (q *Quasar) ObliterateStream(ctx context.Context, uuid []byte) bte.BTE
  • func (q *Quasar) QueryChangedRanges(ctx context.Context, id uuid.UUID, startgen uint64, endgen uint64, ...) (chan ChangedRange, chan bte.BTE, uint64)
  • func (q *Quasar) QueryGeneration(ctx context.Context, id uuid.UUID) (uint64, bte.BTE)
  • func (q *Quasar) QueryNearestValue(ctx context.Context, id uuid.UUID, time int64, backwards bool, gen uint64) (qtree.Record, bte.BTE, uint64)
  • func (q *Quasar) QueryStatisticalValuesStream(ctx context.Context, id uuid.UUID, start int64, end int64, gen uint64, ...) (chan qtree.StatRecord, chan bte.BTE, uint64)
  • func (q *Quasar) QueryValuesStream(ctx context.Context, id uuid.UUID, start int64, end int64, gen uint64) (chan qtree.Record, chan bte.BTE, uint64)
  • func (q *Quasar) QueryWindow(ctx context.Context, id uuid.UUID, start int64, end int64, gen uint64, ...) (chan qtree.StatRecord, chan bte.BTE, uint64)
  • func (q *Quasar) Rez() *rez.RezManager
  • func (q *Quasar) SetStreamAnnotations(ctx context.Context, uuid []byte, aver uint64, changes map[string]*string) bte.BTE
  • func (q *Quasar) StorageProvider() bprovider.StorageProvider

Constants

const LatestGeneration = bstore.LatestGeneration

const MaximumTime = (48 << 56)

const MinimumTime = -(16 << 56)

Types

type ChangedRange

type ChangedRange struct {
	Start int64
	End   int64
}

type Quasar

type Quasar struct {
	// contains filtered or unexported fields
}

func NewQuasar

func NewQuasar(cfg configprovider.Configuration) (*Quasar, error)

func (*Quasar) CreateStream

func (q *Quasar) CreateStream(ctx context.Context, uuid []byte, collection string, tags map[string]string, annotations map[string]string) bte.BTE

CreateStream makes a stream with the given uuid, collection and tags. Returns an error if the uuid already exists.

func (*Quasar) DeleteRange

func (q *Quasar) DeleteRange(ctx context.Context, id uuid.UUID, start int64, end int64) bte.BTE

func (*Quasar) Flush

func (q *Quasar) Flush(ctx context.Context, id uuid.UUID) bte.BTE

func (*Quasar) GetClusterConfiguration

func (q *Quasar) GetClusterConfiguration() configprovider.ClusterConfiguration

func (*Quasar) GetStreamDescriptor

func (q *Quasar) GetStreamDescriptor(ctx context.Context, uuid []byte) (res *mprovider.LookupResult, err bte.BTE)

Get a stream annotations and tags

func (*Quasar) GetStreamVersion

func (q *Quasar) GetStreamVersion(ctx context.Context, uuid []byte) (ver uint64, err bte.BTE)

Get a stream annotations and tags

func (*Quasar) InitiateShutdown

func (q *Quasar) InitiateShutdown() chan struct{}

func (*Quasar) InsertValues

func (q *Quasar) InsertValues(ctx context.Context, id uuid.UUID, r []qtree.Record) bte.BTE

func (*Quasar) ListCollections

func (q *Quasar) ListCollections(ctx context.Context, prefix string, startingFrom string, limit uint64) ([]string, bte.BTE)

ListCollections returns a list of collections beginning with prefix (which may be "") and starting from the given string. If number is > 0, only that many results will be returned. More can be obtained by re-calling ListCollections with a given startingFrom and number.

func (*Quasar) LookupStreams

func (q *Quasar) LookupStreams(ctx context.Context, collection string, isCollectionPrefix bool, tags map[string]*string, annotations map[string]*string) (chan *mprovider.LookupResult, chan bte.BTE)

Return back all streams in all collections beginning with collection (or exactly equal if prefix is false) provided they have the given tags and annotations, where a nil entry in the map means has the tag but the value is irrelevant

func (*Quasar) ObliterateStream

func (q *Quasar) ObliterateStream(ctx context.Context, uuid []byte) bte.BTE

DeleteStream tombstones a stream

func (*Quasar) QueryChangedRanges

func (q *Quasar) QueryChangedRanges(ctx context.Context, id uuid.UUID, startgen uint64, endgen uint64, resolution uint8) (chan ChangedRange, chan bte.BTE, uint64)

Resolution is how far down the tree to go when working out which blocks have changed. Higher resolutions are faster but will give you back coarser results.

func (*Quasar) QueryGeneration

func (q *Quasar) QueryGeneration(ctx context.Context, id uuid.UUID) (uint64, bte.BTE)

func (*Quasar) QueryNearestValue

func (q *Quasar) QueryNearestValue(ctx context.Context, id uuid.UUID, time int64, backwards bool, gen uint64) (qtree.Record, bte.BTE, uint64)

func (*Quasar) QueryStatisticalValuesStream

func (q *Quasar) QueryStatisticalValuesStream(ctx context.Context, id uuid.UUID, start int64, end int64,
	gen uint64, pointwidth uint8) (chan qtree.StatRecord, chan bte.BTE, uint64)

func (*Quasar) QueryValuesStream

func (q *Quasar) QueryValuesStream(ctx context.Context, id uuid.UUID, start int64, end int64, gen uint64) (chan qtree.Record, chan bte.BTE, uint64)

func (*Quasar) QueryWindow

func (q *Quasar) QueryWindow(ctx context.Context, id uuid.UUID, start int64, end int64,
	gen uint64, width uint64, depth uint8) (chan qtree.StatRecord, chan bte.BTE, uint64)

func (*Quasar) Rez

func (q *Quasar) Rez() *rez.RezManager

func (*Quasar) SetStreamAnnotations

func (q *Quasar) SetStreamAnnotations(ctx context.Context, uuid []byte, aver uint64, changes map[string]*string) bte.BTE

Sets the stream annotations. An entry with a nil string implies delete

func (*Quasar) StorageProvider

func (q *Quasar) StorageProvider() bprovider.StorageProvider