bitmapper

Documentation

Index

• Variables
• type Bitmapper
  • func NewBitmapper(ctx context.Context, cfg Config) (*Bitmapper, error)
  • func (b *Bitmapper) GetGroup(ctx context.Context, name string) (*Group, error)
• type Config
• type Entity
• type Group
  • func NewGroup(ctx context.Context, cfg GroupConfig) (*Group, error)
  • func (g *Group) GetEntity(ctx context.Context, uid uint32) (*Entity, error)
  • func (g *Group) GetShardSize() uint32
  • func (g *Group) GetStringID(ctx context.Context, uid uint32) (string, error)
  • func (g *Group) GetUID(ctx context.Context, stringID string, initOnMiss, flush bool) (uint32, bool, error)
  • func (g *Group) PeekNextUID() uint32
  • func (g *Group) UpdateEntity(ctx context.Context, uid uint32, ent *Entity) error
  • func (g *Group) UpdateUIDMapping(ctx context.Context, uid uint32, stringID string) error
• type GroupConfig

Variables

var ErrorUIDNotFound = fmt.Errorf("uid not found")

ErrorUIDNotFound is returned when a UID is not found

var MetaDBName = "meta.db"

MetaDBName is the filename of the meta database

var ShardDBPattern = "shard_%d.db"

ShardDBPattern is the pattern for the shard database filenames

Types

type Bitmapper

type Bitmapper struct {
	CrossGroupLk sync.RWMutex
	// contains filtered or unexported fields
}

Bitmapper is a struct that manages collections of Roaring Bitmaps Collections of bitmaps are organized into Groups that share UID mappings Each Group has its own SQLite database for persisting the UID to string ID mappings as well as a set of sharded SQLite databases for persisting the Roaring Bitmaps

func NewBitmapper

func NewBitmapper(ctx context.Context, cfg Config) (*Bitmapper, error)

NewBitmapper creates a new Bitmapper instance and initializes the groups

func (*Bitmapper) GetGroup

func (b *Bitmapper) GetGroup(ctx context.Context, name string) (*Group, error)

GetGroup retrieves a Group by name

type Config

type Config struct {
	// Root directory for SQLite files, one subfolder per group
	DBDir  string
	Groups []GroupConfig
}

Config is the configuration for a Bitmapper instance

type Entity

type Entity struct {
	LK *sync.RWMutex
	BM *roaring.Bitmap
}

Entity is a member of a Group It contains a Roaring Bitmap and a lock for concurrent access After modifying an Entitiy, use the Group's UpdateEntity method to persist the changes

type Group

type Group struct {
	// contains filtered or unexported fields
}

Group is a struct that manages a collection of Roaring Bitmaps Bitmaps are organized by UID, with mappings to string IDs Each Group has its own SQLite database for persisting the UID to string ID mappings as well as a set of sharded SQLite databases for persisting the Roaring Bitmaps Groups use an ARC cache for lazily loading Bitmaps from disk to keep memory usage manageable Writes to Entities in a Group are immediately persisted to disk

func NewGroup

func NewGroup(ctx context.Context, cfg GroupConfig) (*Group, error)

NewGroup creates a new Group instance

func (*Group) GetEntity

func (g *Group) GetEntity(ctx context.Context, uid uint32) (*Entity, error)

GetEntity retrieves an Entity from the cache or database

func (*Group) GetShardSize

func (g *Group) GetShardSize() uint32

GetShardSize returns the shard size for the group

func (*Group) GetStringID

func (g *Group) GetStringID(ctx context.Context, uid uint32) (string, error)

GetStringID resolves a UID to a string ID

func (*Group) GetUID

func (g *Group) GetUID(ctx context.Context, stringID string, initOnMiss, flush bool) (uint32, bool, error)

GetUID resolves a string ID to a UID If the UID is not found and initOnMiss is true, a new UID is initialized The bool return value indicates whether the UID was initialized

func (*Group) PeekNextUID

func (g *Group) PeekNextUID() uint32

PeekNextUID returns the next UID that will be assigned

func (*Group) UpdateEntity

func (g *Group) UpdateEntity(ctx context.Context, uid uint32, ent *Entity) error

UpdateEntity updates an Entity in the cache and database Only call while holding the Entity's lock

func (*Group) UpdateUIDMapping

func (g *Group) UpdateUIDMapping(ctx context.Context, uid uint32, stringID string) error

UpdateUIDMapping explicitly updates the UID to string ID mapping in the meta database This should only be called by an external process when bulk updating the mapping To initialize a new UID, use GetUID with initOnMiss=true (which will call this function)

type GroupConfig

type GroupConfig struct {
	Name      string
	ShardSize uint32
	CacheSize int
	// contains filtered or unexported fields
}

GroupConfig is the configuration for a Group instance