Documentation
¶
Overview ¶
Package roaring implements roaring bitmaps with support for incremental changes.
Index ¶
- Constants
- Variables
- type Bitmap
- func (b *Bitmap) Add(a ...uint64) (changed bool, err error)
- func (b *Bitmap) Check() error
- func (b *Bitmap) Clone() *Bitmap
- func (b *Bitmap) Contains(v uint64) bool
- func (b *Bitmap) Count() (n uint64)
- func (b *Bitmap) CountRange(start, end uint64) (n uint64)
- func (b *Bitmap) Difference(other *Bitmap) *Bitmap
- func (b *Bitmap) DirectAdd(v uint64) bool
- func (b *Bitmap) Flip(start, end uint64) *Bitmap
- func (b *Bitmap) ForEach(fn func(uint64))
- func (b *Bitmap) ForEachRange(start, end uint64, fn func(uint64))
- func (b *Bitmap) Info() bitmapInfo
- func (b *Bitmap) Intersect(other *Bitmap) *Bitmap
- func (b *Bitmap) IntersectionCount(other *Bitmap) uint64
- func (b *Bitmap) Iterator() *Iterator
- func (b *Bitmap) Max() uint64
- func (b *Bitmap) OffsetRange(offset, start, end uint64) *Bitmap
- func (b *Bitmap) Optimize()
- func (b *Bitmap) Remove(a ...uint64) (changed bool, err error)
- func (b *Bitmap) Reset()
- func (b *Bitmap) Slice() []uint64
- func (b *Bitmap) SliceRange(start, end uint64) []uint64
- func (b *Bitmap) Union(others ...*Bitmap) *Bitmap
- func (b *Bitmap) UnionInPlace(others ...*Bitmap)
- func (b *Bitmap) UnmarshalBinary(data []byte) error
- func (b *Bitmap) WriteTo(w io.Writer) (n int64, err error)
- func (b *Bitmap) Xor(other *Bitmap) *Bitmap
- type Container
- func (c *Container) Clone() *Container
- func (c *Container) Contains(v uint16) bool
- func (c *Container) Mapped() bool
- func (c *Container) N() int32
- func (c *Container) Repair()
- func (c *Container) Reset()
- func (c *Container) Update(containerType byte, n int32, mapped bool)
- func (c *Container) WriteTo(w io.Writer) (n int64, err error)
- type ContainerIterator
- type ContainerPoolingConfiguration
- type Containers
- type ErrorList
- type Iterator
Constants ¶
const ArrayMaxSize = 4096
ArrayMaxSize represents the maximum size of array containers.
Variables ¶
var NewFileBitmap func(a ...uint64) *Bitmap = NewBitmap
NewFileBitmap returns a Bitmap with an initial set of values, used for file storage. By default, this is a copy of NewBitmap, but is replaced with B+Tree in server/enterprise.go
Functions ¶
This section is empty.
Types ¶
type Bitmap ¶
type Bitmap struct {
Containers Containers
// Writer where operations are appended to.
OpWriter io.Writer
// contains filtered or unexported fields
}
Bitmap represents a roaring bitmap.
func NewBitmapWithDefaultPooling ¶
NewBitmapWithDefaultPooling returns a new bitmap with the default pooling configuration. See the comment for NewBitmapWithPooling for more details about the pooling implementation.
func NewBitmapWithPooling ¶
func NewBitmapWithPooling(pooling ContainerPoolingConfiguration, a ...uint64) *Bitmap
NewBitmapWithPooling returns a new Bitmap with the provided container pooling configuration and initial set of values.
Container Pooling is useful for reusing short lived Bitmaps (common in the situation where temporary bitmaps are being created for the sake of computation instead of storage). In that case, allocating new containers over and over again is unecessarily expensive. Instead, when you need an empty bitmap, you can call the Reset() method on an existing one. That will clear all the data it contains and return its containers (up to the configured maximum) to its pool so that when you start adding new data, the already allocated containers can be reused.
In exchange for reduced memory pressure / allocations, bitmaps with pooling enabled will use significantly more memory. This is for two reasons:
- Even when there is no data in the bitmap, a configurable number of containers have already been pre-allocated and are waiting in reserve.
- Every container that is allocated when pooling is enabled is pre-allocated such that it can seamlessly switch between a run, array, or a bitmap with zero allocations. This means it can be used for performing calculations very quickly and without causing G.C pressure, but it will use much more space.
func (*Bitmap) Clone ¶
Clone returns a heap allocated copy of the bitmap. Note: The OpWriter IS NOT copied to the new bitmap.
func (*Bitmap) CountRange ¶
CountRange returns the number of bits set between [start, end).
func (*Bitmap) Difference ¶
Difference returns the difference of b and other.
func (*Bitmap) ForEachRange ¶
ForEachRange executes fn for each value in the bitmap between [start, end).
func (*Bitmap) IntersectionCount ¶
IntersectionCount returns the number of set bits that would result in an intersection between b and other. It is more efficient than actually intersecting the two and counting the result.
func (*Bitmap) Max ¶
Max returns the highest value in the bitmap. Returns zero if the bitmap is empty.
func (*Bitmap) OffsetRange ¶
OffsetRange returns a new bitmap with a containers offset by start.
func (*Bitmap) Optimize ¶
func (b *Bitmap) Optimize()
Optimize converts array and bitmap containers to run containers as necessary.
func (*Bitmap) Reset ¶
func (b *Bitmap) Reset()
Reset reset the bitmap and the underlying containers for re-use.
func (*Bitmap) SliceRange ¶
SliceRange returns a slice of integers between [start, end).
func (*Bitmap) UnionInPlace ¶
UnionInPlace returns the bitwise union of b and others, modifying b in place.
func (*Bitmap) UnmarshalBinary ¶
UnmarshalBinary decodes b from a binary-encoded byte slice. data can be in either official roaring format or Pilosa's roaring format.
type Container ¶
type Container struct {
// contains filtered or unexported fields
}
Container represents a Container for uint16 integers.
These are used for storing the low bits of numbers in larger sets of uint64. The high bits are stored in a Container's key which is tracked by a separate data structure. Integers in a Container can be encoded in one of three ways - the encoding used is usually whichever is most compact, though any Container type should be able to encode any set of integers safely. For containers with less than 4,096 values, an array is often used. Containers with long runs of integers would use run length encoding, and more random data usually uses bitmap encoding.
func NewContainer ¶
func NewContainer() *Container
newContainer returns a new instance of container.
func NewContainerWithPooling ¶
func NewContainerWithPooling(poolingConfig ContainerPoolingConfiguration) *Container
NewContainerWithPooling creates a new container with the provided pooling configuration.
func (*Container) Repair ¶
func (c *Container) Repair()
Repair repairs the cardinality of c if it has been corrupted by optimized operations.
func (*Container) Reset ¶
func (c *Container) Reset()
Reset the container so it can be reused while maintaining any allocated datastructures.
type ContainerIterator ¶
type ContainerPoolingConfiguration ¶
type ContainerPoolingConfiguration struct {
// Maximum size of the allocated array that will be maintained in the pool.
MaxArraySize int
// Whether a bitmap should be allocated for each pooled container.
AllocateBitmap bool
// Maximum size of the allocated runs that will be maintained in the pool.
MaxRunsSize int
// Maximum number of containers to pool.
MaxCapacity int
// Maximum size of keys and containers slice to maintain after calls to Reset().
MaxKeysAndContainersSliceLength int
}
ContainerPoolingConfiguration represents the configuration for container pooling.
func NewDefaultContainerPoolingConfiguration ¶
func NewDefaultContainerPoolingConfiguration(maxCapacity int) ContainerPoolingConfiguration
NewDefaultContainerPoolingConfiguration creates a ContainerPoolingConfiguration with default configuration.
type Containers ¶
type Containers interface {
// Get returns nil if the key does not exist.
Get(key uint64) *Container
// Put adds the container at key.
Put(key uint64, c *Container)
// PutContainerValues updates an existing container at key.
// If a container does not exist for key, a new one is allocated.
// TODO(2.0) make n int32
PutContainerValues(key uint64, containerType byte, n int, mapped bool)
// Remove takes the container at key out.
Remove(key uint64)
// GetOrCreate returns the container at key, creating a new empty container if necessary.
GetOrCreate(key uint64) *Container
// Clone does a deep copy of Containers, including cloning all containers contained.
Clone() Containers
// Last returns the highest key and associated container.
Last() (key uint64, c *Container)
// Size returns the number of containers stored.
Size() int
// Iterator returns a Contiterator which after a call to Next(), a call to Value() will
// return the first container at or after key. found will be true if a
// container is found at key.
Iterator(key uint64) (citer ContainerIterator, found bool)
Count() uint64
// Reset clears the containers collection to allow for recycling during snapshot
Reset()
// Repair will repair the cardinality of any containers whose cardinality were corrupted
// due to optimized operations.
Repair()
}
type ErrorList ¶
type ErrorList []error
ErrorList represents a list of errors.
func (*ErrorList) Append ¶
Append appends an error to the list. If err is an ErrorList then all errors are appended.
func (*ErrorList) AppendWithPrefix ¶
AppendWithPrefix appends an error to the list and includes a prefix.
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator represents an iterator over a Bitmap.
func (*Iterator) Next ¶
Next returns the next value in the bitmap. Returns eof as true if there are no values left in the iterator.
Source Files