pkoracle

README

pkoracle package

This package defines the interface and implementations for segments bloom filter sets of flushcommon metacache.

BloomFilterSet

The basic implementation with segment statslog. A group of bloom filter to perdict whethe some pks exists in the segment.

LazyPkStats

A wrapper for lazy loading PkStats. The inner PkStats could be added async.

CANNOT be used for growing segment.

Documentation

Index

• type BloomFilterSet
  • func NewBloomFilterSet(historyEntries ...*storage.PkStatistics) *BloomFilterSet
  • func NewBloomFilterSetWithBatchSize(batchSize uint, historyEntries ...*storage.PkStatistics) *BloomFilterSet
  • func (bfs *BloomFilterSet) BatchPkExist(lc *storage.BatchLocationsCache) []bool
  • func (bfs *BloomFilterSet) BatchPkExistWithHits(lc *storage.BatchLocationsCache, hits []bool) []bool
  • func (bfs *BloomFilterSet) GetHistory() []*storage.PkStatistics
  • func (bfs *BloomFilterSet) PkExists(lc *storage.LocationsCache) bool
  • func (bfs *BloomFilterSet) Roll(newStats ...*storage.PrimaryKeyStats)
  • func (bfs *BloomFilterSet) UpdatePKRange(ids storage.FieldData) error
• type LazyPkStats
  • func NewLazyPkstats() *LazyPkStats
  • func (s *LazyPkStats) BatchPkExist(lc *storage.BatchLocationsCache) []bool
  • func (s *LazyPkStats) BatchPkExistWithHits(lc *storage.BatchLocationsCache, hits []bool) []bool
  • func (s *LazyPkStats) GetHistory() []*storage.PkStatistics
  • func (s *LazyPkStats) PkExists(lc *storage.LocationsCache) bool
  • func (s *LazyPkStats) Roll(newStats ...*storage.PrimaryKeyStats)
  • func (s *LazyPkStats) SetPkStats(pk PkStat)
  • func (s *LazyPkStats) UpdatePKRange(ids storage.FieldData) error
• type PkStat

Types

type BloomFilterSet

type BloomFilterSet struct {
	// contains filtered or unexported fields
}

BloomFilterSet is a struct with multiple `storage.PkStatstics`. it maintains bloom filter generated from segment primary keys. it may be updated with new insert FieldData when serving growing segments.

func NewBloomFilterSet

func NewBloomFilterSet(historyEntries ...*storage.PkStatistics) *BloomFilterSet

NewBloomFilterSet returns a BloomFilterSet with provided historyEntries. Shall serve Flushed segments only. For growing segments, use `NewBloomFilterSetWithBatchSize` instead.

func NewBloomFilterSetWithBatchSize

func NewBloomFilterSetWithBatchSize(batchSize uint, historyEntries ...*storage.PkStatistics) *BloomFilterSet

NewBloomFilterSetWithBatchSize returns a BloomFilterSet. The batchSize parameter is used to initialize new bloom filter. It shall be the estimated row count per batch for segment to sync with.

func (*BloomFilterSet) BatchPkExist

func (bfs *BloomFilterSet) BatchPkExist(lc *storage.BatchLocationsCache) []bool

func (*BloomFilterSet) BatchPkExistWithHits

func (bfs *BloomFilterSet) BatchPkExistWithHits(lc *storage.BatchLocationsCache, hits []bool) []bool

func (*BloomFilterSet) GetHistory

func (bfs *BloomFilterSet) GetHistory() []*storage.PkStatistics

func (*BloomFilterSet) PkExists

func (bfs *BloomFilterSet) PkExists(lc *storage.LocationsCache) bool

func (*BloomFilterSet) Roll

func (bfs *BloomFilterSet) Roll(newStats ...*storage.PrimaryKeyStats)

func (*BloomFilterSet) UpdatePKRange

func (bfs *BloomFilterSet) UpdatePKRange(ids storage.FieldData) error

type LazyPkStats

type LazyPkStats struct {
	// contains filtered or unexported fields
}

LazyPkStats

func NewLazyPkstats

func NewLazyPkstats() *LazyPkStats

func (*LazyPkStats) BatchPkExist

func (s *LazyPkStats) BatchPkExist(lc *storage.BatchLocationsCache) []bool

func (*LazyPkStats) BatchPkExistWithHits

func (s *LazyPkStats) BatchPkExistWithHits(lc *storage.BatchLocationsCache, hits []bool) []bool

func (*LazyPkStats) GetHistory

func (s *LazyPkStats) GetHistory() []*storage.PkStatistics

func (*LazyPkStats) PkExists

func (s *LazyPkStats) PkExists(lc *storage.LocationsCache) bool

func (*LazyPkStats) Roll

func (s *LazyPkStats) Roll(newStats ...*storage.PrimaryKeyStats)

func (*LazyPkStats) SetPkStats

func (s *LazyPkStats) SetPkStats(pk PkStat)

func (*LazyPkStats) UpdatePKRange

func (s *LazyPkStats) UpdatePKRange(ids storage.FieldData) error

type PkStat

type PkStat interface {
	PkExists(lc *storage.LocationsCache) bool
	BatchPkExist(lc *storage.BatchLocationsCache) []bool
	BatchPkExistWithHits(lc *storage.BatchLocationsCache, hits []bool) []bool
	UpdatePKRange(ids storage.FieldData) error
	Roll(newStats ...*storage.PrimaryKeyStats)
	GetHistory() []*storage.PkStatistics
}

PK stat is the interface for pk statistics.