bloombits

Documentation

Overview

Package bloombits implements bloom filtering on batches of data.

Index

• type Generator
  • func NewGenerator(sections uint) (*Generator, error)
  • func (b *Generator) AddBloom(index uint, bloom types.Bloom) error
  • func (b *Generator) Bitset(idx uint) ([]byte, error)
• type Matcher
  • func NewMatcher(sectionSize uint64, filters [][][]byte) *Matcher
  • func (m *Matcher) Start(ctx context.Context, begin, end uint64, results chan uint64) (*MatcherSession, error)
• type MatcherSession
  • func (s *MatcherSession) Close()
  • func (s *MatcherSession) Error() error
  • func (s *MatcherSession) Multiplex(batch int, wait time.Duration, mux chan chan *Retrieval)
• type Retrieval

Types

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

Generator takes a number of bloom filters and generates the rotated bloom bits to be used for batched filtering. 用来聚合一组布隆过滤器,可以快速查询这一组过滤器的指定位的结果

func NewGenerator

func NewGenerator(sections uint) (*Generator, error)

NewGenerator creates a rotated bloom generator that can iteratively fill a batched bloom filter's bits. 生成Generator对象,传入容量,容量必须是8的倍数

func (*Generator) AddBloom

func (b *Generator) AddBloom(index uint, bloom types.Bloom) error

AddBloom takes a single bloom filter and sets the corresponding bit column in memory accordingly. index代表要添加第几个布隆过滤器

func (*Generator) Bitset

func (b *Generator) Bitset(idx uint) ([]byte, error)

Bitset returns the bit vector belonging to the given bit index after all blooms have been added. 查询前必须将生成器填充满,要查询的比特位一定是[0,2047] 返回结果第i位就代表第i个布隆过滤器第idx位的结果

type Matcher

type Matcher struct {
	// contains filtered or unexported fields
}

Matcher is a pipelined system of schedulers and logic matchers which perform binary AND/OR operations on the bit-streams, creating a stream of potential blocks to inspect for data content.

func NewMatcher

func NewMatcher(sectionSize uint64, filters [][][]byte) *Matcher

NewMatcher creates a new pipeline for retrieving bloom bit streams and doing address and topic filtering on them. Setting a filter component to `nil` is allowed and will result in that filter rule being skipped (OR 0x11...1). 设地址为ax,topic为tx, filters[i]称为filter filters的格式为 [ [a1,a2,a3], [t11,t12], [t21,t22,t23]] 只有filters[0]保存了地址列表,后面都是topic列表 满足filters的条件是每个filter中的值都至少被找到一个 输入filters生成一个Matcher

1. 将原始filters转换成三数字类型的filters
2. filters可以得到需要查询的比特位,每个比特位生成一个scheduler

func (*Matcher) Start

func (m *Matcher) Start(ctx context.Context, begin, end uint64, results chan uint64) (*MatcherSession, error)

Start starts the matching process and returns a stream of bloom matches in a given range of blocks. If there are no more matches in the range, the result channel is closed. 客户端启动一个查询过程，指定了开始和结束的区块号，从results中接收查询结果

type MatcherSession

type MatcherSession struct {
	// contains filtered or unexported fields
}

MatcherSession is returned by a started matcher to be used as a terminator for the actively running matching operation. 调用Matcher.Start后返回MatcherSession对象 可以使用MatcherSession.Close方法来终止查询过程

func (*MatcherSession) Close

func (s *MatcherSession) Close()

Close stops the matching process and waits for all subprocesses to terminate before returning. The timeout may be used for graceful shutdown, allowing the currently running retrievals to complete before this time. 关闭一个匹配器的匹配过程

func (*MatcherSession) Error

func (s *MatcherSession) Error() error

Error returns any failure encountered during the matching session. 返回匹配会话中发生的错误

func (*MatcherSession) Multiplex

func (s *MatcherSession) Multiplex(batch int, wait time.Duration, mux chan chan *Retrieval)

Multiplex polls the matcher session for retrieval tasks and multiplexes it into the requested retrieval queue to be serviced together with other sessions.

This method will block for the lifetime of the session. Even after termination of the session, any request in-flight need to be responded to! Empty responses are fine though in that case. batch代表一次发送请求的个数，wait代表如果个数不足batch等待的时间

type Retrieval

type Retrieval struct {
	// 要查询区块段中区块的布隆过滤器的哪个比特位
	Bit uint
	// 要查询哪些区块段
	Sections []uint64
	// 服务端查询完成后将多个位集结果保存在这里
	Bitsets [][]byte

	Context context.Context
	Error   error
}

Retrieval represents a request for retrieval task assignments for a given bit with the given number of fetch elements, or a response for such a request. It can also have the actual results set to be used as a delivery data struct.

The contest and error fields are used by the light client to terminate matching early if an error is encountered on some path of the pipeline. 代表需要服务端执行的一次查询任务