fuzzyfilter

Documentation

Overview

Copyright 2023 Matrix Origin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

• Constants
• func IfCanUseRoaringFilter(t types.T) bool
• type Argument
  • func NewArgument() *Argument
  • func (arg *Argument) Call(proc *process.Process) (vm.CallResult, error)
  • func (arg *Argument) Free(proc *process.Process, pipelineFailed bool, err error)
  • func (arg *Argument) GetOperatorBase() *vm.OperatorBase
  • func (arg *Argument) Prepare(proc *process.Process) (err error)
  • func (arg *Argument) Release()
  • func (arg *Argument) Reset(proc *process.Process, pipelineFailed bool, err error)
  • func (arg *Argument) String(buf *bytes.Buffer)
  • func (arg Argument) TypeName() string

Constants

const (
	Build = iota
	HandleRuntimeFilter
	Probe
	End
)

Functions

func IfCanUseRoaringFilter

func IfCanUseRoaringFilter(t types.T) bool

Types

type Argument

type Argument struct {

	// Estimates of the number of data items obtained from statistical information
	N                  float64
	PkName             string
	PkTyp              plan.Type
	BuildIdx           int
	IfInsertFromUnique bool

	RuntimeFilterSpec *plan.RuntimeFilterSpec
	vm.OperatorBase
	// contains filtered or unexported fields
}

func NewArgument

func NewArgument() *Argument

func (*Argument) Call

func (arg *Argument) Call(proc *process.Process) (vm.CallResult, error)

opt3 : As mentioned before, you should think of fuzzy as a special kind of join, which also has a Build phase and a Probe phase.

The previous pseudo-code has no problem with correctness, but the memory overhead in some scenarios can be significant, especially when the sink scan has much LARGER data than the table scan. Therefore, build stage also needs to be built on smaller children.

# Flow of optimized pseudo-code if Stats(Table Scan) > Stats(Sink Scan)

Build on Sink scan
	Test and Add
	-> can be optimized to Add if the sinkScan data can guarantee uniqueness
Probe on Table scan
	Test

else

Build on Table scan
	Add
Probe on Sink scan
	Test and Add
	-> can be optimized to Test if the sinkScan data can guarantee uniqueness

func (*Argument) Free

func (arg *Argument) Free(proc *process.Process, pipelineFailed bool, err error)

func (*Argument) GetOperatorBase

func (arg *Argument) GetOperatorBase() *vm.OperatorBase

func (*Argument) Prepare

func (arg *Argument) Prepare(proc *process.Process) (err error)

func (*Argument) Release

func (arg *Argument) Release()

func (*Argument) Reset

func (arg *Argument) Reset(proc *process.Process, pipelineFailed bool, err error)

func (*Argument) String

func (arg *Argument) String(buf *bytes.Buffer)

func (Argument) TypeName

func (arg Argument) TypeName() string