filter

package
v1.0.0-beta.79 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 3, 2023 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source
const (
	EQ       = "$eq"
	GT       = "$gt"
	LT       = "$lt"
	GTE      = "$gte"
	LTE      = "$lte"
	NOT      = "$not"
	REGEX    = "$regex"
	CONTAINS = "$contains"
)

Variables

This section is empty.

Functions

func IndexTypePrimary

func IndexTypePrimary(indexType IndexType) bool

func IndexTypeSecondary

func IndexTypeSecondary(indexType IndexType) bool

func MatcherForArray

func MatcherForArray(matcher ValueMatcher) bool

func None

func None(reqFilter []byte) bool

func PKBuildIndexPartsFunc

func PKBuildIndexPartsFunc(name string, value value.Value) []interface{}

func StringContains

func StringContains(s string, substr string, collation *value.Collation) bool

Types

type AndFilter

type AndFilter struct {
	// contains filtered or unexported fields
}

AndFilter performs a logical AND operation on an array of two or more expressions. The and filter looks like this, {"$and": [{"f1":1}, {"f2": 3}....]} It can be nested i.e. a top level $and can have multiple nested $and/$or.

func NewAndFilter

func NewAndFilter(filter []Filter) (*AndFilter, error)

func (*AndFilter) GetFilters

func (a *AndFilter) GetFilters() []Filter

GetFilters returns all the nested filters for AndFilter.

func (*AndFilter) IsSearchIndexed

func (a *AndFilter) IsSearchIndexed() bool

func (*AndFilter) Matches

func (a *AndFilter) Matches(doc []byte, metadata []byte) bool

Matches returns true if the input doc matches this filter.

func (*AndFilter) MatchesDoc

func (a *AndFilter) MatchesDoc(doc map[string]interface{}) bool

func (*AndFilter) String

func (a *AndFilter) String() string

String a helpful method for logging.

func (*AndFilter) ToSearchFilter

func (a *AndFilter) ToSearchFilter() string

func (*AndFilter) Type

func (a *AndFilter) Type() LogicalOP

type ArrayMatcher

type ArrayMatcher interface {
	ArrMatches(value []any) bool
}

type BuildIndexPartsFunc

type BuildIndexPartsFunc func(fieldName string, val value.Value) []interface{}

type ContainsMatcher

type ContainsMatcher struct {
	// contains filtered or unexported fields
}

ContainsMatcher implements "$contains" operand.

func (*ContainsMatcher) Matches

func (c *ContainsMatcher) Matches(docValue any) bool

func (*ContainsMatcher) String

func (c *ContainsMatcher) String() string

func (*ContainsMatcher) Type

func (c *ContainsMatcher) Type() string

type EmptyFilter

type EmptyFilter struct{}

func (*EmptyFilter) IsSearchIndexed

func (f *EmptyFilter) IsSearchIndexed() bool

func (*EmptyFilter) Matches

func (f *EmptyFilter) Matches(_ []byte, _ []byte) bool

func (*EmptyFilter) MatchesDoc

func (f *EmptyFilter) MatchesDoc(_ map[string]interface{}) bool

func (*EmptyFilter) ToSearchFilter

func (f *EmptyFilter) ToSearchFilter() string

type EqualityMatcher

type EqualityMatcher struct {
	Value value.Value
}

EqualityMatcher implements "$eq" operand.

func NewEqualityMatcher

func NewEqualityMatcher(v value.Value) *EqualityMatcher

NewEqualityMatcher returns EqualityMatcher object.

func (*EqualityMatcher) ArrMatches

func (e *EqualityMatcher) ArrMatches(arr []any) bool

func (*EqualityMatcher) GetValue

func (e *EqualityMatcher) GetValue() value.Value

func (*EqualityMatcher) Matches

func (e *EqualityMatcher) Matches(input value.Value) bool

func (*EqualityMatcher) String

func (e *EqualityMatcher) String() string

func (*EqualityMatcher) Type

func (e *EqualityMatcher) Type() string

type Factory

type Factory struct {
	// contains filtered or unexported fields
}

func NewFactory

func NewFactory(fields []*schema.QueryableField, collation *value.Collation) *Factory

func NewFactoryForSecondaryIndex

func NewFactoryForSecondaryIndex(fields []*schema.QueryableField) *Factory

func (*Factory) Factorize

func (factory *Factory) Factorize(reqFilter []byte) ([]Filter, error)

func (*Factory) ParseSelector

func (factory *Factory) ParseSelector(k []byte, v []byte, dataType jsonparser.ValueType) (Filter, error)

ParseSelector is a short-circuit for Selector i.e. when we know the filter passed is not logical then we directly call this because if it is not logical then it is simply a Selector filter.

func (*Factory) UnmarshalAnd

func (factory *Factory) UnmarshalAnd(input jsoniter.RawMessage) (Filter, error)

func (*Factory) UnmarshalFilter

func (factory *Factory) UnmarshalFilter(input jsoniter.RawMessage) (expression.Expr, error)

func (*Factory) UnmarshalOr

func (factory *Factory) UnmarshalOr(input jsoniter.RawMessage) (Filter, error)

func (*Factory) WrappedFilter

func (factory *Factory) WrappedFilter(reqFilter []byte) (*WrappedFilter, error)

type Filter

type Filter interface {
	// Matches returns true if the input doc passes the filter, otherwise false
	Matches(doc []byte, metadata []byte) bool
	// MatchesDoc similar to Matches but used when document is already parsed
	MatchesDoc(doc map[string]interface{}) bool
	ToSearchFilter() string
	// IsSearchIndexed to let caller knows if there is any fields in the query not indexed in search. This
	// will trigger full scan.
	IsSearchIndexed() bool
}

A Filter represents a query filter that can have any multiple conditions, logical filtering, nested conditions, etc. On a high level, a filter from a user query will map like this

{Selector} --> Filter with a single condition
{Selector, Selector, LogicalOperator} --> Filter with two condition and a logicalOperator
{Selector, LogicalOperator} --> Filter with single condition and a logicalOperator
and so on...

The JSON representation for these filters will look like below, "filter: {"f1": 10} "filter": [{"f1": 10}, {"f2": {"$gt": 10}}] "filter": [{"f1": 10}, {"f2": 10}, {"$or": [{"f3": 20}, {"$and": [{"f4":5}, {"f5": 6}]}]}]

The default rule applied between filters are "$and and the default selector is "$eq".

type GreaterThanEqMatcher

type GreaterThanEqMatcher struct {
	Value value.Value
}

GreaterThanEqMatcher implements "$gte" operand.

func (*GreaterThanEqMatcher) ArrMatches

func (g *GreaterThanEqMatcher) ArrMatches(arr []any) bool

ArrMatches returns true for "GreaterThanEqMatcher" if "v" is lower than any one of the elements of the array.

func (*GreaterThanEqMatcher) GetValue

func (g *GreaterThanEqMatcher) GetValue() value.Value

func (*GreaterThanEqMatcher) Matches

func (g *GreaterThanEqMatcher) Matches(input value.Value) bool

func (*GreaterThanEqMatcher) String

func (g *GreaterThanEqMatcher) String() string

func (*GreaterThanEqMatcher) Type

func (g *GreaterThanEqMatcher) Type() string

type GreaterThanMatcher

type GreaterThanMatcher struct {
	Value value.Value
}

GreaterThanMatcher implements "$gt" operand.

func (*GreaterThanMatcher) ArrMatches

func (g *GreaterThanMatcher) ArrMatches(arr []any) bool

func (*GreaterThanMatcher) GetValue

func (g *GreaterThanMatcher) GetValue() value.Value

func (*GreaterThanMatcher) Matches

func (g *GreaterThanMatcher) Matches(input value.Value) bool

func (*GreaterThanMatcher) String

func (g *GreaterThanMatcher) String() string

func (*GreaterThanMatcher) Type

func (g *GreaterThanMatcher) Type() string

type IndexType

type IndexType uint8
const (
	Unknown IndexType = iota
	PrimaryIndex
	SecondaryIndex
)

type KeyBuilder

type KeyBuilder[F fieldable] struct {
	// contains filtered or unexported fields
}

KeyBuilder is responsible for building internal Keys. A composer is caller by the builder to build the internal keys based on the Composer logic. KeyBuilder uses generics so that it can accept either schema.QueryableField or schema.Field so that it can build a query plan for primay or secondary indexes.

func NewKeyBuilder

func NewKeyBuilder[F fieldable](composer KeyComposer[F], indexType IndexType) *KeyBuilder[F]

NewKeyBuilder returns a KeyBuilder.

func NewPrimaryKeyEqBuilder

func NewPrimaryKeyEqBuilder(keyEncodingFunc KeyEncodingFunc) *KeyBuilder[*schema.Field]

NewPrimaryKeyEqBuilder returns a KeyBuilder for use with schema.Field to build a primary key query plan.

func NewRangeKeyBuilder

func NewRangeKeyBuilder(composer KeyComposer[*schema.QueryableField], indexType IndexType) *KeyBuilder[*schema.QueryableField]

NewRangeKeyBuilder returns a KeyBuilder for use with schema.QueryableField.

func NewSecondaryKeyEqBuilder

func NewSecondaryKeyEqBuilder[F fieldable](keyEncodingFunc KeyEncodingFunc, buildIndexPartsFunc BuildIndexPartsFunc) *KeyBuilder[F]

NewSecondaryKeyEqBuilder returns a KeyBuilder for use with the secondary index.

func (*KeyBuilder[F]) Build

func (k *KeyBuilder[F]) Build(filters []Filter, userDefinedKeys []F) ([]QueryPlan, error)

Build is responsible for building the internal keys from the user filter and using the keys defined in the schema and passed by the caller in this method. The build is doing a level by level traversal to build the internal Keys. On each level multiple keys can be formed because the user can specify ranges. The builder is not deciding the logic of key generation, the builder is simply traversing on the filters and calling compose where the logic resides. If the build is for a primary key, the query plans are merged into a single plan.

type KeyComposer

type KeyComposer[F fieldable] interface {
	Compose(level []*Selector, userDefinedKeys []F, parent LogicalOP) ([]QueryPlan, error)
}

KeyComposer needs to be implemented to have a custom Compose method with different constraints.

type KeyEncodingFunc

type KeyEncodingFunc func(indexParts ...interface{}) (keys.Key, error)

type LessThanEqMatcher

type LessThanEqMatcher struct {
	Value value.Value
}

LessThanEqMatcher implements "$lte" operand.

func (*LessThanEqMatcher) ArrMatches

func (l *LessThanEqMatcher) ArrMatches(arr []any) bool

ArrMatches returns true for "LessThanEqMatcher" if "v" is greater than any one of the array element.

func (*LessThanEqMatcher) GetValue

func (l *LessThanEqMatcher) GetValue() value.Value

func (*LessThanEqMatcher) Matches

func (l *LessThanEqMatcher) Matches(input value.Value) bool

func (*LessThanEqMatcher) String

func (l *LessThanEqMatcher) String() string

func (*LessThanEqMatcher) Type

func (l *LessThanEqMatcher) Type() string

type LessThanMatcher

type LessThanMatcher struct {
	Value value.Value
}

LessThanMatcher implements "$lt" operand.

func (*LessThanMatcher) ArrMatches

func (l *LessThanMatcher) ArrMatches(arr []any) bool

func (*LessThanMatcher) GetValue

func (l *LessThanMatcher) GetValue() value.Value

func (*LessThanMatcher) Matches

func (l *LessThanMatcher) Matches(input value.Value) bool

func (*LessThanMatcher) String

func (l *LessThanMatcher) String() string

func (*LessThanMatcher) Type

func (l *LessThanMatcher) Type() string

type LikeFilter

type LikeFilter struct {
	Field   *schema.QueryableField
	Matcher LikeMatcher
}

LikeFilter creates a filter that offers "like" semantics i.e. "regex"/"contains"/"not". It is not used to create any key. It is always used to post-process the records.

func NewLikeFilter

func NewLikeFilter(field *schema.QueryableField, matcher LikeMatcher) *LikeFilter

func (*LikeFilter) IsSearchIndexed

func (s *LikeFilter) IsSearchIndexed() bool

func (*LikeFilter) Matches

func (s *LikeFilter) Matches(doc []byte, metadata []byte) bool

Matches returns true if the doc value matches this filter.

func (*LikeFilter) MatchesDoc

func (s *LikeFilter) MatchesDoc(doc map[string]any) bool

func (*LikeFilter) String

func (s *LikeFilter) String() string

func (*LikeFilter) ToSearchFilter

func (s *LikeFilter) ToSearchFilter() string

type LikeMatcher

type LikeMatcher interface {
	Matcher

	Matches(value any) bool
}

func NewContainsMatcher

func NewContainsMatcher(value string, collation *value.Collation) (LikeMatcher, error)

func NewLikeMatcher

func NewLikeMatcher(key string, input string, collation *value.Collation) (LikeMatcher, error)

func NewNotMatcher

func NewNotMatcher(value string, collation *value.Collation) (LikeMatcher, error)

func NewRegexMatcher

func NewRegexMatcher(value string, collation *value.Collation) (LikeMatcher, error)

type LogicalFilter

type LogicalFilter interface {
	GetFilters() []Filter
	Type() LogicalOP
	Filter
}

LogicalFilter (or boolean) are the filters that evaluates to True or False. A logical operator can have the following form inside the JSON

{"$and": [{"f1":1}, {"f2": 3}]}
{"$or": [{"f1":1}, {"f2": 3}]}

type LogicalOP

type LogicalOP string
const (
	AndOP LogicalOP = "$and"
	OrOP  LogicalOP = "$or"
)

type Matcher

type Matcher interface {
	// Type return the type of the value matcher, syntactic sugar for logging, etc
	Type() string
}

type NotMatcher

type NotMatcher struct {
	// contains filtered or unexported fields
}

NotMatcher implements "$not" operand.

func (*NotMatcher) Matches

func (n *NotMatcher) Matches(docValue any) bool

func (*NotMatcher) String

func (n *NotMatcher) String() string

func (*NotMatcher) Type

func (n *NotMatcher) Type() string

type OrFilter

type OrFilter struct {
	// contains filtered or unexported fields
}

OrFilter performs a logical OR operation on an array of two or more expressions. The or filter looks like this, {"$or": [{"f1":1}, {"f2": 3}....]} It can be nested i.e. a top level "$or" can have multiple nested $and/$or.

func NewOrFilter

func NewOrFilter(filter []Filter) (*OrFilter, error)

func (*OrFilter) GetFilters

func (o *OrFilter) GetFilters() []Filter

GetFilters returns all the nested filters for OrFilter.

func (*OrFilter) IsSearchIndexed

func (o *OrFilter) IsSearchIndexed() bool

func (*OrFilter) Matches

func (o *OrFilter) Matches(doc []byte, metadata []byte) bool

Matches returns true if the input doc matches this filter.

func (*OrFilter) MatchesDoc

func (o *OrFilter) MatchesDoc(doc map[string]interface{}) bool

func (*OrFilter) String

func (o *OrFilter) String() string

String a helpful method for logging.

func (*OrFilter) ToSearchFilter

func (o *OrFilter) ToSearchFilter() string

func (*OrFilter) Type

func (o *OrFilter) Type() LogicalOP

type QueryPlan

type QueryPlan struct {
	QueryType QueryPlanType
	FieldName string
	DataType  schema.FieldType
	Keys      []keys.Key
	Ascending bool
	IndexType IndexType
	From      keys.Key
}

QueryPlan is returned by KeyBuilder that contains the keys and type of query against fdb.

func NewQueryPlan

func NewQueryPlan(queryType QueryPlanType, fieldName string, dataType schema.FieldType, keys []keys.Key, indexType IndexType) QueryPlan

func QueryPlanFromSort

func QueryPlanFromSort(sortFields *[]tsort.SortField, indexableFields []*schema.QueryableField, encoder KeyEncodingFunc, buildIndexParts BuildIndexPartsFunc, indexType IndexType) (*QueryPlan, error)

func SortQueryPlans

func SortQueryPlans(queries []QueryPlan) []QueryPlan

SortQueryPlans creates a simple way to choose a best query plan based on the queryType.

func (QueryPlan) GetKeyInterfaceParts

func (q QueryPlan) GetKeyInterfaceParts() [][]interface{}

func (QueryPlan) Reverse

func (q QueryPlan) Reverse() bool

type QueryPlanType

type QueryPlanType uint8
const (
	EQUAL QueryPlanType = iota
	RANGE
	FULLRANGE
)

type RangeKeyComposer

type RangeKeyComposer[F fieldable] struct {
	// contains filtered or unexported fields
}

RangeKeyComposer will generate a range key set on the user defined keys It will set the KeyQuery to `FullRange` if the start or end key is not defined in the query if there is a defined start and end key for a range then `Range` is set.

func NewRangeKeyComposer

func NewRangeKeyComposer[F fieldable](keyEncodingFunc KeyEncodingFunc, buildIndexParts BuildIndexPartsFunc, indexType IndexType) *RangeKeyComposer[F]

func (*RangeKeyComposer[F]) Compose

func (s *RangeKeyComposer[F]) Compose(selectors []*Selector, userDefinedKeys []F, parent LogicalOP) ([]QueryPlan, error)

type RegexMatcher

type RegexMatcher struct {
	// contains filtered or unexported fields
}

RegexMatcher implements "$regex" operand. When matching against text, the regexp returns a match that begins as early as possible in the input (leftmost), and among those it chooses the one that a backtracking search would have found first. This so-called leftmost-first matching is the same semantics that Perl, Python, and other implementations use.

func (*RegexMatcher) Matches

func (c *RegexMatcher) Matches(docValue any) bool

func (*RegexMatcher) String

func (c *RegexMatcher) String() string

func (*RegexMatcher) Type

func (c *RegexMatcher) Type() string

type Selector

type Selector struct {
	Matcher   ValueMatcher
	Parent    *schema.QueryableField
	Field     *schema.QueryableField
	Collation *value.Collation
}

Selector is a condition defined inside a filter. It has a field which corresponding the field on which condition is defined and then Matcher. A matcher is formed from the user condition i.e. if the condition is on "$eq" then a EqualityMatcher is created. The matcher implements a Match method which is used to know if the input document passes the condition.

A Selector can have this form inside the input JSON

{f:{$eq:1}}
{f:20} (default is "$eq" so we automatically append EqualityMatcher for this case in parser)
{f:<Expr>}

func NewSelector

func NewSelector(parent *schema.QueryableField, field *schema.QueryableField, matcher ValueMatcher, collation *value.Collation) *Selector

NewSelector returns Selector object.

func (*Selector) IsSearchIndexed

func (s *Selector) IsSearchIndexed() bool

func (*Selector) Matches

func (s *Selector) Matches(doc []byte, metadata []byte) bool

Matches returns true if the input doc matches this filter. To note around the order of checking for not exist and error logging An error is returned if the field does not exist and that is an acceptable error, so return false Only log an error that is unexpected.

func (*Selector) MatchesDoc

func (s *Selector) MatchesDoc(doc map[string]any) bool

func (*Selector) String

func (s *Selector) String() string

String a helpful method for logging.

func (*Selector) ToSearchFilter

func (s *Selector) ToSearchFilter() string

type StrictEqKeyComposer

type StrictEqKeyComposer[F fieldable] struct {
	// contains filtered or unexported fields
}

StrictEqKeyComposer works in to ways to generate internal keys if the condition is equality.

  1. When `matchAll=true`, it will generate internal keys of equality on the fields of the schema if all these fields are present in the filters. The following rules are applied for StrictEqKeyComposer: - The userDefinedKeys(indexes defined in the schema) passed in parameter should be present in the filter - For AND filters it is possible to build internal keys for composite indexes, for OR it is not possible.

2. When `matchAll=false`, it will treat all userDefined as individual and generate an `$eq` query plan for each one that is found.

For OR filter an error is returned if it is used for indexes that are composite.

func NewStrictEqKeyComposer

func NewStrictEqKeyComposer[F fieldable](keyEncodingFunc KeyEncodingFunc, buildIndexPartsFunc BuildIndexPartsFunc, matchAll bool, indexType IndexType) *StrictEqKeyComposer[F]

func (*StrictEqKeyComposer[F]) Compose

func (s *StrictEqKeyComposer[F]) Compose(selectors []*Selector, userDefinedKeys []F, parent LogicalOP) ([]QueryPlan, error)

Compose is implementing the logic of composing keys.

type TableScanPlan

type TableScanPlan struct {
	Table   []byte
	From    keys.Key
	Reverse bool
}

type ValueMatcher

type ValueMatcher interface {
	Matcher
	ArrayMatcher

	// Matches returns true if the receiver has the value object that has the same value as input
	Matches(input value.Value) bool
	// GetValue returns the value on which the Matcher is operating
	GetValue() value.Value
}

ValueMatcher is an interface that has method like Matches.

func NewMatcher

func NewMatcher(key string, v value.Value) (ValueMatcher, error)

NewMatcher returns ValueMatcher that is derived from the key.

type WrappedFilter

type WrappedFilter struct {
	Filter
	// contains filtered or unexported fields
}

func NewWrappedFilter

func NewWrappedFilter(filters []Filter) *WrappedFilter

func (*WrappedFilter) IsSearchIndexed

func (w *WrappedFilter) IsSearchIndexed() bool

func (*WrappedFilter) None

func (w *WrappedFilter) None() bool

func (*WrappedFilter) SearchFilter

func (w *WrappedFilter) SearchFilter() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL