Documentation
¶
Overview ¶
Package filter provides generic rule-based filtering capabilities for struct slices.
Large sets of data can be filtered down to a subset of elements using a set of rules.
The filter can be specified as a slice of slices of rules or as JSON (see filter_schema.json for the JSON schema). The first slice contains the rule sets that will be combined with a boolean AND. The sub-slices contain the rules that will be combined with a boolean OR.
A common application consists of users specifying a filter in a URL query parameter to reduce the amount of data to download on a GET request. The rules are parsed and applied server-side.
Example: ¶
The following pretty-printed JSON:
[
[
{
"field": "name",
"type": "==",
"value": "doe"
},
{
"field": "age",
"type": "<=",
"value": 42
}
],
[
{
"field": "address.country",
"type": "regexp",
"value": "^EN$|^FR$"
}
]
]
can be represented in one line as:
[[{"field":"name","type":"==","value":"doe"},{"field":"age","type":"<=","value":42}],[{"field":"address.country","type":"regexp","value":"^EN$|^FR$"}]]
and URL-encoded as a query parameter:
filter=%5B%5B%7B%22field%22%3A%22name%22%2C%22type%22%3A%22%3D%3D%22%2C%22value%22%3A%22doe%22%7D%2C%7B%22field%22%3A%22age%22%2C%22type%22%3A%22%3C%3D%22%2C%22value%22%3A42%7D%5D%2C%5B%7B%22field%22%3A%22address.country%22%2C%22type%22%3A%22regexp%22%2C%22value%22%3A%22%5EEN%24%7C%5EFR%24%22%7D%5D%5D
the equivalent logic is:
((name==doe OR age<=42) AND (address.country match "EN" or "FR"))
The supported rule types are listed in the rule.go file:
- "regexp" : matches the value against a reference regular expression.
- "==" : Equal to - matches exactly the reference value.
- "=" : Equal fold - matches when strings, interpreted as UTF-8, are equal under simple Unicode case-folding, which is a more general form of case-insensitivity. For example "AB" will match "ab".
- "^=" : Starts with - (strings only) matches when the value begins with the reference string.
- "=$" : Ends with - (strings only) matches when the value ends with the reference string.
- "~=" : Contains -(strings only) matches when the reference string is a sub-string of the value.
- "<" : Less than - matches when the value is less than the reference.
- "<=" : Less than or equal to - matches when the value is less than or equal the reference.
- ">" : Greater than - matches when the value is greater than reference.
- ">=" : Greater than or equal to - matches when the value is greater than or equal the reference.
Every rule type can be prefixed with the symbol "!" to get the negated value. For example "!==" is equivalent to "Not Equal", matching values that are different.
Index ¶
Examples ¶
Constants ¶
const ( // MaxResults is the maximum number of results that can be returned. MaxResults = 1<<31 - 1 // math.MaxInt32 // DefaultMaxResults is the default number of results for Apply. // Can be overridden with WithMaxResults(). DefaultMaxResults = MaxResults // DefaultMaxRules is the default maximum number of rules. // Can be overridden with WithMaxRules(). DefaultMaxRules = 3 // DefaultURLQueryFilterKey is the default URL query key used by Processor.ParseURLQuery(). // Can be customized with WithQueryFilterKey(). DefaultURLQueryFilterKey = "filter" )
const ( // TypePrefixNot is a prefix that can be added to any type to get the negated value (opposite match). TypePrefixNot = "!" // TypeRegexp is a filter type that matches the value against a reference regular expression. // The reference value must be a regular expression that can compile. // Works only with strings (anything else will evaluate to false). TypeRegexp = "regexp" // TypeEqual is a filter type that matches exactly the reference value. TypeEqual = "==" // TypeEqualFold is a filter type that matches when strings, interpreted as UTF-8, are equal under simple Unicode case-folding, which is a more general form of case-insensitivity. For example "AB" will match "ab". TypeEqualFold = "=" // TypeHasPrefix is a filter type that matches when the value begins with the reference string. TypeHasPrefix = "^=" // TypeHasSuffix is a filter type that matches when the value ends with the reference string. TypeHasSuffix = "=$" // TypeContains is a filter type that matches when the reference string is a sub-string of the value. TypeContains = "~=" // TypeLT is a filter type that matches when the value is less than reference. TypeLT = "<" // TypeLTE is a filter type that matches when the value is less than or equal the reference. TypeLTE = "<=" // TypeGT is a filter type that matches when the value is greater than reference. TypeGT = ">" // TypeGTE is a filter type that matches when the value is greater than or equal the reference. TypeGTE = ">=" )
const (
// FieldNameSeparator is the separator for Rule fields.
FieldNameSeparator = "."
)
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Evaluator ¶
type Evaluator interface {
// Evaluate determines if two given values match.
Evaluate(value any) bool
}
Evaluator is the interface to provide functions for a filter type.
type Option ¶
Option is the function that allows to set configuration options.
func WithFieldNameTag ¶
WithFieldNameTag allows to use the field names specified by the tag instead of the original struct names.
Returns an error if the tag is empty.
func WithMaxResults ¶
WithMaxResults sets the maximum length of the slice returned by Apply() and ApplySubset().
func WithMaxRules ¶
WithMaxRules sets the maximum number of rules to pass to the Processor.Apply() function without errors. If this option is not set, it defaults to 3.
Return an error if max is less than 1.
func WithQueryFilterKey ¶
WithQueryFilterKey sets the query parameter key that Processor.ParseURLQuery() looks for.
type Processor ¶
type Processor struct {
// contains filtered or unexported fields
}
Processor provides the filtering logic and methods.
func New ¶
New returns a new Processor with the rules and the given options.
The first level of rules is matched with an AND operator and the second level with an OR.
"[a,[b,c],d]" evaluates to "a AND (b OR c) AND d".
func (*Processor) Apply ¶
Apply filters the slice to remove elements not matching the defined rules. The slice parameter must be a pointer to a slice and is filtered *in place*.
This is a shortcut to ApplySubset with 0 offset and maxResults length.
Returns the length of the filtered slice, the total number of elements that matched the filter, and the eventual error.
Example ¶
package main
import (
"fmt"
"log"
"net/url"
"github.com/Vonage/gosrvlib/pkg/filter"
)
// Address is an example structure type used to test nested structures.
type Address struct {
Country string `json:"country"`
}
// ID is an example structure type.
type ID struct {
Name string `json:"name"`
Age int `json:"age"`
Addr Address `json:"address"`
}
func main() {
// Simulate an encoded query passed in the http.Request of a http.Handler
encodedJSONFilter := "%5B%5B%7B%22field%22%3A%22name%22%2C%22type%22%3A%22%3D%3D%22%2C%22value%22%3A%22doe%22%7D%2C%7B%22field%22%3A%22age%22%2C%22type%22%3A%22%3C%3D%22%2C%22value%22%3A42%7D%5D%2C%5B%7B%22field%22%3A%22address.country%22%2C%22type%22%3A%22regexp%22%2C%22value%22%3A%22%5EEN%24%7C%5EFR%24%22%7D%5D%5D"
u, err := url.Parse("https://server.com/items?filter=" + encodedJSONFilter)
if err != nil {
log.Fatal(err)
}
// Initialize the filter with options
// * WithJSONValues: We want to be lenient on the typing since we create the filter from JSON which handles a few types
// * WithFieldNameTag: to express the filter based on JSON tags and not the actual field names
f, err := filter.New(
filter.WithFieldNameTag("json"),
)
if err != nil {
log.Fatal(err)
}
// The filter matches the following pretty printed json:
//
// [
// [
// {
// "field": "name",
// "type": "==",
// "value": "doe"
// },
// {
// "field": "age",
// "type": "<=",
// "value": 42
// }
// ],
// [
// {
// "field": "address.country",
// "type": "regexp",
// "value": "^EN$|^FR$"
// }
// ]
// ]
//
// can be represented in one line as:
//
// [[{"field":"name","type":"==","value":"doe"},{"field":"age","type":"<=","value":42}],[{"field":"address.country","type":"regexp","value":"^EN$|^FR$"}]]
//
// and URL-encoded as a query parameter:
//
// filter=%5B%5B%7B%22field%22%3A%22name%22%2C%22type%22%3A%22%3D%3D%22%2C%22value%22%3A%22doe%22%7D%2C%7B%22field%22%3A%22age%22%2C%22type%22%3A%22%3C%3D%22%2C%22value%22%3A42%7D%5D%2C%5B%7B%22field%22%3A%22address.country%22%2C%22type%22%3A%22regexp%22%2C%22value%22%3A%22%5EEN%24%7C%5EFR%24%22%7D%5D%5D
//
// the equivalent logic is:
//
// ((name==doe OR age<=42) AND (address.country match "EN" or "FR"))
rules, err := f.ParseURLQuery(u.Query())
if err != nil {
log.Fatal(err)
}
// Given this list, the last item will be filtered
list := []ID{
{
Name: "doe",
Age: 55,
Addr: Address{
Country: "EN",
},
},
{
Name: "dupont",
Age: 42,
Addr: Address{
Country: "FR",
},
},
{
Name: "doe",
Age: 41,
Addr: Address{
Country: "US",
},
},
}
// Filters the list in place
sliceLen, totalMatches, err := f.Apply(rules, &list)
if err != nil {
log.Fatal(err)
}
fmt.Println(sliceLen)
fmt.Println(totalMatches)
for _, id := range list {
fmt.Println(id)
}
}
Output: 2 2 {doe 55 {EN}} {dupont 42 {FR}}
func (*Processor) ApplySubset ¶
func (p *Processor) ApplySubset(rules [][]Rule, slicePtr any, offset, length uint) (uint, uint, error)
ApplySubset filters the slice to remove elements not matching the defined rules. The slice parameter must be a pointer to a slice and is filtered *in place*.
Depending on offset, the first results are filtered even if they match Depending on length, the filtered slice will only contain a set number of elements.
Returns the length of the filtered slice, the total number of elements that matched the filter, and the eventual error.
func (*Processor) ParseURLQuery ¶
ParseURLQuery parses and returns the defined query parameter from a *url.URL. Defaults to DefaultURLQueryFilterKey and can be customized with WithQueryFilterKey().
If the query parameter is empty or missing, will return a nil slice. If there is a value which is invalid, will return an error.
type Rule ¶
type Rule struct {
// Field is a dot separated selector that is used to target a specific field of the evaluated value.
//
// * "Age" will select the Age field of a structure
// * "Address.Country" will select the Country subfield of the Address structure
// * "" will select the whole value (e.g. to filter a []string)
Field string `json:"field"`
// Type controls the evaluation to apply.
// An invalid value will cause Evaluate() to return an error.
// See the Type* constants of this package for valid values.
Type string `json:"type"`
// Value is the reference value to evaluate against.
// Its type should be accepted by the chosen Type.
Value any `json:"value"`
// contains filtered or unexported fields
}
Rule is an individual filter that can be evaluated against any value.
Source Files