filter

Documentation

Overview

Package filter provides generic filtering capabilities for struct slices. The filter can be specified as a slice of slices. The first slice contains the rule sets that will be combined with a boolean AND. The sub slices contains the rules that will be combined with boolean OR.

Example: The following pretty-printed JSON:

[
  [
    {
      "field": "name",
      "type": "==",
      "value": "doe"
    },
    {
      "field": "age",
      "type": "<=",
      "value": 42
    }
  ],
  [
    {
      "field": "address.country",
      "type": "regexp",
      "value": "^EN$|^FR$"
    }
  ]
]

can be represented in one line as:

[[{"field":"name","type":"==","value":"doe"},{"field":"age","type":"<=","value":42}],[{"field":"address.country","type":"regexp","value":"^EN$|^FR$"}]]

and URL-encoded as a query parameter:

filter=%5B%5B%7B%22field%22%3A%22name%22%2C%22type%22%3A%22%3D%3D%22%2C%22value%22%3A%22doe%22%7D%2C%7B%22field%22%3A%22age%22%2C%22type%22%3A%22%3C%3D%22%2C%22value%22%3A42%7D%5D%2C%5B%7B%22field%22%3A%22address.country%22%2C%22type%22%3A%22regexp%22%2C%22value%22%3A%22%5EEN%24%7C%5EFR%24%22%7D%5D%5D

the equivalent logic is:

((name==doe OR age<=42) AND (address.country match "EN" or "FR"))

The supported rule types are listed in the rule.go file:

• "regexp" : matches the value against a reference regular expression.
• "==" : Equal to - matches exactly the reference value.
• "=" : Equal fold - matches when strings, interpreted as UTF-8, are equal under simple Unicode case-folding, which is a more general form of case-insensitivity. For example "AB" will match "ab".
• "^=" : Starts with - (strings only) matches when the value begins with the reference string.
• "=$" : Ends with - (strings only) matches when the value ends with the reference string.
• "~=" : Contains -(strings only) matches when the reference string is a sub-string of the value.
• "<" : Less than - matches when the value is less than the reference.
• "<=" : Less than or equal to - matches when the value is less than or equal the reference.
• ">" : Greater than - matches when the value is greater than reference.
• ">=" : Greater than or equal to - matches when the value is greater than or equal the reference.

Every rule type can be prefixed with "!" to get the negated value. For example "!==" is equivalent to "Not Equal", matching values that are different.

Index

• Constants
• func ParseJSON(s string) ([][]Rule, error)
• type Evaluator
• type Option
  • func WithFieldNameTag(tag string) Option
  • func WithMaxResults(max uint) Option
  • func WithMaxRules(max uint) Option
  • func WithQueryFilterKey(key string) Option
• type Processor
  • func New(opts ...Option) (*Processor, error)
  • func (p *Processor) Apply(rules [][]Rule, slicePtr interface{}) (uint, uint, error)
  • func (p *Processor) ApplySubset(rules [][]Rule, slicePtr interface{}, offset, length uint) (uint, uint, error)
  • func (p *Processor) ParseURLQuery(q url.Values) ([][]Rule, error)
• type Rule
  • func (r *Rule) Evaluate(value interface{}) (bool, error)

Examples

• Processor.Apply

Constants

const (
	// MaxResults is the maximum number of results that can be returned.
	MaxResults = 1<<31 - 1 // math.MaxInt32

	// DefaultMaxResults is the default number of results for Apply.
	// Can be overridden with WithMaxResults().
	DefaultMaxResults = MaxResults

	// DefaultMaxRules is the default maximum number of rules.
	// Can be overridden with WithMaxRules().
	DefaultMaxRules = 3

	// DefaultURLQueryFilterKey is the default URL query key used by Processor.ParseURLQuery().
	// Can be customized with WithQueryFilterKey().
	DefaultURLQueryFilterKey = "filter"
)

const (
	// TypePrefixNot is a prefix that can be added to any type to get the negated value (opposite match).
	TypePrefixNot = "!"

	// TypeRegexp is a filter type that matches the value against a reference regular expression.
	// The reference value must be a regular expression that can compile.
	// Works only with strings (anything else will evaluate to false).
	TypeRegexp = "regexp"

	// TypeEqual is a filter type that matches exactly the reference value.
	TypeEqual = "=="

	// TypeEqualFold is a filter type that matches when strings, interpreted as UTF-8, are equal under simple Unicode case-folding, which is a more general form of case-insensitivity. For example "AB" will match "ab".
	TypeEqualFold = "="

	// TypeHasPrefix is a filter type that matches when the value begins with the reference string.
	TypeHasPrefix = "^="

	// TypeHasSuffix  is a filter type that matches when the value ends with the reference string.
	TypeHasSuffix = "=$"

	// TypeContains  is a filter type that matches when the reference string is a sub-string of the value.
	TypeContains = "~="

	// TypeLT is a filter type that matches when the value is less than reference.
	TypeLT = "<"

	// TypeLTE is a filter type that matches when the value is less than or equal the reference.
	TypeLTE = "<="

	// TypeGT is a filter type that matches when the value is greater than reference.
	TypeGT = ">"

	// TypeGTE is a filter type that matches when the value is greater than or equal the reference.
	TypeGTE = ">="
)

const (
	// FieldNameSeparator is the separator for Rule fields.
	FieldNameSeparator = "."
)

Functions

func ParseJSON

func ParseJSON(s string) ([][]Rule, error)

ParseJSON parses and returns a [][]Rule from its JSON representation.

Types

type Evaluator

type Evaluator interface {
	// Evaluate determines if two given values match.
	Evaluate(value interface{}) bool
}

Evaluator is the interface to provide functions for a filter type.

type Option

type Option func(p *Processor) error

Option is the function that allows to set configuration options.

func WithFieldNameTag

func WithFieldNameTag(tag string) Option

WithFieldNameTag allows to use the field names specified by the tag instead of the original struct names.

Returns an error if the tag is empty.

func WithMaxResults

func WithMaxResults(max uint) Option

WithMaxResults sets the maximum length of the slice returned by Apply() and ApplySubset().

func WithMaxRules

func WithMaxRules(max uint) Option

WithMaxRules sets the maximum number of rules to pass to the Processor.Apply() function without errors. If this option is not set, it defaults to 3.

Return an error if max is less than 1.

func WithQueryFilterKey

func WithQueryFilterKey(key string) Option

WithQueryFilterKey sets the query parameter key that Processor.ParseURLQuery() looks for.

type Processor

type Processor struct {
	// contains filtered or unexported fields
}

Processor provides the filtering logic and methods.

func New

func New(opts ...Option) (*Processor, error)

New returns a new Processor with the rules and the given options.

The first level of rules is matched with an AND operator and the second level with an OR.

"[a,[b,c],d]" evaluates to "a AND (b OR c) AND d".

func (*Processor) Apply

func (p *Processor) Apply(rules [][]Rule, slicePtr interface{}) (uint, uint, error)

Apply filters the slice to remove elements not matching the defined rules. The slice parameter must be a pointer to a slice and is filtered *in place*.

This is a shortcut to ApplySubset with 0 offset and maxResults length.

Returns the length of the filtered slice, the total number of elements that matched the filter, and the eventual error.

Example

package main

import (
	"fmt"
	"log"
	"net/url"

	"github.com/Vonage/gosrvlib/pkg/filter"
)

// Address is an example structure type used to test nested structures.
type Address struct {
	Country string `json:"country"`
}

// ID is an example structure type.
type ID struct {
	Name string  `json:"name"`
	Age  int     `json:"age"`
	Addr Address `json:"address"`
}

func main() {
	// Simulate an encoded query passed in the http.Request of a http.Handler
	encodedJSONFilter := "%5B%5B%7B%22field%22%3A%22name%22%2C%22type%22%3A%22%3D%3D%22%2C%22value%22%3A%22doe%22%7D%2C%7B%22field%22%3A%22age%22%2C%22type%22%3A%22%3C%3D%22%2C%22value%22%3A42%7D%5D%2C%5B%7B%22field%22%3A%22address.country%22%2C%22type%22%3A%22regexp%22%2C%22value%22%3A%22%5EEN%24%7C%5EFR%24%22%7D%5D%5D"

	u, err := url.Parse("https://server.com/items?filter=" + encodedJSONFilter)
	if err != nil {
		log.Fatal(err)
	}

	// Initialize the filter with options
	// * WithJSONValues: We want to be lenient on the typing since we create the filter from JSON which handles a few types
	// * WithFieldNameTag: to express the filter based on JSON tags and not the actual field names
	f, err := filter.New(
		filter.WithFieldNameTag("json"),
	)
	if err != nil {
		log.Fatal(err)
	}

	// The filter matches the following pretty printed json:
	//
	//	[
	//	  [
	//	    {
	//	      "field": "name",
	//	      "type": "==",
	//	      "value": "doe"
	//	    },
	//	    {
	//	      "field": "age",
	//	      "type": "<=",
	//	      "value": 42
	//	    }
	//	  ],
	//	  [
	//	    {
	//	      "field": "address.country",
	//	      "type": "regexp",
	//	      "value": "^EN$|^FR$"
	//	    }
	//	  ]
	//	]
	//
	// can be represented in one line as:
	//
	//	[[{"field":"name","type":"==","value":"doe"},{"field":"age","type":"<=","value":42}],[{"field":"address.country","type":"regexp","value":"^EN$|^FR$"}]]
	//
	// and URL-encoded as a query parameter:
	//
	//	filter=%5B%5B%7B%22field%22%3A%22name%22%2C%22type%22%3A%22%3D%3D%22%2C%22value%22%3A%22doe%22%7D%2C%7B%22field%22%3A%22age%22%2C%22type%22%3A%22%3C%3D%22%2C%22value%22%3A42%7D%5D%2C%5B%7B%22field%22%3A%22address.country%22%2C%22type%22%3A%22regexp%22%2C%22value%22%3A%22%5EEN%24%7C%5EFR%24%22%7D%5D%5D
	//
	// the equivalent logic is:
	//
	//	((name==doe OR age<=42) AND (address.country match "EN" or "FR"))
	rules, err := f.ParseURLQuery(u.Query())
	if err != nil {
		log.Fatal(err)
	}

	// Given this list, the last item will be filtered
	list := []ID{
		{
			Name: "doe",
			Age:  55,
			Addr: Address{
				Country: "EN",
			},
		},
		{
			Name: "dupont",
			Age:  42,
			Addr: Address{
				Country: "FR",
			},
		},
		{
			Name: "doe",
			Age:  41,
			Addr: Address{
				Country: "US",
			},
		},
	}

	// Filters the list in place
	sliceLen, totalMatches, err := f.Apply(rules, &list)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(sliceLen)
	fmt.Println(totalMatches)

	for _, id := range list {
		fmt.Println(id)
	}

}

Output:

2
2
{doe 55 {EN}}
{dupont 42 {FR}}

func (*Processor) ApplySubset

func (p *Processor) ApplySubset(rules [][]Rule, slicePtr interface{}, offset, length uint) (uint, uint, error)

ApplySubset filters the slice to remove elements not matching the defined rules. The slice parameter must be a pointer to a slice and is filtered *in place*.

Depending on offset, the first results are filtered even if they match Depending on length, the filtered slice will only contain a set number of elements.

Returns the length of the filtered slice, the total number of elements that matched the filter, and the eventual error.

func (*Processor) ParseURLQuery

func (p *Processor) ParseURLQuery(q url.Values) ([][]Rule, error)

ParseURLQuery parses and returns the defined query parameter from a *url.URL. Defaults to DefaultURLQueryFilterKey and can be customized with WithQueryFilterKey().

If the query parameter is empty or missing, will return a nil slice. If there is a value which is invalid, will return an error.

type Rule

type Rule struct {
	// Field is a dot separated selector that is used to target a specific field of the evaluated value.
	//
	// * "Age" will select the Age field of a structure
	// * "Address.Country" will select the Country subfield of the Address structure
	// * "" will select the whole value (e.g. to filter a []string)
	Field string `json:"field"`

	// Type controls the evaluation to apply.
	// An invalid value will cause Evaluate() to return an error.
	// See the Type* constants of this package for valid values.
	Type string `json:"type"`

	// Value is the reference value to evaluate against.
	// Its type should be accepted by the chosen Type.
	Value interface{} `json:"value"`
	// contains filtered or unexported fields
}

Rule is an individual filter that can be evaluated against any value.

func (*Rule) Evaluate

func (r *Rule) Evaluate(value interface{}) (bool, error)

Evaluate returns whether the value matches the rule or not.

Returns an error if the Type is invalid, a misconfiguration (e.g. invalid regexp) or the value is invalid (e.g. evaluating an int with a regexp).