govaluate

README

govaluate

Provides support for evaluating arbitrary C-like artithmetic/string expressions.

Why can't you just write these expressions in code?

Sometimes, you can't know ahead-of-time what an expression will look like, or you want those expressions to be configurable. Perhaps you've got a set of data running through your application, and you want to allow your users to specify some validations to run on it before committing it to a database. Or maybe you've written a monitoring framework which is capable of gathering a bunch of metrics, then evaluating a few expressions to see if any metrics should be alerted upon, but the conditions for alerting are different for each monitor.

A lot of people wind up writing their own half-baked style of evaluation language that fits their needs, but isn't complete. Or they wind up baking the expression into the actual executable, even if they know it's subject to change. These strategies may work, but they take time to implement, time for users to learn, and induce technical debt as requirements change. This library is meant to cover all the normal C-like expressions, so that you don't have to reinvent one of the oldest wheels on a computer.

How do I use it?

You create a new EvaluableExpression, then call "Evaluate" on it.

	expression, err := govaluate.NewEvaluableExpression("10 > 0");
	result, err := expression.Evaluate(nil);
	// result is now set to "true", the bool value.

Cool, but how about with parameters?

	expression, err := govaluate.NewEvaluableExpression("foo > 0");

	parameters := make(map[string]interface{}, 8)
	parameters["foo"] = -1;

	result, err := expression.Evaluate(parameters);
	// result is now set to "false", the bool value.

That's cool, but we can almost certainly have done all that in code. What about a complex use case that involves some math?

	expression, err := govaluate.NewEvaluableExpression("(requests_made * requests_succeeded / 100) >= 90");

	parameters := make(map[string]interface{}, 8)
	parameters["requests_made"] = 100;
	parameters["requests_succeeded"] = 80;

	result, err := expression.Evaluate(parameters);
	// result is now set to "false", the bool value.

Or maybe you want to check the status of an alive check ("smoketest") page, which will be a string?

	expression, err := govaluate.NewEvaluableExpression("http_response_body == 'service is ok'");

	parameters := make(map[string]interface{}, 8)
	parameters["http_response_body"] = "service is ok";

	result, err := expression.Evaluate(parameters);
	// result is now set to "true", the bool value.

These examples have all returned boolean values, but it's equally possible to return numeric ones.

	expression, err := govaluate.NewEvaluableExpression("(mem_used / total_mem) * 100");

	parameters := make(map[string]interface{}, 8)
	parameters["total_mem"] = 1024;
	parameters["mem_used"] = 512;

	result, err := expression.Evaluate(parameters);
	// result is now set to "50.0", the float64 value.

You can also do date parsing, though the formats are somewhat limited. Stick to RF3339, ISO8061, unix date, or ruby date formats. If you're having trouble getting a date string to parse, check the list of formats actually used: parsing.go:248.

	expression, err := govaluate.NewEvaluableExpression("'2014-01-02' > '2014-01-01 23:59:59'");
	result, err := expression.Evaluate(nil);

	// result is now set to true

Expressions are parsed once, and can be re-used multiple times. Parsing is the compute-intensive phase of the process, so if you intend to use the same expression with different parameters, just parse it once. Like so;

	expression, err := govaluate.NewEvaluableExpression("response_time <= 100");
	parameters := make(map[string]interface{}, 8)

	for {
		parameters["response_time"] = pingSomething();
		result, err := expression.Evaluate(parameters)
	}

The normal C-standard order of operators is respected. When writing an expression, be sure that you either order the operators correctly, or use parenthesis to clarify which portions of an expression should be run first.

Escaping characters

Sometimes you'll have parameters that have spaces, slashes, pluses, ampersands or some other character that this library interprets as something special. For example, the following expression will not act as one might expect:

"response-time < 100"

As written, the library will parse it as "[response] minus [time] is less than 100". In reality, "response-time" is meant to be one variable that just happens to have a dash in it.

There are two ways to work around this. First, you can escape the entire parameter name:

"[response-time] < 100"

Or you can use backslashes to escape only the minus sign.

"response\\-time < 100"

Backslashes can be used anywhere in an expression to escape the very next character. Square bracketed parameter names can be used instead of plain parameter names at any time.

What operators and types does this support?

• Modifiers: + - / * & | ^ ** % >> <<
• Comparators: > >= < <= == != =~ !~
• Logical ops: || &&
• Numeric constants, as 64-bit floating point (12345.678)
• String constants (single quotes: 'foobar')
• Date constants (single quotes, using any permutation of RFC3339, ISO8601, ruby date, or unix date; date parsing is automatically tried with any string constant)
• Boolean constants: true false
• Parenthesis to control order of evaluation ( )
• Prefixes: ! - ~
• Ternary conditional ? :

Note: for those not familiar, =~ is "regex-equals" and !~ is "regex-not-equals".

If a ternary operator resolves to false with no else case, it returns nil. So false ? 10 will return nil, whereas true ? 10 will return 10.0.

Types

Some operators don't make sense when used with some types. For instance, what does it mean to get the modulo of a string? What happens if you check to see if two numbers are logically AND'ed together?

Everyone has a different intuition about the answers to these questions. To prevent confusion, this library will refuse to operate upon types for which there is not an unambiguous meaning for the operation. The table is listed below.

Any time you attempt to use an operator on a type which doesn't explicitly support it (indicated by a bold "X" in the table below), the expression will fail to evaluate, and return an error indicating the problem.

Note that this table shows what each type supports - if you use an operator then both types need to support the operator, otherwise an error will be returned.

	Number/Date	String	Boolean
+	Adds	Concatenates	X
-	Subtracts	X	X
/	Divides	X	X
*	Multiplies	X	X
&	Bitwise and	X	X
|	Bitwise or	X	X
^	Bitwise xor	X	X
<<	Bitwise left shift	X	X
>>	Bitwise right shift	X	X
**	Takes to the power of	X	X
%	Modulo	X	X
Greater/Lesser (> >= < <=)	Compares	X	X
Equality (== !=)	Checks by value	Checks by value	Checks by value
Ternary (? :)	X	X	Checks by value
Regex (=~ !~)	X	Regex	X
!	X	X	Inverts
Negate (-)	Multiplies by -1	X	X
~	Bitwise not	X	X

It may, at first, not make sense why a Date supports all the same things as a number. In this library, dates are treated as the unix time. That is, the number of seconds since epoch. In practice this means that sub-second precision with this library is impossible (drop an issue in Github if this is a deal-breaker for you). It also, by association, means that you can do operations that you may not expect, like taking a date to the power of two. The author sees no harm in this. Your date probably appreciates it.

Complex types, arrays, and structs are not supported as literals nor parameters. All numeric constants and variables are converted to float64 for evaluation.

Benchmarks

If you're concerned about the overhead of this library, a good range of benchmarks are built into this repo. You can run them with go test -bench=.. The library is built with an eye towards being quick, but has not been aggressively profiled and optimized. For most applications, though, it is completely fine.

For a very rough idea of performance, here are the results output from a benchmark run on my 3rd-gen Macbook Pro (Linux Mint 17.1).

BenchmarkSingleParse-12                          2000000               768 ns/op
BenchmarkSimpleParse-12                           200000              6842 ns/op
BenchmarkFullParse-12                             200000             12791 ns/op
BenchmarkEvaluationSingle-12                    10000000               142 ns/op
BenchmarkEvaluationNumericLiteral-12             3000000               577 ns/op
BenchmarkEvaluationLiteralModifiers-12           2000000               675 ns/op
BenchmarkEvaluationParameters-12                 2000000               883 ns/op
BenchmarkEvaluationParametersModifiers-12        1000000              1305 ns/op
BenchmarkComplexExpression-12                    1000000              1308 ns/op
BenchmarkRegexExpression-12                       100000             22751 ns/op
BenchmarkConstantRegexExpression-12               500000              2599 ns/op
ok

API Breaks

While this library has very few cases which will ever result in an API break, it can (and has) happened. If you are using this in production, vendor the commit you've tested against. Master branch (while infrequent) may at some point contain API breaking changes, and the author will have no way to communicate these to downstreams.

Releases will explicitly state when an API break happens, and if they do not specify an API break it should be safe to upgrade.

Branching

I use green masters, and heavily develop with private feature branches. Full releases are pinned and unchangeable, representing the best available version with the best documentation and test coverage. Master branch, however, should always have all tests pass and implementations considered "working", even if it's just a first pass. Master should never panic.

License

This project is licensed under the MIT general use license. You're free to integrate, fork, and play with this code as you feel fit without consulting the author, as long as you provide proper credit to the author in your works.

Activity

If this repository hasn't been updated in a while, it's probably because I don't have any outstanding issues to work on - it's not because I've abandoned the project. If you have questions, issues, or patches; I'm completely open to pull requests, issues opened on github, or emails from out of the blue.

Documentation

Index

• Constants
• Variables
• func GetTokenKindString(kind TokenKind) string
• type EvaluableExpression
  • func NewEvaluableExpression(expression string) (*EvaluableExpression, error)
  • func (this EvaluableExpression) Eval(parameters Parameters) (interface{}, error)
  • func (this EvaluableExpression) Evaluate(parameters map[string]interface{}) (interface{}, error)
  • func (this EvaluableExpression) String() string
  • func (this EvaluableExpression) ToSQLQuery() (string, error)
  • func (this EvaluableExpression) Tokens() []ExpressionToken
• type ExpressionToken
• type MapParameters
  • func (p MapParameters) Get(name string) (interface{}, error)
• type OperatorPrecedence
• type OperatorSymbol
  • func (this OperatorSymbol) IsModifierType(candidate []OperatorSymbol) bool
  • func (this OperatorSymbol) String() string
• type Parameters
• type TokenKind

Constants

const (
	TYPEERROR_LOGICAL    string = "Value '%v' cannot be used with the logical operator '%v', it is not a bool"
	TYPEERROR_MODIFIER   string = "Value '%v' cannot be used with the modifier '%v', it is not a number"
	TYPEERROR_COMPARATOR string = "Value '%v' cannot be used with the comparator '%v', it is not a number"
	TYPEERROR_TERNARY    string = "Value '%v' cannot be used with the ternary operator '%v', it is not a bool"
	TYPEERROR_PREFIX     string = "Value '%v' cannot be used with the prefix '%v'"
)

Variables

var ADDITIVE_MODIFIERS = []OperatorSymbol{
	PLUS, MINUS,
}

var ADDITIVE_SYMBOLS = map[string]OperatorSymbol{
	"+": PLUS,
	"-": MINUS,
}

var BITWISE_MODIFIERS = []OperatorSymbol{
	BITWISE_AND, BITWISE_OR, BITWISE_XOR,
}

var BITWISE_SHIFT_MODIFIERS = []OperatorSymbol{
	BITWISE_LSHIFT, BITWISE_RSHIFT,
}

var BITWISE_SHIFT_SYMBOLS = map[string]OperatorSymbol{
	">>": BITWISE_RSHIFT,
	"<<": BITWISE_LSHIFT,
}

var BITWISE_SYMBOLS = map[string]OperatorSymbol{
	"^": BITWISE_XOR,
	"&": BITWISE_AND,
	"|": BITWISE_OR,
}

var COMPARATOR_SYMBOLS = map[string]OperatorSymbol{
	"==": EQ,
	"!=": NEQ,
	">":  GT,
	">=": GTE,
	"<":  LT,
	"<=": LTE,
	"=~": REQ,
	"!~": NREQ,
}

Map of all valid comparators, and their string equivalents. Used during parsing of expressions to determine if a symbol is, in fact, a comparator. Also used during evaluation to determine exactly which comparator is being used.

var DUMMY_PARAMETERS = MapParameters(map[string]interface{}{})

var EXPONENTIAL_MODIFIERS = []OperatorSymbol{
	EXPONENT,
}

var EXPONENTIAL_SYMBOLS = map[string]OperatorSymbol{
	"**": EXPONENT,
}

var LOGICAL_SYMBOLS = map[string]OperatorSymbol{
	"&&": AND,
	"||": OR,
}

var MODIFIER_SYMBOLS = map[string]OperatorSymbol{
	"+":  PLUS,
	"-":  MINUS,
	"*":  MULTIPLY,
	"/":  DIVIDE,
	"%":  MODULUS,
	"**": EXPONENT,
	"&":  BITWISE_AND,
	"|":  BITWISE_OR,
	"^":  BITWISE_XOR,
	">>": BITWISE_RSHIFT,
	"<<": BITWISE_LSHIFT,
}

this is defined separately from ADDITIVE_SYMBOLS et al because it's needed for parsing, not stage planning.

var MULTIPLICATIVE_MODIFIERS = []OperatorSymbol{
	MULTIPLY, DIVIDE, MODULUS,
}

var MULTIPLICATIVE_SYMBOLS = map[string]OperatorSymbol{
	"*": MULTIPLY,
	"/": DIVIDE,
	"%": MODULUS,
}

var NUMERIC_COMPARATORS = []OperatorSymbol{
	GT, GTE, LT, LTE,
}

var PREFIX_MODIFIERS = []OperatorSymbol{
	NEGATE, INVERT, BITWISE_NOT,
}

var PREFIX_SYMBOLS = map[string]OperatorSymbol{
	"-": NEGATE,
	"!": INVERT,
	"~": BITWISE_NOT,
}

var STRING_COMPARATORS = []OperatorSymbol{
	REQ, NREQ,
}

var TERNARY_SYMBOLS = map[string]OperatorSymbol{
	"?": TERNARY_TRUE,
	":": TERNARY_FALSE,
}

Functions

func GetTokenKindString

func GetTokenKindString(kind TokenKind) string

GetTokenKindString returns a string that describes the given TokenKind. e.g., when passed the NUMERIC TokenKind, this returns the string "NUMERIC".

Types

type EvaluableExpression

type EvaluableExpression struct {

	/*
		Represents the query format used to output dates. Typically only used when creating SQL or Mongo queries from an expression.
		Defaults to the complete ISO8601 format, including nanoseconds.
	*/
	QueryDateFormat string
	// contains filtered or unexported fields
}

EvaluableExpression represents a set of ExpressionTokens which, taken together, are an expression that can be evaluated down into a single value.

func NewEvaluableExpression

func NewEvaluableExpression(expression string) (*EvaluableExpression, error)

Parses a new EvaluableExpression from the given [expression] string. Returns an error if the given expression has invalid syntax.

func (EvaluableExpression) Eval

func (this EvaluableExpression) Eval(parameters Parameters) (interface{}, error)

Runs the entire expression using the given [parameters]. e.g., If the expression contains a reference to the variable "foo", it will be taken from `parameters.Get("foo")`.

This function returns errors if the combination of expression and parameters cannot be run, such as if a variable in the expression is not present in [parameters].

In all non-error circumstances, this returns the single value result of the expression and parameters given. e.g., if the expression is "1 + 1", this will return 2.0. e.g., if the expression is "foo + 1" and parameters contains "foo" = 2, this will return 3.0

func (EvaluableExpression) Evaluate

func (this EvaluableExpression) Evaluate(parameters map[string]interface{}) (interface{}, error)

Same as `Eval`, but automatically wraps a map of parameters into a `govalute.Parameters` structure.

func (EvaluableExpression) String

func (this EvaluableExpression) String() string

Returns the original expression used to create this EvaluableExpression.

func (EvaluableExpression) ToSQLQuery

func (this EvaluableExpression) ToSQLQuery() (string, error)

Returns a string representing this expression as if it were written in SQL. This function assumes that all parameters exist within the same table, and that the table essentially represents a serialized object of some sort (e.g., hibernate). If your data model is more normalized, you may need to consider iterating through each actual token given by `Tokens()` to create your query.

Boolean values are considered to be "1" for true, "0" for false.

Times are formatted according to this.QueryDateFormat.

func (EvaluableExpression) Tokens

func (this EvaluableExpression) Tokens() []ExpressionToken

Returns an array representing the ExpressionTokens that make up this expression.

type ExpressionToken

type ExpressionToken struct {
	Kind  TokenKind
	Value interface{}
}

Represents a single parsed token.

type MapParameters

type MapParameters map[string]interface{}

MapParameters is an implementation of the Parameters interface using a map.

func (MapParameters) Get

func (p MapParameters) Get(name string) (interface{}, error)

Get implemetns the method from Parameters

type OperatorPrecedence

type OperatorPrecedence int

const (
	VALUE_PRECEDENCE OperatorPrecedence = iota
	PREFIX_PRECEDENCE
	EXPONENTIAL_PRECEDENCE
	ADDITIVE_PRECEDENCE
	BITWISE_PRECEDENCE
	BITWISE_SHIFTPRECEDENCE
	MULTIPLICATIVE_PRECEDENCE
	COMPARATOR_PRECEDENCE
	TERNARY_PRECEDENCE
	LOGICAL_PRECEDENCE
)

type OperatorSymbol

type OperatorSymbol int

Represents the valid symbols for operators.

const (
	NOOP OperatorSymbol = iota
	EQ
	NEQ
	GT
	LT
	GTE
	LTE
	REQ
	NREQ

	AND
	OR

	PLUS
	MINUS
	BITWISE_AND
	BITWISE_OR
	BITWISE_XOR
	BITWISE_LSHIFT
	BITWISE_RSHIFT
	MULTIPLY
	DIVIDE
	MODULUS
	EXPONENT

	NEGATE
	INVERT
	BITWISE_NOT

	TERNARY_TRUE
	TERNARY_FALSE
)

func (OperatorSymbol) IsModifierType

func (this OperatorSymbol) IsModifierType(candidate []OperatorSymbol) bool

Returns true if this operator is contained by the given array of candidate symbols. False otherwise.

func (OperatorSymbol) String

func (this OperatorSymbol) String() string

Generally used when formatting type check errors. We could store the stringified symbol somewhere else and not require a duplicated codeblock to translate OperatorSymbol to string, but that would require more memory, and another field somewhere. Adding operators is rare enough that we just stringify it here instead.

type Parameters

type Parameters interface {
	// Get gets the parameter of the given name
	Get(name string) (interface{}, error)
}

Parameters is a collection of named parameters that are accessible via the Get method.

type TokenKind

type TokenKind int

Represents all valid types of tokens that a token can be.

const (
	UNKNOWN TokenKind = iota

	PREFIX
	NUMERIC
	BOOLEAN
	STRING
	PATTERN
	TIME
	VARIABLE

	COMPARATOR
	LOGICALOP
	MODIFIER

	CLAUSE
	CLAUSE_CLOSE

	TERNARY
)