hypermatch

README

Introduction

Hypermatch is a high-performance Go library that enables rapid matching of a large number of rules against events. Designed for speed and efficiency, hypermatch handles thousands of events per second with low latency, making it ideal for real-time systems.

• Fast Matching: Matches events to a large set of rules in-memory with minimal delay ... it's really fast! (Benchmark)
• Readable Rule Format: Serialize rules into human-readable JSON objects.
• Flexible Rule Syntax: Supports various matching conditions, including equals, prefix, suffix, wildcard, anything-but, all-of, and any-of.

An event consists of a list of fields, provided as name/value pairs. A rule links these event fields to patterns that determine whether the event matches.

Quick Start

import (
    hypermatch "github.com/SchwarzIT/hypermatch"
)

func main() {
    //Initialize hypermatch
    hm := hypermatch.NewHyperMatch()
    
    //Add a rule
    if err := hm.AddRule("markus_rule", hypermatch.ConditionSet{
        hypermatch.Condition{Path: "firstname", Pattern: hypermatch.Pattern{Type: hypermatch.PatternEquals, Value: "markus"}},
        hypermatch.Condition{Path: "lastname", Pattern: hypermatch.Pattern{Type: hypermatch.PatternEquals, Value: "troßbach"}},
        }); err != nil {
            panic(err)
    }
    
    //Test with match
    matchedRules := hm.Match([]hypermatch.Property{
        {Path: "firstname", Values: []string{"markus"}},
        {Path: "lastname", Values: []string{"troßbach"}},
    })
    log.Printf("Following rules matches: %v", matchedRules)
    
    //Test without match
    matchedRules = hm.Match([]hypermatch.Property{
        {Path: "firstname", Values: []string{"john"}},
        {Path: "lastname", Values: []string{"doe"}},
    })
    log.Printf("Following rules matches: %v", matchedRules)
}

Documentation

Example Event

An event is represented as a JSON object with various fields. Here’s a sample event:

{
        "name": "Too many parallel requests on system xy",
        "severity": "critical",
        "status": "firing",
        "message": "Lorem ipsum dolor sit amet, consetetur sadipscing elitr.",
        "team": "awesome-team",
        "application": "webshop",
        "component": "backend-service",
        "tags": [
            "shop",
            "backend"
        ]   
}

This example will be referenced throughout the documentation.

Matching Basics

Rules in Hypermatch are composed of conditions defined by the ConditionSet type.

• Case-Insensitive Matching: All value comparisons are case-insensitive.
• Supported Types: Currently, only strings and string arrays are supported.

Each condition includes:

• Path: The field in the event to match against.
• Pattern: The pattern used to match the value at the specified path.

Here’s an example rule that matches the event above:

ConditionSet{
    {
        Path: "status",
        Pattern: Pattern{Type: PatternEquals, Value: "firing"},
    },
    {
        Path: "name",
        Pattern: Pattern{Type: PatternAnythingBut, Sub: []Pattern{
                {Type: PatternWildcard, Value: "TEST*"},
            },
        },
    },
    {
        Path: "severity",
        Pattern: Pattern{ Type: PatternAnyOf,
            Sub: []Pattern{
                {Type: PatternEquals, Value: "critical"},
                {Type: PatternEquals, Value: "warning"},
            },
        },
    },
    {
        Path: "tags",
        Pattern: Pattern{ Type: PatternAllOf,
            Sub: []Pattern{
                {Type: PatternEquals, Value: "shop"},
                {Type: PatternEquals, Value: "backend"},
            },
        },
    },
}

The rules and conditions are also expressible as JSON objects. The following JSON is the equivalent of the above Go notation for a ConditionSet:

{
    "status": {
        "equals": "firing"
    },
    "name": {
        "anythingBut": [
            {"wildcard": "TEST*"}
        ]
    },
    "severity": {
        "anyOf": [
            {"equals": "critical"},
            {"equals": "warning"}
        ]
    },
    "tags": {
        "allOf": [
            {"equals": "shop"},
            {"equals": "backend"}
        ]
    }
}

Note: For simplicity, all examples in this documentation will be presented in JSON format.

Matching syntax

"equals" matching

The equals condition checks if an attribute of the event matches a specified value, case-insensitively.

{
    "status": {
        "equals": "firing"
    }
}

If the attribute value is type of:

• String: Checks if the value is equal to "firing"
• String array: Checks if the array contains an element equal to "firing"

"prefix" matching

The prefix condition checks if an attribute starts with a specified prefix, case-insensitively.

{
    "status": {
        "prefix": "fir"
    }
}

If the attribute value is type of:

• String: Checks if the value begins with "fir"
• String array: Checks if the array contains an element that begins with "fir"

"suffix" matching

The suffix condition checks if an attribute ends with a specified suffix, case-insensitively.

{
    "status": {
        "suffix": "ing"
    }
}

If the attribute value is type of:

• String: Checks if the value ends with "ing"
• String array: Checks if the array contains an element that ends with "ing"

"wildcard" matching

The wildcard condition uses wildcards to match the value of an attribute, ignoring case.

• Use * as a wildcard to match any number of characters (including none).
• You cannot place wildcards directly next to each other.

{
    "name": {
        "wildcard": "*parallel requests*"
    }
}

If the attribute value is type of:

• String: Checks if the value matches the pattern *parallel requests*
• String array: Checks if any value in the array matches the pattern

"anythingBut" matching

The anythingBut condition negates the match, triggering only if the specified condition is not met.

{
    "status": {
        "anythingBut": [
            {"equals": "firing"}
        ]
    }
}

If the attribute value is type of:

• String: Checks if the value is anything other than "firing"
• String array: Checks if the array does not contain an element equal to "firing"

"anyOf" matching

anyOf does correspond to a boolean "inclusive-or". It checks multiple conditions and matches if any of the conditions are true.

{
    "status": {
        "anyOf": [
            {"equals": "firing"},
            {"equals": "resolved"}
        ]
    }
}

If the attribute value is type of:

• String: Checks if the value is either "firing" or "resolved"
• String array: Checks if the array contains an element equal to "firing" or "resolved" or both.

"allOf" matching

allOf does correspond to a boolean "and". It checks multiple conditions and matches if all the conditions are true.

{
    "tags": {
        "allOf": [
            {"equals": "shop"},
            {"equals": "backend"}
        ]
    }
}

If the attribute value is type of:

• String: This condition makes no sense, as it checks if the value is equal to "shop" and "backend"
• String array: Checks if the array contains both "shop" and "backend"

Performance

hypermatch is designed to be blazing fast with very large numbers of rules. Nevertheless, there are a few things to consider to get maximum performance:

• Shorten the number of fields inside the rules, the fewer conditions, the shorter is the path to find them out.
• Try to make the paths as diverse as possible in events and rules. The more heterogeneous fields, the higher the performance.
• Reduce the number of anyOf conditions wherever possible

Documentation

Overview

Package hypermatch is a high-performance Go library that enables rapid matching of a large number of rules against events. Designed for speed and efficiency, hypermatch handles thousands of events per second with low latency.

Index

• func ValidateRule(set ConditionSet) error
• type Condition
  • func (c Condition) MarshalJSON() ([]byte, error)
  • func (c *Condition) UnmarshalJSON(data []byte) error
• type ConditionSet
  • func (c ConditionSet) MarshalJSON() ([]byte, error)
  • func (c *ConditionSet) UnmarshalJSON(data []byte) error
• type HyperMatch
  • func NewHyperMatch() *HyperMatch
  • func (m *HyperMatch) AddRule(id RuleIdentifier, conditionSet ConditionSet) error
  • func (m *HyperMatch) Match(properties []Property) []RuleIdentifier
• type Pattern
  • func (p Pattern) MarshalJSON() ([]byte, error)
  • func (p *Pattern) UnmarshalJSON(data []byte) error
• type PatternType
  • func PatternTypeFromString(input string) PatternType
  • func (p PatternType) AllValues() []PatternType
  • func (p PatternType) HasLiteralValue() bool
  • func (p PatternType) String() string
• type Property
• type RuleIdentifier

Functions

func ValidateRule

func ValidateRule(set ConditionSet) error

ValidateRule validates the given condition set. Returns an error if validation fails.

Types

type Condition

type Condition struct {
	Path    string  `json:"path"`
	Pattern Pattern `json:"pattern"`
}

Condition represents a single condition inside a ConditionSet. It defines a Path (=reference to property in an event) and a Pattern to check against the value.

func (Condition) MarshalJSON

func (c Condition) MarshalJSON() ([]byte, error)

MarshalJSON marshals a Condition into an easy readable JSON object

func (*Condition) UnmarshalJSON

func (c *Condition) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshal the JSON back to a Condition

type ConditionSet

type ConditionSet []Condition

ConditionSet represents a rule and consists of one or more items of type Condition

func (ConditionSet) MarshalJSON

func (c ConditionSet) MarshalJSON() ([]byte, error)

MarshalJSON marshals a ConditionSet into an easy readable JSON object

func (*ConditionSet) UnmarshalJSON

func (c *ConditionSet) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshal the JSON back to a ConditionSet

type HyperMatch

type HyperMatch struct {
	// contains filtered or unexported fields
}

func NewHyperMatch

func NewHyperMatch() *HyperMatch

func (*HyperMatch) AddRule

func (m *HyperMatch) AddRule(id RuleIdentifier, conditionSet ConditionSet) error

AddRule adds a new rule to the HyperMatch instance.

The rule is defined by a unique identifier and a set of conditions. The conditions define the properties that must be matched in order for the rule to be triggered.

If the rule is successfully added, the function returns nil. Otherwise, an error is returned.

func (*HyperMatch) Match

func (m *HyperMatch) Match(properties []Property) []RuleIdentifier

Match takes a list of properties and returns a list of rule identifiers that match those properties

type Pattern

type Pattern struct {
	Type  PatternType `json:"type"`
	Value string      `json:"value,omitempty"`
	Sub   []Pattern   `json:"sub,omitempty"`
}

Pattern defines how a value should be compared. It consists of a Type and either a Value or Sub-patterns depending on the used Type.

func (Pattern) MarshalJSON

func (p Pattern) MarshalJSON() ([]byte, error)

MarshalJSON marshals a Pattern into an easy readable JSON object

func (*Pattern) UnmarshalJSON

func (p *Pattern) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshal the JSON back to a Pattern

type PatternType

type PatternType int

const (
	PatternEquals PatternType = iota
	PatternPrefix
	PatternSuffix
	PatternWildcard

	PatternAnythingBut
	PatternAnyOf
	PatternAllOf

	PatternUnknown
)

func PatternTypeFromString

func PatternTypeFromString(input string) PatternType

func (PatternType) AllValues

func (p PatternType) AllValues() []PatternType

func (PatternType) HasLiteralValue

func (p PatternType) HasLiteralValue() bool

func (PatternType) String

func (p PatternType) String() string

type Property

type Property struct {
	Path   string
	Values []string
}

Property represents a property and a slice of values inside an event. An Event is defined as a slice of Property objects.

type RuleIdentifier

type RuleIdentifier any

RuleIdentifier is a type alias to represent the identifier for a rule, so that the user of hypermatch get identify which of the rules matches an event.