elastic_proxy

package
v0.0.0-...-0c02112 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2023 License: AGPL-3.0 Imports: 20 Imported by: 0

README

Converting Elastic queries into Sneller SQL

Elastic queries are JSON objects that have a lot of functionality and options. Most basic queries can be translated into Sneller SQL, but it sometimes requires multiple queries.

Filter queries

Elastic supports a wide variety of query filters to reduce the amount of data that needs to be processed. The Elastic Proxy currently supports a subset of the queries and will throw an error if an unsupported query filter is used.

Effectively the query filter defines the WHERE clause of the SQL statement. It's important to filter the data to reduce the amount of data that needs to be scanned. Elastic also supports a Lucene-style query-string that is also supported.

Aggregations

Elasticsearch organizes aggregations into three categories:

  • Metric aggregations that calculate metrics, such as a sum or average, from field values.
  • Bucket aggregations that group documents into buckets, also called bins, based on field values, ranges, or other criteria.
  • Pipeline aggregations that take input from other aggregations instead of documents or fields.

Pipeline aggregations are currently not supported, so we'll focus on bucket and metric aggregations.

Bucket aggregations

There are a lot of bucket aggregations in Elastic, but currently only the following aggregations are supported:

Filter and Filters aggregation

These aggregations result in a single bucket and create a bucket that contains the result of the aggregation on the filtered data.

{
  "count": 0,
  "aggs": {
    "bucketTShirts": {
      "filter": { "term": { "type": "t-shirt" } }
    }
  }
}

translates to:

SELECT COUNT(*) FROM "table" WHERE "type" = 't-shirt'

The filters aggregation is functionally identical to the filter aggregation, but it creates a bucket for each filter, so:

{
  "size": 0,
  "aggs" : {
    "messages" : {
      "filters" : {
        "filters" : {
          "errors" :   { "match" : { "body" : "error"   }},
          "warnings" : { "match" : { "body" : "warning" }}
        }
      }
    }
  }
}

translates to two SQL queries:

SELECT COUNT(*) FROM "table" WHERE "body" = 'error'
SELECT COUNT(*) FROM "table" WHERE "body" = 'warning'
Terms and Multi terms aggregations

Terms bucket aggregations are essentially the GROUP BY functionality. The most common bucket aggregation is a terms aggregation that returns a list of buckets for each value in the GROUP BY, so

{
  "count": 0,
  "aggs": {
    "bucketOrigin": {
      "terms": { "field": "OriginCountry" }
    }
  }
}

translates to:

SELECT "OriginCountry", COUNT(*) FROM "table" GROUP BY "OriginCountry"

Note that Elastic always returns a document count for each bucket, so we always need to include COUNT(*) in the projection. The multi terms aggregation is functionally identical, but it allows to specify multiple fields to group by.

Bucket aggregations can be nested, so it's valid to use this:

{
  "count": 0,
  "aggs": {
    "bucketOrigin": {
      "terms": { "field": "OriginCountry" },
      "aggs": {
        "bucketDest": {
          "terms": { "field": "DestCountry" }
        }
      }
    }
  }
}

This translates to the following two queries:

SELECT "OriginCountry", COUNT(*) FROM "table" GROUP BY "OriginCountry"
SELECT "OriginCountry", "DestCountry", COUNT(*) FROM "table" GROUP BY "OriginCountry", "DestCountry"

Note that this first query is obsolete, because the OriginCountry elements and the count per OriginCountry can be calculated during postprocessing, based on the second query. We'll get to these optimizations later.

Instead of using a nested terms aggregation, it would also have been possible to use a single multi terms aggregation instead. The difference is subtle, but the multi terms aggregation returns a single array of buckets. The nested terms aggregation also returns a nested array of buckets.

Metric aggregations

There are a lot of metric aggregations in Elastic, but currently only the following aggregations are supported:

  • Min, Max and Avg return the min/max/average value.
  • Value count returns the number of values. Note that this may be different than the doc_count value, because the value count counts the number of times that the value was actually in the data. If a field doesn't exist in a particular record, then the record is counted in doc_count, but the record won't increase the value count.
  • Cardinality returns the number of unique items.

These metrics aggregations can be translated into SQL using the basic SQL functions: MIN(field) (min), MAX(field) (max), AVG(field) (avg), COUNT(field) (value count) and COUNT(DISTINCT field) (cardinality). Metric aggregations can be used as a top-level aggregation, such as:

{
  "count": 0,
  "aggs": {
    "minPrice": { "min": { "field": "price" } },
    "avgPrice": { "avg": { "field": "price" } },
    "maxPrice": { "max": { "field": "price" } }
  }
}

translates to:

SELECT COUNT(*), MIN(price), AVG(price), MAX(price) FROM "table"

Note that COUNT(*) is always added to return the doc_count per bucket.

Metrics aggregations can be used as a nested aggregation, but it's not possible to nest an aggregation under a metrics aggregation. When nested, the translation is a bit different:

{
  "count": 0,
  "aggs": {
    "bucketOrigin": {
      "terms": { "field": "product" },
      "aggs": {
        "minPrice": { "min": { "field": "price" } },
        "avgPrice": { "avg": { "field": "price" } },
        "maxPrice": { "max": { "field": "price" } }
      }
    }
  }
}

translates to:

SELECT "product", COUNT(*), MIN(price), AVG(price), MAX(price) FROM "table" GROUP BY "product"

Metric aggregations can be added to the bucket aggregation query, so it's fairly easy to implement.

Cardinality aggregation

The cardinality aggregation using COUNT(DISTINCT field) and (for now) can't be combined with other aggregations. Although the Sneller VM supports multiple hash-sets, it's currently not able to combine them with other metrics. So the following SQL query will return an error:

SELECT COUNT(*), COUNT(DISTINCT "price") FROM "table" GROUP BY "product"

For now, this is solved by creating two queries:

SELECT COUNT(*) FROM "table" GROUP BY "product"
SELECT COUNT(DISTINCT "price") FROM "table" GROUP BY "product"

Optimizing query execution

The basic Elastic proxy translates all aggregations into separate queries. Consider the following Elastic query:

{
  "count": 0,
  "aggs": {
    "categories": {
      "terms": { "field": "category" },
      "avgPrice": { "avg": { "field": "price" } },
      "aggs": {
        "products": {
          "terms": { "field": "product" },
          "aggs": {
            "minPrice": { "min": { "field": "price" } },
            "avgPrice": { "avg": { "field": "price" } },
            "maxPrice": { "max": { "field": "price" } },
            "prices": { "cardinality": { "field": "price" } }
          }
        }
      }
    }
  }
}

This translates to the following queries:

SELECT "category", COUNT(*)                           FROM "table" GROUP BY "category"
SELECT "category", AVG("price")                       FROM "table" GROUP BY "category"
SELECT "category", "product", COUNT(*)                FROM "table" GROUP BY "category", "product"
SELECT "category", "product", MIN("price")            FROM "table" GROUP BY "category", "product"
SELECT "category", "product", AVG("price")            FROM "table" GROUP BY "category", "product"
SELECT "category", "product", MAX("price")            FROM "table" GROUP BY "category", "product"
SELECT "category", "product", COUNT(DISTINCT "price") FROM "table" GROUP BY "category", "product"

Because most operators can be combined, this can be reduced to the following three queries:

SELECT "category", COUNT(*), AVG("price") FROM "table" GROUP BY "category"
SELECT "category", "product", COUNT(*), MIN("price"), AVG("price"), MAX("price") FROM "table" GROUP BY "category", "product"
SELECT "category", "product", COUNT(DISTINCT "price") FROM "table" GROUP BY "category", "product"

If the average price at the category level isn't required, then the first query can be completely eliminated and the doc_count per category could be determined from the output of the second aggregation.

The Sneller SQL engine is more efficient when all queries are executed in a single roundtrip. Instead of sending three separate queries, it will send the following query:

SELECT
 (SELECT "category", COUNT(*), AVG("price") FROM "table" GROUP BY "category") AS "q1",
 (SELECT "category", "product", COUNT(*), MIN("price"), AVG("price"), MAX("price") FROM "table" GROUP BY "category", "product") AS "q2",
 (SELECT "category", "product", COUNT(DISTINCT "price") FROM "table" GROUP BY "category", "product") AS "q3"

Another advantage of this approach is that the query runs with the same set of data during each query. When the queries are fired sequentially, then data might have been changed between two queries and give inconsistent results.

Combining SQL results back to Elastic responses

The SQL results return all the required data, but it's not in the format that is used by the Elastic Proxy. The returned PartiQL data needs to be translated back into an Elastic response. To make sure that this is possible, all the SQL queries are annotated, so the results can be related to the original Elastic query.

Instead of using the following queries:

SELECT
 (SELECT "category", COUNT(*), AVG("price") FROM "table" GROUP BY "category") AS "q1",
 (SELECT "category", "product", COUNT(*), MIN("price"), AVG("price"), MAX("price") FROM "table" GROUP BY "category", "product") AS "q2",
 (SELECT "category", "product", COUNT(DISTINCT "price") FROM "table" GROUP BY "category", "product") AS "q3"

It will be sent like this:

SELECT
(
  SELECT "category" AS "$key:categories%0",
         COUNT(*) AS "doc_count",
         AVG("price") AS "avgPrice"
  FROM "table"
  GROUP BY "category"
) AS "$bucket:categories%0",
(
  SELECT "category" AS "$key:categories%0",
         "product" AS "$key:categories:products%0",
         COUNT(*) AS "doc_count",
         MIN("price") AS "minPrice",
         AVG("price") AS "avgPrice",
         MAX("price") AS "maxPrice"
  FROM "table"
  GROUP BY "category", "product"
) AS "$bucket:categories:products%0",
(
  SELECT "category" AS "$key:categories%0",
         "product" AS "$key:categories:products%0",
         COUNT(DISTINCT "price") AS "prices"
  FROM "table"
  GROUP BY "category", "product"
) AS "$bucket:categories:products%1"

Each query runs for a particular bucket aggregation, but a bucket aggregation can run multiple queries. Each SELECT statement is assigned the name $bucket:<aggName>[:<aggName>]*%<index>. If a bucket aggregation uses multiple queries, then they only differ by the index number.

The first step is to combine all results per bucket aggregation. In the example above, the data of $bucket:categories:products%0 and $bucket:categories:products%0 can be merged into a single "table". These tables are merged based on the $key:<column>%<index> values.

The second step is to fill produce "missing" data. The following Elastic query:

{
  "count": 0,
  "aggs": {
    "categories": {
      "terms": { "field": "category" },
      "aggs": {
        "products": {
          "terms": { "field": "product" }
        }
      }
    }
  }
}

Translates to the following SQL query:

SELECT
(
  SELECT "category" AS "$key:categories%0",
         "product" AS "$key:categories:products%0",
         COUNT(*) AS "doc_count"
  FROM "table"
  GROUP BY "category", "product"
) AS "$bucket:categories:products%0"

The calculation of the COUNT(*) for the top-level aggregation was optimized, because it can be calculated from the nested query. So if the result processing detects a nested bucket without results for the bucket above, it will bottom-up produce the top-level data.

The last step is to convert the actual bucket data into the Elastic JSON response by combining the proper buckets and nested buckets.

Documentation

Overview

Package elastic_proxy provides facilities to translate queries for Elastic Search JSON and SQL-style queries for Sneller engine.

Index

Constants

View Source 
const (
	TotalCountBucket  = "$total_count"
	HitsBucket        = "$hits"
	KeyPrefix         = "$key"
	BucketPrefix      = "$bucket"
	DocCount          = "$doc_count"
	DefaultSource     = "$source"
	SourceAliasPrefix = "$source:"
)
View Source 
const (
	LatExt = ".lat"
	LonExt = ".lon"
)
View Source 
const (
	OrderAscending  = "ASC"
	OrderDescending = "DESC"
)
View Source 
const DefaultElasticUser = "elastic"
View Source 
const (
	DefaultHTTPTimeout time.Duration = 30 * time.Second
)
View Source 
const DummyAlias = "$dummy$"

Variables

View Source 
var (
	ErrNoFieldName          = errors.New("no fieldname set")
	ErrInvalidRegexOperator = errors.New("invalid regex operator")
)
View Source 
var (
	ErrNotSupported        = errors.New("unsupported element")
	ErrUnsupportedNumber   = errors.New("unsupported number")
	ErrTermOnlySingleField = errors.New("term supports only a single field")
)
View Source 
var DefaultTrackTotalHits = TrackTotalHits{
	Enabled: true,
	Limit:   10000,
}
View Source 
var (
	ErrInvalidUnit = errors.New("invalid unit")
)
View Source 
var (
	ErrUnsupportedLiteralType = errors.New("unsupported literal type")
)

Functions

func ConvertION 

func ConvertION(v any) any

func ExecuteQuery 

func ExecuteQuery(client *http.Client, u *url.URL, token, sql string) (*http.Response, error)

func NewElasticFloat 

func NewElasticFloat(v any) (elasticFloat, error)

func ParseDateMath 

func ParseDateMath(data string, now time.Time) (time.Time, error)

func ParseExprFieldName 

func ParseExprFieldName(qc *QueryContext, fieldName string) *exprFieldName

func ParseExprFieldNameAlias 

func ParseExprFieldNameAlias(qc *QueryContext, alias, fieldName string) *exprFieldNameAlias

func ParseExprFieldNameParts 

func ParseExprFieldNameParts(qc *QueryContext, fields []string) *exprFieldName

func ParseExprSourceName 

func ParseExprSourceName(qc *QueryContext, source string) *exprSourceName

func ParseExprSourceNameWithAlias 

func ParseExprSourceNameWithAlias(qc *QueryContext, source, alias string) *exprSourceName

func PrintExpr 

func PrintExpr(e expression) string

func PrintExprPretty 

func PrintExprPretty(e expression) string

Types

type AggregationStrategy 

type AggregationStrategy int
const (
	//
	// Perform single scan over the data while
	// building full tree for all aggregation
	// levels and prune back when done.
	// (Consumes more memory and typically
	//  faster)
	SingleScan AggregationStrategy = iota
	//
	// Perform more than one scan and prune
	// back interim results after each pass.
	// (Consumes more memory and typically
	//  slower)
	MultiScan
)

type ElasticJSON 

type ElasticJSON struct {
	From           *int                   `json:"from"`
	Size           *int                   `json:"size"`
	Aggregations   map[string]aggregation `json:"aggs"`
	Sort           []SortField            `json:"sort"`
	Query          *Query                 `json:"query"`
	Version        *bool                  `json:"version"` // indicates it the version should be included in the hit
	Source         *source                `json:"_source"` // indicates if source record should be included in the hit
	Fields         []projectedField       `json:"fields"`
	TrackTotalHits *TrackTotalHits        `json:"track_total_hits"`
}

func (*ElasticJSON) ConvertResult 

func (ej *ElasticJSON) ConvertResult(qc *QueryContext, snellerResult map[string]any) (*ElasticResult, map[string]any, error)

func (*ElasticJSON) SQL 

func (ej *ElasticJSON) SQL(qc *QueryContext) (*exprSelect, error)

type ElasticMapping 

type ElasticMapping struct {
	Properties Properties `json:"properties"`
}

ElasticMapping represents type mappings used by ElasticSearch

func DataShapeToElasticMapping 

func DataShapeToElasticMapping(fields map[string]any) *ElasticMapping

DataShapeToElasticMapping translates raw 'fields' output from SNELLER_DATASHAPE into Elastic's Mapping structure.

type ElasticResult 

type ElasticResult struct {
	TimedOut     bool                `json:"timed_out"`
	Hits         *elasticResultHits  `json:"hits"`
	Shards       elasticResultShards `json:"_shards"`
	Count        *int64              `json:"count,omitempty"` // only for Counting API
	Took         int                 `json:"took"`
	Aggregations *map[string]any     `json:"aggregations,omitempty"`
}

type JSONLiteral 

type JSONLiteral struct {
	// string, bool or float64
	Value any
}

func NewJSONLiteral 

func NewJSONLiteral(v any) (JSONLiteral, error)

func (*JSONLiteral) String 

func (jl *JSONLiteral) String() string

func (*JSONLiteral) UnmarshalJSON 

func (jl *JSONLiteral) UnmarshalJSON(data []byte) error

type MappingValue 

type MappingValue struct {
	Type       string     `json:"type"`
	Indexed    bool       `json:"indexed,omitempty"`
	Properties Properties `json:"properties,omitempty"`
}

type Operator 

type Operator string
const (
	OperatorOr  Operator = "or"
	OperatorAnd Operator = "and"
)

type Ordering 

type Ordering string

func (*Ordering) UnmarshalJSON 

func (o *Ordering) UnmarshalJSON(data []byte) error

type Properties 

type Properties map[string]MappingValue

type Query 

type Query struct {
	// https://www.elastic.co/guide/en/elasticsearch/reference/current/compound-queries.html
	Bool          *boolean       `json:"bool"`           // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html
	Boosting      *notSupported  `json:"boosting"`       // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-boosting-query.html
	ConstantScore *constantScore `json:"constant_score"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-constant-score-query.html
	DisMax        *notSupported  `json:"dis_max"`        // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-dis-max-query.html
	FunctionScore *notSupported  `json:"function_score"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/full-text-queries.html
	Intervals         *notSupported     `json:"intervals"`           // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-intervals-query.html
	Match             *match            `json:"match"`               // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html
	MatchBoolPrefix   *map[string]field `json:"match_bool_prefix"`   // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-bool-prefix-query.html
	MatchPhrase       *matchPhrase      `json:"match_phrase"`        // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query-phrase.html
	MatchPhrasePrefix *map[string]field `json:"match_phrase_prefix"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query-phrase-prefix.html
	CombinedFields    *map[string]field `json:"combined_fields"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-combined-fields-query.html
	MultiMatch        *map[string]field `json:"multi_match"`         // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html
	QueryString       *QueryString      `json:"query_string"`        // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
	SimpleQueryString *notSupported     `json:"simple_query_string"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-queries.html
	GeoBoundingBox *geoBoundingBox `json:"geo_bounding_box"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-bounding-box-query.html
	GeoDistance    *notSupported   `json:"geo_distance"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-query.html
	GeoPolygon     *notSupported   `json:"geo_polygon"`      // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-polygon-query.html
	GeoShape       *notSupported   `json:"geo_shape"`        // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/shape-queries.html
	Shape *notSupported `json:"shape"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-shape-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/joining-queries.html
	Nested    *notSupported `json:"nested"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-nested-query.html
	HasChild  *notSupported `json:"has_child"`  // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-has-child-query.html
	HasParent *notSupported `json:"has_parent"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-has-parent-query.html
	ParentID  *notSupported `json:"parent_id"`  // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-parent-id-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html
	MatchAll  *matchAll  `json:"match_all"`  // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html#query-dsl-match-all-query
	MatchNone *matchNone `json:"match_none"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html#query-dsl-match-none-query

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/span-queries.html
	SpanContaining   *notSupported `json:"span_containing"`    // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-containing-query.html
	SpanFieldMasking *notSupported `json:"span_field_masking"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-field-masking-query.html
	SpanFirst        *notSupported `json:"span_first"`         // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-first-query.html
	SpanMulti        *notSupported `json:"span_multi"`         // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-multi-term-query.html
	SpanNear         *notSupported `json:"span_near"`          // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-near-query.html
	SpanNot          *notSupported `json:"span_not"`           // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-not-query.html
	SpanOr           *notSupported `json:"span_or"`            // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-or-query.html
	SpanTerm         *notSupported `json:"span_term"`          // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-term-query.html
	SpanWithin       *notSupported `json:"span_within"`        // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-within-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/specialized-queries.html
	DistanceFeature *notSupported `json:"distance_feature"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-distance-feature-query.html
	MoreLikeThis    *notSupported `json:"more_like_this"`   // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-mlt-query.html
	Percolate       *notSupported `json:"percolate"`        // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-percolate-query.html
	RankFeature     *notSupported `json:"rank_feature"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-rank-feature-query.html
	Script          *notSupported `json:"script"`           // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-query.html
	ScriptScore     *notSupported `json:"script_score"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html
	Wrapper         *notSupported `json:"wrapper"`          // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-wrapper-query.html
	Pinned          *notSupported `json:"pinned"`           // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-pinned-query.html

	// https://www.elastic.co/guide/en/elasticsearch/reference/current/term-level-queries.html
	Exists   *exists       `json:"exists"`    // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
	Fuzzy    *notSupported `json:"fuzzy"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-fuzzy-query.html
	Prefix   *notSupported `json:"prefix"`    // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-ids-query.html
	Range    *ranges       `json:"range"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html
	Regexp   *notSupported `json:"regexp"`    // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-regexp-query.html
	Term     *term         `json:"term"`      // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html
	Terms    *terms        `json:"terms"`     // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html
	TermsSet *notSupported `json:"terms_set"` // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-set-query.html
	Wildcard *wildCard     `json:"wildcard"`  // https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html
}

Query implements the parsing and expression logic for the Elastic Query.

The parser canonicalizes all the input, so all deprecated or shortened representations should be handled by the `UnmarshalJSON` method.

func (*Query) Expression 

func (q *Query) Expression(qc *QueryContext) (expression, error)

Expression obtains the full expression (that can be converted) to a SQL WHERE clause for the query

type QueryContext 

type QueryContext struct {
	Query                  ElasticJSON
	Index                  string
	TableSources           []TableSource
	Sources                []expression
	IgnoreTotalHits        bool
	IgnoreSumOtherDocCount bool
	TypeMapping            map[string]TypeMapping
}

type QueryString 

type QueryString struct {
	Query                           string      `json:"query"`
	DefaultField                    *string     `json:"default_field"`
	AllowLeadingWildcard            *bool       `json:"allow_leading_wildcard"`
	AnalyzeWildcard                 *bool       `json:"analyze_wildcard"`
	Analyzer                        *string     `json:"analyzer"`
	AutoGenerateSynonymsPhraseQuery *bool       `json:"auto_generate_synonyms_phrase_query"`
	Boost                           *boostValue `json:"boost"`
	DefaultOperator                 *string     `json:"default_operator"`
	EnablePositionIncrements        *bool       `json:"enable_position_increments"`
	Fields                          *[]string   `json:"fields"`
	Fuzziness                       *string     `json:"fuzziness"`
	FuzzyMaxExpansions              *int        `json:"fuzzy_max_expansions"`
	FuzzyPrefixLength               *int        `json:"fuzzy_prefix_length"`
	FuzzyTranspositions             *bool       `json:"fuzzy_transpositions"`
	Lenient                         *bool       `json:"lenient"`
	MaxDeterminizedStates           *int        `json:"max_determinized_states"`
	MinimumShouldMatch              *string     `json:"minimum_should_match"`
	QuoteAnalyzer                   *string     `json:"quote_analyzer"`
	PhraseSlop                      *int        `json:"phrase_slop"`
	QuoteFieldSuffix                *string     `json:"quote_field_suffix"`
	Rewrite                         *string     `json:"rewrite"`
	TimeZone                        *string     `json:"time_zone"`
}

func (*QueryString) Expression 

func (qs *QueryString) Expression(qc *QueryContext) (expression, error)

type Range 

type Range struct {
	GreaterThan          *JSONLiteral `json:"gt"`
	GreaterThanOrEqualTo *JSONLiteral `json:"gte"`
	LessThan             *JSONLiteral `json:"lt"`
	LessThanOrEqualTo    *JSONLiteral `json:"lte"`
	Format               *string      `json:"format"`
	Relation             *string      `json:"relation"`
	TimeZone             *string      `json:"time_zone"`
	Boost                *boostValue  `json:"boost"`
}

func (*Range) UnmarshalJSON 

func (r *Range) UnmarshalJSON(data []byte) error

type SortField 

type SortField struct {
	Field  string
	Format string
	Order  Ordering
}

func (*SortField) MarshalJSON 

func (sf *SortField) MarshalJSON() ([]byte, error)

func (*SortField) UnmarshalJSON 

func (sf *SortField) UnmarshalJSON(data []byte) error

type TableSource 

type TableSource struct {
	Database, Table string
}

type TrackTotalHits 

type TrackTotalHits struct {
	Enabled bool  // set if hits need to be returned
	Limit   int64 // max nr. of items (-1: exact count)
}

func (*TrackTotalHits) UnmarshalJSON 

func (sf *TrackTotalHits) UnmarshalJSON(data []byte) error

type TypeMapping 

type TypeMapping struct {
	Type   string            `json:"type"`
	Fields map[string]string `json:"fields,omitempty"`
}

func (*TypeMapping) UnmarshalJSON 

func (tm *TypeMapping) UnmarshalJSON(data []byte) error

type ZeroTermsQuery 

type ZeroTermsQuery string
const (
	ZeroTermsQueryNone ZeroTermsQuery = "none"
	ZeroTermsQueryAll  ZeroTermsQuery = "all"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.