golastic

package module
v0.0.0-...-912a459 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 28, 2020 License: MIT Imports: 11 Imported by: 6

README

Build Status Go Report Card GoDoc GitHub license

Golastic is meant to be a simple and intuitive programmatic query builder implementation for Elasticsearch. It intends to provide a convenient and fluent interface for creating and running Elasticsearch queries as well as for performing different indices operations.

Getting Started

To start using this package in your application simply run: go get github.com/alejandro-carstens/golastic

Usage

Establish a connection:


import (
    "os"
    
    "github.com/alejandro-carstens/golastic"
)

func main() {
	connection := golastic.NewConnection(
		&golastic.ConnectionContext{
			Urls:                []string{os.Getenv("ELASTICSEARCH_URI")},
			Password:            os.Getenv("ELASTICSEARCH_PASSWORD"),
			Username:            os.Getenv("ELASTICSEARCH_USERNAME"),
			HealthCheckInterval: 30,
		},
	)

        if err := connection.Connect(); err != nil {
                // Handle error
        }
  
        // Do something else here
}

Create a builder:

	doc := &Example{
		Id:          "awesome_unique_id",
		Description: "This is an awesome description",
	}
	
	builder := connection.Builder("your_index")
	
	response, err := builder.Insert(doc)
	
	// Handle response and error

Create an indexer:

	config := map[string]interface{}{
		"settings": // settings...,
		"mappings": // mappings...,
	}
	
	schema, err := json.Marshal(config)
	
	if err != nil {
		// Handle error
	}
	
	options := &golastic.IndexOptions{
		WaitForCompletion: true,
		IgnoreUnavailable: true,
		// More options...
	}
	
	indexer := connection.Indexer(options)
	
	if err := indexer.CreateIndex("your_index", string(schema)); err != nil {
		// Handle error
	}

Using the Indexer

Please checkout the godoc Indexer section for more detailed documentation on the Indexer API.

Building Queries

Golastic provides the following clauses for building queries:

Where

Where clauses map to must + term queries in Elasticsearch, meaning that there will be a look up for the exact search term on an inverted index

  • Where (=, <>, >, <, <=, >=)
  • WhereIn
  • WhereNotIn
	players := []interface{}{"player1", "player2", "palyer3"}
	games := []interface{}{"game4", "game5"}
	
	builder := connection.Builder("your_index")
	
	builder.Where("level", ">", 2).WhereIn("player", players).WhereNotIn("game", games)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
Match

Match clauses map to must + match queries in Elasticsearch, which means that an analyzer will be applied to the search term and will therefore try to match what is stored on a given index

  • Match (=, <>)
  • MatchIn
  • MatchNotIn
	players := []interface{}{"player1", "player2", "palyer3"}
	games := []interface{}{"game4", "game5"}
	
	builder := connection.Builder("your_index")
	
	builder.Match("level", "<>", 7).MatchIn("player", players).MatchNotIn("game", games)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
Filter

Filter clauses map to filter + term queries in Elasticsearch. Filter queries do not use the _score field for returned results, they just return the results that match the query criteria

  • Filter (=, <>, >, <, <=, >=)
  • FilterIn
	players := []interface{}{"player1", "player2", "palyer3"}
	
	builder := connection.Builder("your_index")
	
	builder.Filter("level", ">=", 7).FilterIn("player", players)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
Nested

Golastic provides the ability to perform nested queries by using all the previous clauses nested counterparts WhereNested, WhereInNested, WhereNotInNested, FilterNested, FilterInNested, MatchNested, MatchInNested & MatchNotInNested. Nested clauses are subjected to the same rules as their non-nested counter parts. However, it is important to specify the nested path using dot notation such as attribute.value.

	players := []interface{}{"player1", "player2", "palyer3"}
	
	builder := connection.Builder("your_index")
	
	builder.FilterNested("video_game.level", ">=", 7).WhereNotInNested("video_game.player", players)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
From

From clauses set the offset from which the query will return documents

	players := []interface{}{"player1", "player2", "palyer3"}
	
	builder := connection.Builder("your_index")
	
	builder.Filter("level", ">=", 7).OrderBy("player", true).From(5).Limit(5)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
Limit

Limit clauses set the limit for the maximum number of documents to be returned

	players := []interface{}{"player1", "player2", "palyer3"}
	
	builder := connection.Builder("your_index")
	
	builder.Filter("level", ">=", 7).WhereIn("player", players).Limit(10)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
OrderBy

OrderBy clauses set the sorting order in which the documents need to be returned. Use true for ascending and false for descending.

	builder := connection.Builder("your_index")
	
	builder.Filter("level", ">=", 7).OrderBy("player", true).Limit(10)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
OrderByNested

OrderByNested clauses allows for sorting by nested fields. As its non-nested counter part please use true for ascending and false for descending.

	players := []interface{}{"player1", "player2", "palyer3"}
	
	builder := connection.Builder("your_index")
	
	builder.Filter("level", ">=", 7).OrderByNested("attributes.price", true).Limit(10)
	
	response := []Response{} // It can also be map[string]interface{}{}
	
	if err := builder.Get(&response); err != nil {
		// Handle error
	}
GroupBy

The GroupBy clause is used for performing aggregations. This clause won't have any effect on a Get, Execute, or Destroy query. It will only produce results when an Aggregate query is issued. This clause makes it is possible to aggregate by one or more paramters. The first parameter will be the main aggregation and each subsequent field will become a sub-aggregation of the previous aggregation. It is important to mention that aggregations for nested fields have not been yet implemented by Golastic. Please refer to the following example for a better understanding.

	builder.
		WhereIn("rating", []interface{}{"R"}).
		WhereNested("cast.director", "=", "James Cameron").
		GroupBy("rating", "views")
		
	aggregations, err := builder.Aggregate()
	
	if err != nil {
		// Handle error
	}

Example of the aggregations response:

{  
   "rating":{  
      "doc_count_error_upper_bound":0,
      "sum_other_doc_count":0,
      "buckets":[  
         {  
            "key":"R",
            "doc_count":330,
            "items":{  
               "views":{  
                  "doc_count_error_upper_bound":0,
                  "sum_other_doc_count":0,
                  "buckets":[  
                     {  
                        "key":500000,
                        "doc_count":330,
                        "items":null
                     }
                  ]
               }
            }
         }
      ]
   }
}

Using the Builder to Execute Queries

Please refer to the godoc Builder section for detailed documentation of the methods available to run queries. For further reference on functionality please look at the examples folder or take a look at the tests.

Contributing

By using this package you are already contributing, if you would like to go a bit further simply give the project a star. Otherwise, find an area you can help with and do it. Open source is about collaboration and open participation. Try to make your code look like what already exists or hopefully better and submit a pull request. Also, if you have any ideas on how to make the code better or on improving its scope and functionality please raise an issue and I will do my best to address it in a timely manner.

Liscense

MIT.

Documentation

Index

Constants

View Source 
const CONCURRENT_BATCH int = 10

CONCURRENT_BATCH is the default concurrent response proccesing batch size

View Source 
const LIMIT int = 10000

LIMIT is the default limit of documents to be returned by elasticsearch

View Source 
const VALUE string = "value"

VALUE self EXPLANATORY

View Source 
const VALUE_AS_STRING string = "value_as_string"

VALUE_AS_STRING self explanatory

Variables

This section is empty.

Functions

This section is empty.

Types

type AggregationBucket 

type AggregationBucket struct {
	Key      interface{}                     `json:"key"`
	DocCount int                             `json:"doc_count"`
	Items    map[string]*AggregationResponse `json:"items"`
}

AggregationBucket represents a bucket within an AggregationResponse

type AggregationResponse 

type AggregationResponse struct {
	DocCountErrorUpperBound int                  `json:"doc_count_error_upper_bound"`
	SumOtherDocCount        int                  `json:"sum_other_doc_count"`
	Buckets                 []*AggregationBucket `json:"buckets"`
}

AggregationResponse represents an aggregation's query response

func (*AggregationResponse) ToGabsContainer 

func (ab *AggregationResponse) ToGabsContainer() (*gabs.Container, error)

ToGabsContainer converts a response to a *gabs.Container instance

type AggregationResponses 

type AggregationResponses map[string]*AggregationResponse

AggregationResponses represents a map for *AggregationResponse

func (*AggregationResponses) ToGabsContainer 

func (ar *AggregationResponses) ToGabsContainer() (*gabs.Container, error)

ToGabsContainer converts a response to a *gabs.Container instance

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder represents the struct in charge of building and executing elasticsearch queries

func (*Builder) Aggregate 

func (b *Builder) Aggregate() (map[string]*AggregationResponse, error)

Aggregate retrieves all the queries aggregations

func (*Builder) AggregateRaw 

func (b *Builder) AggregateRaw() (*gabs.Container, error)

AggregateRaw returns raw aggregation results

func (*Builder) CancelTask 

func (b *Builder) CancelTask(taskId string) (*gabs.Container, error)

CancelTask cancels a task provided a taskId

func (*Builder) Clear 

func (qb *Builder) Clear() *queryBuilder

func (*Builder) ClearScroll 

func (b *Builder) ClearScroll() error

ClearScroll cancel's the current scroll operation

func (*Builder) Client 

func (b *Builder) Client() *elastic.Client

Client returns an instance of *elastic.Client

func (*Builder) Count 

func (b *Builder) Count() (int64, error)

Count retrieves the number of elements that match the query

func (*Builder) Cursor 

func (b *Builder) Cursor(offset int, sortValues []interface{}, items interface{}) ([]interface{}, error)

Cursor paginates based on searching after the last returned sortValues

func (*Builder) Delete 

func (b *Builder) Delete(ids ...string) (*gabs.Container, error)

Delete deletes one or multiple documents by id from the corresponding elasticsearch index

func (*Builder) Destroy 

func (b *Builder) Destroy() (*gabs.Container, error)

Destroy executes a delete by query

func (*Builder) DestroyAsync 

func (b *Builder) DestroyAsync() (*gabs.Container, error)

DestroyAsync executes a delete by query asynchronously

func (*Builder) Execute 

func (b *Builder) Execute(params map[string]interface{}) (*gabs.Container, error)

Execute executes an update by query

func (*Builder) ExecuteAsync 

func (b *Builder) ExecuteAsync(scrollSize int, params map[string]interface{}) (*gabs.Container, error)

ExecuteAsync executes an update by query asynchronously

func (*Builder) Filter 

func (qb *Builder) Filter(field string, operand string, value interface{}) *queryBuilder

func (*Builder) FilterIn 

func (qb *Builder) FilterIn(field string, values []interface{}) *queryBuilder

func (*Builder) FilterInNested 

func (qb *Builder) FilterInNested(field string, values []interface{}) *queryBuilder

func (*Builder) FilterNested 

func (qb *Builder) FilterNested(field string, operand string, value interface{}) *queryBuilder

func (*Builder) Find 

func (b *Builder) Find(id string, item interface{}) error

Find retrieves an instance of a model for the specified Id from the corresponding elasticsearch index

func (*Builder) From 

func (qb *Builder) From(value int) *queryBuilder

func (*Builder) Get 

func (b *Builder) Get(items interface{}) error

Get executes the search query and retrieves the results

func (*Builder) GetTask 

func (b *Builder) GetTask(taskId string, waitForCompletion bool) (*gabs.Container, error)

GetTask retrieves a task given a taskId

func (*Builder) GroupBy 

func (qb *Builder) GroupBy(fields ...string) *queryBuilder

func (*Builder) InitScroller 

func (b *Builder) InitScroller(size int, scroll string) *Builder

InitScroller initializes the scroller

func (*Builder) InitSlicedScroller 

func (b *Builder) InitSlicedScroller(id, max, size int, scroll string) *Builder

InitSlicedScroller boots a sliced scroller

func (*Builder) Insert 

func (b *Builder) Insert(items ...interface{}) (*gabs.Container, error)

Insert inserts one or multiple documents into the corresponding elasticsearch index

func (*Builder) InsertWithOverwrittenId 

func (b *Builder) InsertWithOverwrittenId(items map[string]interface{}) (*gabs.Container, error)

InsertWithOverwrittenId allows to overwrite the of the given document on creation

func (*Builder) Limit 

func (qb *Builder) Limit(value int) *queryBuilder

func (*Builder) Match 

func (qb *Builder) Match(field string, operand string, value interface{}) *queryBuilder

func (*Builder) MatchIn 

func (qb *Builder) MatchIn(field string, values []interface{}) *queryBuilder

func (*Builder) MatchInNested 

func (qb *Builder) MatchInNested(field string, values []interface{}) *queryBuilder

func (*Builder) MatchNested 

func (qb *Builder) MatchNested(field string, operand string, value interface{}) *queryBuilder

func (*Builder) MatchNotIn 

func (qb *Builder) MatchNotIn(field string, values []interface{}) *queryBuilder

func (*Builder) MatchNotInNested 

func (qb *Builder) MatchNotInNested(field string, values []interface{}) *queryBuilder

func (*Builder) MatchPhrase 

func (qb *Builder) MatchPhrase(field string, operand string, value interface{}) *queryBuilder

func (*Builder) MatchPhraseIn 

func (qb *Builder) MatchPhraseIn(field string, values []interface{}) *queryBuilder

func (*Builder) MatchPhraseInNested 

func (qb *Builder) MatchPhraseInNested(field string, values []interface{}) *queryBuilder

func (*Builder) MatchPhraseNested 

func (qb *Builder) MatchPhraseNested(field string, operand string, value interface{}) *queryBuilder

func (*Builder) MatchPhraseNotIn 

func (qb *Builder) MatchPhraseNotIn(field string, values []interface{}) *queryBuilder

func (*Builder) MatchPhraseNotInNested 

func (qb *Builder) MatchPhraseNotInNested(field string, values []interface{}) *queryBuilder

func (*Builder) MinMax 

func (b *Builder) MinMax(field string, isDateField bool) (*MinMaxResponse, error)

MinMax returns the minimum and maximum values for a given field on an index

func (*Builder) OrderBy 

func (qb *Builder) OrderBy(field string, asc bool) *queryBuilder

func (*Builder) OrderByNested 

func (qb *Builder) OrderByNested(path string, order bool) *queryBuilder

func (*Builder) RawQuery 

func (b *Builder) RawQuery(query string) elastic.Query

RawQuery return an elastic raw query

func (*Builder) Scroll 

func (b *Builder) Scroll() (*gabs.Container, error)

Scroll executes the scrolling

func (*Builder) Stats 

func (qb *Builder) Stats(fields ...string) *queryBuilder

func (*Builder) Update 

func (b *Builder) Update(items ...interface{}) (*gabs.Container, error)

Update updates one or multiple documents from the corresponding elasticsearch index

func (*Builder) Where 

func (qb *Builder) Where(field string, operand string, value interface{}) *queryBuilder

func (*Builder) WhereIn 

func (qb *Builder) WhereIn(field string, values []interface{}) *queryBuilder

func (*Builder) WhereInNested 

func (qb *Builder) WhereInNested(field string, values []interface{}) *queryBuilder

func (*Builder) WhereNested 

func (qb *Builder) WhereNested(field string, operand string, value interface{}) *queryBuilder

func (*Builder) WhereNotIn 

func (qb *Builder) WhereNotIn(field string, values []interface{}) *queryBuilder

func (*Builder) WhereNotInNested 

func (qb *Builder) WhereNotInNested(field string, values []interface{}) *queryBuilder

type Connection 

type Connection struct {
	// contains filtered or unexported fields
}

Connection represents an elasticsearch connection

func NewConnection 

func NewConnection(context *ConnectionContext) *Connection

NewConnection creates an elasticsearch connection

func (*Connection) Builder 

func (c *Connection) Builder(index string) *Builder

Builder creates a new Builder

func (*Connection) Connect 

func (c *Connection) Connect() error

Connect initializes an Elastic Client

func (*Connection) Indexer 

func (c *Connection) Indexer(options *IndexOptions) *Indexer

Indexer creates a new indexer

type ConnectionContext 

type ConnectionContext struct {
	Urls                []string
	Sniff               bool
	HealthCheckInterval int64
	ErrorLogPrefix      string
	InfoLogPrefix       string
	Password            string
	Username            string
	Context             context.Context
}

ConnectionContext sets the connection configuration

type IndexOptions 

type IndexOptions struct {
	WaitForCompletion  bool
	IgnoreUnavailable  bool
	IncludeGlobalState bool
	Partial            bool
	IncludeAliases     bool
	Timeout            string
	RenamePattern      string
	RenameReplacement  string
	Indices            []string
	IndexSettings      map[string]interface{}
}

IndexOptions are options that can be passed as elasticsearch parameters

type Indexer 

type Indexer struct {
	// contains filtered or unexported fields
}

Indexer represents a struct that performs different actions on elasticsearch indices

func (*Indexer) AddAlias 

func (i *Indexer) AddAlias(indexName string, aliasName string) (*gabs.Container, error)

AddAlias adds an alias to a given elasticsearch index

func (*Indexer) AddAliasByAction 

func (i *Indexer) AddAliasByAction(aliasAction *elastic.AliasAddAction) (*gabs.Container, error)

AddAliasByAction adds an alias by *elastic.AliasAddAction

func (*Indexer) AliasAddAction 

func (i *Indexer) AliasAddAction(alias string) *elastic.AliasAddAction

AliasAddAction returns an instances of *elastic.AliasAddAction

func (*Indexer) Aliases 

func (i *Indexer) Aliases(indices ...string) (*gabs.Container, error)

Aliases retrieves the aliases for a given set of indices

func (*Indexer) AliasesCat 

func (i *Indexer) AliasesCat() (*gabs.Container, error)

AliasesCat retrives information associated to all current index aliases

func (*Indexer) Close 

func (i *Indexer) Close(name string) (*gabs.Container, error)

Close closes an elasticsearch index

func (*Indexer) CreateIndex 

func (i *Indexer) CreateIndex(name string, schema string) error

CreateIndex creates and ElasticSearch index

func (*Indexer) CreateRepository 

func (i *Indexer) CreateRepository(repository string, repoType string, verify bool, settings map[string]interface{}) (*gabs.Container, error)

CreateRepository creates a snapshot repository

func (*Indexer) DeleteIndex 

func (i *Indexer) DeleteIndex(name string) error

DeleteIndex deletes an ElasticSearch Index

func (*Indexer) DeleteRepositories 

func (i *Indexer) DeleteRepositories(repositories ...string) (*gabs.Container, error)

DeleteRepositories deletes one or many snapshot repositories

func (*Indexer) DeleteSnapshot 

func (i *Indexer) DeleteSnapshot(repository string, name string) (*gabs.Container, error)

DeleteSnapshot deletes a snapshot for a given repository

func (*Indexer) Exists 

func (i *Indexer) Exists(name string) (bool, error)

Exists checks if a given index exists on ElasticSearch

func (*Indexer) FieldMappings 

func (i *Indexer) FieldMappings(indices ...string) (*gabs.Container, error)

FieldMappings returns the field mappings for the specified indices

func (*Indexer) GetClusterHealth 

func (i *Indexer) GetClusterHealth(indices ...string) (*gabs.Container, error)

GetClusterHealth returns the health status of the cluster

func (*Indexer) GetIndices 

func (i *Indexer) GetIndices(indices ...string) (*gabs.Container, error)

GetIndices returns index information for the provided indices

func (*Indexer) GetSnapshots 

func (i *Indexer) GetSnapshots(repository string, snapshot string) (*gabs.Container, error)

GetSnapshots retrives information regarding snapshots in a given repository

func (*Indexer) GetTask 

func (i *Indexer) GetTask(taskId string) (*gabs.Container, error)

GetTask retrieves the status of a given task by task id

func (*Indexer) IndexCat 

func (i *Indexer) IndexCat(name string) (*gabs.Container, error)

IndexCat retrieves information associated to the given index

func (*Indexer) IndexStats 

func (i *Indexer) IndexStats(indices ...string) (map[string]*gabs.Container, error)

IndexStats retrieves the statistics for the given indices

func (*Indexer) ListAllIndices 

func (i *Indexer) ListAllIndices() ([]string, error)

ListAllIndices lists all indices on and elasticsearch cluster

func (*Indexer) ListIndices 

func (i *Indexer) ListIndices() ([]string, error)

ListIndices lists all open inidces on an elsasticsearch cluster

func (*Indexer) ListSnapshots 

func (i *Indexer) ListSnapshots(repository string) ([]string, error)

ListSnapshots returns a list of snapshots for the given repository

func (*Indexer) Mappings 

func (i *Indexer) Mappings(indices ...string) (*gabs.Container, error)

Mappings returns the mappings for the specified indices

func (*Indexer) Open 

func (i *Indexer) Open(name string) (*gabs.Container, error)

Open opens an elasticsearch index

func (*Indexer) PutSettings 

func (i *Indexer) PutSettings(body string, indices ...string) (*gabs.Container, error)

PutSettings updates elasticsearch indices settings

func (*Indexer) Recovery 

func (i *Indexer) Recovery(indices ...string) (map[string]*gabs.Container, error)

Recovery checks the indices recovery status when restoring a snapshot

func (*Indexer) RemoveIndexFromAlias 

func (i *Indexer) RemoveIndexFromAlias(index string, alias string) (*gabs.Container, error)

RemoveIndexFromAlias removes an index from a given alias

func (*Indexer) Rollover 

func (i *Indexer) Rollover(alias, newIndex, maxAge, maxSize string, maxDocs int64, settings map[string]interface{}) (*gabs.Container, error)

Rollover executes an index rollover if the given conditions are met

func (*Indexer) SetOptions 

func (i *Indexer) SetOptions(options *IndexOptions)

SetOptions sets the index options for the action to be performed

func (*Indexer) Settings 

func (i *Indexer) Settings(names ...string) (map[string]*gabs.Container, error)

Settings gets the index settings for the specified indices

func (*Indexer) Snapshot 

func (i *Indexer) Snapshot(repository string, snapshot string, indices ...string) (*gabs.Container, error)

Snapshot takes a snapshot of one or more indices and stores it in the provided repository

func (*Indexer) SnapshotRestore 

func (i *Indexer) SnapshotRestore(repository string, snapshot string) (*gabs.Container, error)

SnapshotRestore restores a snapshot from the specified repository

func (*Indexer) Templates 

func (i *Indexer) Templates(indexTemplates ...string) (*gabs.Container, error)

Templates returns the templates associated to the specified index templates

type MinMaxResponse 

type MinMaxResponse struct {
	Min interface{} `json:"min"`
	Max interface{} `json:"max"`
}

MinMaxResponse is the response for the MinMax builder call

func (*MinMaxResponse) ToGabsContainer 

func (mmr *MinMaxResponse) ToGabsContainer() (*gabs.Container, error)

ToGabsContainer converts a response to a *gabs.Container instance

Source Files

View all Source files

Directories

Path Synopsis
cursor
delete
simple
update

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.