sajari

package module
v0.0.0-...-9a18aee Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 27, 2017 License: MIT Imports: 16 Imported by: 8

README

Sajari APIs Go Client · Build Status

This repository provides functionality for interacting with Sajari APIs.

Installation

If you haven't setup Go before, you need to first set a GOPATH (see https://golang.org/doc/code.html#GOPATH).

To fetch and build the code:

$ go get code.sajari.com/sajari-sdk-go/...

This will also build the command line tools (in particular query, csv-importer, schema and pipeline which can be used to interaction with Sajari collections) into $GOPATH/bin (assumed to be in your PATH already).

Getting Started

package main

import (
	"log"

	"golang.org/x/net/context"

	"code.sajari.com/sajari-sdk-go"
)

func main() {
	creds := sajari.KeyCredentials("<key-id>", "<key-secret>")
	client, err := sajari.New("<project>", "<collection>", sajari.WithCredentials(creds))
	if err != nil {
		log.Fatalf("error creating client: %v", err)
	}
	defer client.Close()

	// Initialise the "website" pipeline. Ideal starting point for querying
	// website collections.
	pipeline := client.Pipeline("website")

	// Setup parameters for the search.
	values := map[string]string{
		"q":              "search terms",
		"resultsPerPage": "10",
	}

	// Setup tracking for the search results.
	t := sajari.Tracking{
		Type:  sajari.TrackingClick, // Enable click tracking.
		Field: "url",         // Use the url field for identifying records.
	}

	// Perform the search.
	resp, _, err := pipeline.Search(context.Background(), values, t)
	if err != nil {
		log.Fatalf("error performing search: %v", err)
	}

	for _, r := range resp.Results {
		log.Printf("Values: %v", r.Values)
		log.Println("Tokens: %v", r.Tokens)
	}
}

Documentation

Overview

Package sajari provides functionality for interacting with Sajari APIs.

Index

Constants

View Source 
const (
	// BodyField is the name of the internal field which is used to represent the
	// record body.
	BodyField = "_body"

	// IDField is the name of the internal identifier field which is added to
	// each record.
	IDField = "_id"
)

Field constants for internal fields. NB: All field names prefixed with _ (underscore) are reserved for internal use.

Variables

View Source 
var DefaultAddTransforms = []Transform{
	SplitStopStemIndexedFieldsTranform,
}

DefaultAddTransforms is the default list of transforms which are used when adding records.

View Source 
var ErrNoSuchRecord = errors.New("sajari: no such record")

ErrNoSuchRecord is returned when a requested record cannot be found.

Functions

This section is empty.

Types

type Aggregate 

type Aggregate interface {
	// contains filtered or unexported methods
}

Aggregate is an interface which is implemented by all aggregate types in this package.

func AvgAggregate 

func AvgAggregate(field string) Aggregate

AvgAggregate computes the average value of a numeric field over a result set.

func BucketAggregate 

func BucketAggregate(bs ...Bucket) Aggregate

BucketAggregate is an aggregate which counts records which fall into a defined set of buckets.

func CountAggregate 

func CountAggregate(field string) Aggregate

CountAggregate is an aggregate which counts unique field values.

func MaxAggregate 

func MaxAggregate(field string) Aggregate

MaxAggregate computes the maximum value of a numeric field over a result set.

func MinAggregate 

func MinAggregate(field string) Aggregate

MinAggregate computes the minimum value of a numeric field over a result set.

func SumAggregate 

func SumAggregate(field string) Aggregate

SumAggregate computes the sum of a numeric field over a result set.

type Body 

type Body struct {
	// Text to search for.
	Text string

	// Weight (significance) of text.
	Weight float64
}

Body is weighted free text.

type Bucket 

type Bucket struct {
	// Name of the bucket.
	Name string

	// Filter that records must satisfy to be included in the bucket.
	Filter Filter
}

Bucket is a container used to classify records in a result set (see BucketAggregate). A record is included in the bucket if it satisfies the filter.

type BucketResponse 

type BucketResponse struct {
	// Name of the bucket.
	Name string

	// Number of records.
	Count int
}

type BucketsResponse 

type BucketsResponse map[string]BucketResponse

BucketsResponse is a type returned from a query performing bucket aggregate.

type Client 

type Client struct {
	Project    string
	Collection string

	ClientConn *grpc.ClientConn
	// contains filtered or unexported fields
}

Client is a type which makes requests to the Sajari Engine.

func New 

func New(project, collection string, opts ...Opt) (*Client, error)

New creates a new Client which can be used to make requests to Sajari services.

func (*Client) Add 

func (c *Client) Add(ctx context.Context, r Record, ts ...Transform) (*Key, error)

Add adds a record to a collection, returning a key which can be used to retrieve the record. If no transforms are specified then DefaultAddTransforms is used.

func (*Client) AddMulti 

func (c *Client) AddMulti(ctx context.Context, rs []Record, ts ...Transform) ([]*Key, error)

AddMulti adds records to the underlying collection, returning a list of Keys which can be used to retrieve the respective record. If any of the adds fail then a MultiError will be returned with errors set in the respective indexes. If no transforms are specified then DefaultAddTransforms is used.

func (*Client) Close 

func (c *Client) Close() error

Close releases all resources held by the Client.

func (*Client) Delete 

func (c *Client) Delete(ctx context.Context, k *Key) error

Delete removes the record identified by key k.

func (*Client) DeleteMulti 

func (c *Client) DeleteMulti(ctx context.Context, ks []*Key) error

DeleteMulti removes the records identified by the keys k. Returns non-nil error if there was a communication problem, but fails silently if any key doesn't have a corresponding record.

func (*Client) Exists 

func (c *Client) Exists(ctx context.Context, k *Key) (bool, error)

Exists returns true iff a record identified by k exists or non-nil error if there was a problem completing the request.

func (*Client) ExistsMulti 

func (c *Client) ExistsMulti(ctx context.Context, k []*Key) ([]bool, error)

ExistsMulti checks whether records identified by keys exist. Returns a slice of bool values indiciating the existence of each.

func (*Client) Get 

func (c *Client) Get(ctx context.Context, k *Key) (Record, error)

Get returns the record identified by the Key.

func (*Client) GetMulti 

func (c *Client) GetMulti(ctx context.Context, k []*Key) ([]Record, error)

GetMulti retrieves the records identified by the keys k.

func (*Client) Learn 

func (c *Client) Learn(ctx context.Context, k *Key, r Request, count int, score float32) error

Learn takes a record identified by k and a query request r and applies pos and neg weighting to the intersections of r and the record.

func (*Client) LearnMulti 

func (c *Client) LearnMulti(ctx context.Context, ks []*Key, r Request, counts []int, scores []float32) error

Learn takes a list of records identified by keys ks and a query request r and applies pos and neg weighting to the intersections of r and the record.

func (*Client) Mutate 

func (c *Client) Mutate(ctx context.Context, k *Key, m ...FieldMutation) error

Patch makes changes to a record identified by k in-place (where possible). To remove a value set the corresponding field name to nil in values map.

func (*Client) MutateMulti 

func (c *Client) MutateMulti(ctx context.Context, rms ...RecordMutation) error

func (*Client) Pipeline 

func (c *Client) Pipeline(name string) *Pipeline

Pipeline returns a Pipeline for querying a collection.

func (*Client) Query 

func (c *Client) Query() *Query

Query returns a handler for running queries using the Client.

func (*Client) Schema 

func (c *Client) Schema() *Schema

Schema returns the schema (list of fields) for the collection.

type CountResponse 

type CountResponse map[string]int

CountResponse is a type returned from a query which has performed a count aggregate.

type Credentials 

type Credentials interface {
	// contains filtered or unexported methods
}

Credentials is an interface which is implemented by types providing credential information used in requests.

func KeyCredentials 

func KeyCredentials(keyID, keySecret string) Credentials

KeyCredentials defines a Credential which uses a Key ID-Secret pair.

type FeatureFieldBoost 

type FeatureFieldBoost struct {
	// contains filtered or unexported fields
}

FeatureFieldBoost is a wrapper which turns a FieldBoost into a feature. See NewFeatureFieldBoost.

func NewFeatureFieldBoost 

func NewFeatureFieldBoost(b FieldBoost, value float64) FeatureFieldBoost

FeatureFieldBoost uses the normalised form of the FieldBoost b (computed internally) to count for a portion (value between 0 and 1) of the overall record score.

type FeatureQuery 

type FeatureQuery struct {
	// FieldBoosts is a list of FeatureFieldBoosts which contribute
	// to the feature score of a record in a query.
	FieldBoosts []FeatureFieldBoost
}

FeatureQuery is a feature-based query which contributes to the scoring of records.

type Field 

type Field struct {
	// Name is the name used to identify the field.
	Name string

	// Description is a description of the field.
	Description string

	// Type defines the type of the field.
	Type Type

	// Repeated indicates that this field can hold a list of values.
	Repeated bool

	// Required indicates that this field should always be set on all records.
	Required bool

	// Indexed indicates that the field should be indexed.  This is only valid for
	// String or StringArray fields (see TypeString, TypeStringArray).
	Indexed bool

	// Unique indicates that the field is unique (and this will
	// be encoforced when new records are added).  Unique fields can
	// be used to retrieve/delete records.
	Unique bool
}

Field represents a meta field which can be assigned in a collection record.

type FieldBoost 

type FieldBoost interface {
	// contains filtered or unexported methods
}

FieldBoost is an interface satisfied by field-based boosting types.

FieldBoosts are a way to influence the scoring of a record during a search based on the record's field data. All boosts are all normalised internally (converted to a number between 0 and 1).

func ElementFieldBoost 

func ElementFieldBoost(field string, elts []string) FieldBoost

ElementFieldBoost represents an element-based boosting for repeated field values.

The resulting boost is the proportion of elements in elts that are also in the field value.

func FilterFieldBoost 

func FilterFieldBoost(f Filter, value float64) FieldBoost

FilterFieldBoost is a boost which is applied to records which satisfy the filter. Value must be greater than 0. Records which match the filter will receive a boost of Value

func IntervalFieldBoost 

func IntervalFieldBoost(field string, values ...IntervalPoint) FieldBoost

IntervalFieldBoost represents an interval-based boost for numeric field values.

An interval field boost is defined by a list of points with corresponding boost values. When a field value falls between between two IntervalPoints.Point values is computed linearly.

func TextFieldBoost 

func TextFieldBoost(field string, text string) FieldBoost

TextFieldBoost represents a text-based boosting for string fields.

It compares the text gainst the record field using a bag-of-words model.

type FieldMutation 

type FieldMutation interface {
	// contains filtered or unexported methods
}

FieldMutation is an interface satisfied by all field mutations defined in this package.

func SetField 

func SetField(field string, value interface{}) FieldMutation

SetField is a FieldMutation which sets field to value. If value is nil then this unsets field.

func SetFields 

func SetFields(m map[string]interface{}) []FieldMutation

SetFields converts the map of field-value pairs into field mutations for use in Mutate.

type Filter 

type Filter interface {
	// contains filtered or unexported methods
}

Filter is an interface satisified by filter types that can defined in this package.

func AllFilters 

func AllFilters(filters ...Filter) Filter

AllFilters returns a filter which matches documensts that satisfy all of the supplied filters. Equivalent to AND.

func AnyFilter 

func AnyFilter(filters ...Filter) Filter

AnyFilter returns a filter which matches records that satisfy any of the given filters. Equivalent to OR.

func FieldFilter 

func FieldFilter(fieldOp string, value interface{}) Filter

FieldFilter creates a field-based filter. The fieldOp argument must be a field name followed by optional space and then one of "=", "!=", ">", ">=" "<", "<=", "~" (contains), "!~" (does not contain), "^" (prefix) or "$" (suffix).

Filter which matches records where the field 'url' begins with "https://www.sajari.com": 

FieldFilter("url ^", "https://www.sajari.com")

Filter which matches records where the field 'name' contains "Sajari": 

FieldFilter("name ~", "Sajari")

Filter which matches records where the field 'count' is greater than or equal to 10: 

FieldFilter("count >=", 10)

func GeoFilter 

func GeoFilter(fieldLat, fieldLng string, lat, lng, radius float64, region GeoFilterRegion) Filter

GeoFilter is a geo-based boost for records with numeric fields containing latitude/longitude. 

// Construct a geo-filter on fields "lat" and "lng" which define a location
// within 10km of Sydney (33.8688° S, 151.2093° E).
GeoFilter("lat", "lng", -33.8688, 151.2093, 10.00, GeoFilterInside)

func NoneOfFilters 

func NoneOfFilters(filters ...Filter) Filter

NoneOfFilters returns a filter which matches records that do not satisfy any of the given filters. Equivalent to NAND.

func OneOfFilters 

func OneOfFilters(filters ...Filter) Filter

OneOfFilters returns a filter which matches records that satisfy only one of the given filters. Equivalent to XOR.

type GeoFilterRegion 

type GeoFilterRegion int

GeoFilterRegion is an enumeration of region values for specifying regions in GeoFilters 

const (
	// GeoFilterInside is used to configure a geo filter to be
	// applied to all points within the radius.
	GeoFilterInside GeoFilterRegion = iota

	// GeoFilterOutside is used to contigure a geo boost to be
	// applied to all points outside the radius.
	GeoFilterOutside
)

Constants for use with GeoFilter.Region.

type IndexQuery 

type IndexQuery struct {
	// Text is the default body (free-text) of the query.
	// If set this has weight 1.0.
	Text string

	// Body is a list of weighted free-text.
	Body []Body

	// Terms is a list of pre-split terms.
	Terms []Term

	// FieldBoosts to be applied to the index score.
	FieldBoosts []FieldBoost

	// InstanceBoosts to be applied (change scoring of records).
	InstanceBoosts []InstanceBoost
}

IndexQuery is a query run against

type InstanceBoost 

type InstanceBoost interface {
	// contains filtered or unexported methods
}

InstanceBoost is an interface satisfied by instance-based boosting types defined in this package.

InstanceBoosts are a way to influence the scoring of a record during a search by changing the importance of term instances in indexed records.

func FieldInstanceBoost 

func FieldInstanceBoost(field string, value float64) InstanceBoost

FieldInstanceBoost is a boost applied to index terms which originate in a particular field.

func ScoreInstanceBoost 

func ScoreInstanceBoost(minCount int, threshold float64) InstanceBoost

ScoreInstanceBoost is a boost applied to index terms which have interaction data set. For an instance score boost to take effect, the instance must have received at least minCount score updates (i.e. count). If an item is performing as it should then its score will be 1. If the score is below threshold (0 < threshold < 1) then the score will be applied.

type IntervalPoint 

type IntervalPoint struct {
	// Point is a field value.
	Point float64

	// Value of boost to assign at this point.
	Value float64
}

IntervalPoint is point-value pair used to construct an IntervalFieldBoost.

It defines the boost value at a particular point in an interval.

type Key 

type Key struct {
	// contains filtered or unexported fields
}

Key is a unique identifier for a stored record.

func NewKey 

func NewKey(field string, value interface{}) *Key

NewKey creates a new Key with a field and value. Field must be marked as unique in the collection schema.

func (*Key) String 

func (k *Key) String() string

String implements Stringer.

type MultiError 

type MultiError []error

MultiError can be returned from calls which make multiple requests (see AddMulti, GetMulti, etc).

func (MultiError) Error 

func (me MultiError) Error() string

Error implements error.

type Mutation 

type Mutation interface {
	// contains filtered or unexported methods
}

Mutation is an interface which is satisfied by schema field mutations.

func IndexedMutation 

func IndexedMutation(indexed bool) Mutation

IndexedMutation creates a schema field mutation which changes the indexed property on a field.

func NameMutation 

func NameMutation(name string) Mutation

NameMutation creates a schema field mutation which changes the name of a field.

func RepeatedMutation 

func RepeatedMutation(repeated bool) Mutation

RepeatedMutation creates a schema field mutation which changes the repeated property on a field.

func RequiredMutation 

func RequiredMutation(required bool) Mutation

RequiredMutation creates a schema field mutation which changes the required constraint on a field.

func TypeMutation 

func TypeMutation(ty Type) Mutation

TypeMutation creates a schema field mutation which changes the type of a field.

func UniqueMutation 

func UniqueMutation(unique bool) Mutation

UniqueMutation creates a schema field mutation which changes the unique constraint on a field.

type Opt 

type Opt func(c *Client)

Opt is a type which defines Client options.

func WithCredentials 

func WithCredentials(c Credentials) Opt

WithCredentials sets the client credentials used in each request.

func WithEndpoint 

func WithEndpoint(endpoint string) Opt

WithEndpoint configures the client to use a custom endpoint.

func WithGRPCDialOption 

func WithGRPCDialOption(opt grpc.DialOption) Opt

WithGRPCDialOption returns an Opt which appends a new grpc.DialOption to an underlying gRPC dial.

type Pipeline 

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline is a handler for a named pipeline.

func (*Pipeline) Search 

func (p *Pipeline) Search(ctx context.Context, values map[string]string, tracking Tracking) (*Results, map[string]string, error)

Search runs a search query defined by a pipline with the given values and tracking configuration. Returns the query results and returned values (which could have been modified in the pipeline).

type Query 

type Query struct {
	// contains filtered or unexported fields
}

Query is a handler which runs queries on a collection.

func (*Query) Analyse 

func (q *Query) Analyse(ctx context.Context, k *Key, r Request) ([]string, error)

Analyse returns the list of overlapping terms between the record identified by k and the search search request r.

func (*Query) AnalyseMulti 

func (q *Query) AnalyseMulti(ctx context.Context, ks []*Key, r Request) ([][]string, error)

AnalyseMulti performs Analysis on multiple records against the same query request.

func (*Query) Search 

func (q *Query) Search(ctx context.Context, r *Request) (*Results, error)

Search performs an engine search with the Request r, returning a set of Results and non-nil error if there was a problem.

type Record 

type Record map[string]interface{}

Record is a set of key-value pairs.

func NewRecord 

func NewRecord(body string, values map[string]interface{}) Record

NewRecord creates a new Record with the given body and field values.

type RecordMutation 

type RecordMutation struct {
	// Key identifies the record to mutate.
	Key *Key

	// FieldMutations to apply to the record.
	FieldMutations []FieldMutation
}

RecordMutation is a mutation to apply to a Record.

type Request 

type Request struct {
	// Tracking configuration.
	Tracking Tracking

	// Filter to be applied (exclude records from results).
	Filter Filter

	// IndexQuery defines a query to be run against the search index.
	IndexQuery IndexQuery

	// FeatureQuery
	FeatureQuery FeatureQuery

	// Offset of results to return.
	Offset int

	// Limit is the number of results to return.
	Limit int

	// Ordering applied to results.
	Sort []Sort

	// Fields returned in results, if empty will return all fields.
	Fields []string

	// Aggregates is a set of Aggregates to run against a result set.
	Aggregates map[string]Aggregate

	// Transforms is a list of transforms to be applied to the query before it is run.
	Transforms []Transform
}

Request is an API representation of a search request.

type Result 

type Result struct {
	// Values are field values of records.
	Values map[string]interface{}

	// Tokens contains any tokens associated with this Result.
	Tokens map[string]interface{}

	// Score is the overall score of this Result.
	Score float64

	// IndexScore is the index-matched score of this Result.
	IndexScore float64
}

Result is an individual query result.

type Results 

type Results struct {
	// Reads is the total number of index values read.
	Reads int

	// TotalResults is the total number of results for the query.
	TotalResults int

	// Time taken to perform the query.
	Time time.Duration

	// Aggregates computed on the query results (see Aggregate).
	Aggregates map[string]interface{}

	// Results of the query.
	Results []Result
}

Results is a collection of results from a Search.

type Schema 

type Schema struct {
	// contains filtered or unexported fields
}

Schema provides methods for managing collection schemas. Use Client.Schema to create one for a collection.

func (*Schema) Add 

func (s *Schema) Add(ctx context.Context, fs ...Field) error

Add adds Fields to the collection schema.

func (*Schema) Fields 

func (s *Schema) Fields(ctx context.Context) ([]Field, error)

Fields returns the fields in the collection.

func (*Schema) MutateField 

func (s *Schema) MutateField(ctx context.Context, name string, muts ...Mutation) error

MutateField mutates a field identifier by name. Each mutation is performed in the order in which it is specified. If any fail, then the rest are ignored.

type Sort 

type Sort interface {
	// contains filtered or unexported methods
}

Sort is an interface satisfied by all types which produce sort config.

type SortByField 

type SortByField string

SortByField defines a sort order using a field.

type Term 

type Term struct {
	Value  string  // String representation of the field
	Field  string  // Meta field
	Pos    uint16  // Number of positive interactions
	Neg    uint16  // Number of negative interactions
	Weight float64 // Significance of term
	WOff   uint16  // Word offset
	POff   uint16  // Paragraph offset
}

Term is a scored term.

type Tracking 

type Tracking struct {
	// Tracking specifies which kind (if any) tokens should be generated and returned
	// with the query results.
	Type TrackingType

	// QueryID is a unique identifier for a single search query.  In the
	// case of live querying this is defined to be multiple individual queries
	// (i.e. as a user types the query is re-run).
	QueryID string

	// Sequence (i.e. sequential identifier) of this  in the context of a
	// sequence of queries.
	Sequence int

	// Field is the field to be used for adding identifier information to
	// generated tokens (see TrackingType).
	Field string

	// Data are values which will be recorded along with tracking data produced
	// for the request.
	Data map[string]string
}

Tracking configures tracking for a search query.

type TrackingType 

type TrackingType string

TrackingType defines different modes of tracking which can be applied to search requests. 

const (
	TrackingNone   TrackingType = ""        // No tracking is enabled.
	TrackingClick  TrackingType = "CLICK"   // Click tracking is enabled, Click tokens will be returned with results.
	TrackingPosNeg TrackingType = "POS_NEG" // Positive/negative interaction tokens should be returned with results.
)

TrackingType constants.

type Transform 

type Transform string

Transform is a definition of a transformation applied to a Request which is applied before the Request is executed. 

const (
	// SplitStopStemIndexedFieldsTransform splits indexed fields into terms,
	// removes stop words and stems what remains.
	// This is the default setting for transforms.
	SplitStopStemIndexedFieldsTranform Transform = "split-stop-stem-indexed-fields"

	// StopStemmerTransform removes stop terms and stems terms.
	StopStemTransform Transform = "stop-stem"

	// SplitIndexFields splits index fields into terms.
	SplitIndexedFieldsTransform Transform = "split-indexed-fields"
)

Transforms commonly used when adding records.

type Type 

type Type string

Type defines field data types. 

const (
	TypeString    Type = "STRING"
	TypeInteger   Type = "INTEGER"
	TypeFloat     Type = "FLOAT"
	TypeBoolean   Type = "BOOLEAN"
	TypeTimestamp Type = "TIMESTAMP"
)

Source Files

View all Source files

Directories

Path Synopsis
Package autocomplete provides methods for interacting directly with auto complete models.
 Package autocomplete provides methods for interacting directly with auto complete models.
Package bayes provides methods for creating, training and querying bayes models.
 Package bayes provides methods for creating, training and querying bayes models.
cmd
autocomplete-train
csv-import
pipeline
query
record
schema

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.