azureaisearch

Documentation

Overview

Package azureaisearch contains an implementation of the VectorStore interface that connects to Azure AI search.

Index

• Constants
• Variables
• func WithFilters(filters any) vectorstores.Option
• type FieldType
  • func CollectionField(fieldType FieldType) FieldType
• type IndexOption
• type Option
  • func WithAPIKey(azureAISearchAPIKey string) Option
  • func WithEmbedder(e embeddings.Embedder) Option
  • func WithHTTPClient(client *http.Client) Option
• type QueryCaptions
• type QueryType
• type SearchDocumentsRequestInput
• type SearchDocumentsRequestInputVector
• type SearchDocumentsRequestOuput
• type SpellerType
• type Store
  • func New(opts ...Option) (Store, error)
  • func (s *Store) AddDocuments(ctx context.Context, docs []schema.Document, options ...vectorstores.Option) ([]string, error)
  • func (s *Store) CreateIndex(ctx context.Context, indexName string, opts ...IndexOption) error
  • func (s *Store) CreateIndexAPIRequest(ctx context.Context, indexName string, payload any) error
  • func (s *Store) DeleteIndex(ctx context.Context, indexName string) error
  • func (s *Store) ListIndexes(ctx context.Context, output *map[string]interface{}) error
  • func (s *Store) RetrieveIndex(ctx context.Context, indexName string, output *map[string]interface{}) error
  • func (s *Store) SearchDocuments(ctx context.Context, indexName string, payload SearchDocumentsRequestInput, ...) error
  • func (s *Store) SimilaritySearch(ctx context.Context, query string, numDocuments int, ...) ([]schema.Document, error)
  • func (s *Store) UploadDocument(ctx context.Context, id string, indexName string, text string, ...) error
  • func (s *Store) UploadDocumentAPIRequest(ctx context.Context, indexName string, document any) error

Constants

const (
	// EnvironmentVariableEndpoint environment variable to set azure ai search endpoint.
	EnvironmentVariableEndpoint string = "AZURE_AI_SEARCH_ENDPOINT"
	// EnvironmentVariableAPIKey environment variable to set azure ai api key.
	EnvironmentVariableAPIKey string = "AZURE_AI_SEARCH_API_KEY"
)

Variables

var (
	// ErrNumberOfVectorDoesNotMatch when providing documents,
	// the number of vectors generated should be equal to the number of docs.
	ErrNumberOfVectorDoesNotMatch = errors.New(
		"number of vectors from embedder does not match number of documents",
	)
	// ErrAssertingMetadata SearchScore is stored as float64.
	ErrAssertingSearchScore = errors.New(
		"couldn't assert @search.score to float64",
	)
	// ErrAssertingMetadata Metadata is stored as string.
	ErrAssertingMetadata = errors.New(
		"couldn't assert metadata to string",
	)
	// ErrAssertingContent Content is stored as string.
	ErrAssertingContent = errors.New(
		"couldn't assert content to string",
	)
)

var (
	// ErrMissingEnvVariableAzureAISearchEndpoint environment variable to set azure ai search endpoint missing.
	ErrMissingEnvVariableAzureAISearchEndpoint = errors.New(
		"missing azureAISearchEndpoint",
	)
	// ErrMissingEmbedded embedder is missing, one should be set when instantiating the vectorstore.
	ErrMissingEmbedded = errors.New(
		"missing embedder",
	)
)

var ErrSendingRequest = errors.New(
	"error sedding request",
)

ErrSendingRequest basic error when the request failed.

Functions

func WithFilters

func WithFilters(filters any) vectorstores.Option

WithFilters can set the filter property in search document payload.

Types

type FieldType

type FieldType = string

FieldType type for pseudo enum.

const (
	FieldTypeString         FieldType = "Edm.String"
	FieldTypeSingle         FieldType = "Edm.Single"
	FieldTypeInt32          FieldType = "Edm.Int32"
	FieldTypeInt64          FieldType = "Edm.Int64"
	FieldTypeDouble         FieldType = "Edm.Double"
	FieldTypeBoolean        FieldType = "Edm.Boolean"
	FieldTypeDatetimeOffset FieldType = "Edm.DateTimeOffset"
	FieldTypeComplexType    FieldType = "Edm.ComplexType"
)

Pseudo enum for all the different FieldType.

func CollectionField

func CollectionField(fieldType FieldType) FieldType

CollectionField allows to define a fieldtype as a collection.

type IndexOption

type IndexOption func(indexMap *map[string]interface{})

IndexOption is used to customise the index when creating the index useful if you use differemt embedder than text-embedding-ada-002.

type Option

type Option func(p *Store)

Option is a function type that can be used to modify the client.

func WithAPIKey

func WithAPIKey(azureAISearchAPIKey string) Option

WithAPIKey is an option for setting the azure AI search API Key.

func WithEmbedder

func WithEmbedder(e embeddings.Embedder) Option

WithEmbedder is an option for setting the embedder to use.

func WithHTTPClient

func WithHTTPClient(client *http.Client) Option

WithEmbedder is an option for setting the http client, the vectorstore uses the REST API, default http client is set but can be overridden by this option.

type QueryCaptions

type QueryCaptions string

QueryCaptions pseudo enum for SearchDocumentsRequestInput queryCaptions property.

const (
	QueryTypeExtractive QueryCaptions = "extractive"
	QueryTypeNone       QueryCaptions = "none"
)

type QueryType

type QueryType string

QueryType pseudo enum for SearchDocumentsRequestInput queryType property.

const (
	QueryTypeSimple   QueryType = "simple"
	QueryTypeFull     QueryType = "full"
	QueryTypeSemantic QueryType = "semantic"
)

type SearchDocumentsRequestInput

type SearchDocumentsRequestInput struct {
	Count                 bool                                `json:"count,omitempty"`
	Captions              QueryCaptions                       `json:"captions,omitempty"`
	Facets                []string                            `json:"facets,omitempty"`
	Filter                string                              `json:"filter,omitempty"`
	Highlight             string                              `json:"highlight,omitempty"`
	HighlightPostTag      string                              `json:"highlightPostTag,omitempty"`
	HighlightPreTag       string                              `json:"highlightPreTag,omitempty"`
	MinimumCoverage       int16                               `json:"minimumCoverage,omitempty"`
	Orderby               string                              `json:"orderby,omitempty"`
	QueryType             QueryType                           `json:"queryType,omitempty"`
	QueryLanguage         string                              `json:"queryLanguage,omitempty"`
	Speller               SpellerType                         `json:"speller,omitempty"`
	SemanticConfiguration string                              `json:"semanticConfiguration,omitempty"`
	ScoringParameters     []string                            `json:"scoringParameters,omitempty"`
	ScoringProfile        string                              `json:"scoringProfile,omitempty"`
	Search                string                              `json:"search,omitempty"`
	SearchFields          string                              `json:"searchFields,omitempty"`
	SearchMode            string                              `json:"searchMode,omitempty"`
	SessionID             string                              `json:"sessionId,omitempty"`
	ScoringStatistics     string                              `json:"scoringStatistics,omitempty"`
	Select                string                              `json:"select,omitempty"`
	Skip                  int                                 `json:"skip,omitempty"`
	Top                   int                                 `json:"top,omitempty"`
	Vectors               []SearchDocumentsRequestInputVector `json:"vectors,omitempty"`
	VectorFilterMode      string                              `json:"vectorFilterMode,omitempty"`
}

SearchDocumentsRequestInput is the input struct to format a payload in order to search for a document.

type SearchDocumentsRequestInputVector

type SearchDocumentsRequestInputVector struct {
	Kind       string    `json:"kind,omitempty"`
	Value      []float32 `json:"value,omitempty"`
	Fields     string    `json:"fields,omitempty"`
	K          int       `json:"k,omitempty"`
	Exhaustive bool      `json:"exhaustive,omitempty"`
}

SearchDocumentsRequestInputVector is the input struct for vector search.

type SearchDocumentsRequestOuput

type SearchDocumentsRequestOuput struct {
	OdataCount   int `json:"@odata.count,omitempty"`
	SearchFacets struct {
		Category []struct {
			Count int    `json:"count,omitempty"`
			Value string `json:"value,omitempty"`
		} `json:"category,omitempty"`
	} `json:"@search.facets,omitempty"`
	SearchNextPageParameters SearchDocumentsRequestInput `json:"@search.nextPageParameters,omitempty"`
	Value                    []map[string]interface{}    `json:"value,omitempty"`
	OdataNextLink            string                      `json:"@odata.nextLink,omitempty"`
}

SearchDocumentsRequestOuput is the output struct for search.

type SpellerType

type SpellerType string

SpellerType pseudo enum for SearchDocumentsRequestInput spellerType property.

const (
	SpellerTypeLexicon SpellerType = "lexicon"
	SpellerTypeNone    SpellerType = "none"
)

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a wrapper to use azure AI search rest API.

func New

func New(opts ...Option) (Store, error)

New creates a vectorstore for azure AI search and returns the `Store` object needed by the other accessors.

func (*Store) AddDocuments

func (s *Store) AddDocuments(
	ctx context.Context,
	docs []schema.Document,
	options ...vectorstores.Option,
) ([]string, error)

AddDocuments adds the text and metadata from the documents to the Chroma collection associated with 'Store'. and returns the ids of the added documents.

func (*Store) CreateIndex

func (s *Store) CreateIndex(ctx context.Context, indexName string, opts ...IndexOption) error

CreateIndex defines a default index (default one is made for text-embedding-ada-002) but can be customised through IndexOption functions.

func (*Store) CreateIndexAPIRequest

func (s *Store) CreateIndexAPIRequest(ctx context.Context, indexName string, payload any) error

CreateIndexAPIRequest send a request to azure AI search Rest API for creating an index.

func (*Store) DeleteIndex

func (s *Store) DeleteIndex(ctx context.Context, indexName string) error

CreateIndexAPIRequest send a request to azure AI search Rest API for deleting an index.

func (*Store) ListIndexes

func (s *Store) ListIndexes(ctx context.Context, output *map[string]interface{}) error

ListIndexes send a request to azure AI search Rest API for creatin an index, helper function.

func (*Store) RetrieveIndex

func (s *Store) RetrieveIndex(ctx context.Context, indexName string, output *map[string]interface{}) error

RetrieveIndex send a request to azure AI search Rest API for retrieving an index, helper function.

func (*Store) SearchDocuments

func (s *Store) SearchDocuments(
	ctx context.Context,
	indexName string,
	payload SearchDocumentsRequestInput,
	output *SearchDocumentsRequestOuput,
) error

SearchDocuments send a request to azure AI search Rest API for searching documents.

func (*Store) SimilaritySearch

func (s *Store) SimilaritySearch(
	ctx context.Context,
	query string,
	numDocuments int,
	options ...vectorstores.Option,
) ([]schema.Document, error)

SimilaritySearch creates a vector embedding from the query using the embedder and queries to find the most similar documents.

func (*Store) UploadDocument

func (s *Store) UploadDocument(
	ctx context.Context,
	id string,
	indexName string,
	text string,
	vector []float32,
	metadata map[string]any,
) error

UploadDocument format document for similiraty search and upload it.

func (*Store) UploadDocumentAPIRequest

func (s *Store) UploadDocumentAPIRequest(ctx context.Context, indexName string, document any) error

UploadDocumentAPIRequest makes a request to azure AI search to upload a document. tech debt: should use SDK when available: https://azure.github.io/azure-sdk/releases/latest/go.html