azureaisearch

package
v0.1.6-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2024 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package azureaisearch contains an implementation of the VectorStore interface that connects to Azure AI search.

Index

Constants

View Source 
const (
	// EnvironmentVariableEndpoint environment variable to set azure ai search endpoint.
	EnvironmentVariableEndpoint string = "AZURE_AI_SEARCH_ENDPOINT"
	// EnvironmentVariableAPIKey environment variable to set azure ai api key.
	EnvironmentVariableAPIKey string = "AZURE_AI_SEARCH_API_KEY"
)

Variables

View Source 
var (
	// ErrNumberOfVectorDoesNotMatch when providing documents,
	// the number of vectors generated should be equal to the number of docs.
	ErrNumberOfVectorDoesNotMatch = errors.New(
		"number of vectors from embedder does not match number of documents",
	)
	// ErrAssertingMetadata SearchScore is stored as float64.
	ErrAssertingSearchScore = errors.New(
		"couldn't assert @search.score to float64",
	)
	// ErrAssertingMetadata Metadata is stored as string.
	ErrAssertingMetadata = errors.New(
		"couldn't assert metadata to string",
	)
	// ErrAssertingContent Content is stored as string.
	ErrAssertingContent = errors.New(
		"couldn't assert content to string",
	)
)
View Source 
var (
	// ErrMissingEnvVariableAzureAISearchEndpoint environment variable to set azure ai search endpoint missing.
	ErrMissingEnvVariableAzureAISearchEndpoint = errors.New(
		"missing azureAISearchEndpoint",
	)
	// ErrMissingEmbedded embedder is missing, one should be set when instantiating the vectorstore.
	ErrMissingEmbedded = errors.New(
		"missing embedder",
	)
)
View Source 
var ErrSendingRequest = errors.New(
	"error sedding request",
)

ErrSendingRequest basic error when the request failed.

Functions

func WithFilters 

func WithFilters(filters any) vectorstores.Option

WithFilters can set the filter property in search document payload.

Types

type FieldType 

type FieldType = string

FieldType type for pseudo enum. 

const (
	FieldTypeString         FieldType = "Edm.String"
	FieldTypeSingle         FieldType = "Edm.Single"
	FieldTypeInt32          FieldType = "Edm.Int32"
	FieldTypeInt64          FieldType = "Edm.Int64"
	FieldTypeDouble         FieldType = "Edm.Double"
	FieldTypeBoolean        FieldType = "Edm.Boolean"
	FieldTypeDatetimeOffset FieldType = "Edm.DateTimeOffset"
	FieldTypeComplexType    FieldType = "Edm.ComplexType"
)

Pseudo enum for all the different FieldType.

func CollectionField 

func CollectionField(fieldType FieldType) FieldType

CollectionField allows to define a fieldtype as a collection.

type IndexOption 

type IndexOption func(indexMap *map[string]interface{})

IndexOption is used to customise the index when creating the index useful if you use differemt embedder than text-embedding-ada-002.

type Option 

type Option func(p *Store)

Option is a function type that can be used to modify the client.

func WithAPIKey 

func WithAPIKey(azureAISearchAPIKey string) Option

WithAPIKey is an option for setting the azure AI search API Key.

func WithEmbedder 

func WithEmbedder(e embeddings.Embedder) Option

WithEmbedder is an option for setting the embedder to use.

func WithHTTPClient 

func WithHTTPClient(client *http.Client) Option

WithEmbedder is an option for setting the http client, the vectorstore uses the REST API, default http client is set but can be overridden by this option.

type QueryCaptions 

type QueryCaptions string

QueryCaptions pseudo enum for SearchDocumentsRequestInput queryCaptions property. 

const (
	QueryTypeExtractive QueryCaptions = "extractive"
	QueryTypeNone       QueryCaptions = "none"
)

type QueryType 

type QueryType string

QueryType pseudo enum for SearchDocumentsRequestInput queryType property. 

const (
	QueryTypeSimple   QueryType = "simple"
	QueryTypeFull     QueryType = "full"
	QueryTypeSemantic QueryType = "semantic"
)

type SearchDocumentsRequestInput 

type SearchDocumentsRequestInput struct {
	Count                 bool                                `json:"count,omitempty"`
	Captions              QueryCaptions                       `json:"captions,omitempty"`
	Facets                []string                            `json:"facets,omitempty"`
	Filter                string                              `json:"filter,omitempty"`
	Highlight             string                              `json:"highlight,omitempty"`
	HighlightPostTag      string                              `json:"highlightPostTag,omitempty"`
	HighlightPreTag       string                              `json:"highlightPreTag,omitempty"`
	MinimumCoverage       int16                               `json:"minimumCoverage,omitempty"`
	Orderby               string                              `json:"orderby,omitempty"`
	QueryType             QueryType                           `json:"queryType,omitempty"`
	QueryLanguage         string                              `json:"queryLanguage,omitempty"`
	Speller               SpellerType                         `json:"speller,omitempty"`
	SemanticConfiguration string                              `json:"semanticConfiguration,omitempty"`
	ScoringParameters     []string                            `json:"scoringParameters,omitempty"`
	ScoringProfile        string                              `json:"scoringProfile,omitempty"`
	Search                string                              `json:"search,omitempty"`
	SearchFields          string                              `json:"searchFields,omitempty"`
	SearchMode            string                              `json:"searchMode,omitempty"`
	SessionID             string                              `json:"sessionId,omitempty"`
	ScoringStatistics     string                              `json:"scoringStatistics,omitempty"`
	Select                string                              `json:"select,omitempty"`
	Skip                  int                                 `json:"skip,omitempty"`
	Top                   int                                 `json:"top,omitempty"`
	Vectors               []SearchDocumentsRequestInputVector `json:"vectors,omitempty"`
	VectorFilterMode      string                              `json:"vectorFilterMode,omitempty"`
}

SearchDocumentsRequestInput is the input struct to format a payload in order to search for a document.

type SearchDocumentsRequestInputVector 

type SearchDocumentsRequestInputVector struct {
	Kind       string    `json:"kind,omitempty"`
	Value      []float32 `json:"value,omitempty"`
	Fields     string    `json:"fields,omitempty"`
	K          int       `json:"k,omitempty"`
	Exhaustive bool      `json:"exhaustive,omitempty"`
}

SearchDocumentsRequestInputVector is the input struct for vector search.

type SearchDocumentsRequestOuput 

type SearchDocumentsRequestOuput struct {
	OdataCount   int `json:"@odata.count,omitempty"`
	SearchFacets struct {
		Category []struct {
			Count int    `json:"count,omitempty"`
			Value string `json:"value,omitempty"`
		} `json:"category,omitempty"`
	} `json:"@search.facets,omitempty"`
	SearchNextPageParameters SearchDocumentsRequestInput `json:"@search.nextPageParameters,omitempty"`
	Value                    []map[string]interface{}    `json:"value,omitempty"`
	OdataNextLink            string                      `json:"@odata.nextLink,omitempty"`
}

SearchDocumentsRequestOuput is the output struct for search.

type SpellerType 

type SpellerType string

SpellerType pseudo enum for SearchDocumentsRequestInput spellerType property. 

const (
	SpellerTypeLexicon SpellerType = "lexicon"
	SpellerTypeNone    SpellerType = "none"
)

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is a wrapper to use azure AI search rest API.

func New 

func New(opts ...Option) (Store, error)

New creates a vectorstore for azure AI search and returns the `Store` object needed by the other accessors.

func (*Store) AddDocuments 

func (s *Store) AddDocuments(
	ctx context.Context,
	docs []schema.Document,
	options ...vectorstores.Option,
) ([]string, error)

AddDocuments adds the text and metadata from the documents to the Chroma collection associated with 'Store'. and returns the ids of the added documents.

func (*Store) CreateIndex 

func (s *Store) CreateIndex(ctx context.Context, indexName string, opts ...IndexOption) error

CreateIndex defines a default index (default one is made for text-embedding-ada-002) but can be customised through IndexOption functions.

func (*Store) CreateIndexAPIRequest 

func (s *Store) CreateIndexAPIRequest(ctx context.Context, indexName string, payload any) error

CreateIndexAPIRequest send a request to azure AI search Rest API for creating an index.

func (*Store) DeleteIndex 

func (s *Store) DeleteIndex(ctx context.Context, indexName string) error

CreateIndexAPIRequest send a request to azure AI search Rest API for deleting an index.

func (*Store) ListIndexes 

func (s *Store) ListIndexes(ctx context.Context, output *map[string]interface{}) error

ListIndexes send a request to azure AI search Rest API for creatin an index, helper function.

func (*Store) RetrieveIndex 

func (s *Store) RetrieveIndex(ctx context.Context, indexName string, output *map[string]interface{}) error

RetrieveIndex send a request to azure AI search Rest API for retrieving an index, helper function.

func (*Store) SearchDocuments 

func (s *Store) SearchDocuments(
	ctx context.Context,
	indexName string,
	payload SearchDocumentsRequestInput,
	output *SearchDocumentsRequestOuput,
) error

SearchDocuments send a request to azure AI search Rest API for searching documents.

func (*Store) SimilaritySearch 

func (s *Store) SimilaritySearch(
	ctx context.Context,
	query string,
	numDocuments int,
	options ...vectorstores.Option,
) ([]schema.Document, error)

SimilaritySearch creates a vector embedding from the query using the embedder and queries to find the most similar documents.

func (*Store) UploadDocument 

func (s *Store) UploadDocument(
	ctx context.Context,
	id string,
	indexName string,
	text string,
	vector []float32,
	metadata map[string]any,
) error

UploadDocument format document for similiraty search and upload it.

func (*Store) UploadDocumentAPIRequest 

func (s *Store) UploadDocumentAPIRequest(ctx context.Context, indexName string, document any) error

UploadDocumentAPIRequest makes a request to azure AI search to upload a document. tech debt: should use SDK when available: https://azure.github.io/azure-sdk/releases/latest/go.html

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.