txtai

README

txtai: AI-powered search engine for Go

txtai builds an AI-powered index over sections of text. txtai supports building text indices to perform similarity searches and create extractive question-answering based systems. txtai also has functionality for zero-shot classification.

This repository contains Go bindings for the txtai API. Full txtai functionality is supported.

Installation

txtai.go can be installed as follows:

go get -u github.com/neuml/txtai.go

Alternatively, adding the following import within a module will also download txtai.go

import "github.com/neuml/txtai.go"

Examples

The examples directory has a series of examples that give an overview of txtai. See the list of examples below.

Example	Description
Introducing txtai	Overview of the functionality provided by txtai
Extractive QA with txtai	Extractive question-answering with txtai
Labeling with zero-shot classification	Labeling with zero-shot classification

txtai.go connects to a txtai api instance. See this link for details on how to start a new api instance.

Once an api instance is running, do the following to run the examples.

git clone https://github.com/neuml/txtai.go
cd txtai.go/examples
make embeddings|extractor|labels

Documentation

Overview

Base API definition

Embeddings definition

Extractor definition

Labels definition

Index

• type API
  • func (api *API) Get(method string, params map[string]string, result interface{})
  • func (api *API) Post(method string, params interface{}, result interface{})
  • func (api *API) Request(method string, result interface{}) *resty.Request
• type Answer
• type Document
• type EmbeddingsAPI
  • func Embeddings(url string) EmbeddingsAPI
  • func (embeddings *EmbeddingsAPI) Add(documents interface{})
  • func (embeddings *EmbeddingsAPI) Embeddings(t string) []float64
  • func (embeddings *EmbeddingsAPI) Index()
  • func (embeddings *EmbeddingsAPI) Search(q string, n int) []Score
  • func (embeddings *EmbeddingsAPI) Similarity(search string, data []string) []float64
• type ExtractorAPI
  • func Extractor(url string) ExtractorAPI
  • func (extractor *ExtractorAPI) Extract(documents []Section, queue []Question) []Answer
• type LabelsAPI
  • func Labels(url string) LabelsAPI
  • func (label *LabelsAPI) Label(text string, labels []string) []Score
• type Question
• type Score
• type Section

Types

type API

type API struct {
	// contains filtered or unexported fields
}

Base API definition

func (*API) Get

func (api *API) Get(method string, params map[string]string, result interface{})

Executes a GET request. Loads response into result.

func (*API) Post

func (api *API) Post(method string, params interface{}, result interface{})

Executes a POST request. Loads response into result.

func (*API) Request

func (api *API) Request(method string, result interface{}) *resty.Request

Creates a new Resty request and returns it.

type Answer

type Answer struct {
	Question string `json:"question"`
	Answer   string `json:"answer"`
}

Answer response

type Document

type Document struct {
	Id   string `json:"id"`
	Text string `json:"text"`
}

Base input Document

type EmbeddingsAPI

type EmbeddingsAPI struct {
	// contains filtered or unexported fields
}

Embeddings definition

func Embeddings

func Embeddings(url string) EmbeddingsAPI

Creates an Embeddings instance.

func (*EmbeddingsAPI) Add

func (embeddings *EmbeddingsAPI) Add(documents interface{})

Adds a batch of documents for indexing.

func (*EmbeddingsAPI) Embeddings

func (embeddings *EmbeddingsAPI) Embeddings(t string) []float64

Transforms text into an embeddings array.

func (*EmbeddingsAPI) Index

func (embeddings *EmbeddingsAPI) Index()

Builds an embeddings index. No further documents can be added after this call.

func (*EmbeddingsAPI) Search

func (embeddings *EmbeddingsAPI) Search(q string, n int) []Score

Runs an Embeddings search. Returns []Score.

func (*EmbeddingsAPI) Similarity

func (embeddings *EmbeddingsAPI) Similarity(search string, data []string) []float64

Calculates the similarity between search and list of elements in data.

type ExtractorAPI

type ExtractorAPI struct {
	// contains filtered or unexported fields
}

Extractor definition

func Extractor

func Extractor(url string) ExtractorAPI

Creates an Extractor instance.

func (*ExtractorAPI) Extract

func (extractor *ExtractorAPI) Extract(documents []Section, queue []Question) []Answer

Extracts answers to input questions.

type LabelsAPI

type LabelsAPI struct {
	// contains filtered or unexported fields
}

Labels definition

func Labels

func Labels(url string) LabelsAPI

Creates a Labels instance.

func (*LabelsAPI) Label

func (label *LabelsAPI) Label(text string, labels []string) []Score

Applies a zero shot classifier to a text section using a list of labels.

type Question

type Question struct {
	Name     string `json:"name"`
	Query    string `json:"query"`
	Question string `json:"question"`
	Snippet  bool   `json:"snippet"`
}

Question parameters

type Score

type Score struct {
	Id    string  `json:"id"`
	Score float64 `json:"score"`
}

API score result

type Section

type Section struct {
	Id   int    `json:"id"`
	Text string `json:"text"`
}

Text section