textencoding

Documentation

Index

• Constants
• Variables
• type Interface
• type Response

Constants

const (
	// DefaultModel is a sentence-transformers model:
	// It maps sentences & paragraphs to dense vector space and can be used for tasks like clustering or semantic search.
	// Model card: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
	DefaultModel = "sentence-transformers/all-MiniLM-L6-v2"

	// DefaultModelMulti it's like the model before but multilingual:
	// it can be used to map 109 languages to a shared vector space.
	// It works well for finding translation pairs in multiple languages.
	// It doesn't perform so well for assessing the similarity of sentence pairs that are not translations of each other.
	// Model card: https://huggingface.co/sentence-transformers/LaBSE
	DefaultModelMulti = "sentence-transformers/LaBSE"
)

Variables

var ErrInputSequenceTooLong = errors.New("input sequence too long")

ErrInputSequenceTooLong means that pre-processing the input text produced a sequence that exceeds the maximum allowed length.

Types

type Interface

type Interface interface {
	// Encode returns the encoded representation of the given example.
	Encode(ctx context.Context, text string, poolingStrategy int) (Response, error)
}

Interface defines the main functions for text encoding task.

type Response

type Response struct {
	// the encoded representation
	Vector mat.Matrix
}

Response contains the response from text classification.