documenttransformer

Documentation

Index

• type AmazonTranslate
  • func NewAmazonTranslate(client AmazonTranslateClient, targetLanguageCode string, ...) *AmazonTranslate
  • func (t *AmazonTranslate) Transform(ctx context.Context, docs []schema.Document) ([]schema.Document, error)
• type AmazonTranslateClient
• type AmazonTranslateOptions
• type MetaDataTagger
  • func NewMetaDataTagger(chatModel schema.ChatModel, tags any, optFns ...func(o *MetaDataTaggerOptions)) (*MetaDataTagger, error)
  • func (t *MetaDataTagger) Transform(ctx context.Context, docs []schema.Document) ([]schema.Document, error)
• type MetaDataTaggerOptions

Types

type AmazonTranslate

type AmazonTranslate struct {
	// contains filtered or unexported fields
}

AmazonTranslate is a transformer that uses Amazon Translate to translate text in documents to a target language.

func NewAmazonTranslate

func NewAmazonTranslate(client AmazonTranslateClient, targetLanguageCode string, optFns ...func(o *AmazonTranslateOptions)) *AmazonTranslate

NewAmazonTranslate creates a new instance of the AmazonTranslate transformer.

func (*AmazonTranslate) Transform

func (t *AmazonTranslate) Transform(ctx context.Context, docs []schema.Document) ([]schema.Document, error)

Transform translates the text in a slice of documents to the target language using Amazon Translate.

type AmazonTranslateClient

type AmazonTranslateClient interface {
	TranslateText(ctx context.Context, params *translate.TranslateTextInput, optFns ...func(*translate.Options)) (*translate.TranslateTextOutput, error)
}

AmazonTranslateClient is an interface for the Amazon Translate client.

type AmazonTranslateOptions

type AmazonTranslateOptions struct {
	// MaxConcurrency sets the maximum number of concurrent translation requests.
	MaxConcurrency int
	// SourceLanguageCode sets the source language code for translation. Default is "auto".
	SourceLanguageCode string
	// IncludeSourceText indicates whether to include the source text in document metadata.
	IncludeSourceText bool
}

AmazonTranslateOptions contains options for configuring the AmazonTranslate transformer.

type MetaDataTagger

type MetaDataTagger struct {
	// contains filtered or unexported fields
}

MetaDataTagger is a document transformer that adds metadata to documents using a tagging chain.

func NewMetaDataTagger

func NewMetaDataTagger(chatModel schema.ChatModel, tags any, optFns ...func(o *MetaDataTaggerOptions)) (*MetaDataTagger, error)

NewMetaDataTagger creates a new MetaDataTagger instance.

func (*MetaDataTagger) Transform

func (t *MetaDataTagger) Transform(ctx context.Context, docs []schema.Document) ([]schema.Document, error)

Transform transforms a slice of documents by adding metadata using the tagging chain.

type MetaDataTaggerOptions

type MetaDataTaggerOptions struct {
	MaxConcurrency int
}

MetaDataTaggerOptions represents the options for the MetaDataTagger.