llm

package
v0.0.36 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 11, 2023 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package llm provides functionalities for working with Large Language Models (LLMs).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Cohere 

type Cohere struct {
	schema.Tokenizer
	// contains filtered or unexported fields
}

Cohere represents the Cohere language model.

func NewCohere 

func NewCohere(apiKey string, optFns ...func(o *CohereOptions)) (*Cohere, error)

NewCohere creates a new Cohere instance using the provided API key and optional configuration options. It internally creates a Cohere client using the provided API key and initializes the Cohere struct.

func NewCohereFromClient added in v0.0.31 

func NewCohereFromClient(client CohereClient, optFns ...func(o *CohereOptions)) (*Cohere, error)

NewCohereFromClient creates a new Cohere instance using the provided Cohere client and optional configuration options.

func (*Cohere) Callbacks 

func (l *Cohere) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*Cohere) Generate 

func (l *Cohere) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*Cohere) InvocationParams added in v0.0.27 

func (l *Cohere) InvocationParams() map[string]any

InvocationParams returns the parameters used in the llm model invocation.

func (*Cohere) Type 

func (l *Cohere) Type() string

Type returns the type of the model.

func (*Cohere) Verbose 

func (l *Cohere) Verbose() bool

Verbose returns the verbosity setting of the model.

type CohereClient added in v0.0.31 

type CohereClient interface {
	Generate(opts cohere.GenerateOptions) (*cohere.GenerateResponse, error)
}

CohereClient is an interface for the Cohere client.

type CohereOptions 

type CohereOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`

	// Model represents the name or identifier of the Cohere language model to use.
	Model string `map:"model,omitempty"`

	// NumGenerations denotes the maximum number of generations that will be returned.                   string
	NumGenerations int `map:"num_generations"`

	// MaxTokens denotes the number of tokens to predict per generation.
	MaxTokens uint `map:"max_tokens"`

	// Temperatur is a non-negative float that tunes the degree of randomness in generation.
	Temperatur float64 `map:"temperature"`

	// K specifies the number of top most likely tokens to consider for generation at each step.
	K int `map:"k"`

	// P is a probability value between 0.0 and 1.0. It ensures that only the most likely tokens,
	// with a total probability mass of P, are considered for generation at each step.
	P float64 `map:"p"`

	// FrequencyPenalty is used to reduce repetitiveness of generated tokens. A higher value applies
	// a stronger penalty to previously present tokens, proportional to how many times they have
	// already appeared in the prompt or prior generation.
	FrequencyPenalty float64 `map:"frequency_penalty"`

	// PresencePenalty is used to reduce repetitiveness of generated tokens. It applies a penalty
	// equally to all tokens that have already appeared, regardless of their exact frequencies.
	PresencePenalty float64 `map:"presence_penalty"`

	// ReturnLikelihoods specifies whether and how the token likelihoods are returned with the response.
	// It can be set to "GENERATION", "ALL", or "NONE". If "GENERATION" is selected, the token likelihoods
	// will only be provided for generated text. If "ALL" is selected, the token likelihoods will be
	// provided for both the prompt and the generated text.
	ReturnLikelihoods string `map:"return_likelihoods,omitempty"`

	// MaxRetries represents the maximum number of retries to make when generating.
	MaxRetries uint `map:"max_retries,omitempty"`
}

CohereOptions contains options for configuring the Cohere LLM model.

type Fake added in v0.0.14 

type Fake struct {
	schema.Tokenizer
	// contains filtered or unexported fields
}

Fake is a fake LLM model that generates text based on a provided response function.

func NewFake added in v0.0.14 

func NewFake(responseFunc FakeResponseFunc, optFns ...func(o *FakeOptions)) *Fake

NewFake creates a new instance of the Fake LLM model with the provided response function and options.

func (*Fake) Callbacks added in v0.0.14 

func (l *Fake) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*Fake) Generate added in v0.0.14 

func (l *Fake) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*Fake) InvocationParams added in v0.0.27 

func (l *Fake) InvocationParams() map[string]any

InvocationParams returns the parameters used in the model invocation.

func (*Fake) Type added in v0.0.14 

func (l *Fake) Type() string

Type returns the type of the model.

func (*Fake) Verbose added in v0.0.14 

func (l *Fake) Verbose() bool

Verbose returns the verbosity setting of the model.

type FakeOptions added in v0.0.31 

type FakeOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`
}

FakeOptions contains options for configuring the Fake LLM model.

type FakeResponseFunc added in v0.0.35 

type FakeResponseFunc func(prompt string) string

FakeResponseFunc is a function type for generating fake responses based on a prompt.

type HuggingFaceHub added in v0.0.14 

type HuggingFaceHub struct {
	schema.Tokenizer
	// contains filtered or unexported fields
}

HuggingFaceHub represents the Hugging Face Hub LLM model.

func NewHuggingFaceHub added in v0.0.14 

func NewHuggingFaceHub(apiToken string, optFns ...func(o *HuggingFaceHubOptions)) (*HuggingFaceHub, error)

NewHuggingFaceHub creates a new instance of the HuggingFaceHub model using the provided API token and options.

func NewHuggingFaceHubFromClient added in v0.0.31 

func NewHuggingFaceHubFromClient(client HuggingFaceHubClient, optFns ...func(o *HuggingFaceHubOptions)) (*HuggingFaceHub, error)

NewHuggingFaceHubFromClient creates a new instance of the HuggingFaceHub model using the provided client and options.

func (*HuggingFaceHub) Callbacks added in v0.0.14 

func (l *HuggingFaceHub) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*HuggingFaceHub) Generate added in v0.0.14 

func (l *HuggingFaceHub) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*HuggingFaceHub) InvocationParams added in v0.0.27 

func (l *HuggingFaceHub) InvocationParams() map[string]any

InvocationParams returns the parameters used in the model invocation.

func (*HuggingFaceHub) Type added in v0.0.14 

func (l *HuggingFaceHub) Type() string

Type returns the type of the model.

func (*HuggingFaceHub) Verbose added in v0.0.14 

func (l *HuggingFaceHub) Verbose() bool

Verbose returns the verbosity setting of the model.

type HuggingFaceHubClient added in v0.0.31 

type HuggingFaceHubClient interface {
	// TextGeneration performs text generation based on the provided request and returns the response.
	TextGeneration(ctx context.Context, req *huggingface.TextGenerationRequest) (huggingface.TextGenerationResponse, error)

	// Text2TextGeneration performs text-to-text generation based on the provided request and returns the response.
	Text2TextGeneration(ctx context.Context, req *huggingface.Text2TextGenerationRequest) (huggingface.Text2TextGenerationResponse, error)

	// Summarization performs text summarization based on the provided request and returns the response.
	Summarization(ctx context.Context, req *huggingface.SummarizationRequest) (huggingface.SummarizationResponse, error)

	// SetModel sets the model to be used for inference.
	SetModel(model string)
}

HuggingFaceHubClient represents the client for interacting with the Hugging Face Hub service.

type HuggingFaceHubOptions added in v0.0.14 

type HuggingFaceHubOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`
	Model                   string `map:"model,omitempty"`
	Task                    string `map:"task,omitempty"`
}

HuggingFaceHubOptions contains options for configuring the Hugging Face Hub LLM model.

type LLMContentHandler 

type LLMContentHandler struct {
	// contains filtered or unexported fields
}

LLMContentHandler handles content transformation for the LLM model.

func NewLLMContentHandler 

func NewLLMContentHandler(contentType, accept string, transformer Transformer) *LLMContentHandler

NewLLMContentHandler creates a new LLMContentHandler instance.

func (*LLMContentHandler) Accept 

func (ch *LLMContentHandler) Accept() string

Accept returns the accept type of the LLMContentHandler.

func (*LLMContentHandler) ContentType 

func (ch *LLMContentHandler) ContentType() string

ContentType returns the content type of the LLMContentHandler.

func (*LLMContentHandler) TransformInput 

func (ch *LLMContentHandler) TransformInput(prompt string) ([]byte, error)

TransformInput transforms the input prompt using the LLMContentHandler's transformer.

func (*LLMContentHandler) TransformOutput 

func (ch *LLMContentHandler) TransformOutput(output []byte) (string, error)

TransformOutput transforms the output from the LLM model using the LLMContentHandler's transformer.

type OpenAI 

type OpenAI struct {
	schema.Tokenizer
	// contains filtered or unexported fields
}

OpenAI is an implementation of the LLM interface for the OpenAI language model.

func NewOpenAI 

func NewOpenAI(apiKey string, optFns ...func(o *OpenAIOptions)) (*OpenAI, error)

NewOpenAI creates a new OpenAI instance with the provided API key and options.

func NewOpenAIFromClient added in v0.0.31 

func NewOpenAIFromClient(client OpenAIClient, optFns ...func(o *OpenAIOptions)) (*OpenAI, error)

NewOpenAIFromClient creates a new OpenAI instance with the provided client and options.

func (*OpenAI) Callbacks 

func (l *OpenAI) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*OpenAI) Generate 

func (l *OpenAI) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*OpenAI) InvocationParams added in v0.0.27 

func (l *OpenAI) InvocationParams() map[string]any

InvocationParams returns the parameters used in the model invocation.

func (*OpenAI) Type 

func (l *OpenAI) Type() string

Type returns the type of the model.

func (*OpenAI) Verbose 

func (l *OpenAI) Verbose() bool

Verbose returns the verbosity setting of the model.

type OpenAIClient added in v0.0.31 

type OpenAIClient interface {
	// CreateCompletionStream creates a streaming completion request with the provided completion request.
	// It returns a completion stream for receiving streamed completion responses from the OpenAI API.
	// The `CompletionStream` should be closed after use.
	CreateCompletionStream(ctx context.Context, request openai.CompletionRequest) (stream *openai.CompletionStream, err error)

	// CreateCompletion sends a completion request to the OpenAI API and returns the completion response.
	// It blocks until the response is received from the API.
	CreateCompletion(ctx context.Context, request openai.CompletionRequest) (response openai.CompletionResponse, err error)
}

OpenAIClient represents the interface for interacting with the OpenAI API.

type OpenAIOptions 

type OpenAIOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`
	// ModelName is the name of the OpenAI language model to use.
	ModelName string `map:"model_name,omitempty"`
	// Temperature is the sampling temperature to use during text generation.
	Temperatur float32 `map:"temperatur,omitempty"`
	// MaxTokens is the maximum number of tokens to generate in the completion.
	MaxTokens int `map:"max_tokens,omitempty"`
	// TopP is the total probability mass of tokens to consider at each step.
	TopP float32 `map:"top_p,omitempty"`
	// PresencePenalty penalizes repeated tokens.
	PresencePenalty float32 `map:"presence_penalty,omitempty"`
	// FrequencyPenalty penalizes repeated tokens according to frequency.
	FrequencyPenalty float32 `map:"frequency_penalty,omitempty"`
	// N is the number of completions to generate for each prompt.
	N int `map:"n,omitempty"`
	// BestOf selects the best completion from multiple completions.
	BestOf int `map:"best_of,omitempty"`
	// LogitBias adjusts the probability of specific tokens being generated.
	LogitBias map[string]int `map:"logit_bias,omitempty"`
	// Stream indicates whether to stream the results or not.
	Stream bool `map:"stream,omitempty"`
	// MaxRetries represents the maximum number of retries to make when generating.
	MaxRetries uint `map:"max_retries,omitempty"`
}

OpenAIOptions contains options for configuring the OpenAI LLM model.

type Palm added in v0.0.36 

type Palm struct {
	// contains filtered or unexported fields
}

Palm is a struct representing the PALM language model.

func NewPalm added in v0.0.36 

func NewPalm(client PalmClient, optFns ...func(o *PalmOptions)) (*Palm, error)

NewPalm creates a new instance of the PALM language model.

func (*Palm) Callbacks added in v0.0.36 

func (l *Palm) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*Palm) Generate added in v0.0.36 

func (l *Palm) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*Palm) InvocationParams added in v0.0.36 

func (l *Palm) InvocationParams() map[string]any

InvocationParams returns the parameters used in the model invocation.

func (*Palm) Type added in v0.0.36 

func (l *Palm) Type() string

Type returns the type of the model.

func (*Palm) Verbose added in v0.0.36 

func (l *Palm) Verbose() bool

Verbose returns the verbosity setting of the model.

type PalmClient added in v0.0.36 

type PalmClient interface {
	GenerateText(ctx context.Context, req *generativelanguagepb.GenerateTextRequest, opts ...gax.CallOption) (*generativelanguagepb.GenerateTextResponse, error)
}

PalmClient is the interface for the PALM client.

type PalmOptions added in v0.0.36 

type PalmOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`

	// ModelName is the name of the Palm language model to use.
	ModelName string `map:"model_name,omitempty"`

	// Temperature is the sampling temperature to use during text generation.
	Temperatur float32 `map:"temperatur,omitempty"`

	// TopP is the total probability mass of tokens to consider at each step.
	TopP float32 `map:"top_p,omitempty"`

	// TopK determines how the model selects tokens for output.
	TopK int32 `map:"top_k"`

	// MaxOutputTokens specifies the maximum number of output tokens for text generation.
	MaxOutputTokens int32 `map:"max_output_tokens"`

	// CandidateCount specifies the number of candidates to generate during text completion.
	CandidateCount int32 `map:"candidate_count"`
}

PalmOptions is the options struct for the PALM language model.

type SagemakerEndpoint 

type SagemakerEndpoint struct {
	schema.Tokenizer
	// contains filtered or unexported fields
}

SagemakerEndpoint represents an LLM model deployed on AWS SageMaker.

func NewSagemakerEndpoint 

func NewSagemakerEndpoint(client SagemakerRuntimeClient, endpointName string, contenHandler *LLMContentHandler, optFns ...func(o *SagemakerEndpointOptions)) (*SagemakerEndpoint, error)

NewSagemakerEndpoint creates a new SagemakerEndpoint instance.

func (*SagemakerEndpoint) Callbacks 

func (l *SagemakerEndpoint) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*SagemakerEndpoint) Generate 

func (l *SagemakerEndpoint) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*SagemakerEndpoint) InvocationParams added in v0.0.27 

func (l *SagemakerEndpoint) InvocationParams() map[string]any

InvocationParams returns the parameters used in the model invocation.

func (*SagemakerEndpoint) Type 

func (l *SagemakerEndpoint) Type() string

Type returns the type of the model.

func (*SagemakerEndpoint) Verbose 

func (l *SagemakerEndpoint) Verbose() bool

Verbose returns the verbosity setting of the model.

type SagemakerEndpointOptions 

type SagemakerEndpointOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`
}

SagemakerEndpointOptions contains options for configuring the SagemakerEndpoint.

type SagemakerRuntimeClient added in v0.0.34 

type SagemakerRuntimeClient interface {
	// InvokeEndpoint invokes an endpoint in the SageMaker Runtime service with the specified input parameters.
	// It returns the output of the endpoint invocation or an error if the invocation fails.
	InvokeEndpoint(ctx context.Context, params *sagemakerruntime.InvokeEndpointInput, optFns ...func(*sagemakerruntime.Options)) (*sagemakerruntime.InvokeEndpointOutput, error)
}

SagemakerRuntimeClient is an interface that represents the client for interacting with the SageMaker Runtime service.

type Transformer 

type Transformer interface {
	// Transforms the input to a format that model can accept
	// as the request Body. Should return bytes or seekable file
	// like object in the format specified in the content_type
	// request header.
	TransformInput(prompt string) ([]byte, error)

	// Transforms the output from the model to string that
	// the LLM class expects.
	TransformOutput(output []byte) (string, error)
}

Transformer defines the interface for transforming input and output data for the LLM model.

type VertexAI added in v0.0.31 

type VertexAI struct {
	schema.Tokenizer
	// contains filtered or unexported fields
}

VertexAI represents the VertexAI language model.

func NewVertexAI added in v0.0.31 

func NewVertexAI(client VertexAIClient, endpoint string, optFns ...func(o *VertexAIOptions)) (*VertexAI, error)

NewVertexAI creates a new VertexAI instance with the provided client and endpoint.

func (*VertexAI) Callbacks added in v0.0.31 

func (l *VertexAI) Callbacks() []schema.Callback

Callbacks returns the registered callbacks of the model.

func (*VertexAI) Generate added in v0.0.31 

func (l *VertexAI) Generate(ctx context.Context, prompt string, optFns ...func(o *schema.GenerateOptions)) (*schema.ModelResult, error)

Generate generates text based on the provided prompt and options.

func (*VertexAI) InvocationParams added in v0.0.31 

func (l *VertexAI) InvocationParams() map[string]any

InvocationParams returns the parameters used in the model invocation.

func (*VertexAI) Type added in v0.0.31 

func (l *VertexAI) Type() string

Type returns the type of the model.

func (*VertexAI) Verbose added in v0.0.31 

func (l *VertexAI) Verbose() bool

Verbose returns the verbosity setting of the model.

type VertexAIClient added in v0.0.31 

type VertexAIClient interface {
	// Predict sends a prediction request to the Vertex AI service.
	// It takes a context, predict request, and optional call options.
	// It returns the predict response or an error if the prediction fails.
	Predict(ctx context.Context, req *aiplatformpb.PredictRequest, opts ...gax.CallOption) (*aiplatformpb.PredictResponse, error)
}

VertexAIClient represents the interface for interacting with Vertex AI.

type VertexAIOptions added in v0.0.31 

type VertexAIOptions struct {
	*schema.CallbackOptions `map:"-"`
	schema.Tokenizer        `map:"-"`

	// Temperature is the sampling temperature to use during text generation.
	Temperatur float32 `map:"temperatur"`

	// MaxOutputTokens determines the maximum amount of text output from one prompt.
	MaxOutputTokens int `map:"max_output_tokens"`

	// TopP is the total probability mass of tokens to consider at each step.
	TopP float32 `map:"top_p"`

	// TopK determines how the model selects tokens for output.
	TopK int `map:"top_k"`
}

VertexAIOptions contains options for configuring the VertexAI language model.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.