llms

package
v0.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 19, 2024 License: MIT Imports: 6 Imported by: 276

Documentation

Overview

Package llms provides unified support for interacting with different Language Models (LLMs) from various providers. Designed with an extensible architecture, the package facilitates seamless integration of LLMs with a focus on modularity, encapsulation, and easy configurability.

The package includes the following subpackages for LLM providers: 1. Hugging Face: llms/huggingface/ 2. Local LLM: llms/local/ 3. OpenAI: llms/openai/ 4. Google AI: llms/googleai/ 5. Cohere: llms/cohere/

Each subpackage includes provider-specific LLM implementations and helper files for communication with supported LLM providers. The internal directories within these subpackages contain provider-specific client and API implementations.

The `llms.go` file contains the types and interfaces for interacting with different LLMs.

The `options.go` file provides various options and functions to configure the LLMs.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateMaxTokens 

func CalculateMaxTokens(model, text string) int

CalculateMaxTokens calculates the max number of tokens that could be added to a text.

func CountTokens 

func CountTokens(model, text string) int

CountTokens gets the number of tokens the text contains.

func GenerateFromSinglePrompt added in v0.1.4 

func GenerateFromSinglePrompt(ctx context.Context, llm Model, prompt string, options ...CallOption) (string, error)

GenerateFromSinglePrompt is a convenience function for calling an LLM with a single string prompt, expecting a single string response. It's useful for simple, string-only interactions and provides a slightly more ergonomic API than the more general llms.Model.GenerateContent.

func GetModelContextSize 

func GetModelContextSize(model string) int

ModelContextSize gets the max number of tokens for a language model. If the model name isn't recognized the default value 2048 is returned.

Types

type BinaryContent added in v0.1.3 

type BinaryContent struct {
	MIMEType string
	Data     []byte
}

BinaryContent is content holding some binary data with a MIME type.

func BinaryPart added in v0.1.4 

func BinaryPart(mime string, data []byte) BinaryContent

BinaryPart creates a new BinaryContent from the given MIME type (e.g. "image/png" and binary data).

type CallOption 

type CallOption func(*CallOptions)

CallOption is a function that configures a CallOptions.

func WithCandidateCount added in v0.1.4 

func WithCandidateCount(c int) CallOption

WithCandidateCount specifies the number of response candidates to generate.

func WithFrequencyPenalty 

func WithFrequencyPenalty(frequencyPenalty float64) CallOption

WithFrequencyPenalty will add an option to set the frequency penalty for sampling.

func WithFunctionCallBehavior 

func WithFunctionCallBehavior(behavior FunctionCallBehavior) CallOption

WithFunctionCallBehavior will add an option to set the behavior to use when calling functions.

func WithFunctions 

func WithFunctions(functions []FunctionDefinition) CallOption

WithFunctions will add an option to set the functions to include in the request.

func WithJSONMode added in v0.1.6 

func WithJSONMode() CallOption

WithJSONMode will add an option to set the response format to JSON. This is useful for models that return structured data.

func WithMaxLength 

func WithMaxLength(maxLength int) CallOption

WithMaxLength will add an option to set the maximum length of the generated text.

func WithMaxTokens 

func WithMaxTokens(maxTokens int) CallOption

WithMaxTokens specifies the max number of tokens to generate.

func WithMinLength 

func WithMinLength(minLength int) CallOption

WithMinLength will add an option to set the minimum length of the generated text.

func WithModel 

func WithModel(model string) CallOption

WithModel specifies which model name to use.

func WithN 

func WithN(n int) CallOption

WithN will add an option to set how many chat completion choices to generate for each input message.

func WithOptions 

func WithOptions(options CallOptions) CallOption

WithOptions specifies options.

func WithPresencePenalty 

func WithPresencePenalty(presencePenalty float64) CallOption

WithPresencePenalty will add an option to set the presence penalty for sampling.

func WithRepetitionPenalty 

func WithRepetitionPenalty(repetitionPenalty float64) CallOption

WithRepetitionPenalty will add an option to set the repetition penalty for sampling.

func WithSeed 

func WithSeed(seed int) CallOption

WithSeed will add an option to use deterministic sampling.

func WithStopWords 

func WithStopWords(stopWords []string) CallOption

WithStopWords specifies a list of words to stop generation on.

func WithStreamingFunc 

func WithStreamingFunc(streamingFunc func(ctx context.Context, chunk []byte) error) CallOption

WithStreamingFunc specifies the streaming function to use.

func WithTemperature 

func WithTemperature(temperature float64) CallOption

WithTemperature specifies the model temperature, a hyperparameter that regulates the randomness, or creativity, of the AI's responses.

func WithTopK 

func WithTopK(topK int) CallOption

WithTopK will add an option to use top-k sampling.

func WithTopP 

func WithTopP(topP float64) CallOption

WithTopP will add an option to use top-p sampling.

type CallOptions 

type CallOptions struct {
	// Model is the model to use.
	Model string `json:"model"`
	// CandidateCount is the number of response candidates to generate.
	CandidateCount int `json:"candidate_count"`
	// MaxTokens is the maximum number of tokens to generate.
	MaxTokens int `json:"max_tokens"`
	// Temperature is the temperature for sampling, between 0 and 1.
	Temperature float64 `json:"temperature"`
	// StopWords is a list of words to stop on.
	StopWords []string `json:"stop_words"`
	// StreamingFunc is a function to be called for each chunk of a streaming response.
	// Return an error to stop streaming early.
	StreamingFunc func(ctx context.Context, chunk []byte) error
	// TopK is the number of tokens to consider for top-k sampling.
	TopK int `json:"top_k"`
	// TopP is the cumulative probability for top-p sampling.
	TopP float64 `json:"top_p"`
	// Seed is a seed for deterministic sampling.
	Seed int `json:"seed"`
	// MinLength is the minimum length of the generated text.
	MinLength int `json:"min_length"`
	// MaxLength is the maximum length of the generated text.
	MaxLength int `json:"max_length"`
	// N is how many chat completion choices to generate for each input message.
	N int `json:"n"`
	// RepetitionPenalty is the repetition penalty for sampling.
	RepetitionPenalty float64 `json:"repetition_penalty"`
	// FrequencyPenalty is the frequency penalty for sampling.
	FrequencyPenalty float64 `json:"frequency_penalty"`
	// PresencePenalty is the presence penalty for sampling.
	PresencePenalty float64 `json:"presence_penalty"`

	// JSONMode is a flag to enable JSON mode.
	JSONMode bool `json:"json"`

	// Function defitions to include in the request.
	Functions []FunctionDefinition `json:"functions"`
	// FunctionCallBehavior is the behavior to use when calling functions.
	//
	// If a specific function should be invoked, use the format:
	// `{"name": "my_function"}`
	FunctionCallBehavior FunctionCallBehavior `json:"function_call"`
}

CallOptions is a set of options for calling models. Not all models support all options.

type ContentChoice added in v0.1.3 

type ContentChoice struct {
	// Content is the textual content of a response
	Content string

	// StopReason is the reason the model stopped generating output.
	StopReason string

	// GenerationInfo is arbitrary information the model adds to the response.
	GenerationInfo map[string]any

	// FuncCall is non-nil when the model asks to invoke a function/tool.
	FuncCall *schema.FunctionCall
}

ContentChoice is one of the response choices returned by GenerateContent calls.

type ContentPart added in v0.1.3 

type ContentPart interface {
	// contains filtered or unexported methods
}

ContentPart is an interface all parts of content have to implement.

type ContentResponse added in v0.1.3 

type ContentResponse struct {
	Choices []*ContentChoice
}

ContentResponse is the response returned by a GenerateContent call. It can potentially return multiple content choices.

type FunctionCallBehavior 

type FunctionCallBehavior string

FunctionCallBehavior is the behavior to use when calling functions. 

const (
	// FunctionCallBehaviorNone will not call any functions.
	FunctionCallBehaviorNone FunctionCallBehavior = "none"
	// FunctionCallBehaviorAuto will call functions automatically.
	FunctionCallBehaviorAuto FunctionCallBehavior = "auto"
)

type FunctionDefinition 

type FunctionDefinition struct {
	// Name is the name of the function.
	Name string `json:"name"`
	// Description is a description of the function.
	Description string `json:"description"`
	// Parameters is a list of parameters for the function.
	Parameters any `json:"parameters"`
}

FunctionDefinition is a definition of a function that can be called by the model.

type ImageURLContent added in v0.1.3 

type ImageURLContent struct {
	URL string
}

ImageURLContent is content with an URL pointing to an image.

func ImageURLPart added in v0.1.4 

func ImageURLPart(url string) ImageURLContent

ImageURLPart creates a new ImageURLContent from the given URL.

func (ImageURLContent) MarshalJSON added in v0.1.3 

func (iuc ImageURLContent) MarshalJSON() ([]byte, error)

type LLM deprecated 

type LLM = Model

LLM is an alias for model, for backwards compatibility.

Deprecated: This alias may be removed in the future; please use Model instead.

type MessageContent added in v0.1.4 

type MessageContent struct {
	Role  schema.ChatMessageType
	Parts []ContentPart
}

MessageContent is the content of a message sent to a LLM. It has a role and a sequence of parts. For example, it can represent one message in a chat session sent by the user, in which case Role will be schema.ChatMessageTypeHuman and Parts will be the sequence of items sent in this specific message.

func TextParts added in v0.1.4 

func TextParts(role schema.ChatMessageType, parts ...string) MessageContent

TextParts is a helper function to create a MessageContent with a role and a list of text parts.

type Model added in v0.1.3 

type Model interface {
	// GenerateContent asks the model to generate content from a sequence of
	// messages. It's the most general interface for multi-modal LLMs that support
	// chat-like interactions.
	GenerateContent(ctx context.Context, messages []MessageContent, options ...CallOption) (*ContentResponse, error)

	// Call is a simplified interface for a text-only Model, generating a single
	// string response from a single string prompt.
	//
	// Deprecated: this method is retained for backwards compatibility. Use the
	// more general [GenerateContent] instead. You can also use
	// the [GenerateFromSinglePrompt] function which provides a similar capability
	// to Call and is built on top of the new interface.
	Call(ctx context.Context, prompt string, options ...CallOption) (string, error)
}

Model is an interface multi-modal models implement.

type TextContent added in v0.1.3 

type TextContent struct {
	Text string
}

TextContent is content with some text.

func TextPart added in v0.1.4 

func TextPart(s string) TextContent

TextPart creates TextContent from a given string.

func (TextContent) MarshalJSON added in v0.1.3 

func (tc TextContent) MarshalJSON() ([]byte, error)

Source Files

View all Source files

Directories

Path Synopsis
internal/anthropicclient
internal/bedrockclient
internal/cohereclient
Package ernie wrapper around the Baidu Large Language Model Platform APIs.
 Package ernie wrapper around the Baidu Large Language Model Platform APIs.
internal/ernieclient
package googleai implements a langchaingo provider for Google AI LLMs.
 package googleai implements a langchaingo provider for Google AI LLMs.
internal/cmd
Code generator for vertex.go from googleai.go nolint
 Code generator for vertex.go from googleai.go nolint
internal/palmclient
palm
package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models.
 package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models.
vertex
package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models.
 package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models.
internal/huggingfaceclient
internal/llamafileclient
internal/localclient
Package localclient provides a client for local LLMs.
 Package localclient provides a client for local LLMs.
internal/ollamaclient
internal/openaiclient

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.