local

package
v0.1.10-pre.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 7, 2024 License: MIT Imports: 9 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrEmptyResponse is returned when the local LLM binary returns an empty response.
	ErrEmptyResponse = errors.New("no response")
	// ErrMissingBin is returned when the LOCAL_LLM_BIN environment variable is not set.
	ErrMissingBin = errors.New("missing the local LLM binary path, set the LOCAL_LLM_BIN environment variable")
)

Functions

This section is empty.

Types

type LLM

type LLM struct {
	CallbacksHandler callbacks.Handler
	// contains filtered or unexported fields
}

LLM is a local LLM implementation.

func New

func New(opts ...Option) (*LLM, error)

New creates a new local LLM implementation.

func (*LLM) Call

func (o *LLM) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error)

Call calls the local LLM binary with the given prompt.

func (*LLM) GenerateContent added in v0.1.4

func (o *LLM) GenerateContent(ctx context.Context, messages []llms.MessageContent, options ...llms.CallOption) (*llms.ContentResponse, error)

GenerateContent implements the Model interface.

type Option

type Option func(*options)

func WithArgs

func WithArgs(args string) Option

WithArgs passes the CLI arguments to the local LLM binary. If not set, then will be used the LOCAL_LLM_ARGS environment variable.

func WithBin

func WithBin(bin string) Option

WithBin passes the path to the local LLM binary to the client. If not set, then will be used the LOCAL_LLM_BIN environment variable.

func WithGlobalAsArgs

func WithGlobalAsArgs() Option

WithGlobalAsArgs passes the CLI arguments to the local LLM binary formed from global llms.Options.

Directories

Path Synopsis
internal
localclient
Package localclient provides a client for local LLMs.
Package localclient provides a client for local LLMs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL