localclient

Documentation

Overview

Package localclient provides a client for local LLMs.

Index

• Variables
• type Client
  • func New(binPath string, globalAsArgs bool, args ...string) (*Client, error)
  • func (c *Client) CreateCompletion(ctx context.Context, r *CompletionRequest) (*Completion, error)
• type Completion
• type CompletionRequest

Variables

var ErrEmptyResponse = errors.New("empty response")

ErrEmptyResponse is returned when the OpenAI API returns an empty response.

Types

type Client

type Client struct {
	BinPath      string
	Args         []string
	GlobalAsArgs bool
}

Client is a client for a local LLM.

func New

func New(binPath string, globalAsArgs bool, args ...string) (*Client, error)

New returns a new local client.

func (*Client) CreateCompletion

func (c *Client) CreateCompletion(ctx context.Context, r *CompletionRequest) (*Completion, error)

CreateCompletion creates a completion.

type Completion

type Completion struct {
	Text string `json:"text"`
}

Completion is a completion.

type CompletionRequest

type CompletionRequest struct {
	Prompt string `json:"prompt"`
}

CompletionRequest is a request to create a completion.