api

package
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 8, 2024 License: MIT Imports: 20 Imported by: 74

Documentation

Overview

Package api implements the client-side API for code wishing to interact with the ollama service. The methods of the Client type correspond to the ollama REST API as described in the API documentation. The ollama command-line client itself uses this package to interact with the backend service.

Examples

Several examples of using this package are available in the GitHub repository.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FormatParams 

func FormatParams(params map[string][]string) (map[string]interface{}, error)

FormatParams converts specified parameter options to their correct types

Types

type ChatRequest 

type ChatRequest struct {
	// Model is the model name, as in [GenerateRequest].
	Model string `json:"model"`

	// Messages is the messages of the chat - can be used to keep a chat memory.
	Messages []Message `json:"messages"`

	// Stream enable streaming of returned response; true by default.
	Stream *bool `json:"stream,omitempty"`

	// Format is the format to return the response in (e.g. "json").
	Format string `json:"format"`

	// KeepAlive controls how long the model will stay loaded into memory
	// followin the request.
	KeepAlive *Duration `json:"keep_alive,omitempty"`

	// Tools is an optional list of tools the model has access to.
	Tools `json:"tools,omitempty"`

	// Options lists model-specific options.
	Options map[string]interface{} `json:"options"`
}

ChatRequest describes a request sent by Client.Chat.

type ChatResponse 

type ChatResponse struct {
	Model      string    `json:"model"`
	CreatedAt  time.Time `json:"created_at"`
	Message    Message   `json:"message"`
	DoneReason string    `json:"done_reason,omitempty"`

	Done bool `json:"done"`

	Metrics
}

ChatResponse is the response returned by Client.Chat. Its fields are similar to GenerateResponse.

type ChatResponseFunc 

type ChatResponseFunc func(ChatResponse) error

ChatResponseFunc is a function that Client.Chat invokes every time a response is received from the service. If this function returns an error, Client.Chat will stop generating and return this error.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client encapsulates client state for interacting with the ollama service. Use ClientFromEnvironment to create new Clients.

func ClientFromEnvironment 

func ClientFromEnvironment() (*Client, error)

ClientFromEnvironment creates a new Client using configuration from the environment variable OLLAMA_HOST, which points to the network host and port on which the ollama service is listenting. The format of this variable is: 

<scheme>://<host>:<port>

If the variable is not specified, a default ollama host and port will be used.

func NewClient added in v0.1.33 

func NewClient(base *url.URL, http *http.Client) *Client

func (*Client) Chat 

func (c *Client) Chat(ctx context.Context, req *ChatRequest, fn ChatResponseFunc) error

Chat generates the next message in a chat. ChatRequest may contain a sequence of messages which can be used to maintain chat history with a model. fn is called for each response (there may be multiple responses, e.g. if case streaming is enabled).

func (*Client) Copy 

func (c *Client) Copy(ctx context.Context, req *CopyRequest) error

Copy copies a model - creating a model with another name from an existing model.

func (*Client) Create 

func (c *Client) Create(ctx context.Context, req *CreateRequest, fn CreateProgressFunc) error

Create creates a model from a Modelfile. fn is a progress function that behaves similarly to other methods (see Client.Pull).

func (*Client) CreateBlob 

func (c *Client) CreateBlob(ctx context.Context, digest string, r io.Reader) error

CreateBlob creates a blob from a file on the server. digest is the expected SHA256 digest of the file, and r represents the file.

func (*Client) Delete 

func (c *Client) Delete(ctx context.Context, req *DeleteRequest) error

Delete deletes a model and its data.

func (*Client) Embed added in v0.2.6 

func (c *Client) Embed(ctx context.Context, req *EmbedRequest) (*EmbedResponse, error)

Embed generates embeddings from a model.

func (*Client) Embeddings 

func (c *Client) Embeddings(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)

Embeddings generates an embedding from a model.

func (*Client) Generate 

func (c *Client) Generate(ctx context.Context, req *GenerateRequest, fn GenerateResponseFunc) error

Generate generates a response for a given prompt. The req parameter should be populated with prompt details. fn is called for each response (there may be multiple responses, e.g. in case streaming is enabled).

func (*Client) Heartbeat 

func (c *Client) Heartbeat(ctx context.Context) error

Heartbeat checks if the server has started and is responsive; if yes, it returns nil, otherwise an error.

func (*Client) List 

func (c *Client) List(ctx context.Context) (*ListResponse, error)

List lists models that are available locally.

func (*Client) ListRunning added in v0.1.38 

func (c *Client) ListRunning(ctx context.Context) (*ProcessResponse, error)

ListRunning lists running models.

func (*Client) Pull 

func (c *Client) Pull(ctx context.Context, req *PullRequest, fn PullProgressFunc) error

Pull downloads a model from the ollama library. fn is called each time progress is made on the request and can be used to display a progress bar, etc.

func (*Client) Push 

func (c *Client) Push(ctx context.Context, req *PushRequest, fn PushProgressFunc) error

Push uploads a model to the model library; requires registering for ollama.ai and adding a public key first. fn is called each time progress is made on the request and can be used to display a progress bar, etc.

func (*Client) Show 

func (c *Client) Show(ctx context.Context, req *ShowRequest) (*ShowResponse, error)

Show obtains model information, including details, modelfile, license etc.

func (*Client) Version 

func (c *Client) Version(ctx context.Context) (string, error)

Version returns the Ollama server version as a string.

type CopyRequest 

type CopyRequest struct {
	Source      string `json:"source"`
	Destination string `json:"destination"`
}

CopyRequest is the request passed to Client.Copy.

type CreateProgressFunc 

type CreateProgressFunc func(ProgressResponse) error

CreateProgressFunc is a function that Client.Create invokes when progress is made. It's similar to other progress function types like PullProgressFunc.

type CreateRequest 

type CreateRequest struct {
	Model     string `json:"model"`
	Modelfile string `json:"modelfile"`
	Stream    *bool  `json:"stream,omitempty"`
	Quantize  string `json:"quantize,omitempty"`

	// Deprecated: set the model name with Model instead
	Name string `json:"name"`

	// Deprecated: set the file content with Modelfile instead
	Path string `json:"path"`

	// Deprecated: use Quantize instead
	Quantization string `json:"quantization,omitempty"`
}

CreateRequest is the request passed to Client.Create.

type DeleteRequest 

type DeleteRequest struct {
	Model string `json:"model"`

	// Deprecated: set the model name with Model instead
	Name string `json:"name"`
}

DeleteRequest is the request passed to Client.Delete.

type Duration 

type Duration struct {
	time.Duration
}

func (Duration) MarshalJSON added in v0.1.34 

func (d Duration) MarshalJSON() ([]byte, error)

func (*Duration) UnmarshalJSON 

func (d *Duration) UnmarshalJSON(b []byte) (err error)

type EmbedRequest added in v0.2.6 

type EmbedRequest struct {
	// Model is the model name.
	Model string `json:"model"`

	// Input is the input to embed.
	Input any `json:"input"`

	// KeepAlive controls how long the model will stay loaded in memory following
	// this request.
	KeepAlive *Duration `json:"keep_alive,omitempty"`

	Truncate *bool `json:"truncate,omitempty"`

	// Options lists model-specific options.
	Options map[string]interface{} `json:"options"`
}

EmbedRequest is the request passed to Client.Embed.

type EmbedResponse added in v0.2.6 

type EmbedResponse struct {
	Model      string      `json:"model"`
	Embeddings [][]float32 `json:"embeddings"`

	TotalDuration   time.Duration `json:"total_duration,omitempty"`
	LoadDuration    time.Duration `json:"load_duration,omitempty"`
	PromptEvalCount int           `json:"prompt_eval_count,omitempty"`
}

EmbedResponse is the response from Client.Embed.

type EmbeddingRequest 

type EmbeddingRequest struct {
	// Model is the model name.
	Model string `json:"model"`

	// Prompt is the textual prompt to embed.
	Prompt string `json:"prompt"`

	// KeepAlive controls how long the model will stay loaded in memory following
	// this request.
	KeepAlive *Duration `json:"keep_alive,omitempty"`

	// Options lists model-specific options.
	Options map[string]interface{} `json:"options"`
}

EmbeddingRequest is the request passed to Client.Embeddings.

type EmbeddingResponse 

type EmbeddingResponse struct {
	Embedding []float64 `json:"embedding"`
}

EmbeddingResponse is the response from Client.Embeddings.

type GenerateRequest 

type GenerateRequest struct {
	// Model is the model name; it should be a name familiar to Ollama from
	// the library at https://ollama.com/library
	Model string `json:"model"`

	// Prompt is the textual prompt to send to the model.
	Prompt string `json:"prompt"`

	// Suffix is the text that comes after the inserted text.
	Suffix string `json:"suffix"`

	// System overrides the model's default system message/prompt.
	System string `json:"system"`

	// Template overrides the model's default prompt template.
	Template string `json:"template"`

	// Context is the context parameter returned from a previous call to
	// Generate call. It can be used to keep a short conversational memory.
	Context []int `json:"context,omitempty"`

	// Stream specifies whether the response is streaming; it is true by default.
	Stream *bool `json:"stream,omitempty"`

	// Raw set to true means that no formatting will be applied to the prompt.
	Raw bool `json:"raw,omitempty"`

	// Format specifies the format to return a response in.
	Format string `json:"format"`

	// KeepAlive controls how long the model will stay loaded in memory following
	// this request.
	KeepAlive *Duration `json:"keep_alive,omitempty"`

	// Images is an optional list of base64-encoded images accompanying this
	// request, for multimodal models.
	Images []ImageData `json:"images,omitempty"`

	// Options lists model-specific options. For example, temperature can be
	// set through this field, if the model supports it.
	Options map[string]interface{} `json:"options"`
}

GenerateRequest describes a request sent by Client.Generate. While you have to specify the Model and Prompt fields, all the other fields have reasonable defaults for basic uses.

type GenerateResponse 

type GenerateResponse struct {
	// Model is the model name that generated the response.
	Model string `json:"model"`

	// CreatedAt is the timestamp of the response.
	CreatedAt time.Time `json:"created_at"`

	// Response is the textual response itself.
	Response string `json:"response"`

	// Done specifies if the response is complete.
	Done bool `json:"done"`

	// DoneReason is the reason the model stopped generating text.
	DoneReason string `json:"done_reason,omitempty"`

	// Context is an encoding of the conversation used in this response; this
	// can be sent in the next request to keep a conversational memory.
	Context []int `json:"context,omitempty"`

	Metrics
}

GenerateResponse is the response passed into GenerateResponseFunc.

type GenerateResponseFunc 

type GenerateResponseFunc func(GenerateResponse) error

GenerateResponseFunc is a function that Client.Generate invokes every time a response is received from the service. If this function returns an error, Client.Generate will stop generating and return this error.

type ImageData 

type ImageData []byte

ImageData represents the raw binary data of an image file.

type ListModelResponse added in v0.1.42 

type ListModelResponse struct {
	Name       string       `json:"name"`
	Model      string       `json:"model"`
	ModifiedAt time.Time    `json:"modified_at"`
	Size       int64        `json:"size"`
	Digest     string       `json:"digest"`
	Details    ModelDetails `json:"details,omitempty"`
}

ListModelResponse is a single model description in ListResponse.

type ListResponse 

type ListResponse struct {
	Models []ListModelResponse `json:"models"`
}

ListResponse is the response from Client.List.

type Message 

type Message struct {
	Role      string      `json:"role"`
	Content   string      `json:"content"`
	Images    []ImageData `json:"images,omitempty"`
	ToolCalls []ToolCall  `json:"tool_calls,omitempty"`
}

Message is a single message in a chat sequence. The message contains the role ("system", "user", or "assistant"), the content and an optional list of images.

func (*Message) UnmarshalJSON added in v0.2.6 

func (m *Message) UnmarshalJSON(b []byte) error

type Metrics 

type Metrics struct {
	TotalDuration      time.Duration `json:"total_duration,omitempty"`
	LoadDuration       time.Duration `json:"load_duration,omitempty"`
	PromptEvalCount    int           `json:"prompt_eval_count,omitempty"`
	PromptEvalDuration time.Duration `json:"prompt_eval_duration,omitempty"`
	EvalCount          int           `json:"eval_count,omitempty"`
	EvalDuration       time.Duration `json:"eval_duration,omitempty"`
}

func (*Metrics) Summary 

func (m *Metrics) Summary()

type ModelDetails 

type ModelDetails struct {
	ParentModel       string   `json:"parent_model"`
	Format            string   `json:"format"`
	Family            string   `json:"family"`
	Families          []string `json:"families"`
	ParameterSize     string   `json:"parameter_size"`
	QuantizationLevel string   `json:"quantization_level"`
}

ModelDetails provides details about a model.

type Options 

type Options struct {
	Runner

	// Predict options used at runtime
	NumKeep          int      `json:"num_keep,omitempty"`
	Seed             int      `json:"seed,omitempty"`
	NumPredict       int      `json:"num_predict,omitempty"`
	TopK             int      `json:"top_k,omitempty"`
	TopP             float32  `json:"top_p,omitempty"`
	MinP             float32  `json:"min_p,omitempty"`
	TFSZ             float32  `json:"tfs_z,omitempty"`
	TypicalP         float32  `json:"typical_p,omitempty"`
	RepeatLastN      int      `json:"repeat_last_n,omitempty"`
	Temperature      float32  `json:"temperature,omitempty"`
	RepeatPenalty    float32  `json:"repeat_penalty,omitempty"`
	PresencePenalty  float32  `json:"presence_penalty,omitempty"`
	FrequencyPenalty float32  `json:"frequency_penalty,omitempty"`
	Mirostat         int      `json:"mirostat,omitempty"`
	MirostatTau      float32  `json:"mirostat_tau,omitempty"`
	MirostatEta      float32  `json:"mirostat_eta,omitempty"`
	PenalizeNewline  bool     `json:"penalize_newline,omitempty"`
	Stop             []string `json:"stop,omitempty"`
}

Options specified in GenerateRequest, if you add a new option here add it to the API docs also.

func DefaultOptions 

func DefaultOptions() Options

DefaultOptions is the default set of options for GenerateRequest; these values are used unless the user specifies other values explicitly.

func (*Options) FromMap 

func (opts *Options) FromMap(m map[string]interface{}) error

type ProcessModelResponse added in v0.1.42 

type ProcessModelResponse struct {
	Name      string       `json:"name"`
	Model     string       `json:"model"`
	Size      int64        `json:"size"`
	Digest    string       `json:"digest"`
	Details   ModelDetails `json:"details,omitempty"`
	ExpiresAt time.Time    `json:"expires_at"`
	SizeVRAM  int64        `json:"size_vram"`
}

ProcessModelResponse is a single model description in ProcessResponse.

type ProcessResponse added in v0.1.42 

type ProcessResponse struct {
	Models []ProcessModelResponse `json:"models"`
}

ProcessResponse is the response from [Client.Process].

type ProgressResponse 

type ProgressResponse struct {
	Status    string `json:"status"`
	Digest    string `json:"digest,omitempty"`
	Total     int64  `json:"total,omitempty"`
	Completed int64  `json:"completed,omitempty"`
}

ProgressResponse is the response passed to progress functions like PullProgressFunc and PushProgressFunc.

type PullProgressFunc 

type PullProgressFunc func(ProgressResponse) error

PullProgressFunc is a function that Client.Pull invokes every time there is progress with a "pull" request sent to the service. If this function returns an error, Client.Pull will stop the process and return this error.

type PullRequest 

type PullRequest struct {
	Model    string `json:"model"`
	Insecure bool   `json:"insecure,omitempty"`
	Username string `json:"username"`
	Password string `json:"password"`
	Stream   *bool  `json:"stream,omitempty"`

	// Deprecated: set the model name with Model instead
	Name string `json:"name"`
}

PullRequest is the request passed to Client.Pull.

type PushProgressFunc 

type PushProgressFunc func(ProgressResponse) error

PushProgressFunc is a function that Client.Push invokes when progress is made. It's similar to other progress function types like PullProgressFunc.

type PushRequest 

type PushRequest struct {
	Model    string `json:"model"`
	Insecure bool   `json:"insecure,omitempty"`
	Username string `json:"username"`
	Password string `json:"password"`
	Stream   *bool  `json:"stream,omitempty"`

	// Deprecated: set the model name with Model instead
	Name string `json:"name"`
}

PushRequest is the request passed to Client.Push.

type RetrieveModelResponse added in v0.2.0 

type RetrieveModelResponse struct {
	Id      string `json:"id"`
	Object  string `json:"object"`
	Created int64  `json:"created"`
	OwnedBy string `json:"owned_by"`
}

type Runner 

type Runner struct {
	NumCtx    int   `json:"num_ctx,omitempty"`
	NumBatch  int   `json:"num_batch,omitempty"`
	NumGPU    int   `json:"num_gpu,omitempty"`
	MainGPU   int   `json:"main_gpu,omitempty"`
	LowVRAM   bool  `json:"low_vram,omitempty"`
	F16KV     bool  `json:"f16_kv,omitempty"` // Deprecated: This option is ignored
	LogitsAll bool  `json:"logits_all,omitempty"`
	VocabOnly bool  `json:"vocab_only,omitempty"`
	UseMMap   *bool `json:"use_mmap,omitempty"`
	UseMLock  bool  `json:"use_mlock,omitempty"`
	NumThread int   `json:"num_thread,omitempty"`
}

Runner options which must be set when the model is loaded into memory

type ShowRequest 

type ShowRequest struct {
	Model  string `json:"model"`
	System string `json:"system"`

	// Template is deprecated
	Template string `json:"template"`
	Verbose  bool   `json:"verbose"`

	Options map[string]interface{} `json:"options"`

	// Deprecated: set the model name with Model instead
	Name string `json:"name"`
}

ShowRequest is the request passed to Client.Show.

type ShowResponse 

type ShowResponse struct {
	License       string         `json:"license,omitempty"`
	Modelfile     string         `json:"modelfile,omitempty"`
	Parameters    string         `json:"parameters,omitempty"`
	Template      string         `json:"template,omitempty"`
	System        string         `json:"system,omitempty"`
	Details       ModelDetails   `json:"details,omitempty"`
	Messages      []Message      `json:"messages,omitempty"`
	ModelInfo     map[string]any `json:"model_info,omitempty"`
	ProjectorInfo map[string]any `json:"projector_info,omitempty"`
	ModifiedAt    time.Time      `json:"modified_at,omitempty"`
}

ShowResponse is the response returned from Client.Show.

type StatusError 

type StatusError struct {
	StatusCode   int
	Status       string
	ErrorMessage string `json:"error"`
}

StatusError is an error with and HTTP status code.

func (StatusError) Error 

func (e StatusError) Error() string

type TokenResponse 

type TokenResponse struct {
	Token string `json:"token"`
}

type Tool added in v0.2.6 

type Tool struct {
	Type     string       `json:"type"`
	Function ToolFunction `json:"function"`
}

func (Tool) String added in v0.3.1 

func (t Tool) String() string

type ToolCall added in v0.2.6 

type ToolCall struct {
	Function ToolCallFunction `json:"function"`
}

type ToolCallFunction added in v0.2.7 

type ToolCallFunction struct {
	Name      string                    `json:"name"`
	Arguments ToolCallFunctionArguments `json:"arguments"`
}

type ToolCallFunctionArguments added in v0.2.7 

type ToolCallFunctionArguments map[string]any

func (*ToolCallFunctionArguments) String added in v0.2.7 

func (t *ToolCallFunctionArguments) String() string

type ToolFunction added in v0.2.7 

type ToolFunction struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Parameters  struct {
		Type       string   `json:"type"`
		Required   []string `json:"required"`
		Properties map[string]struct {
			Type        string   `json:"type"`
			Description string   `json:"description"`
			Enum        []string `json:"enum,omitempty"`
		} `json:"properties"`
	} `json:"parameters"`
}

func (*ToolFunction) String added in v0.2.7 

func (t *ToolFunction) String() string

type Tools added in v0.2.7 

type Tools []Tool

func (Tools) String added in v0.2.7 

func (t Tools) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.