provider

Documentation

Overview

Package provider implements an abstraction layer for interacting with AI model providers (like OpenAI, Anthropic, etc.) in a consistent way. It defines interfaces and types for streaming AI completions while handling provider-specific implementation details.

Design decisions:

• Provider abstraction: Single interface that different AI providers can implement
• Streaming first: Built around streaming responses for real-time interaction
• Type-safe events: Generic types ensure compile-time correctness of response handling
• Structured metadata: Each event includes run/turn IDs and timestamps for tracking
• Error handling: Dedicated error type that preserves context and metadata
• Memory management: Integration with short-term memory for context preservation

Key concepts:

• Provider: Interface defining the contract for AI model providers
• StreamEvent: Base interface for all streaming events (chunks, responses, errors)
• CompletionParams: Configuration for chat completion requests
• Checkpoint: Captures conversation state for context management

The streaming architecture uses four main event types:

1. Delim: Delimiter events marking stream boundaries
2. Chunk: Incremental response fragments
3. Response: Complete responses with checkpoints
4. Error: Error events with preserved context

Example usage:

provider := openai.NewProvider(config)
params := CompletionParams{
    RunID:        uuid.New(),
    Instructions: "You are a helpful assistant",
    Stream:       true,
    Tools:        []tool.Definition{...},
}

events, err := provider.ChatCompletion(ctx, params)
if err != nil {
    return err
}

for event := range events {
    switch e := event.(type) {
    case Chunk[messages.AssistantMessage]:
        // Handle incremental response
    case Response[messages.AssistantMessage]:
        // Handle complete response
    case Error:
        // Handle error with context
    }
}

The package is designed to be extensible, allowing new providers to be added by implementing the Provider interface while maintaining consistent behavior and error handling across different AI model providers.

Index

• func ChunkToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Chunk[T])
• func ResponseToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Response[T])
• type Chunk
  • func (c Chunk[T]) MarshalJSON() ([]byte, error)
  • func (c *Chunk[T]) UnmarshalJSON(data []byte) error
• type CompletionParams
• type Delim
  • func (d Delim) MarshalJSON() ([]byte, error)
  • func (d *Delim) UnmarshalJSON(data []byte) error
• type Error
  • func (e Error) Error() string
  • func (e Error) MarshalJSON() ([]byte, error)
  • func (e *Error) UnmarshalJSON(data []byte) error
• type Provider
• type Response
  • func (r Response[T]) MarshalJSON() ([]byte, error)
  • func (r *Response[T]) UnmarshalJSON(data []byte) error
• type StreamEvent

Functions

func ChunkToMessage

func ChunkToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Chunk[T])

func ResponseToMessage

func ResponseToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Response[T])

Types

type Chunk

type Chunk[T messages.Response] struct {
	RunID     uuid.UUID       `json:"run_id"`
	TurnID    uuid.UUID       `json:"turn_id"`
	Chunk     T               `json:"chunk"`
	Timestamp strfmt.DateTime `json:"timestamp,omitempty"`
	Meta      gjson.Result    `json:"meta,omitempty"`
}

func (Chunk[T]) MarshalJSON

func (c Chunk[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Chunk[T]

func (*Chunk[T]) UnmarshalJSON

func (c *Chunk[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Chunk[T]

type CompletionParams

type CompletionParams struct {
	RunID          uuid.UUID
	Instructions   string
	Thread         *shorttermmemory.Aggregator
	Stream         bool
	ResponseSchema *jsonschema.Schema
	Model          interface {
		Name() string
		Provider() Provider
	}
	Tools []tool.Definition
	// contains filtered or unexported fields
}

type Delim

type Delim struct {
	RunID  uuid.UUID `json:"run_id"`
	TurnID uuid.UUID `json:"turn_id"`
	Delim  string    `json:"delim"`
}

func (Delim) MarshalJSON

func (d Delim) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Delim

func (*Delim) UnmarshalJSON

func (d *Delim) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Delim

type Error

type Error struct {
	RunID     uuid.UUID       `json:"run_id"`
	TurnID    uuid.UUID       `json:"turn_id"`
	Err       error           `json:"error"`
	Timestamp strfmt.DateTime `json:"timestamp,omitempty"`
	Meta      gjson.Result    `json:"meta,omitempty"`
}

func (Error) Error

func (e Error) Error() string

func (Error) MarshalJSON

func (e Error) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Error

func (*Error) UnmarshalJSON

func (e *Error) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Error

type Provider

type Provider interface {
	ChatCompletion(context.Context, CompletionParams) (<-chan StreamEvent, error)
}

type Response

type Response[T messages.Response] struct {
	RunID      uuid.UUID                  `json:"run_id"`
	TurnID     uuid.UUID                  `json:"turn_id"`
	Checkpoint shorttermmemory.Checkpoint `json:"checkpoint"`
	Response   T                          `json:"response"`
	Timestamp  strfmt.DateTime            `json:"timestamp,omitempty"`
	Meta       gjson.Result               `json:"meta,omitempty"`
}

func (Response[T]) MarshalJSON

func (r Response[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Response[T]

func (*Response[T]) UnmarshalJSON

func (r *Response[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Response[T]

type StreamEvent

type StreamEvent interface {
	// contains filtered or unexported methods
}