provider

Documentation

Overview

Package provider implements an abstraction layer for interacting with AI model providers (like OpenAI, Anthropic, etc.) in a consistent way. It defines interfaces and types for streaming AI completions while handling provider-specific implementation details.

Design decisions:

• Provider abstraction: Single interface that different AI providers can implement
• Streaming first: Built around streaming responses for real-time interaction
• Type-safe events: Generic types ensure compile-time correctness of response handling
• Structured metadata: Each event includes run/turn IDs and timestamps for tracking
• Error handling: Dedicated error type that preserves context and metadata
• Memory management: Integration with short-term memory for context preservation

Key concepts:

• Provider: Interface defining the contract for AI model providers
• StreamEvent: Base interface for all streaming events (chunks, responses, errors)
• CompletionParams: Configuration for chat completion requests
• Checkpoint: Captures conversation state for context management

The streaming architecture uses four main event types:

1. Delim: Delimiter events marking stream boundaries
2. Chunk: Incremental response fragments
3. Response: Complete responses with checkpoints
4. Error: Error events with preserved context

Example usage:

provider := openai.NewProvider(config)
params := CompletionParams{
    RunID:        uuid.New(),
    Instructions: "You are a helpful assistant",
    Stream:       true,
    Tools:        []tool.Definition{...},
}

events, err := provider.ChatCompletion(ctx, params)
if err != nil {
    return err
}

for event := range events {
    switch e := event.(type) {
    case Chunk[messages.AssistantMessage]:
        // Handle incremental response
    case Response[messages.AssistantMessage]:
        // Handle complete response
    case Error:
        // Handle error with context
    }
}

The package is designed to be extensible, allowing new providers to be added by implementing the Provider interface while maintaining consistent behavior and error handling across different AI model providers.

Index

• func ChunkToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Chunk[T])
• func ResponseToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Response[T])
• type Chunk
  • func (c Chunk[T]) MarshalJSON() ([]byte, error)
  • func (c *Chunk[T]) UnmarshalJSON(data []byte) error
• type CompletionParams
• type Delim
  • func (d Delim) MarshalJSON() ([]byte, error)
  • func (d *Delim) UnmarshalJSON(data []byte) error
• type Error
  • func (e Error) Error() string
  • func (e Error) MarshalJSON() ([]byte, error)
  • func (e *Error) UnmarshalJSON(data []byte) error
• type Provider
• type Response
  • func (r Response[T]) MarshalJSON() ([]byte, error)
  • func (r *Response[T]) UnmarshalJSON(data []byte) error
• type StreamEvent
• type StructuredOutput

Functions

func ChunkToMessage

func ChunkToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Chunk[T])

func ResponseToMessage

func ResponseToMessage[T messages.Response, M messages.ModelMessage](dst *messages.Message[M], src Response[T])

Types

type Chunk

type Chunk[T messages.Response] struct {
	RunID     uuid.UUID       `json:"run_id"`
	TurnID    uuid.UUID       `json:"turn_id"`
	Chunk     T               `json:"chunk"`
	Timestamp strfmt.DateTime `json:"timestamp,omitempty"`
	Meta      gjson.Result    `json:"meta,omitempty"`
}

func (Chunk[T]) MarshalJSON

func (c Chunk[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Chunk[T]

func (*Chunk[T]) UnmarshalJSON

func (c *Chunk[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Chunk[T]

type CompletionParams

type CompletionParams struct {
	// RunID uniquely identifies this completion request for tracking and debugging
	RunID uuid.UUID

	// Instructions provide the system prompt or role instructions for the AI
	Instructions string

	// Thread contains the conversation history and context
	Thread *shorttermmemory.Aggregator

	// Stream indicates whether to receive responses as a stream of chunks
	// When true, responses come incrementally. When false, wait for complete response.
	Stream bool

	// ResponseSchema defines the structure for formatted output
	// When provided, the AI will attempt to format its response according to this schema
	ResponseSchema *StructuredOutput

	// Model specifies which AI model to use for this completion
	// It must provide its name and associated provider
	Model interface {
		Name() string
		Provider() Provider
	}

	// Tools defines the available functions/capabilities the AI can use
	Tools []tool.Definition
	// contains filtered or unexported fields
}

CompletionParams encapsulates all parameters needed for a chat completion request. It provides configuration for how the AI model should process the request and structure its response.

type Delim

type Delim struct {
	RunID  uuid.UUID `json:"run_id"`
	TurnID uuid.UUID `json:"turn_id"`
	Delim  string    `json:"delim"`
}

func (Delim) MarshalJSON

func (d Delim) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Delim

func (*Delim) UnmarshalJSON

func (d *Delim) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Delim

type Error

type Error struct {
	RunID     uuid.UUID       `json:"run_id"`
	TurnID    uuid.UUID       `json:"turn_id"`
	Err       error           `json:"error"`
	Timestamp strfmt.DateTime `json:"timestamp,omitempty"`
	Meta      gjson.Result    `json:"meta,omitempty"`
}

func (Error) Error

func (e Error) Error() string

func (Error) MarshalJSON

func (e Error) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Error

func (*Error) UnmarshalJSON

func (e *Error) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Error

type Provider

type Provider interface {
	ChatCompletion(context.Context, CompletionParams) (<-chan StreamEvent, error)
}

Provider defines the interface for AI model providers (e.g., OpenAI, Anthropic). Implementations of this interface handle the specifics of communicating with different AI services while maintaining a consistent interface for the rest of the application.

type Response

type Response[T messages.Response] struct {
	RunID      uuid.UUID                  `json:"run_id"`
	TurnID     uuid.UUID                  `json:"turn_id"`
	Checkpoint shorttermmemory.Checkpoint `json:"checkpoint"`
	Response   T                          `json:"response"`
	Timestamp  strfmt.DateTime            `json:"timestamp,omitempty"`
	Meta       gjson.Result               `json:"meta,omitempty"`
}

func (Response[T]) MarshalJSON

func (r Response[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for Response[T]

func (*Response[T]) UnmarshalJSON

func (r *Response[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for Response[T]

type StreamEvent

type StreamEvent interface {
	// contains filtered or unexported methods
}

type StructuredOutput

type StructuredOutput struct {
	// Name identifies this output format
	Name string

	// Description explains the purpose and usage of this format
	Description string

	// Schema defines the JSON structure that responses should follow
	Schema *jsonschema.Schema
}

StructuredOutput defines a schema for formatted AI responses. This allows requesting responses in specific formats for easier parsing and validation.