tokens

Documentation

Overview

Package tokens provides utilities for interacting with large language models from different providers such as OpenAI, Cohere, and Google AI. The package includes functions for counting tokens from prompts and facilitating model-specific interactions. This enables standardized processing of prompts across multiple AI providers.

Index

• type RealTokenCounter
  • func (rtc RealTokenCounter) GetNumTokensFromPrompt(prompt string, provider string, model string, key string) int
• type TokenCounter

Types

type RealTokenCounter

type RealTokenCounter struct{}

RealTokenCounter is an implementation of the TokenCounter interface that uses actual APIs. It supports multiple providers by making HTTP requests to their respective APIs to calculate the number of tokens in given text prompts.

func (RealTokenCounter) GetNumTokensFromPrompt

func (rtc RealTokenCounter) GetNumTokensFromPrompt(prompt string, provider string, model string, key string) int

GetNumTokensFromPrompt calculates the number of tokens in a given prompt. This method dispatches requests to various provider APIs based on the provider specified, and extracts the token count from the API's response.

Arguments:

• prompt: The input text to be analyzed.
• provider: The name of the AI provider, such as "OpenAI", "Cohere", or "GoogleAI".
• model: The specific model to use for the token calculation, relevant to the provider.
• key: The API key used for authentication with the provider's services.

Returns:

• An integer representing the number of tokens in the prompt, or zero if the provider is unsupported.

The function logs an error and returns zero if the provider is not supported.

type TokenCounter

type TokenCounter interface {
	GetNumTokensFromPrompt(prompt string, provider string, model string, key string) int
}

TokenCounter defines an interface for counting tokens in text prompts. It requires an implementation that can handle different providers and models, using provider-specific logic to interact with APIs or SDKs.