fireworksai

package
v0.29.0-beta Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 30, 2024 License: MIT Imports: 8 Imported by: 0

README 

---
title: "Fireworks AI"
lang: "en-US"
draft: false
description: "Learn about how to set up a VDP Fireworks AI component https://github.com/instill-ai/instill-core"
---

The Fireworks AI component is an AI component that allows users to connect the AI models served on the Fireworks AI Platform.
It can carry out the following tasks:
- [Text Generation Chat](#text-generation-chat)
- [Text Embeddings](#text-embeddings)

## Release Stage

`Alpha`

## Configuration

The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/component/blob/main/ai/fireworksai/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/component/blob/main/ai/fireworksai/v0/config/tasks.json) files respectively.

## Setup


In order to communicate with Fireworks AI, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's `setup` block, or you can create a **Connection** from
the [**Integration Settings**](https://www.instill.tech/docs/vdp/integration)
page and reference the whole `setup` as `setup:
${connection.<my-connection-id>}`.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| API Key | `api-key` | string | Fill in your Fireworks AI API key. To find your keys, visit the Fireworks AI API Keys page.  |

</div>




## Supported Tasks

### Text Generation Chat

Fireworks AI's text generation models (often called generative pre-trained transformers or large language models) have been trained to understand natural language, code, and images. The models provide text outputs in response to their inputs. The inputs to these models are also referred to as "prompts". Designing a prompt is essentially how you “program” a large language model model, usually by providing instructions or some examples of how to successfully complete a task

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_TEXT_GENERATION_CHAT` |
| Model Name (required) | `model` | string | The OSS model to be used |
| Prompt (required) | `prompt` | string | The prompt text |
| System message | `system-message` | string | The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. By default, the model’s behavior is set using a generic message as "You are a helpful assistant." |
| Prompt Images | `prompt-images` | array[string] | The prompt images (Note: According to Fireworks AI documentation on 2024-07-24, the total number of images included in a single API request should not exceed 30, and all the images should be smaller than 5MB in size) |
| [Chat history](#text-generation-chat-chat-history) | `chat-history` | array[object] | Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : \{"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"\} |
| Seed | `seed` | integer | The seed |
| Temperature | `temperature` | number | The temperature for sampling |
| Top K | `top-k` | integer | Integer to define the top tokens considered within the sample operation to create new text |
| Max new tokens | `max-new-tokens` | integer | The maximum number of tokens for model to generate |
| Top P | `top-p` | number | Float to define the tokens that are within the sample operation of text generation. Add tokens in the sample for more probable to least probable until the sum of the probabilities is greater than top-p (default=0.5) |
</div>


<details>
<summary> Input Objects in Text Generation Chat</summary>

<h4 id="text-generation-chat-chat-history">Chat History</h4>

Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : \{"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"\}

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| [Content](#text-generation-chat-content) | `content` | array | The message content  |
| Role | `role` | string | The message role, i.e. 'system', 'user' or 'assistant'  |
</div>
<h4 id="text-generation-chat-content">Content</h4>

The message content

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| [Image URL](#text-generation-chat-image-url) | `image-url` | object | The image URL  |
| Text | `text` | string | The text content.  |
| Type | `type` | string | The type of the content part.  <br/><details><summary><strong>Enum values</strong></summary><ul><li>`text`</li><li>`image_url`</li></ul></details>  |
</div>
<h4 id="text-generation-chat-image-url">Image URL</h4>

The image URL

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| URL | `url` | string | Either a URL of the image or the base64 encoded image data.  |
</div>
</details>



<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Text | `text` | string | Model Output |
| [Usage](#text-generation-chat-usage) (optional) | `usage` | object | Token usage on the Fireworks AI platform text generation models |
</div>

<details>
<summary> Output Objects in Text Generation Chat</summary>

<h4 id="text-generation-chat-usage">Usage</h4>

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Input Tokens | `input-tokens` | number | The input tokens used by Fireworks AI models |
| Output Tokens | `output-tokens` | number | The output tokens generated by Fireworks AI models |
</div>
</details>

### Text Embeddings

An embedding is a list of floating point numbers that captures semantic information about the text that it represents.

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_TEXT_EMBEDDINGS` |
| Model Name (required) | `model` | string | The OSS embedding model to be used |
| Text (required) | `text` | string | The text |
</div>






<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Embedding | `embedding` | array[number] | Embedding of the input text |
| [Usage](#text-embeddings-usage) (optional) | `usage` | object | Token usage on the Fireworks AI platform embedding models |
</div>

<details>
<summary> Output Objects in Text Embeddings</summary>

<h4 id="text-embeddings-usage">Usage</h4>

<div class="markdown-col-no-wrap" data-col-1 data-col-2>

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Token Count | `tokens` | number | The token count used by Fireworks AI models |
</div>
</details>
## Example Recipes

Recipe for the [Fireworks Chinese Content Writer](https://instill.tech/instill-ai/pipelines/fireworks-chinese-content-writer/playground) pipeline.

```yaml
version: v1beta
component:
  fireworks-0:
    type: fireworks-ai
    task: TASK_TEXT_GENERATION_CHAT
    input:
      max-new-tokens: 200
      model: qwen2-72b-instruct
      prompt: ${variable.prompt}
      system-message: You are a expert social media content writing assistant. Output the result in chinese for 小紅書.
      temperature: 0.05
      top-k: 10
      top-p: 0.5
    setup:
      api-key: ${secret.INSTILL_SECRET}
variable:
  prompt:
    title: prompt
    description: input prompt i.e. "寫一份生椰拿鐵文案", "write about horses"
    instill-format: string
output:
  output:
    title: output
    value: ${fireworks-0.output.text}
```

Documentation

Index

Constants

View Source 
const (
	TaskTextGenerationChat = "TASK_TEXT_GENERATION_CHAT"
	TaskTextEmbeddings     = "TASK_TEXT_EMBEDDINGS"
)

Variables

This section is empty.

Functions

func Init 

func Init(bc base.Component) *component

Types

type ChatMessage 

type ChatMessage struct {
	Role    string              `json:"role"`
	Content []MultiModalContent `json:"content"`
}

type ChatRequest 

type ChatRequest struct {
	// reference: https://docs.fireworks.ai/api-reference/post-chatcompletions on 2024-07-23
	Model             string                        `json:"model"`
	Messages          []FireworksChatRequestMessage `json:"messages"`
	Tools             *[]FireworksTool              `json:"tools,omitempty"`
	MaxTokens         int                           `json:"max_tokens"`
	PromptTruncateLen int                           `json:"prompt_truncate_len,omitempty"`
	Temperature       float32                       `json:"temperature,omitempty"`
	TopP              float32                       `json:"top_p,omitempty"`
	TopK              int                           `json:"top_k,omitempty"`
	FrequencyPenalty  float32                       `json:"frequency_penalty,omitempty"`
	PresencePenalty   float32                       `json:"presence_penalty,omitempty"`
	N                 int                           `json:"n,omitempty"`
	User              string                        `json:"user,omitempty"`
}

type ChatResponse 

type ChatResponse struct {
	ID      string             `json:"id"`
	Object  FireworksObject    `json:"object"`
	Created int64              `json:"created"`
	Model   string             `json:"model"`
	Choices []FireWorksChoice  `json:"choices"`
	Usage   FireworksChatUsage `json:"usage"`
}

type EmbedRequest 

type EmbedRequest struct {
	// reference: https://docs.fireworks.ai/api-reference/creates-an-embedding-vector-representing-the-input-text on 2024-07-24
	Model      string `json:"model"`
	Input      string `json:"input"`
	Dimensions int    `json:"dimensions,omitempty"`
}

type EmbedResponse 

type EmbedResponse struct {
	Model  string               `json:"model"`
	Data   []FireworksEmbedData `json:"data"`
	Usage  FireworksEmbedUsage  `json:"usage"`
	Object FireworksObject      `json:"object"`
}

type FireWorksChoice 

type FireWorksChoice struct {
	Index        int                          `json:"index"`
	FinishReason FireworksFinishReason        `json:"finish_reason"`
	Message      FireworksChatResponseMessage `json:"message"`
}

type FireworksChatMessageRole 

type FireworksChatMessageRole string
const (
	FireworksChatMessageRoleUser      FireworksChatMessageRole = "user"
	FireworksChatMessageRoleSystem    FireworksChatMessageRole = "system"
	FireworksChatMessageRoleAssistant FireworksChatMessageRole = "assistant"
)

type FireworksChatRequestMessage 

type FireworksChatRequestMessage struct {
	Role    FireworksChatMessageRole     `json:"role"`
	Content []FireworksMultiModalContent `json:"content"`
	Name    string                       `json:"name,omitempty"`
}

type FireworksChatResponseMessage 

type FireworksChatResponseMessage struct {
	Role      FireworksChatMessageRole `json:"role"`
	Content   string                   `json:"content"`
	ToolCalls []FireworksToolCall      `json:"tool_calls"`
}

type FireworksChatUsage 

type FireworksChatUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

type FireworksClient 

type FireworksClient struct {
	// contains filtered or unexported fields
}

func (FireworksClient) Chat 

func (c FireworksClient) Chat(request ChatRequest) (ChatResponse, error)

func (FireworksClient) Embed 

func (c FireworksClient) Embed(request EmbedRequest) (EmbedResponse, error)

type FireworksClientInterface 

type FireworksClientInterface interface {
	Chat(ChatRequest) (ChatResponse, error)
	Embed(EmbedRequest) (EmbedResponse, error)
}

type FireworksContentType 

type FireworksContentType string
const (
	FireworksContentTypeText     FireworksContentType = "text"
	FireworksContentTypeImageURL FireworksContentType = "image_url"
)

type FireworksEmbedData 

type FireworksEmbedData struct {
	Index     int             `json:"index"`
	Embedding []float32       `json:"embedding"`
	Object    FireworksObject `json:"object"`
}

type FireworksEmbedUsage 

type FireworksEmbedUsage struct {
	PromptTokens int `json:"prompt_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

type FireworksFinishReason 

type FireworksFinishReason string
const (
	FireworksFinishReasonStop   FireworksFinishReason = "stop"
	FireworksFinishReasonLength FireworksFinishReason = "length"
)

type FireworksMultiModalContent 

type FireworksMultiModalContent struct {
	ImageURL FireworksURL         `json:"image_url,omitempty"`
	Text     string               `json:"text,omitempty"`
	Type     FireworksContentType `json:"type"`
}

type FireworksObject 

type FireworksObject string
const (
	FireworksResponseObjectChatCompletion FireworksObject = "chat.completion"
	FireworksResponseObjectEmbedding      FireworksObject = "embedding"
	FireworksObjectList                   FireworksObject = "list"
)

type FireworksTool 

type FireworksTool struct {
	Type     FireworksToolType     `json:"type"`
	Function FireworksToolFunction `json:"function"`
}

type FireworksToolCall 

type FireworksToolCall struct {
	ID       string            `json:"id"`
	Type     FireworksToolType `json:"type"`
	Function struct {
		Name      string `json:"name"`
		Arguments string `json:"arguments"`
	} `json:"function"`
}

type FireworksToolFunction 

type FireworksToolFunction struct {
	Description string         `json:"description"`
	Name        string         `json:"name"`
	Parameters  map[string]any `json:"parameters"`
}

type FireworksToolType 

type FireworksToolType string
const (
	FireworksToolTypeFunction FireworksToolType = "function"
)

type FireworksURL 

type FireworksURL struct {
	URL string `json:"url"`
}

type MultiModalContent 

type MultiModalContent struct {
	ImageURL URL    `json:"image-url"`
	Text     string `json:"text"`
	Type     string `json:"type"`
}

type TaskTextEmbeddingsInput 

type TaskTextEmbeddingsInput struct {
	Text  string `json:"text"`
	Model string `json:"model"`
}

type TaskTextEmbeddingsOutput 

type TaskTextEmbeddingsOutput struct {
	Embedding []float32               `json:"embedding"`
	Usage     TaskTextEmbeddingsUsage `json:"usage"`
}

type TaskTextEmbeddingsUsage 

type TaskTextEmbeddingsUsage struct {
	Tokens int `json:"tokens"`
}

type TaskTextGenerationChatInput 

type TaskTextGenerationChatInput struct {
	ChatHistory  []ChatMessage `json:"chat-history"`
	MaxNewTokens int           `json:"max-new-tokens"`
	Model        string        `json:"model"`
	Prompt       string        `json:"prompt"`
	PromptImages []string      `json:"prompt-images"`
	Seed         int           `json:"seed"`
	SystemMsg    string        `json:"system-message"`
	Temperature  float32       `json:"temperature"`
	TopK         int           `json:"top-k"`
	TopP         float32       `json:"top-p"`
}

type TaskTextGenerationChatOuput 

type TaskTextGenerationChatOuput struct {
	Text  string                      `json:"text"`
	Usage TaskTextGenerationChatUsage `json:"usage"`
}

type TaskTextGenerationChatUsage 

type TaskTextGenerationChatUsage struct {
	InputTokens  int `json:"input-tokens"`
	OutputTokens int `json:"output-tokens"`
}

type URL 

type URL struct {
	URL string `json:"url"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.