weaviate

README

---
title: "Weaviate"
lang: "en-US"
draft: false
description: "Learn about how to set up a VDP Weaviate component https://github.com/instill-ai/instill-core"
---

The Weaviate component is a data component that allows users to build and search vector datasets.
It can carry out the following tasks:

- [Vector Search](#vector-search)
- [Batch Insert](#batch-insert)
- [Insert](#insert)
- [Update](#update)
- [Delete](#delete)
- [Delete Collection](#delete-collection)



## Release Stage

`Alpha`



## Configuration

The component configuration is defined and maintained [here](https://github.com/instill-ai/component/blob/main/data/weaviate/v0/config/definition.json).




## Setup




In order to communicate with Weaviate, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
withing the component's `setup` block, or you can create a **Connection** from
the [**Integration Settings**](https://www.instill.tech/docs/vdp/integration)
page and reference the whole `setup` as `setup:
${connection.<my-connection-id>}`.

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| API Key (required) | `api-key` | string | Fill in your Weaviate API key. Please refer to clusters in Weaviate Console |
| Weaviate URL Endpoint (required) | `url` | string | Fill in your Weaviate base URL without https:// or http://. Please refer to clusters in Weaviate Console |




## Supported Tasks

### Vector Search

Perform a vector search on a collection


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_VECTOR_SEARCH` |
| Collection Name (required) | `collection-name` | string | The name of the collection to perform vector search on |
| Vector | `vector` | array[number] | An array of dimensions for the vector query, optional. If not provided, all objects will be returned |
| Limit | `limit` | integer | The limit of objects, empty for all objects |
| Fields | `fields` | array[string] | The fields to return in the objects. If empty then all fields will be returned |
| Filter | `filter` | object | The properties filter to be applied to the data with GraphQL queries, which starts with WHERE field, please refer to https://weaviate.io/developers/weaviate/search/filters |
| Tenant | `tenant` | string | The tenant to perform the vector search on |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Result | `result` | object | Result of the vector search operation |
| Status | `status` | string | Upsert status |






### Batch Insert

Insert multiple vector objects into a collection


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_BATCH_INSERT` |
| Collection Name (required) | `collection-name` | string | The name of the collection to upsert the object into |
| Array ID | `array-id` | array[string] | The array of id |
| Array Metadata (required) | `array-metadata` | array[object] | The array of vector metadata properties |
| Array Vector (required) | `array-vector` | array[array] | The array of vector values |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Status | `status` | string | Batch insert status |






### Insert

Insert a vector object into a collection


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_INSERT` |
| Collection Name (required) | `collection-name` | string | The name of the collection to upsert the object into |
| ID | `id` | string | The ID of the object |
| Vector (required) | `vector` | array[number] | An array of dimensions for the vector value |
| Metadata (required) | `metadata` | object | The vector metadata properties |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Status | `status` | string | Upsert status |






### Update

Update vector object in a collection


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_UPDATE` |
| Collection Name (required) | `collection-name` | string | The name of the collection to update the object in |
| ID (required) | `id` | string | The ID of the object |
| Update Vector | `update-vector` | array[number] | The updated vector value, optional |
| Update Metadata | `update-metadata` | object | The updated vector metadata properties, optional |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Status | `status` | string | Update status |






### Delete

Delete vector objects from a collection


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DELETE` |
| Collection Name (required) | `collection-name` | string | The name of the collection to delete the object from |
| ID | `id` | string | The ID of the object |
| Filter | `filter` | object | The properties filter to be applied to the data with GraphQL queries, which starts with WHERE field, please refer to https://weaviate.io/developers/weaviate/search/filters |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Status | `status` | string | Delete status |






### Delete Collection

Delete a collection


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_DELETE_COLLECTION` |
| Collection Name (required) | `collection-name` | string | The name of the collection to delete |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Status | `status` | string | Delete collection status |

Documentation

Index

• Constants
• func Init(bc base.Component) *component
• func VectorSearch(ctx context.Context, client weaviate.Client, inputStruct VectorSearchInput) ([]map[string]any, error)
• type BatchInsertInput
• type BatchInsertOutput
• type DeleteCollectionInput
• type DeleteCollectionOutput
• type DeleteInput
• type DeleteOutput
• type InsertInput
• type InsertOutput
• type MockWeaviateClient
• type Result
• type UpdateInput
• type UpdateOutput
• type VectorSearchInput
• type VectorSearchOutput

Constants

const (
	TaskVectorSearch     = "TASK_VECTOR_SEARCH"
	TaskInsert           = "TASK_INSERT"
	TaskUpdate           = "TASK_UPDATE"
	TaskDelete           = "TASK_DELETE"
	TaskBatchInsert      = "TASK_BATCH_INSERT"
	TaskDeleteCollection = "TASK_DELETE_COLLECTION"
)

Functions

func Init

func Init(bc base.Component) *component

func VectorSearch

func VectorSearch(ctx context.Context, client weaviate.Client, inputStruct VectorSearchInput) ([]map[string]any, error)

vector is optional, nil will return all objects fields is optional, nil will return all objects limit is optional, 0 will return all objects tenant is optional, required for multi-tenancy filter is optional, nil will have no filter

Types

type BatchInsertInput

type BatchInsertInput struct {
	ArrayID        []string         `json:"array-id"`
	CollectionName string           `json:"collection-name"`
	ArrayMetadata  []map[string]any `json:"array-metadata"`
	ArrayVector    [][]float32      `json:"array-vector"`
}

type BatchInsertOutput

type BatchInsertOutput struct {
	Status string `json:"status"`
}

type DeleteCollectionInput

type DeleteCollectionInput struct {
	CollectionName string `json:"collection-name"`
}

type DeleteCollectionOutput

type DeleteCollectionOutput struct {
	Status string `json:"status"`
}

type DeleteInput

type DeleteInput struct {
	ID             string         `json:"id"`
	CollectionName string         `json:"collection-name"`
	Filter         map[string]any `json:"filter"`
}

type DeleteOutput

type DeleteOutput struct {
	Status string `json:"status"`
}

type InsertInput

type InsertInput struct {
	ID             string         `json:"id"`
	CollectionName string         `json:"collection-name"`
	Vector         []float32      `json:"vector"`
	Metadata       map[string]any `json:"metadata"`
}

type InsertOutput

type InsertOutput struct {
	Status string `json:"status"`
}

type MockWeaviateClient

type MockWeaviateClient struct {
	Successful   int
	VectorSearch Result
}

type Result

type Result struct {
	IDs      []string         `json:"ids"`
	Objects  []map[string]any `json:"objects"`
	Vectors  [][]float32      `json:"vectors"`
	Metadata []map[string]any `json:"metadata"`
}

type UpdateInput

type UpdateInput struct {
	ID             string         `json:"id"`
	CollectionName string         `json:"collection-name"`
	Metadata       map[string]any `json:"update-metadata"`
	Vector         []float32      `json:"update-vector"`
}

type UpdateOutput

type UpdateOutput struct {
	Status string `json:"status"`
}

type VectorSearchInput

type VectorSearchInput struct {
	CollectionName string         `json:"collection-name"`
	Vector         []float32      `json:"vector"`
	Filter         map[string]any `json:"filter"`
	Limit          int            `json:"limit"`
	Fields         []string       `json:"fields"`
	Tenant         string         `json:"tenant"`
}

type VectorSearchOutput

type VectorSearchOutput struct {
	Status string `json:"status"`
	Result Result `json:"result"`
}