pinecone

package module
v0.0.0-...-f77c034 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2023 License: MIT Imports: 9 Imported by: 0

README

go-pinecone

Go Reference Go Report Testing Building

This package is an API client for Pinecone, a SaaS vector database. With this package, users can easily perform Index and Vector data operations in Golang projects.

Introduction

Pinecone is a cloud-based vector database that enables users to store large-scale vector data and query them efficiently.

This repo aims to provide a simple and easy-to-use client for Golang users to interface with Pinecone. It supports all the operations that are available through the Pinecone API, such as creating and deleting indexes, inserting and querying vectors, and modifying index metadata, among other functionalities.

Installation

To install, simply run the following command:

go get -u github.com/nekomeowww/go-pinecone

Documentation

For a complete reference of the functions and types, please refer to the godoc documentation.

Get started

Initialize a new Pinecone client
package main

import (
    pinecone "github.com/nekomeowww/go-pinecone"
)

func main() {
    p, err := pinecone.New(
        pinecone.WithAPIKey("YOUR_API_KEY"),
        pinecone.WithEnvironment("YOUR_ACCOUNT_REGION"),
        pinecone.WithProjectName("YOUR_PROJECT_NAME"),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Do something with the client
}
Enables debug and HTTP request dump
package main

import (
    pinecone "github.com/nekomeowww/go-pinecone"
)

func main() {
    p, err := pinecone.New(
        pinecone.WithAPIKey("YOUR_API_KEY"),
        pinecone.WithEnvironment("YOUR_ACCOUNT_REGION"),
        pinecone.WithProjectName("YOUR_PROJECT_NAME"),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Enable debug
    p = p.Debug()
}
Establish a connection to interact with Vectors
package main

import (
    pinecone "github.com/nekomeowww/go-pinecone"
)

func main() {
    p, err := pinecone.New(
        pinecone.WithAPIKey("YOUR_API_KEY"),
        pinecone.WithEnvironment("YOUR_ACCOUNT_REGION"),
        pinecone.WithProjectName("YOUR_PROJECT_NAME"),
    )
    if err != nil {
        log.Fatal(err)
    }
}

Initialize a new Index client

Vector operations are performed on a given index, we initialize a new index client like:

package main

import (
	"context"
	"fmt"
	pinecone "github.com/nekomeowww/go-pinecone"
	"log"
)

func main() {
	client, err := pinecone.NewIndexClient(
		pinecone.WithIndexName("YOUR_INDEX_NAME"),
		pinecone.WithEnvironment("YOUR_ACCOUNT_REGION"),
		pinecone.WithProjectName("YOUR_PROJECT_NAME"),
		pinecone.WithAPIKey("YOUR_API_KEY"),
	)

	if err != nil {
		log.Fatal(err)
	}
}

To describe index stats we can use:

	ctx := context.Background()

	params := pinecone.DescribeIndexStatsParams{}
	resp, err := client.DescribeIndexStats(ctx, params)
	if err != nil {
		panic(err)
	}
	for k, v := range resp.Namespaces {
		fmt.Printf("%s: %+v\n", k, v)
	}

To Upsert Vectors we do such as:

    // Use your own embedding client/library such as OpenAI embedding API
    myVectorEmbedding := embedding.New("this is a sentence I want to embed")
	
    ctx := context.Background()
    params := pinecone.UpsertVectorsParams{
        Vectors: []*pinecone.Vector{
            {
                ID: "YOUR_VECTOR_ID",
                Values: myVectorEmbedding,
                Metadata: map[string]any{"foo": "bar"}
            },
        },
    }
    resp, err := client.UpsertVectors(ctx, params)
    if err != nil {
        panic(err)
    }
    fmt.Printf("%+v\n", resp)

For a complete reference of the functions and types, please refer to the godoc documentation.

Contributing

We welcome contributions from the Golang community! If you'd like to contribute, please follow these steps:

  1. Fork this repository
  2. Create a new branch for your changes
  3. Make the changes
  4. Commit your changes with a meaningful commit message
  5. Create a pull request

Acknowledgements

License

Released under the MIT License.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrRequestFailed is returned and wrapped when a request to the Pinecone API fails.
	ErrRequestFailed = errors.New("request failed")
	// ErrIndexNotFound is returned when an index is not found.
	ErrIndexNotFound = errors.New("index not found")
	// ErrInvalidParams is returned when an invalid parameter is passed to a function.
	ErrInvalidParams = errors.New("invalid params")
)

Functions

This section is empty.

Types

type CallOptions 

type CallOptions struct {
	// contains filtered or unexported fields
}

func WithAPIKey 

func WithAPIKey(apiKey string) CallOptions

WithAPIKey sets the API key to use for the call.

func WithEnvironment 

func WithEnvironment(environment string) CallOptions

WithEnvironment sets the environment to use for the call.

func WithIndexName 

func WithIndexName(indexName string) CallOptions

func WithProjectName 

func WithProjectName(projectName string) CallOptions

WithProjectName sets the project name to use for the call.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the main entry point for the Pinecone API.

func New 

func New(callOpts ...CallOptions) (*Client, error)

New creates a new Pinecone client.

func (*Client) ConfigureIndex 

func (c *Client) ConfigureIndex(ctx context.Context, params ConfigureIndexParams) error

ConfigureIndex specifies the pod type and number of replicas for an index.

API Reference: https://docs.pinecone.io/reference/configure_index

func (*Client) CreateIndex 

func (c *Client) CreateIndex(ctx context.Context, params CreateIndexParams) error

CreateIndex creates a Pinecone index. You can use it to specify the measure of similarity, the dimension of vectors to be stored in the index, the numbers of replicas to use, and more.

API Reference: https://docs.pinecone.io/reference/create_index

func (*Client) Debug 

func (c *Client) Debug() *Client

Debug enables debug logging and http dump for the client.

func (*Client) DeleteIndex 

func (c *Client) DeleteIndex(ctx context.Context, indexName string) error

DeleteIndex deletes an existing index.

API Reference: https://docs.pinecone.io/reference/delete_index

func (*Client) DescribeIndex 

func (c *Client) DescribeIndex(ctx context.Context, indexName string) (*DescribeIndexResponse, error)

DescribeIndex gets a description of an index.

API Reference: https://docs.pinecone.io/reference/describe_index

func (*Client) ListIndexes 

func (c *Client) ListIndexes() ([]string, error)

ListIndexes returns a list of your Pinecone indexes.

API Reference: https://docs.pinecone.io/reference/list_indexes

type ConfigureIndexBodyParams 

type ConfigureIndexBodyParams struct {
	Replicas *int    `json:"replicas,omitempty"`
	PodType  *string `json:"pod_type,omitempty"`
}

type ConfigureIndexParams 

type ConfigureIndexParams struct {
	// The name of the index
	IndexName string
	// The number of replicas. Replicas duplicate
	// your index. They provide higher availability
	// and throughput.
	Replicas mo.Option[int]
	// The type of pod to use. One of s1, p1, or p2
	PodType mo.Option[CreateIndexPodType]
	PodSize mo.Option[CreateIndexPodSize]
}

type CreateIndexBodyParams 

type CreateIndexBodyParams struct {
	Name             string             `json:"name"`
	Dimension        int                `json:"dimension"`
	Metric           *CreateIndexMetric `json:"metric,omitempty"`
	Pods             *int               `json:"pods,omitempty"`
	Replicas         *int               `json:"replicas,omitempty"`
	PodType          *string            `json:"pod_type,omitempty"`
	MetadataConfig   map[string]string  `json:"metadata_config,omitempty"`
	SourceCollection *string            `json:"source_collection,omitempty"`
}

type CreateIndexMetric 

type CreateIndexMetric string
const (
	CreateIndexMetricEuclidean  CreateIndexMetric = "euclidean"
	CreateIndexMetricCosine     CreateIndexMetric = "cosine"
	CreateIndexMetricDotProduct CreateIndexMetric = "dotproduct"
)

type CreateIndexParams 

type CreateIndexParams struct {
	// Required. The name of the index to be created.
	// The maximum length is 45 characters.
	Name string
	// Required. The dimensions of the vectors to be
	// inserted in the index
	Dimension int
	// The distance metric to be used for similarity
	// search. You can use 'euclidean', 'cosine', or
	// 'dotproduct'.
	Metric mo.Option[CreateIndexMetric]
	// The number of pods for the index to use,
	// including replicas.
	Pods mo.Option[int]
	// The number of replicas. Replicas duplicate
	// your index. They provide higher availability
	// and throughput.
	Replicas mo.Option[int]
	// The type of pod to use. One of s1, p1, or p2
	PodType mo.Option[CreateIndexPodType]
	PodSize mo.Option[CreateIndexPodSize]
	// Configuration for the behavior of Pinecone's
	// internal metadata index. By default, all
	// metadata is indexed; when metadata_config is
	// present, only specified metadata fields are
	// indexed. To specify metadata fields to index,
	// provide a JSON object of the following form:
	//	{"indexed": ["example_metadata_field"]}
	MetadataConfig mo.Option[map[string]string]
	// The name of the collection to create an index
	// from
	SourceCollection mo.Option[string]
}

type CreateIndexPodSize 

type CreateIndexPodSize string
const (
	CreateIndexPodSize1 CreateIndexPodSize = "1"
	CreateIndexPodSize2 CreateIndexPodSize = "2"
	CreateIndexPodSize4 CreateIndexPodSize = "4"
	CreateIndexPodSize8 CreateIndexPodSize = "8"
)

type CreateIndexPodType 

type CreateIndexPodType string
const (
	CreateIndexPodTypeS1 CreateIndexPodType = "s1"
	CreateIndexPodTypeP1 CreateIndexPodType = "p1"
	CreateIndexPodTypeP2 CreateIndexPodType = "p2"
)

type Database 

type Database struct {
	Name      string `json:"name"`
	Metric    string `json:"metric"`
	Dimension int    `json:"dimension"`
	Replicas  int    `json:"replicas"`
	Shards    int    `json:"shards"`
	Pods      int    `json:"pods"`
	PodType   string `json:"pod_type"`
}

type DeleteVectorsParams 

type DeleteVectorsParams struct {
	IDs       []string       `json:"ids"`
	Namespace string         `json:"namespace"`
	DeleteAll bool           `json:"deleteAll"`
	Filter    map[string]any `json:"filter"`
}

DeleteVectorsParams represents the parameters for a delete vectors request. See https://docs.pinecone.io/reference/delete_post for more information.

type DescribeIndexResponse 

type DescribeIndexResponse struct {
	Database Database `json:"database"`
	Status   Status   `json:"status"`
}

type DescribeIndexStatsParams 

type DescribeIndexStatsParams struct {
	Filter map[string]any `json:"filter"`
}

DescribeIndexStatsParams represents the parameters for a describe index stats request.

type DescribeIndexStatsResponse 

type DescribeIndexStatsResponse struct {
	Namespaces       map[string]*VectorCount `json:"namespaces"`
	Dimensions       int64                   `json:"dimensions"`
	IndexFullness    float32                 `json:"indexFullness"`
	TotalVectorCount int64                   `json:"totalVectorCount"`
}

DescribeIndexStatsResponse represents the response from a describe index stats request.

type FetchVectorsParams 

type FetchVectorsParams struct {
	IDs       []string `json:"ids"`
	Namespace string   `json:"namespace"`
}

FetchVectorsParams represents the parameters for a fetch vectors request.

type FetchVectorsResponse 

type FetchVectorsResponse struct {
	Vectors   map[string]*Vector `json:"vectors"`
	Namespace string             `json:"namespace"`
}

FetchVectorsResponse represents the response from a fetch vectors request.

type IndexClient 

type IndexClient struct {
	// contains filtered or unexported fields
}

IndexClient client for vector operations

func NewIndexClient 

func NewIndexClient(opts ...CallOptions) (*IndexClient, error)

func (*IndexClient) Debug 

func (ic *IndexClient) Debug() *IndexClient

func (*IndexClient) DeleteVectors 

func (ic *IndexClient) DeleteVectors(ctx context.Context, params DeleteVectorsParams) error

DeleteVectors performs a delete vectors request. returns an error if the request fails and nil otherwise.

func (*IndexClient) DescribeIndexStats 

func (ic *IndexClient) DescribeIndexStats(ctx context.Context, params DescribeIndexStatsParams) (*DescribeIndexStatsResponse, error)

DescribeIndexStats returns the index stats for the given index.

func (*IndexClient) FetchVectors 

func (ic *IndexClient) FetchVectors(ctx context.Context, params FetchVectorsParams) (*FetchVectorsResponse, error)

FetchVectors performs a fetch vectors request. See https://docs.pinecone.io/reference/fetch for more information.

func (*IndexClient) Query 

func (ic *IndexClient) Query(ctx context.Context, params QueryParams) (*QueryResponse, error)

Query performs a query request.

func (*IndexClient) UpdateVector 

func (ic *IndexClient) UpdateVector(ctx context.Context, params UpdateVectorParams) error

UpdateVector performs an update vector request. See https://docs.pinecone.io/reference/update for more information.

func (*IndexClient) UpsertVectors 

func (ic *IndexClient) UpsertVectors(ctx context.Context, params UpsertVectorsParams) (*UpsertVectorsResponse, error)

UpsertVectors performs an upsert vectors request. See https://docs.pinecone.io/reference/upsert for more information.

type QueryParams 

type QueryParams struct {
	Filter          map[string]any `json:"filter"`
	IncludeValues   bool           `json:"includeValues"`
	IncludeMetadata bool           `json:"includeMetadata"`
	Vector          []float32      `json:"vector"`
	SparseVector    *SparseVector  `json:"sparseVector"`
	Namespace       string         `json:"namespace"`
	TopK            int64          `json:"topK"`
	ID              string         `json:"id"`
}

QueryParams represents the parameters for a query request. See https://docs.pinecone.io/reference/query for more information.

type QueryResponse 

type QueryResponse struct {
	Matches   []*QueryVector `json:"matches"`
	Namespace string         `json:"namespace"`
}

QueryResponse represents the response from a query request.

type QueryVector 

type QueryVector struct {
	Vector
	Score float32 `json:"score"`
}

QueryVector represents a scored vector.

type SparseVector 

type SparseVector struct {
	Indices []int32   `json:"indices"`
	Values  []float32 `json:"values"`
}

SparseVector represents a sparse vector.

type Status 

type Status struct {
	Waiting []interface{} `json:"waiting"`
	Crashed []interface{} `json:"crashed"`
	Host    string        `json:"host"`
	Port    int           `json:"port"`
	State   string        `json:"state"`
	Ready   bool          `json:"ready"`
}

type UpdateVectorParams 

type UpdateVectorParams struct {
	Values       []float32      `json:"values"`
	SparseValues *SparseVector  `json:"sparseValues"`
	SetMetadata  map[string]any `json:"setMetadata"`
	ID           string         `json:"id"`
	Namespace    string         `json:"namespace"`
}

UpdateVectorParams represents the parameters for an update vector request.

type UpsertVectorsParams 

type UpsertVectorsParams struct {
	Vectors   []*Vector `json:"vectors"`
	Namespace string    `json:"namespace"`
}

UpsertVectorsParams represents the parameters for an upsert vectors request. See https://docs.pinecone.io/reference/upsert for more information.

type UpsertVectorsResponse 

type UpsertVectorsResponse struct {
	UpsertedCount int `json:"upsertedCount"`
}

UpsertVectorsResponse represents the response from an upsert vectors request.

type Vector 

type Vector struct {
	ID           string         `json:"id"`
	Values       []float32      `json:"values"`
	SparseValues *SparseVector  `json:"sparseValues"`
	Metadata     map[string]any `json:"metadata"`
}

Vector represents a struct with shared fields for vectors

type VectorCount 

type VectorCount struct {
	VectorCount int64 `json:"vectorCount"`
}

VectorCount represents the number of vectors in a namespace.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.