embeddings

README

Error Grouping using LLM Embeddings

This a simple demo of some of the error / embedding logic that we’re working on at highlight.io. To visit the demo, see our app here.

Our goal was to build two features: (1) tagging errors (e.g. deeming an error as “authentication error” or a “database error”); and (2) grouping similar errors together (e.g. two errors that have a different stacktrace and body, but are semantically not very different).

Our general approach for classifying/comparing text is as follows. As each set of tokens (i.e a string) comes in, our backend makes a request to an inference endpoint and receives a 1024-dimension float vector as a response. We then store that vector using pgvector. To compare any two sets for similarity, we simply look at the Euclidian distance between their respective embeddings using the ivfflat index implemented by pgvector.

See the full post on hacker news here.

Demo

Visit https://app.highlight.io/error-tags

Documentation

Index

• Constants
• func GetErrorObjectQuery(errorObj *model.ErrorObject) string
• func MatchErrorTag(ctx context.Context, db *gorm.DB, c Client, query string) ([]*modelInputs.MatchedErrorTag, error)
• type Client
  • func New() Client
• type EmbeddingType
• type HuggingfaceModelClient
  • func (c *HuggingfaceModelClient) GetEmbeddings(ctx context.Context, errors []*model.ErrorObject) ([]*model.ErrorObjectEmbeddings, error)
  • func (c *HuggingfaceModelClient) GetErrorTagEmbedding(ctx context.Context, title string, description string) (*model.ErrorTag, error)
  • func (c *HuggingfaceModelClient) GetStringEmbedding(ctx context.Context, input string) ([]float32, error)
• type HuggingfaceModelInputs
• type OpenAIClient
  • func (c *OpenAIClient) GetEmbeddings(ctx context.Context, errors []*model.ErrorObject) ([]*model.ErrorObjectEmbeddings, error)

Constants

const InferenceTimeout = 5 * time.Second

InferenceTimeout is max time to do inference in case api is slow. p95 ~ 0.3s

Functions

func GetErrorObjectQuery

func GetErrorObjectQuery(errorObj *model.ErrorObject) string

func MatchErrorTag

func MatchErrorTag(ctx context.Context, db *gorm.DB, c Client, query string) ([]*modelInputs.MatchedErrorTag, error)

Types

type Client

type Client interface {
	GetEmbeddings(ctx context.Context, errors []*model.ErrorObject) ([]*model.ErrorObjectEmbeddings, error)
	GetErrorTagEmbedding(ctx context.Context, title string, description string) (*model.ErrorTag, error)
	GetStringEmbedding(ctx context.Context, text string) ([]float32, error)
}

func New

func New() Client

type EmbeddingType

type EmbeddingType string

const CombinedEmbedding EmbeddingType = "CombinedEmbedding"

const EventEmbedding EmbeddingType = "EventEmbedding"

const PayloadEmbedding EmbeddingType = "PayloadEmbedding"

const StackTraceEmbedding EmbeddingType = "StackTraceEmbedding"

type HuggingfaceModelClient

type HuggingfaceModelClient struct {
	// contains filtered or unexported fields
}

func (*HuggingfaceModelClient) GetEmbeddings

func (c *HuggingfaceModelClient) GetEmbeddings(ctx context.Context, errors []*model.ErrorObject) ([]*model.ErrorObjectEmbeddings, error)

func (*HuggingfaceModelClient) GetErrorTagEmbedding

func (c *HuggingfaceModelClient) GetErrorTagEmbedding(ctx context.Context, title string, description string) (*model.ErrorTag, error)

func (*HuggingfaceModelClient) GetStringEmbedding

func (c *HuggingfaceModelClient) GetStringEmbedding(ctx context.Context, input string) ([]float32, error)

type HuggingfaceModelInputs

type HuggingfaceModelInputs struct {
	Inputs string `json:"inputs"`
}

type OpenAIClient

type OpenAIClient struct {
	// contains filtered or unexported fields
}

func (*OpenAIClient) GetEmbeddings

func (c *OpenAIClient) GetEmbeddings(ctx context.Context, errors []*model.ErrorObject) ([]*model.ErrorObjectEmbeddings, error)