universe

Documentation

Index

• type Chunk
  • func NewChunk(content []byte) *Chunk
  • func UnmarshalCompressed(data []byte) (c *Chunk, err error)
  • func (c *Chunk) Hash() string
  • func (c *Chunk) MarshalCompressed() (hash string, data []byte, err error)
• type Universe
  • func Open(path string) (u *Universe, err error)
  • func (u *Universe) Close() error

Types

type Chunk

type Chunk struct {
	Content   []byte
	Embedding []float64
	// contains filtered or unexported fields
}

Chunk is a chunk of content.

func NewChunk

func NewChunk(content []byte) *Chunk

NewChunk creates a new chunk from content.

func UnmarshalCompressed

func UnmarshalCompressed(data []byte) (c *Chunk, err error)

UnmarshalCompressed unmarshals a chunk from compressed data.

func (*Chunk) Hash

func (c *Chunk) Hash() string

Hash returns the hash of a chunk.

func (*Chunk) MarshalCompressed

func (c *Chunk) MarshalCompressed() (hash string, data []byte, err error)

MarshalCompressed marshals a chunk to compressed data for storing in the database. Like git, we add an object header, then compute the hash of the header and content, then compress the header and content.

type Universe

type Universe struct {
	// contains filtered or unexported fields
}

func Open

func Open(path string) (u *Universe, err error)

Open opens the universe, creating kv db and its buckets if it doesn't exist.

Buckets: - name: chunk, key: content hash, value: content chunk - name: embedding, key: content hash, value: embedding - name: root, key: document path, value: merkle hash - name: tree, key: merkle hash, value: list of hashes of trees or chunks

func (*Universe) Close

func (u *Universe) Close() error

Close closes the universe.