rupture

README

rupture

An explosive companion to the bleve indexing library

Features

rupture includes the following additions to bleve:

• Flushing batches: Batches of operation which automatically flush to the underlying bleve index.
• Sharded indices: An index-like abstraction built on top of several underlying indices. Sharded indices provide lower write latencies for indices with large amounts of data.
• Index metadata: Track index version for easily managing migrations and schema changes.

Documentation

Index

• func WriteIndexMetadata(path string, meta *IndexMetadata) error
• type FlushingBatch
  • func NewFlushingBatch(index bleve.Index, maxBatchSize int) FlushingBatch
  • func NewShardedFlushingBatch(index ShardedIndex, maxBatchSize int) FlushingBatch
• type IndexMetadata
  • func ReadIndexMetadata(path string) (*IndexMetadata, error)
• type ShardedIndex
  • func NewShardedIndex(path string, mapping mapping.IndexMapping, numShards int) (ShardedIndex, error)
  • func OpenShardedIndex(path string) (ShardedIndex, error)

Functions

func WriteIndexMetadata

func WriteIndexMetadata(path string, meta *IndexMetadata) error

WriteIndexMetadata writes metadata for the index at the specified path.

Types

type FlushingBatch

type FlushingBatch interface {
	// Index adds the specified index operation batch, possibly triggering a
	// flush.
	Index(id string, data interface{}) error
	// Remove adds the specified delete operation to the batch, possibly
	// triggering a flush.
	Delete(id string) error
	// Flush flushes the batch's contents.
	Flush() error
}

FlushingBatch is a batch of operations that automatically flushes to the underlying index once it reaches a certain size.

func NewFlushingBatch

func NewFlushingBatch(index bleve.Index, maxBatchSize int) FlushingBatch

NewFlushingBatch creates a new flushing batch for the specified index. Once the number of operations in the batch reaches the specified limit, the batch automatically flushes its operations to the index.

func NewShardedFlushingBatch

func NewShardedFlushingBatch(index ShardedIndex, maxBatchSize int) FlushingBatch

NewShardedFlushingBatch creates a flushing batch with the specified batch size for the specified sharded index.

type IndexMetadata

type IndexMetadata struct {
	// The version of the data in the index. This can be useful for tracking
	// schema changes or data migrations.
	Version int `json:"version"`
}

IndexMetadata contains metadata about a bleve index.

func ReadIndexMetadata

func ReadIndexMetadata(path string) (*IndexMetadata, error)

ReadIndexMetadata returns the metadata for the index at the specified path. If no such index metadata exists, an empty metadata and a nil error are returned.

type ShardedIndex

type ShardedIndex interface {
	bleve.Index
	// contains filtered or unexported methods
}

ShardedIndex an index that is built onto of multiple underlying bleve indices (i.e. shards). Similar to bleve's index aliases, some methods may not be supported.

func NewShardedIndex

func NewShardedIndex(path string, mapping mapping.IndexMapping, numShards int) (ShardedIndex, error)

NewShardedIndex creates a sharded index at the specified path, with the specified mapping and number of shards.

func OpenShardedIndex

func OpenShardedIndex(path string) (ShardedIndex, error)

OpenShardedIndex opens a sharded index at the specified path.