docstore

README

DocStore Service

This service provides a simple, indexed key/value store for both JSON documents and Binaries. Documents are stored in a BoltDB database, and indexed (and thus searchable) using Bleve full-text search engine.

API

GRPC Handler provides the following service

service DocStore {
    rpc PutDocument (PutDocumentRequest) returns (PutDocumentResponse) {};
    rpc GetDocument (GetDocumentRequest) returns (GetDocumentResponse) {};
    rpc DeleteDocuments (DeleteDocumentsRequest) returns (DeleteDocumentsResponse) {};
    rpc CountDocuments(ListDocumentsRequest) returns (CountDocumentsResponse) {};
    rpc ListDocuments(ListDocumentsRequest) returns (stream ListDocumentsResponse) {};
}

Documents objects are use the following model:

message Document {
    string ID = 2;
    DocumentType Type = 3;
    string Owner = 4;
    bytes Data = 5;
    bytes IndexableMeta = 6;
}

Data can contain a JSON serialized string that will be actually stored, whereas IndexableMeta contains JSON that will be indexed by the search engine. This metadata can have many level depth, and keys can then be searched with Bleve query string like "Key1: value" or "+Key1.SubKey:value*"

Binaries

Binary documents are redirected at the gateway level to a dedicated S3 bucket defined in the configuration. Binary are then served directly via S3.

Documentation

Overview

Package docstore provides an indexed JSON document store.

It is used by various services to store their data instead of implementing yet-another persistence layer. It uses a combination of Bolt for storage and Bleve for indexation.

Index

• Variables
• func Migrate(ctx, fromCtx, toCtx context.Context, dryRun bool, ...) (map[string]int, error)
• type DAO

Variables

var (
	Drivers = service.StorageDrivers{}
)

Functions

func Migrate

func Migrate(ctx, fromCtx, toCtx context.Context, dryRun bool, status chan service.MigratorStatus) (map[string]int, error)

Types

type DAO

type DAO interface {
	PutDocument(ctx context.Context, storeID string, doc *docstore.Document) error
	GetDocument(ctx context.Context, storeID string, docId string) (*docstore.Document, error)
	DeleteDocument(ctx context.Context, storeID string, docID string) error
	DeleteDocuments(ctx context.Context, storeID string, query *docstore.DocumentQuery) (int, error)
	QueryDocuments(ctx context.Context, storeID string, query *docstore.DocumentQuery) (chan *docstore.Document, error)
	CountDocuments(ctx context.Context, storeID string, query *docstore.DocumentQuery) (int, error)
	ListStores(ctx context.Context) ([]string, error)
	CloseAndDrop(ctx context.Context) error
	Reset() error
}