index

Documentation

Overview

Package index provides interfaces for indexing documents metadata and retrieving this metadata back from the index. Currently, there is only one implementation to those interfaces, using Bleve.

Index

• Constants
• func Create(path string) bleve.Index
• func CreateMapping() mapping.IndexMapping
• type Author
  • func (a Author) BleveType() string
• type BleveIndexer
  • func NewBleve(index bleve.Index, fs afero.Fs, libraryPath string, ...) *BleveIndexer
  • func (b *BleveIndexer) AddFile(file string) (string, error)
  • func (b *BleveIndexer) AddLibrary(batchSize int, forceIndexing bool) error
  • func (b *BleveIndexer) Author(slug string) (Author, error)
  • func (b *BleveIndexer) Close() error
  • func (b *BleveIndexer) Count(t string) (uint64, error)
  • func (b *BleveIndexer) Document(slug string) (Document, error)
  • func (b *BleveIndexer) Documents(IDs []string) (map[string]Document, error)
  • func (b *BleveIndexer) IndexingProgress() (Progress, error)
  • func (b *BleveIndexer) RemoveFile(file string) error
  • func (b *BleveIndexer) SameAuthors(slugID string, quantity int) ([]Document, error)
  • func (b *BleveIndexer) SameSeries(slugID string, quantity int) ([]Document, error)
  • func (b *BleveIndexer) SameSubjects(slugID string, quantity int) ([]Document, error)
  • func (b *BleveIndexer) Search(keywords string, page, resultsPerPage int) (result.Paginated[[]Document], error)
  • func (b *BleveIndexer) SearchByAuthor(authorSlug string, page, resultsPerPage int) (result.Paginated[[]Document], error)
  • func (b *BleveIndexer) Slug(document Document, batchSlugs map[string]struct{}) string
• type Document
  • func (d Document) BleveType() string
• type Progress

Constants

const (
	TypeDocument = "document"
	TypeAuthor   = "author"
)

const Version = "v4"

Version identifies the mapping used for indexing. Any changes in the mapping requires an increase of version, to signal that a new index needs to be created.

Functions

func Create

func Create(path string) bleve.Index

func CreateMapping

func CreateMapping() mapping.IndexMapping

Types

type Author

type Author struct {
	Slug string
	Name string
	Type string
}

func (Author) BleveType

func (a Author) BleveType() string

BleveType is part of the bleve.Classifier interface and its purpose is to tell the indexer the type of the document, which will be used to decide which analyzer will parse it.

type BleveIndexer

type BleveIndexer struct {
	// contains filtered or unexported fields
}

func NewBleve

func NewBleve(index bleve.Index, fs afero.Fs, libraryPath string, read map[string]metadata.Reader) *BleveIndexer

NewBleve creates a new BleveIndexer instance using the passed parameters

func (*BleveIndexer) AddFile

func (b *BleveIndexer) AddFile(file string) (string, error)

AddFile adds a file to the index

func (*BleveIndexer) AddLibrary

func (b *BleveIndexer) AddLibrary(batchSize int, forceIndexing bool) error

AddLibrary scans <libraryPath> for documents and adds them to the index in batches of <batchSize> if they haven't been previously indexed or if <forceIndexing> is true

func (*BleveIndexer) Author

func (b *BleveIndexer) Author(slug string) (Author, error)

func (*BleveIndexer) Close

func (b *BleveIndexer) Close() error

Close closes the index

func (*BleveIndexer) Count

func (b *BleveIndexer) Count(t string) (uint64, error)

Count returns the number of indexed documents

func (*BleveIndexer) Document

func (b *BleveIndexer) Document(slug string) (Document, error)

func (*BleveIndexer) Documents

func (b *BleveIndexer) Documents(IDs []string) (map[string]Document, error)

func (*BleveIndexer) IndexingProgress

func (b *BleveIndexer) IndexingProgress() (Progress, error)

func (*BleveIndexer) RemoveFile

func (b *BleveIndexer) RemoveFile(file string) error

RemoveFile removes a file from the index

func (*BleveIndexer) SameAuthors

func (b *BleveIndexer) SameAuthors(slugID string, quantity int) ([]Document, error)

SameAuthors returns an array of metadata of documents by the same authors which does not belong to the same collection

func (*BleveIndexer) SameSeries

func (b *BleveIndexer) SameSeries(slugID string, quantity int) ([]Document, error)

SameSeries returns an array of metadata of documents in the same series

func (*BleveIndexer) SameSubjects

func (b *BleveIndexer) SameSubjects(slugID string, quantity int) ([]Document, error)

SameSubjects returns an array of metadata of documents by other authors, different between each other, which have similar subjects as the passed one and does not belong to the same collection

func (*BleveIndexer) Search

func (b *BleveIndexer) Search(keywords string, page, resultsPerPage int) (result.Paginated[[]Document], error)

Search look for documents which match with the passed keywords. Returns a maximum <resultsPerPage> documents, offset by <page>

func (*BleveIndexer) SearchByAuthor

func (b *BleveIndexer) SearchByAuthor(authorSlug string, page, resultsPerPage int) (result.Paginated[[]Document], error)

func (*BleveIndexer) Slug

func (b *BleveIndexer) Slug(document Document, batchSlugs map[string]struct{}) string

As Bleve index is not updated until the batch is executed, we need to store the slugs processed in the current batch in memory to also compare the current doc slug against them.

type Document

type Document struct {
	metadata.Metadata
	ID            string
	Slug          string
	AuthorsSlugs  []string
	SeriesSlug    string
	SubjectsSlugs []string
	Highlighted   bool
	Type          string
}

func (Document) BleveType

func (d Document) BleveType() string

BleveType is part of the bleve.Classifier interface and its purpose is to tell the indexer the type of the document, which will be used to decide which analyzer will parse it.

type Progress

type Progress struct {
	RemainingTime time.Duration
	Percentage    float64
}