engine

Documentation

Overview

Package engine is riot engine

Index

• Constants
• func GetVersion() string
• type Engine
  • func (engine *Engine) CheckMem()
  • func (engine *Engine) Close()
  • func (engine *Engine) FlushIndex()
  • func (engine *Engine) GetAllDocIds() []uint64
  • func (engine *Engine) IndexDocument(docId uint64, data types.DocIndexData, forceUpdate bool)
  • func (engine *Engine) Indexer(options types.EngineInitOptions)
  • func (engine *Engine) Init(options types.EngineInitOptions)
  • func (engine *Engine) InitStorage()
  • func (engine *Engine) NumDocumentsIndexed() uint64
  • func (engine *Engine) NumDocumentsRemoved() uint64
  • func (engine *Engine) NumTokenIndexAdded() uint64
  • func (engine *Engine) PinYin(hans string) []string
  • func (engine *Engine) Ranker(options types.EngineInitOptions)
  • func (engine *Engine) RemoveDocument(docId uint64, forceUpdate bool)
  • func (engine *Engine) Search(request types.SearchRequest) (output types.SearchResponse)
  • func (engine *Engine) Segment(content string) (keywords []string)
  • func (engine *Engine) Segspl(splData []string, num int) (Map, int)
  • func (engine *Engine) Storage()
• type Map
• type StopTokens
  • func (st *StopTokens) Init(stopTokenFile string)
  • func (st *StopTokens) IsStopToken(token string) bool

Constants

const (

	// NumNanosecondsInAMillisecond nano-seconds in a milli-second num
	NumNanosecondsInAMillisecond = 1000000
	// PersistentStorageFilePrefix persistent storage file prefix
	PersistentStorageFilePrefix = "riot"
)

Functions

func GetVersion

func GetVersion() string

GetVersion get version

Types

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine initialize the engine

func (*Engine) CheckMem

func (engine *Engine) CheckMem()

CheckMem check the memory when the memory is larger than 99.99% using the storage

func (*Engine) Close

func (engine *Engine) Close()

Close close the engine 关闭引擎

func (*Engine) FlushIndex

func (engine *Engine) FlushIndex()

FlushIndex block wait until all indexes are added 阻塞等待直到所有索引添加完毕

func (*Engine) GetAllDocIds

func (engine *Engine) GetAllDocIds() []uint64

GetAllDocIds get all the DocId from the storage database and return 从数据库遍历所有的 DocId, 并返回

func (*Engine) IndexDocument

func (engine *Engine) IndexDocument(docId uint64, data types.DocIndexData, forceUpdate bool)

IndexDocument add the document to the index 将文档加入索引

输入参数：

docId	      标识文档编号，必须唯一，docId == 0 表示非法文档（用于强制刷新索引），[1, +oo) 表示合法文档
data	      见DocIndexData注释
forceUpdate 是否强制刷新 cache，如果设为 true，则尽快添加到索引，否则等待 cache 满之后一次全量添加

注意：

1. 这个函数是线程安全的，请尽可能并发调用以提高索引速度
2. 这个函数调用是非同步的，也就是说在函数返回时有可能文档还没有加入索引中，因此 如果立刻调用Search可能无法查询到这个文档。强制刷新索引请调用FlushIndex函数。

func (*Engine) Indexer

func (engine *Engine) Indexer(options types.EngineInitOptions)

Indexer initialize the indexer channel

func (*Engine) Init

func (engine *Engine) Init(options types.EngineInitOptions)

Init initialize the engine

func (*Engine) InitStorage

func (engine *Engine) InitStorage()

InitStorage initialize the persistent storage channel

func (*Engine) NumDocumentsIndexed

func (engine *Engine) NumDocumentsIndexed() uint64

func (*Engine) NumDocumentsRemoved

func (engine *Engine) NumDocumentsRemoved() uint64

func (*Engine) NumTokenIndexAdded

func (engine *Engine) NumTokenIndexAdded() uint64

func (*Engine) PinYin

func (engine *Engine) PinYin(hans string) []string

PinYin get the Chinese alphabet and abbreviation

func (*Engine) Ranker

func (engine *Engine) Ranker(options types.EngineInitOptions)

Ranker initialize the ranker channel

func (*Engine) RemoveDocument

func (engine *Engine) RemoveDocument(docId uint64, forceUpdate bool)

RemoveDocument remove the document from the index 将文档从索引中删除

输入参数：

docId	      标识文档编号，必须唯一，docId == 0 表示非法文档（用于强制刷新索引），[1, +oo) 表示合法文档
forceUpdate 是否强制刷新 cache，如果设为 true，则尽快删除索引，否则等待 cache 满之后一次全量删除

注意：

1. 这个函数是线程安全的，请尽可能并发调用以提高索引速度
2. 这个函数调用是非同步的，也就是说在函数返回时有可能文档还没有加入索引中，因此 如果立刻调用Search可能无法查询到这个文档。强制刷新索引请调用FlushIndex函数。

func (*Engine) Search

func (engine *Engine) Search(request types.SearchRequest) (output types.SearchResponse)

Search find the document that satisfies the search criteria. This function is thread safe 查找满足搜索条件的文档，此函数线程安全

func (*Engine) Segment

func (engine *Engine) Segment(content string) (keywords []string)

Segment get the word segmentation result of the text 获取文本的分词结果, 只分词与过滤弃用词

func (*Engine) Segspl

func (engine *Engine) Segspl(splData []string, num int) (Map, int)

Segspl split seg

func (*Engine) Storage

func (engine *Engine) Storage()

Storage start the persistent storage work connection

type Map

type Map map[string][]int

Map defines the type map[string][]int

type StopTokens

type StopTokens struct {
	// contains filtered or unexported fields
}

StopTokens stop tokens map

func (*StopTokens) Init

func (st *StopTokens) Init(stopTokenFile string)

Init 从 stopTokenFile 中读入停用词，一个词一行 文档索引建立时会跳过这些停用词

func (*StopTokens) IsStopToken

func (st *StopTokens) IsStopToken(token string) bool

IsStopToken to determine whether to stop token