utils

Documentation

Index

• func Intersection(a []int, b []int) []int
• func LoadDocuments(path string) ([]document, error)
• type Index
  • func (idx Index) Add(docs []document)
  • func (idx Index) Search(text string) []int

Functions

func Intersection

func Intersection(a []int, b []int) []int

intersection returns the set intersection between a and b. a and b have to be sorted in ascending order and contain no duplicates.

func LoadDocuments

func LoadDocuments(path string) ([]document, error)

loadDocuments loads a Wikipedia abstract dump and returns a slice of documents. Dump example: https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-abstract1.xml.gz

Types

type Index

type Index map[string][]int

Index is an inverted index. It maps tokens to document IDs.

func (Index) Add

func (idx Index) Add(docs []document)

add adds documents to the Index.

func (Index) Search

func (idx Index) Search(text string) []int

search queries the Index for the given text.