irtools

package module

v0.0.0-...-ab36419 Latest Latest Go to latest Published: Aug 1, 2014 License: MIT Imports: 3 Imported by: 0

Details

A set of packages for golang for Information Retrieval.

github.com/dchest/stemmer/porter2

This section is empty.

This section is empty.

func Combine(dest, source *map[string]int)

Take two term frequency maps add the source map to the destination map

func Count(terms []string) (counted map[string]int)

Take a list of terms and produce a frequency map for the list

func Difference(m1, m2 map[string]int) (count int, difference []string)

Counts the number of terms in m2 which are not in m1

func ReadLines(path string) ([]string, error)

readLines reads a whole file into memory and returns a slice of its lines.

func URLFilter(text string) (clean string, urls []string)

Removes URLs from a string, and returns the string along with any removed URLs

This section is empty.

Path	Synopsis
filter URL Filter removes URLs from the inpout tokens, and returns the remaining tokens	URL Filter removes URLs from the inpout tokens, and returns the remaining tokens
formats
index A very simply sorted map.	A very simply sorted map.
parse TweetTokenizer is a tokenizer designed explicitly for parsing Tweets and other Twitter content.	TweetTokenizer is a tokenizer designed explicitly for parsing Tweets and other Twitter content.
similarity
transform LowercaseFilter is badly named - rather than filter out lowercase characters, as the name would imply, it converts text to lowercase.	LowercaseFilter is badly named - rather than filter out lowercase characters, as the name would imply, it converts text to lowercase.