rank

Documentation

Index

• func Dedupe(items []*Item, limit int) []string
• func Filter(strs []string) []string
• func Normalize(sanitized string) (string, bool)
• func RegisterStopWords(words []string)
• type Item
  • func ParseHTML(lines []string, verbose bool) []*Item
  • func ParseText(tokens []string) []*Item

Functions

func Dedupe

func Dedupe(items []*Item, limit int) []string

Dedupe ...

func Filter

func Filter(strs []string) []string

Filter ...

func Normalize

func Normalize(sanitized string) (string, bool)

Normalize sanitizes word and tells whether it is allowed token or not.

func RegisterStopWords

func RegisterStopWords(words []string)

RegisterStopWords ...

Types

type Item

type Item struct {
	Value string
	Score float64
}

Item holds some arbitrary string value (e.g. a word) along with some extra data about it.

func ParseHTML

func ParseHTML(lines []string, verbose bool) []*Item

ParseHTML receives lines of raw strings from the Web and produces result of prioritised tags based on the importance of HTML tags which wrap sentences.

Example:

<h1>A story about foo
<p> Foo was a good guy but, had a quite poor time management skills,
therefore he had issues with shipping all his tasks. Though foo had heaps
of other amazing skills, which were appreciated by his management.

Result:

foo: 5 + 1, story: 5, management: 1 + 1, skills: 1 + 1.

func ParseText

func ParseText(tokens []string) []*Item

ParseText ...