textkit

Documentation

Overview

Package textkit provides helpers to normalize and process text.

Index

• func NormalizeHTMLToText(htmlDocument string) string
• func Summarize(text string, keepIfUnder int, truncateAfter int) string
• type TextRanker
  • func NewTextRanker() *TextRanker
  • func (t *TextRanker) RankTopNPhrases(text string, topN int) []string
  • func (t *TextRanker) RankTopNWords(text string, topN int) []string

Functions

func NormalizeHTMLToText

func NormalizeHTMLToText(htmlDocument string) string

func Summarize

func Summarize(text string, keepIfUnder int, truncateAfter int) string

Summarize returns a summary of the given text.

If the text is shorter than keepIfUnder characters, it is returned as is. The summary is then built by keeping the first paragraphs, up to truncateAfter characters.

Types

type TextRanker

type TextRanker struct {
	// contains filtered or unexported fields
}

A TextRanker normalizes and processes text to extract the top ranking phrases and keywords.

func NewTextRanker

func NewTextRanker() *TextRanker

NewTextRanker initializes and returns a new TextRanker.

func (*TextRanker) RankTopNPhrases

func (t *TextRanker) RankTopNPhrases(text string, topN int) []string

RankTopNPhrases extracts the top N ranking phrases using TextRank.

func (*TextRanker) RankTopNWords

func (t *TextRanker) RankTopNWords(text string, topN int) []string