parse

Documentation

Index

• type ParsedSentence
  • func (parsedSentence *ParsedSentence) GetOriginal() string
  • func (parsedSentence *ParsedSentence) GetWords() []string
• type Rule
• type RuleDefault
  • func NewRule() *RuleDefault
  • func (r *RuleDefault) IsSentenceSeparator(rune rune) bool
  • func (r *RuleDefault) IsWordSeparator(rune rune) bool
• type Text
  • func TokenizeText(rawText string, rule Rule) Text
  • func (text *Text) Append(rawSentence string, words []string)
  • func (text *Text) GetSentences() []ParsedSentence

Types

type ParsedSentence

type ParsedSentence struct {
	// contains filtered or unexported fields
}

ParsedSentence struct contains the original raw sentences and their words.

func (*ParsedSentence) GetOriginal

func (parsedSentence *ParsedSentence) GetOriginal() string

GetOriginal method returns the original sentence as a string from a ParsedSentence struct.

func (*ParsedSentence) GetWords

func (parsedSentence *ParsedSentence) GetWords() []string

GetWords methods returns the words string slice of ParsedSentence struct.

type Rule

type Rule interface {
	IsWordSeparator(rune rune) bool
	IsSentenceSeparator(rune rune) bool
}

Rule interface and its methods make possible the polimorf usage of process how Rule retrieve tokens from text.

type RuleDefault

type RuleDefault struct {
	// contains filtered or unexported fields
}

RuleDefault struct implements the Rule interface. It contains the separator characters and can decide a character is separator or not.

func NewRule

func NewRule() *RuleDefault

NewRule constructor retrieves a RuleDefault pointer.

func (*RuleDefault) IsSentenceSeparator

func (r *RuleDefault) IsSentenceSeparator(rune rune) bool

IsSentenceSeparator method retrieves true when a character is a kind of special character and possibly it separates to words from each other.

func (*RuleDefault) IsWordSeparator

func (r *RuleDefault) IsWordSeparator(rune rune) bool

IsWordSeparator method retrieves true when a character is a kind of special character and possibly it separates to words from each other. It also checks for sentence separator by IsSentenceSeparator method.

type Text

type Text struct {
	// contains filtered or unexported fields
}

Text struct contains a parsed text.

func TokenizeText

func TokenizeText(rawText string, rule Rule) Text

TokenizeText function use the given raw text and parses by a Rule object and retrieves the parsed text in a Text struct object.

func (*Text) Append

func (text *Text) Append(rawSentence string, words []string)

Append method creates a sentence and its words and append them to the Text object.

func (*Text) GetSentences

func (text *Text) GetSentences() []ParsedSentence

GetSentences method returns ParsedSentence slice from Text struct.