posseg

Documentation

Overview

Package posseg is the Golang implementation of Jieba's posseg module.

Example

package main

import (
	"fmt"

	"github.com/Soontao/jiebago/posseg"
)

func main() {
	var seg posseg.Segmenter
	seg.LoadDictionary("../dict.txt")

	for segment := range seg.Cut("我爱北京天安门", true) {
		fmt.Printf("%s %s\n", segment.Text(), segment.Pos())
	}
}

Output:

我 r
爱 v
北京 ns
天安门 ns

Index

• type Dictionary
  • func (d *Dictionary) AddToken(token dictionary.Token)
  • func (d *Dictionary) Frequency(key string) (float64, bool)
  • func (d *Dictionary) Load(ch <-chan dictionary.Token)
  • func (d *Dictionary) Pos(key string) (string, bool)
• type Segment
  • func (s Segment) Pos() string
  • func (s Segment) Text() string
• type Segmenter
  • func (seg *Segmenter) Cut(sentence string, hmm bool) <-chan Segment
  • func (seg *Segmenter) LoadDictionary(fileName string) error
  • func (seg *Segmenter) LoadUserDictionary(fileName string) error

Examples

• Package

Types

type Dictionary

type Dictionary struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

A Dictionary represents a thread-safe dictionary used for word segmentation.

func (*Dictionary) AddToken

func (d *Dictionary) AddToken(token dictionary.Token)

AddToken adds one token

func (*Dictionary) Frequency

func (d *Dictionary) Frequency(key string) (float64, bool)

Frequency returns the frequency and existence of give word

func (*Dictionary) Load

func (d *Dictionary) Load(ch <-chan dictionary.Token)

Load loads all tokens from given channel

func (*Dictionary) Pos

func (d *Dictionary) Pos(key string) (string, bool)

Pos returns the POS and existence of give word

type Segment

type Segment struct {
	// contains filtered or unexported fields
}

Segment represents a word with it's POS

func (Segment) Pos

func (s Segment) Pos() string

Pos returns the Segment's POS.

func (Segment) Text

func (s Segment) Text() string

Text returns the Segment's text.

type Segmenter

type Segmenter struct {
	// contains filtered or unexported fields
}

Segmenter is a Chinese words segmentation struct.

func (*Segmenter) Cut

func (seg *Segmenter) Cut(sentence string, hmm bool) <-chan Segment

Cut cuts a sentence into words. Parameter hmm controls whether to use the Hidden Markov Model.

func (*Segmenter) LoadDictionary

func (seg *Segmenter) LoadDictionary(fileName string) error

LoadDictionary loads dictionary from given file name. Everytime LoadDictionary is called, previously loaded dictionary will be cleard.

func (*Segmenter) LoadUserDictionary

func (seg *Segmenter) LoadUserDictionary(fileName string) error

LoadUserDictionary loads a user specified dictionary, it must be called after LoadDictionary, and it will not clear any previous loaded dictionary, instead it will override exist entries.