posseg

Documentation

Overview

Package posseg is the Golang implementation of Jieba's posseg module.

Example

package main

import (
	"fmt"

	"github.com/fumiama/jieba/posseg"
)

func main() {
	seg, err := posseg.LoadDictionaryAt("../dict.txt")
	if err != nil {
		panic(err)
	}

	for _, segment := range seg.Cut("我爱北京天安门", true) {
		fmt.Printf("%s %s\n", segment.Text(), segment.Pos())
	}
}

Output:

我 r
爱 v
北京 ns
天安门 ns

Index

• type Dictionary
  • func (d *Dictionary) AddToken(token dictionary.Token)
  • func (d *Dictionary) Frequency(key string) (float64, bool)
  • func (d *Dictionary) Load(tokens ...dictionary.Token)
  • func (d *Dictionary) Pos(key string) (string, bool)
• type Segment
  • func (s Segment) Pos() string
  • func (s Segment) Text() string
• type Segmenter
  • func LoadDictionary(file io.Reader) (*Segmenter, error)
  • func LoadDictionaryAt(file string) (*Segmenter, error)
  • func (seg *Segmenter) Cut(sentence string, hmm bool) (results []Segment)
  • func (seg *Segmenter) LoadUserDictionary(file io.Reader) error
  • func (seg *Segmenter) LoadUserDictionaryAt(fileName string) error

Examples

• Package

Types

type Dictionary

type Dictionary struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

A Dictionary represents a thread-safe dictionary used for word segmentation.

func (*Dictionary) AddToken

func (d *Dictionary) AddToken(token dictionary.Token)

AddToken adds one token

func (*Dictionary) Frequency

func (d *Dictionary) Frequency(key string) (float64, bool)

Frequency returns the frequency and existence of give word

func (*Dictionary) Load

func (d *Dictionary) Load(tokens ...dictionary.Token)

Load loads all tokens

func (*Dictionary) Pos

func (d *Dictionary) Pos(key string) (string, bool)

Pos returns the POS and existence of give word

type Segment

type Segment struct {
	// contains filtered or unexported fields
}

Segment represents a word with it's POS

func (Segment) Pos

func (s Segment) Pos() string

Pos returns the Segment's POS.

func (Segment) Text

func (s Segment) Text() string

Text returns the Segment's text.

type Segmenter

type Segmenter Dictionary

Segmenter is a Chinese words segmentation struct.

func LoadDictionary

func LoadDictionary(file io.Reader) (*Segmenter, error)

LoadDictionary loads dictionary from given file name. Everytime LoadDictionaryAt is called, previously loaded dictionary will be cleard.

func LoadDictionaryAt

func LoadDictionaryAt(file string) (*Segmenter, error)

LoadDictionaryAt loads dictionary from given file name. Everytime LoadDictionaryAt is called, previously loaded dictionary will be cleard.

func (*Segmenter) Cut

func (seg *Segmenter) Cut(sentence string, hmm bool) (results []Segment)

Cut cuts a sentence into words. Parameter hmm controls whether to use the Hidden Markov Model.

func (*Segmenter) LoadUserDictionary

func (seg *Segmenter) LoadUserDictionary(file io.Reader) error

LoadUserDictionary loads a user specified dictionary, it must be called after LoadDictionary, and it will not clear any previous loaded dictionary, instead it will override exist entries.

func (*Segmenter) LoadUserDictionaryAt

func (seg *Segmenter) LoadUserDictionaryAt(fileName string) error

LoadUserDictionaryAt loads a user specified dictionary, it must be called after LoadDictionary, and it will not clear any previous loaded dictionary, instead it will override exist entries.