ngram

Documentation

Index

• Variables
• func Generator(reader io.Reader, sNgram int) chan Ngram
• type FastGenerator
  • func NewFastGenerator(reader io.ReadSeeker, sNgram int) (fg FastGenerator)
  • func (fg *FastGenerator) Next() (err error)
• type Ngram
  • func NewNgram(buf []byte) Ngram
  • func (ngram *Ngram) Hash() uint64

Variables

var ErrBadNgramSize = errors.New("ErrBadNgramSize: ngram size must be in ]0;MAXINT]")

ErrBadNgramSize is raised when the ngram size is not in the correct range

Functions

func Generator

func Generator(reader io.Reader, sNgram int) chan Ngram

Generator generates Ngrams of size sNgram from a file

Types

type FastGenerator

type FastGenerator struct {
	Init   bool
	Reader io.ReadSeeker
	Ngram  Ngram
}

func NewFastGenerator

func NewFastGenerator(reader io.ReadSeeker, sNgram int) (fg FastGenerator)

func (*FastGenerator) Next

func (fg *FastGenerator) Next() (err error)

type Ngram

type Ngram []byte

func NewNgram

func NewNgram(buf []byte) Ngram

New new ngram from buffer

func (*Ngram) Hash

func (ngram *Ngram) Hash() uint64

Hash hashes a ngram