character

package
v2.0.0-...-6844d4a Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2024 License: Apache-2.0 Imports: 2 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CharacterTokenizer

type CharacterTokenizer struct {
	// contains filtered or unexported fields
}

func NewCharacterTokenizer

func NewCharacterTokenizer(f IsTokenRune) *CharacterTokenizer

func (*CharacterTokenizer) Tokenize

func (c *CharacterTokenizer) Tokenize(input []byte) analysis.TokenStream

type IsTokenRune

type IsTokenRune func(r rune) bool

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL