colltab

Documentation

Index

• type ContractTrieSet
• type Elem
  • func (ce Elem) CCC() uint8
  • func (ce Elem) Primary() int
• type Iter
  • func (i *Iter) Discard()
  • func (i *Iter) Next() bool
  • func (i *Iter) Reset(p int)
  • func (i *Iter) SetInput(s []byte)
• type Table
  • func (t *Table) AppendNext(w []Elem, src []byte) (res []Elem, n int)
• type Trie

Types

type ContractTrieSet

type ContractTrieSet []struct{ L, H, N, I uint8 }

type Elem

type Elem uint32

Elem is a representation of a collation element. This API provides ways to encode and decode Elems. Implementations of collation tables may use values greater or equal to PrivateUse for their own purposes. However, these should never be returned by AppendNext.

func (Elem) CCC

func (ce Elem) CCC() uint8

CCC returns the canonical combining class associated with the underlying character, if applicable, or 0 otherwise.

func (Elem) Primary

func (ce Elem) Primary() int

Primary returns the primary collation weight for ce.

type Iter

type Iter struct {
	Weighter *Table
	Elems    []Elem
	// N is the number of elements in Elems that will not be reordered on
	// subsequent iterations, N <= len(Elems).
	N int
	// contains filtered or unexported fields
}

An Iter incrementally converts chunks of the input text to collation elements, while ensuring that the collation elements are in normalized order (that is, they are in the order as if the input text were normalized first).

func (*Iter) Discard

func (i *Iter) Discard()

Discard removes the collation elements up to N.

func (*Iter) Next

func (i *Iter) Next() bool

Next appends Elems to the internal array. On each iteration, it will either add starters or modifiers. In the majority of cases, an Elem with a primary value > 0 will have a CCC of 0. The CCC values of collation elements are also used to detect if the input string was not normalized and to adjust the result accordingly.

func (*Iter) Reset

func (i *Iter) Reset(p int)

Reset sets the position in the current input text to p and discards any results obtained so far.

func (*Iter) SetInput

func (i *Iter) SetInput(s []byte)

SetInput resets i to input s.

type Table

type Table struct {
	Index Trie // main trie

	// expansion info
	ExpandElem []uint32

	// contraction info
	ContractTries ContractTrieSet
	ContractElem  []uint32
}

Table holds all collation data for a given collation ordering.

func (*Table) AppendNext

func (t *Table) AppendNext(w []Elem, src []byte) (res []Elem, n int)

AppendNext appends the weights corresponding to the next rune or contraction in s. If a contraction is matched to a discontinuous sequence of runes, the weights for the interstitial runes are appended as well. It returns a new slice that includes the appended weights and the number of bytes consumed from s.

type Trie

type Trie struct {
	Index0  []uint16 // index for first byte (0xC0-0xFF)
	Values0 []uint32 // index for first byte (0x00-0x7F)
	Index   []uint16
	Values  []uint32
}