Versions in this module Expand all Collapse all v1 v1.0.1 Feb 3, 2023 Changes in this version + type Encoder struct + func NewEncoder() (*Encoder, error) + func NewEncoderWithVocab(bpeData []byte, jsonEncoder []byte) (*Encoder, error) + func (e *Encoder) Decode(tokens []int) string + func (e *Encoder) Encode(text string) ([]int, error)
Go to main page