scanner

Documentation

Overview

Package scanner implements a lexical scanner for TOML, as defined by the TOML v1 specification https://toml.io/en/v1.0.0.

Index

• func Escape(src string) []byte
• func EscapeMultiline(src string) []byte
• func IsWord(s string) bool
• func Unescape(src []byte) ([]byte, error)
• type LineCol
  • func (lc LineCol) String() string
• type Location
  • func (loc Location) String() string
• type Scanner
  • func New(r io.Reader) *Scanner
  • func (s *Scanner) Err() error
  • func (s *Scanner) Location() Location
  • func (s *Scanner) Next() error
  • func (s *Scanner) Prev() Token
  • func (s *Scanner) Span() Span
  • func (s *Scanner) Text() []byte
  • func (s *Scanner) Token() Token
• type Span
  • func (s Span) String() string
• type Token
  • func (t Token) IsValue() bool
  • func (t Token) String() string

Functions

func Escape

func Escape(src string) []byte

Escape encodes a string to escape characters for a TOML basic string. The result is not quoted, the caller must add quotation marks.

func EscapeMultiline

func EscapeMultiline(src string) []byte

EscapeMultiline encodes a string to escape characters for a TOML multi-line basic string. The result is not quoted, the caller must add quotation marks.

func IsWord

func IsWord(s string) bool

IsWord reports whether s can be encoded as a bare (unquoted) word in a TOML key-value pair or table name.

func Unescape

func Unescape(src []byte) ([]byte, error)

Unescape decodes a byte slice containing a TOML basic string. The input must have the enclosing double quotation marks already removed.

Escape sequences are replaced with their unescaped equivalents. Invalid escapes are replaced by the Unicode replacement rune. Unescape reports an error for an incomplete escape sequence.

Types

type LineCol

type LineCol struct {
	Line   int // line number, 1-based
	Column int // byte offset of column in line, 0-based
}

A LineCol describes the line number and column offset of a location in source text.

func (LineCol) String

func (lc LineCol) String() string

type Location

type Location struct {
	Span
	First, Last LineCol
}

A Location describes the complete location of a range of source text, including line and column offsets.

func (Location) String

func (loc Location) String() string

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

A Scanner reads lexical tokens from an input stream. Each call to Next advances the scanner to the next token, or reports an error.

func New

func New(r io.Reader) *Scanner

New constructs a new lexical scanner that consumes input from r.

func (*Scanner) Err

func (s *Scanner) Err() error

Err returns the last error reported by Next.

func (*Scanner) Location

func (s *Scanner) Location() Location

Location returns the complete location of the current token.

func (*Scanner) Next

func (s *Scanner) Next() error

Next advances s to the next token of the input, or reports an error. At the end of the input, Next returns io.EOF.

func (*Scanner) Prev

func (s *Scanner) Prev() Token

Prev returns the type of the immediately prior token, or Invalid. Whitespace resets the previous token.

func (*Scanner) Span

func (s *Scanner) Span() Span

Span returns the location span of the current token.

func (*Scanner) Text

func (s *Scanner) Text() []byte

Text returns the undecoded text of the current token. The return value is only valid until the next call of Next. The caller must copy the contents of the returned slice if it is needed beyond that.

func (*Scanner) Token

func (s *Scanner) Token() Token

Token returns the type of the current token.

type Span

type Span struct {
	Pos int // the start offset, 0-based
	End int // the end offset, 0-based (noninclusive)
}

A Span describes a contiguous span of a source input.

func (Span) String

func (s Span) String() string

type Token

type Token byte

Token is the type of a lexical token in the TOML grammar.

const (
	Invalid       Token = iota // invalid token
	Comment                    // single line-comment (# xxx)
	Newline                    // line break
	Word                       // unquoted word (foo, 0-bar-1)
	String                     // basic string ("xxx")
	MString                    // multiline basic string ("""xxx""")
	LString                    // literal string ('xxx')
	MLString                   // multiline literal string (”'xxx”')
	Integer                    // integer literal (0x2f, -15)
	Float                      // floating-point literal (6e-9, 0.22)
	DateTime                   // offset date-time (2006-01-02T15:04:05.999999999Z07:00)
	LocalDate                  // local date (2006-01-02)
	LocalTime                  // local time (15:04:05.999999999)
	LocalDateTime              // local date-time (2006-01-02T15:04:05.999999999)
	LBracket                   // left bracket ("[")
	RBracket                   // right bracket ("]")
	LInline                    // inline table open ("{")
	RInline                    // inline table close ("}")
	Equal                      // equal sign ("=")
	Comma                      // comma separator (",")
	Dot                        // key separator (".")

)

Constants defining the valid Token values.

func (Token) IsValue

func (t Token) IsValue() bool

IsValue reports whether t can serve as the value in a TOML table.

func (Token) String

func (t Token) String() string