pageparser

Documentation

Overview

Package pageparser provides a parser for Hugo content files (Markdown, HTML etc.) in Hugo. This implementation is highly inspired by the great talk given by Rob Pike called "Lexical Scanning in Go" It's on YouTube, Google it!. See slides here: https://go.dev/talks/2011/lex.slide#1

Index

• func FormatFromFrontMatterType(typ ItemType) metadecoders.Format
• func HasShortcode(s string) bool
• func IsProbablySourceOfItems(source []byte, items Items) bool
• type Config
• type ContentFrontMatter
  • func ParseFrontMatterAndContent(r io.Reader) (ContentFrontMatter, error)
• type Item
  • func (i Item) IsDone() bool
  • func (i Item) IsEOF() bool
  • func (i Item) IsError() bool
  • func (i Item) IsFrontMatter() bool
  • func (i Item) IsIndentation() bool
  • func (i Item) IsInlineShortcodeName() bool
  • func (i Item) IsLeftShortcodeDelim() bool
  • func (i Item) IsNonWhitespace(source []byte) bool
  • func (i Item) IsRightShortcodeDelim() bool
  • func (i Item) IsShortcodeClose() bool
  • func (i Item) IsShortcodeMarkupDelimiter() bool
  • func (i Item) IsShortcodeName() bool
  • func (i Item) IsShortcodeParam() bool
  • func (i Item) IsShortcodeParamVal() bool
  • func (i Item) IsText() bool
  • func (i Item) Pos() int
  • func (i Item) ToString(source []byte) string
  • func (i Item) Val(source []byte) []byte
  • func (i Item) ValStr(source []byte) string
  • func (i Item) ValTyped(source []byte) any
• type ItemType
  • func (i ItemType) String() string
• type Items
  • func ParseBytes(b []byte, cfg Config) (Items, error)
• type Iterator
  • func NewIterator(items Items) *Iterator
  • func (t *Iterator) Backup()
  • func (t *Iterator) Consume(cnt int)
  • func (t *Iterator) Current() Item
  • func (t *Iterator) IsValueNext() bool
  • func (t *Iterator) LineNumber(source []byte) int
  • func (t *Iterator) Next() Item
  • func (t *Iterator) Peek() Item
  • func (t *Iterator) PeekWalk(walkFn func(item Item) bool)
  • func (t *Iterator) Pos() int
• type Result
  • func ParseMain(r io.Reader, cfg Config) (Result, error)

Functions

func FormatFromFrontMatterType

func FormatFromFrontMatterType(typ ItemType) metadecoders.Format

func HasShortcode

func HasShortcode(s string) bool

HasShortcode returns true if the given string contains a shortcode.

func IsProbablySourceOfItems

func IsProbablySourceOfItems(source []byte, items Items) bool

IsProbablySourceOfItems returns true if the given source looks like original source of the items. There may be some false positives, but that is highly unlikely and good enough for the planned purpose. It will also return false if the last item is not EOF (error situations) and true if both source and items are empty.

Types

type Config

type Config struct {
	NoFrontMatter    bool
	NoSummaryDivider bool
}

type ContentFrontMatter

type ContentFrontMatter struct {
	Content           []byte
	FrontMatter       map[string]any
	FrontMatterFormat metadecoders.Format
}

func ParseFrontMatterAndContent

func ParseFrontMatterAndContent(r io.Reader) (ContentFrontMatter, error)

ParseFrontMatterAndContent is a convenience method to extract front matter and content from a content page.

type Item

type Item struct {
	Type ItemType
	Err  error
	// contains filtered or unexported fields
}

func (Item) IsDone

func (i Item) IsDone() bool

func (Item) IsEOF

func (i Item) IsEOF() bool

func (Item) IsError

func (i Item) IsError() bool

func (Item) IsFrontMatter

func (i Item) IsFrontMatter() bool

func (Item) IsIndentation

func (i Item) IsIndentation() bool

func (Item) IsInlineShortcodeName

func (i Item) IsInlineShortcodeName() bool

func (Item) IsLeftShortcodeDelim

func (i Item) IsLeftShortcodeDelim() bool

func (Item) IsNonWhitespace

func (i Item) IsNonWhitespace(source []byte) bool

func (Item) IsRightShortcodeDelim

func (i Item) IsRightShortcodeDelim() bool

func (Item) IsShortcodeClose

func (i Item) IsShortcodeClose() bool

func (Item) IsShortcodeMarkupDelimiter

func (i Item) IsShortcodeMarkupDelimiter() bool

func (Item) IsShortcodeName

func (i Item) IsShortcodeName() bool

func (Item) IsShortcodeParam

func (i Item) IsShortcodeParam() bool

func (Item) IsShortcodeParamVal

func (i Item) IsShortcodeParamVal() bool

func (Item) IsText

func (i Item) IsText() bool

func (Item) Pos

func (i Item) Pos() int

func (Item) ToString

func (i Item) ToString(source []byte) string

func (Item) Val

func (i Item) Val(source []byte) []byte

func (Item) ValStr

func (i Item) ValStr(source []byte) string

func (Item) ValTyped

func (i Item) ValTyped(source []byte) any

type ItemType

type ItemType int

const (

	// page items
	TypeLeadSummaryDivider ItemType // <!--more-->,  # more
	TypeFrontMatterYAML
	TypeFrontMatterTOML
	TypeFrontMatterJSON
	TypeFrontMatterORG
	TypeIgnore // // The BOM Unicode byte order marker and possibly others

)

func (ItemType) String

func (i ItemType) String() string

type Items

type Items []Item

func ParseBytes

func ParseBytes(b []byte, cfg Config) (Items, error)

ParseBytes parses the page in b according to the given Config.

type Iterator

type Iterator struct {
	// contains filtered or unexported fields
}

An Iterator has methods to iterate a parsed page with support going back if needed.

func NewIterator

func NewIterator(items Items) *Iterator

NewIterator creates a new Iterator.

func (*Iterator) Backup

func (t *Iterator) Backup()

backs up one token.

func (*Iterator) Consume

func (t *Iterator) Consume(cnt int)

Consume is a convenience method to consume the next n tokens, but back off Errors and EOF.

func (*Iterator) Current

func (t *Iterator) Current() Item

Current will repeatably return the current item.

func (*Iterator) IsValueNext

func (t *Iterator) IsValueNext() bool

check for non-error and non-EOF types coming next

func (*Iterator) LineNumber

func (t *Iterator) LineNumber(source []byte) int

LineNumber returns the current line number. Used for logging.

func (*Iterator) Next

func (t *Iterator) Next() Item

consumes and returns the next item

func (*Iterator) Peek

func (t *Iterator) Peek() Item

look at, but do not consume, the next item repeated, sequential calls will return the same item

func (*Iterator) PeekWalk

func (t *Iterator) PeekWalk(walkFn func(item Item) bool)

PeekWalk will feed the next items in the iterator to walkFn until it returns false.

func (*Iterator) Pos

func (t *Iterator) Pos() int

Pos returns the current position in the input.

type Result

type Result interface {
	// Iterator returns a new Iterator positioned at the beginning of the parse tree.
	Iterator() *Iterator
	// Input returns the input to Parse.
	Input() []byte
}

Result holds the parse result.

func ParseMain

func ParseMain(r io.Reader, cfg Config) (Result, error)

ParseMain parses starting with the main section. Used in tests.