parser

Documentation

Overview

Package parser implements parser for markdown text that generates AST (abstract syntax tree).

Index

• Variables
• func IsAlnum(c byte) bool
• func IsCallout(data []byte) (id []byte, consumed int)
• func IsEmpty(data []byte) int
• func IsLetter(c byte) bool
• func IsPunctuation(c byte) bool
• func IsSafeURL(url []byte) bool
• func IsSpace(c byte) bool
• func LinesUntilEmpty(data []byte) int
• func NormalizeNewlines(d []byte) []byte
• type BlockFunc
• type Flags
• type Options
• type Parser
  • func New() *Parser
  • func NewWithExtensions(extension ext.Extensions) *Parser
  • func (p *Parser) AddBlock(n ast.Node) ast.Node
  • func (p *Parser) Block(data []byte)
  • func (p *Parser) Finalize(block ast.Node)
  • func (p *Parser) Inline(currBlock ast.Node, data []byte)
  • func (p *Parser) Parse(input []byte) ast.Node
• type ReadIncludeFunc
• type Reference
• type ReferenceOverrideFunc

Variables

var Paths = [][]byte{
	[]byte("/"),
	[]byte("./"),
	[]byte("../"),
}

var URIs = [][]byte{
	[]byte("http://"),
	[]byte("https://"),
	[]byte("ftp://"),
	[]byte("mailto:"),
}

Functions

func IsAlnum

func IsAlnum(c byte) bool

IsAlnum returns true if c is a digit or letter TODO: check when this is looking for ASCII alnum and when it should use unicode

func IsCallout

func IsCallout(data []byte) (id []byte, consumed int)

IsCallout detects a callout in the following format: <<N>> Where N is a integer > 0.

func IsEmpty

func IsEmpty(data []byte) int

func IsLetter

func IsLetter(c byte) bool

IsLetter returns true if c is ascii letter

func IsPunctuation

func IsPunctuation(c byte) bool

IsPunctuation returns true if c is a punctuation symbol.

func IsSafeURL

func IsSafeURL(url []byte) bool

IsSafeURL returns true if url starts with one of the valid schemes or is a relative path.

func IsSpace

func IsSpace(c byte) bool

IsSpace returns true if c is a white-space charactr

func LinesUntilEmpty

func LinesUntilEmpty(data []byte) int

LinesUntilEmpty scans lines up to the first empty line.

func NormalizeNewlines

func NormalizeNewlines(d []byte) []byte

Types

type BlockFunc

type BlockFunc func(data []byte) (ast.Node, []byte, int)

BlockFunc allows to registration of a parser function. If successful it returns an ast.Node, a buffer that should be parsed as a block and the the number of bytes consumed.

type Flags

type Flags int

Flags control optional behavior of parser.

const (
	FlagsNone        Flags = 0
	SkipFootnoteList Flags = 1 << iota // Skip adding the footnote list (regardless if they are parsed)
)

Parser renderer configuration options.

type Options

type Options struct {
	ParserHook    BlockFunc
	ReadIncludeFn ReadIncludeFunc

	Flags Flags // Flags allow customizing parser's behavior
}

Options is a collection of supplementary parameters tweaking the behavior of various parts of the parser.

type Parser

type Parser struct {

	// ReferenceOverride is an optional function callback that is called every
	// time a reference is resolved. It can be set before starting parsing.
	//
	// In Markdown, the link reference syntax can be made to resolve a link to
	// a reference instead of an inline URL, in one of the following ways:
	//
	//  * [link text][refid]
	//  * [refid][]
	//
	// Usually, the refid is defined at the bottom of the Markdown document. If
	// this override function is provided, the refid is passed to the override
	// function first, before consulting the defined refids at the bottom. If
	// the override function indicates an override did not occur, the refids at
	// the bottom will be used to fill in the link details.
	ReferenceOverride ReferenceOverrideFunc

	// IsSafeURLOverride allows overriding the default URL matcher. URL is
	// safe if the overriding function returns true. Can be used to extend
	// the default list of safe URLs.
	IsSafeURLOverride func(url []byte) bool

	Opts Options

	// after parsing, this is AST root of parsed markdown text
	Doc ast.Node
	// contains filtered or unexported fields
}

Parser is a type that holds extensions and the runtime state used by Parse, and the renderer. You can not use it directly, construct it with New.

func New

func New() *Parser

New creates a markdown parser with CommonExtensions.

You can then call `doc := p.Parse(markdown)` to parse markdown document and `markdown.Render(doc, renderer)` to convert it to another format with a renderer.

func NewWithExtensions

func NewWithExtensions(extension ext.Extensions) *Parser

NewWithExtensions creates a markdown parser with given extensions.

func (*Parser) AddBlock

func (p *Parser) AddBlock(n ast.Node) ast.Node

func (*Parser) Block

func (p *Parser) Block(data []byte)

Parse Block-level data. Note: this function and many that it calls assume that the input buffer ends with a newline.

func (*Parser) Finalize

func (p *Parser) Finalize(block ast.Node)

func (*Parser) Inline

func (p *Parser) Inline(currBlock ast.Node, data []byte)

Inline parses text within a block. Each function returns the number of consumed chars.

func (*Parser) Parse

func (p *Parser) Parse(input []byte) ast.Node

Parse generates AST (abstract syntax tree) representing markdown document.

The result is a root of the tree whose underlying type is *ast.Document

You can then convert AST to html using html.Renderer, to some other format using a custom renderer or transform the tree.

type ReadIncludeFunc

type ReadIncludeFunc func(from, path string, address []byte) []byte

ReadIncludeFunc should read the file under path and returns the read bytes, from will be set to the name of the current file being parsed. Initially this will be empty. address is the optional address specifier of which lines of the file to return. If this function is not set no data will be read.

type Reference

type Reference struct {
	// Link is usually the URL the reference points to.
	Link string
	// Title is the alternate text describing the link in more detail.
	Title string
	// Text is the optional text to override the ref with if the syntax used was
	// [refid][]
	Text string
}

Reference represents the details of a link. See the documentation in Options for more details on use-case.

type ReferenceOverrideFunc

type ReferenceOverrideFunc func(reference string) (ref *Reference, overridden bool)

ReferenceOverrideFunc is expected to be called with a reference string and return either a valid Reference type that the reference string maps to or nil. If overridden is false, the default reference logic will be executed. See the documentation in Options for more details on use-case.