parsers

Documentation

Overview

parsers are interfaces that Colibri can use to parse the content of the responses.

Index

• Constants
• Variables
• func Set[T Element](parsers *Parsers, expr string, parserFunc func(colibri.Response) (T, error)) error
• type Element
• type HTMLElement
  • func ParseHTML(resp colibri.Response) (*HTMLElement, error)
  • func (html *HTMLElement) CSSFind(expr string) (Element, error)
  • func (html *HTMLElement) CSSFindAll(expr string) ([]Element, error)
  • func (html *HTMLElement) Find(expr, exprType string) (Element, error)
  • func (html *HTMLElement) FindAll(expr, exprType string) ([]Element, error)
  • func (html *HTMLElement) Value() any
  • func (html *HTMLElement) XPathFind(expr string) (Element, error)
  • func (html *HTMLElement) XPathFindAll(expr string) ([]Element, error)
• type JSONElement
  • func ParseJSON(resp colibri.Response) (*JSONElement, error)
  • func (json *JSONElement) Find(expr, exprType string) (Element, error)
  • func (json *JSONElement) FindAll(expr, exprType string) ([]Element, error)
  • func (json *JSONElement) Value() any
• type ParserFunc
• type Parsers
  • func New() (*Parsers, error)
  • func (parsers *Parsers) Clear()
  • func (parsers *Parsers) Match(contentType string) bool
  • func (parsers *Parsers) Parse(rules *colibri.Rules, resp colibri.Response) (map[string]any, error)
• type TextElement
  • func ParseText(resp colibri.Response) (*TextElement, error)
  • func (text *TextElement) Find(expr, exprType string) (Element, error)
  • func (text *TextElement) FindAll(expr, exprType string) ([]Element, error)
  • func (text *TextElement) Value() any
• type XMLElement
  • func ParseXML(resp colibri.Response) (*XMLElement, error)
  • func (xml *XMLElement) Find(expr, exprType string) (Element, error)
  • func (xml *XMLElement) FindAll(expr, exprType string) ([]Element, error)
  • func (xml *XMLElement) Value() any

Constants

const (
	XPathExpr   = "xpath"
	CSSSelector = "css"
	RegularExpr = "regular"
)

const HTMLRegexp = `^text\/html`

HTMLRegexp contains a regular expression that matches the HTML MIME type.

const JSONRegexp = `^application\/(json|x-json|([a-z]+\+json))`

JSONRegexp contains a regular expression that matches the JSON MIME type.

const TextRegexp = `^text\/plain`

TextRegexp contains a regular expression that matches the MIME type plain text.

const XMLRegexp = `(?i)((application|image|message|model)/((\w|\.|-)+\+?)?|text/)(wb)?xml`

XMLRegexp contains a regular expression that matches the XML MIME type.

Variables

var (
	// ErrNotMatch is returned when the Content-Tyepe does not match the Paser.
	ErrNotMatch = errors.New("Content-Type does not match")

	// ErrExprType is returned when the type of expression is not compatible with the element.
	ErrExprType = errors.New("ExprType not compatible with Element")
)

Functions

func Set

func Set[T Element](parsers *Parsers, expr string, parserFunc func(colibri.Response) (T, error)) error

Set adds to parsers the regular expression and the corresponding ParserFunc.

Types

type Element

type Element interface {
	// Find finds a child element that matches the expression.
	Find(expr, exprType string) (Element, error)

	// FindAll finds all child elements that match the expression.
	FindAll(expr, exprType string) ([]Element, error)

	// Value returns the value of the element.
	Value() any
}

type HTMLElement

type HTMLElement struct {
	// contains filtered or unexported fields
}

HTMLElement represents an HTML element compatible with XPath expressions and CSS selectors. If the type of expression is not specified, they assume it is an XPath expression.

func ParseHTML

func ParseHTML(resp colibri.Response) (*HTMLElement, error)

ParseHTML parses the content of the response and returns the root element.

func (*HTMLElement) CSSFind

func (html *HTMLElement) CSSFind(expr string) (Element, error)

func (*HTMLElement) CSSFindAll

func (html *HTMLElement) CSSFindAll(expr string) ([]Element, error)

func (*HTMLElement) Find

func (html *HTMLElement) Find(expr, exprType string) (Element, error)

func (*HTMLElement) FindAll

func (html *HTMLElement) FindAll(expr, exprType string) ([]Element, error)

func (*HTMLElement) Value

func (html *HTMLElement) Value() any

func (*HTMLElement) XPathFind

func (html *HTMLElement) XPathFind(expr string) (Element, error)

func (*HTMLElement) XPathFindAll

func (html *HTMLElement) XPathFindAll(expr string) ([]Element, error)

type JSONElement

type JSONElement struct {
	// contains filtered or unexported fields
}

JSONElement represents a JSON element compatible with XPath expressions.

func ParseJSON

func ParseJSON(resp colibri.Response) (*JSONElement, error)

ParseJSON parses the content of the response and returns the root element.

func (*JSONElement) Find

func (json *JSONElement) Find(expr, exprType string) (Element, error)

func (*JSONElement) FindAll

func (json *JSONElement) FindAll(expr, exprType string) ([]Element, error)

func (*JSONElement) Value

func (json *JSONElement) Value() any

type ParserFunc

type ParserFunc func(colibri.Response) (Element, error)

ParserFunc parses the content of the response and returns the root element.

type Parsers

type Parsers struct {
	// contains filtered or unexported fields
}

Parsers stores ParserFunc used to parse the content of the responses. ParserFunc are stored with a regular expression that functions as a key. When a regular expression matches the Content-Type of the response, the content of the response is parsed with the ParserFunc corresponding to the regular expression.

func New

func New() (*Parsers, error)

New returns a new Parsers with ParserFunc to parse HTML, XHML, JSON and Plain Text. See the colibri.Parser interface.

func (*Parsers) Clear

func (parsers *Parsers) Clear()

Clear deletes all stored ParserFunc.

func (*Parsers) Match

func (parsers *Parsers) Match(contentType string) bool

Match returns true if the Content-Type is compatible with the Parser.

func (*Parsers) Parse

func (parsers *Parsers) Parse(rules *colibri.Rules, resp colibri.Response) (map[string]any, error)

Parse parses the response based on the rules.

type TextElement

type TextElement struct {
	// contains filtered or unexported fields
}

TextElement represents a Text element compatible with regular expressions.

func ParseText

func ParseText(resp colibri.Response) (*TextElement, error)

ParseText parses the content of the response and returns the root element.

func (*TextElement) Find

func (text *TextElement) Find(expr, exprType string) (Element, error)

func (*TextElement) FindAll

func (text *TextElement) FindAll(expr, exprType string) ([]Element, error)

func (*TextElement) Value

func (text *TextElement) Value() any

type XMLElement

type XMLElement struct {
	// contains filtered or unexported fields
}

XMLElement represents an XML element compatible with XPath expressions.

func ParseXML

func ParseXML(resp colibri.Response) (*XMLElement, error)

ParseXML parses the content of the response and returns the root element.

func (*XMLElement) Find

func (xml *XMLElement) Find(expr, exprType string) (Element, error)

func (*XMLElement) FindAll

func (xml *XMLElement) FindAll(expr, exprType string) ([]Element, error)

func (*XMLElement) Value

func (xml *XMLElement) Value() any