htmlscrape

README

htmlscrape

A dumping ground of utility functions for I make for scraping HTML pages

Documentation

Index

• func FindNode(n *html.Node, matcher *NodeMatcher) *html.Node
• func New(url string) (*html.Node, error)
• func Table(url string, matcher *NodeMatcher) (*html.Node, error)
• func TableRows(url string, rows chan<- *html.Node, tableMatcher *NodeMatcher) error
• type NodeMatcher
  • func NewNodeMatcher() *NodeMatcher
  • func (m *NodeMatcher) Atom(v atom.Atom) *NodeMatcher
  • func (m *NodeMatcher) Attr(k, v string) *NodeMatcher
  • func (m *NodeMatcher) Attrs(v map[string]string) *NodeMatcher
  • func (m *NodeMatcher) Elem() *NodeMatcher
  • func (m *NodeMatcher) Text() *NodeMatcher
  • func (m *NodeMatcher) Type(v html.NodeType) *NodeMatcher

Functions

func FindNode

func FindNode(n *html.Node, matcher *NodeMatcher) *html.Node

FindNode recursively searches for a node which matches the given NodeMatcher

func New

func New(url string) (*html.Node, error)

New returns an HTML document rooted at the HTML node

func Table

func Table(url string, matcher *NodeMatcher) (*html.Node, error)

Table returns an HTML table which matches the given NodeMatcher

func TableRows

func TableRows(url string, rows chan<- *html.Node, tableMatcher *NodeMatcher) error

TableRows sends to a channel the rows of the HTML table which matches the NodeMatcher

Types

type NodeMatcher

type NodeMatcher struct {
	// contains filtered or unexported fields
}

NodeMatcher allows you to build a set of criteria to match against a node

func NewNodeMatcher

func NewNodeMatcher() *NodeMatcher

NewNodeMatcher initializes a new NodeMatcher object

func (*NodeMatcher) Atom

func (m *NodeMatcher) Atom(v atom.Atom) *NodeMatcher

Atom allows you to add DataAtom as matching criteria

func (*NodeMatcher) Attr

func (m *NodeMatcher) Attr(k, v string) *NodeMatcher

Attr allows you to add an attribute as matching criteria

func (*NodeMatcher) Attrs

func (m *NodeMatcher) Attrs(v map[string]string) *NodeMatcher

Attrs allows you to add a slice of attributes as matching criteria

func (*NodeMatcher) Elem

func (m *NodeMatcher) Elem() *NodeMatcher

Elem specifies ElementNode as matching criteria

func (*NodeMatcher) Text

func (m *NodeMatcher) Text() *NodeMatcher

Text specifies TextNode as matching criteria

func (*NodeMatcher) Type

func (m *NodeMatcher) Type(v html.NodeType) *NodeMatcher

Type allows you to add NodeType as matching criteria