htmlx

README

htmlx

HTML parsing eXtensions, aka Monadic Finder of elements.

Includes handy parsehtml command.

Documentation

Index

• type Finder
  • func FinderFromData(r io.Reader) (Finder, error)
  • func FinderFromNode(h *html.Node) Finder
  • func FinderFromString(s string) (Finder, error)
  • func (f Finder) Attr() attr.List
  • func (f Finder) Copy() Finder
  • func (f Finder) Find(pred pred.Predicate) (r Finder)
  • func (f Finder) FindAll(pred pred.Predicate) FinderStream
  • func (f Finder) FindPrevSibling(pred pred.Predicate) (r Finder)
  • func (f Finder) FindPrevSiblings(pred pred.Predicate) FinderStream
  • func (f Finder) FindSibling(pred pred.Predicate) (r Finder)
  • func (f Finder) FindSiblings(pred pred.Predicate) FinderStream
  • func (f Finder) FindWithSiblings(pred pred.Predicate) FinderStream
  • func (f Finder) FirstChild() Finder
  • func (f Finder) InnerText() string
  • func (f Finder) IsEmpty() bool
  • func (f Finder) LastChild() Finder
  • func (f Finder) NextSibling() Finder
  • func (f Finder) Parent() Finder
  • func (f Finder) PrevSibling() Finder
  • func (f Finder) StreamSelf() FinderStream
  • func (f Finder) String() string
  • func (f Finder) Write(w io.Writer) error
• type FinderStream
  • func AllText(f Finder) FinderStream
  • func Inject(a []Finder) FinderStream
  • func (ff FinderStream) Collect() (res []Finder)
  • func (ff FinderStream) DropN(n int) FinderStream
  • func (ff FinderStream) Filter(p pred.Predicate) FinderStream
  • func (ff FinderStream) First() Finder
  • func (ff FinderStream) Join(split SplitFunc) FinderStream
  • func (ff FinderStream) Last() (f Finder)
  • func (ff FinderStream) Map(m MapFunc) FinderStream
  • func (ff FinderStream) Reduce(combine ReduceFunc) FinderStream
  • func (ff FinderStream) Select(p pred.Predicate) Finder
  • func (ff FinderStream) TakeN(n int) FinderStream
• type MapFunc
• type ReduceFunc
• type SplitFunc

Types

type Finder

type Finder struct {
	*html.Node
}

func FinderFromData

func FinderFromData(r io.Reader) (Finder, error)

func FinderFromNode

func FinderFromNode(h *html.Node) Finder

func FinderFromString

func FinderFromString(s string) (Finder, error)

func (Finder) Attr

func (f Finder) Attr() attr.List

func (Finder) Copy

func (f Finder) Copy() Finder

func (Finder) Find

func (f Finder) Find(pred pred.Predicate) (r Finder)

Find performs depth-first traversal looking for the node satisfying given precicate. Includes current node in the search. Stops at the first found node and returns it, wrapped in a new Finder.

func (Finder) FindAll

func (f Finder) FindAll(pred pred.Predicate) FinderStream

FindAll performs depth-first traversal like Find, but returns a stream of all the found nodes satisfying the predicate, each wrapped in a Finder. Current node is included in the search.

func (Finder) FindPrevSibling

func (f Finder) FindPrevSibling(pred pred.Predicate) (r Finder)

FindPrevSibling performs flat find of the first node satistying the predicate among current node's previous (left) siblings. No recursion. Omits current node, starts from a first sibling.

func (Finder) FindPrevSiblings

func (f Finder) FindPrevSiblings(pred pred.Predicate) FinderStream

FindPrevSiblings performs flat find of all the nodes satistying the predicate among current node's previous (left) siblings. No recursion. Omits current node, starts from a first sibling.

func (Finder) FindSibling

func (f Finder) FindSibling(pred pred.Predicate) (r Finder)

FindSibling performs flat find of the first node satistying the predicate among current node's next (right) siblings. No recursion. Omits current node, starts from a first sibling.

func (Finder) FindSiblings

func (f Finder) FindSiblings(pred pred.Predicate) FinderStream

FindSiblings performs flat find of all the nodes satistying the predicate among current node's next (right) siblings. No recursion. Omits current node, starts from a first sibling.

func (Finder) FindWithSiblings

func (f Finder) FindWithSiblings(pred pred.Predicate) FinderStream

FindWithSiblings performs depth-first travelsal looking for the first of nodes satisfying given predicate. Then it continues flat find of all the siblings satisfying the predicate. All results are pushed into the returted stream of Finders.

func (Finder) FirstChild

func (f Finder) FirstChild() Finder

func (Finder) InnerText

func (f Finder) InnerText() string

func (Finder) IsEmpty

func (f Finder) IsEmpty() bool

func (Finder) LastChild

func (f Finder) LastChild() Finder

func (Finder) NextSibling

func (f Finder) NextSibling() Finder

func (Finder) Parent

func (f Finder) Parent() Finder

func (Finder) PrevSibling

func (f Finder) PrevSibling() Finder

func (Finder) StreamSelf

func (f Finder) StreamSelf() FinderStream

func (Finder) String

func (f Finder) String() string

func (Finder) Write

func (f Finder) Write(w io.Writer) error

type FinderStream

type FinderStream <-chan Finder

func AllText

func AllText(f Finder) FinderStream

AllText is a splitFunc returning a stream of all text nodes found descending from the given node, depth-first. The nodes containing only whitespaces are not included.

func Inject

func Inject(a []Finder) FinderStream

func (FinderStream) Collect

func (ff FinderStream) Collect() (res []Finder)

func (FinderStream) DropN

func (ff FinderStream) DropN(n int) FinderStream

func (FinderStream) Filter

func (ff FinderStream) Filter(p pred.Predicate) FinderStream

func (FinderStream) First

func (ff FinderStream) First() Finder

func (FinderStream) Join

func (ff FinderStream) Join(split SplitFunc) FinderStream

Join performs depth-first join of streams generated by applying the splitFunc on every item from the current stream.

func (FinderStream) Last

func (ff FinderStream) Last() (f Finder)

func (FinderStream) Map

func (ff FinderStream) Map(m MapFunc) FinderStream

Map maps the stream, changing each node wrapped by the finder according to the given function. Note that this function as a signature: `func (Finder)` and is supposed to change the internally copied html.Node struct whose pointer is wrapped by the finder. Also note that such an implicit copy is not made in the Reduce function, where we may return an original Finder.

func (FinderStream) Reduce

func (ff FinderStream) Reduce(combine ReduceFunc) FinderStream

Reduce reduces the stream according to the given combine function. It copies the first item of the stream, then for each item y, uses combine(prev, y) func to decide what should be copied to the output stream. There are three possibilities: (_, false) no item is copied in this iteration (y, true) item x is copied (newy, true) combined item newx is copied. Note: for the last case, combine func should do explicit x.Copy() to create new html.Node struct.

func (FinderStream) Select

func (ff FinderStream) Select(p pred.Predicate) Finder

func (FinderStream) TakeN

func (ff FinderStream) TakeN(n int) FinderStream

type MapFunc

type MapFunc func(Finder)

type ReduceFunc

type ReduceFunc func(prev, y Finder) (Finder, bool)

type SplitFunc

type SplitFunc func(Finder) FinderStream