decoder

Documentation

Overview

Package decoder implements a high performance decoder for wiki pages.

It uses xml.Decoder.RawToken to speed up parsing and Page.UnmarshalXML to reduce allocation.

Index

• type Decoder
  • func New(r io.Reader) (*Decoder, error)
  • func (d *Decoder) Err() error
  • func (d *Decoder) Next() bool
  • func (d *Decoder) Scan(v xml.Unmarshaler) error
• type Page
  • func (p *Page) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error

Types

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder is an XML decoder tailored to the Wikipedia dataset.

func New

func New(r io.Reader) (*Decoder, error)

New instanciates a new Decoder.

It fail if it cannot find the mediawiki and siteinfo elements from the dataset.

func (*Decoder) Err

func (d *Decoder) Err() error

Err returns any error encountered by Next.

func (*Decoder) Next

func (d *Decoder) Next() bool

Next moves to the next element.

func (*Decoder) Scan

func (d *Decoder) Scan(v xml.Unmarshaler) error

Scan scans an element.

Beware, UnmarshalXML must be implemented by calling RawToken.

type Page

type Page struct {
	ID        int64     `xml:"-"`
	UpdatedAt time.Time `xml:"revision>timestamp"`
	Title     string    `xml:"title"`
	Text      string    `xml:"revision>text"`
}

Page represents a page from Wikipedia.

func (*Page) UnmarshalXML

func (p *Page) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error

UnmarshalXML unmarshals an XML element into the page.