parser

README

Run

$ go run ./cmd/parser-breakit/main.go

Command line arguments

Name	Default	Description
d	0	Recursion depth
p	1	Concurrency limit
v	false	Verbose logging
o	-	Output filename

Documentation

Index

• type Article
• type Parser
  • func New(verbose bool) *Parser
  • func (p *Parser) Parse(ctx context.Context, depth int64, concurrency int64) ([]*Article, error)

Types

type Article

type Article struct {
	URL         *url.URL
	PublishedAt time.Time
	Title       string
	Preamble    string
	Summary     *string
	Links       []*url.URL
	Depth       int64
}

Article is a parsed article.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser parses articles from https://breakit.se

func New

func New(verbose bool) *Parser

New creates a new parser.

func (*Parser) Parse

func (p *Parser) Parse(ctx context.Context, depth int64, concurrency int64) ([]*Article, error)

Parse starts parsing.