spider

Documentation

Index

• Variables
• type Result
• type Spider
  • func (spider *Spider) Links(ctx context.Context, resp *crawl.Response) (err error)
  • func (spider *Spider) Register()

Variables

var Links = "github.com/crackcomm/crawl-links/spider.Links"

Links - Gather links from page.

Types

type Result

type Result struct {
	// Results - List of resulted links.
	Results []string `json:"results,omitempty"`
	// Source - URL of the source.
	Source string `json:"source,omitempty"`
}

Result - Links crawl result.

type Spider

type Spider struct {
	crawl.Crawler
	Output func(*Result) error
}

Spider - Links spider.

func (*Spider) Links

func (spider *Spider) Links(ctx context.Context, resp *crawl.Response) (err error)

Links - Crawls Links.

func (*Spider) Register

func (spider *Spider) Register()

Register - Registers spider.