crawler

Documentation

Index

• func NewKatana(show bool)
• type KatanaTask
  • func (t *KatanaTask) StartCrawler()
• type Runner
  • func New(options *types.Options) (*Runner, error)
  • func (r *Runner) Close() error

Functions

func NewKatana

func NewKatana(show bool)

NewKatana 初始化 katana 爬虫，全局只初始化一次，共用一个，不然创建多次，时间长了，目标多了，小水管受不了

Types

type KatanaTask

type KatanaTask struct {
	Target   string
	OnResult types.OnResultCallback
}

func (*KatanaTask) StartCrawler

func (t *KatanaTask) StartCrawler()

type Runner

type Runner struct {
	Crawler engine.Engine
	Options *types.Options
	// contains filtered or unexported fields
}

Runner creates the required resources for crawling and executes the crawl process.

var Katana *Runner

func New

func New(options *types.Options) (*Runner, error)

New returns a new crawl runner structure

func (*Runner) Close

func (r *Runner) Close() error

Close closes the runner releasing resources