engine

Documentation

Index

• type ConcurrentEngine
  • func (e *ConcurrentEngine) Run(seeds ...Request)
• type ParseResult
  • func NilParser([]byte) ParseResult
• type Request
• type Scheduler
• type SimpleEngine
  • func (e SimpleEngine) Run(seeds ...Request)

Types

type ConcurrentEngine

type ConcurrentEngine struct {
	// scheduler 哪里来？肚子里放一个就可以了
	Scheduler   Scheduler
	WorkerCount int
}

func (*ConcurrentEngine) Run

func (e *ConcurrentEngine) Run(seeds ...Request)

自己也最好变成指针类型的接收者

type ParseResult

type ParseResult struct {
	Requests []Request
	Items    []interface{}
}

func NilParser

func NilParser([]byte) ParseResult

type Request

type Request struct {
	Url        string
	ParserFunc func([]byte) ParseResult
}

type Scheduler

type Scheduler interface {
	Submit(Request)
	ConfigureMasterWorkerChan(chan Request)
}

使用者来定义，你去实现

type SimpleEngine

type SimpleEngine struct{}

func (SimpleEngine) Run

func (e SimpleEngine) Run(seeds ...Request)

可以送很多种子