Documentation ¶
Index ¶
Constants ¶
View Source
const ( BASE_PARSE_NAME = "base" SPIDERS_STATUS_INIT = iota SPIDERS_STATUS_RUNNING SPIDERS_STATUS_STOP SPIDERS_BASIC_COOKIE )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Spider ¶
type Spider struct { Name string StartUrls []string ParseMap map[string]func(response *http.Response) ([]*http.Request, error) // defined you own parse function to scrapy response,make sure it return []response }
what a spider do * make start request * define basic parse func
func (*Spider) MakeStartRequests ¶
Click to show internal directories.
Click to hide internal directories.