downloader

package
v0.0.0-...-2d91a95 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2017 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var SurferDownloader = &Surfer{
	surf:    surfer.New(),
	phantom: surfer.NewPhantom(config.PHANTOMJS, config.PHANTOMJS_TEMP),
}

SurferDownloader is a struct ...

Functions

This section is empty.

Types

type Downloader

type Downloader interface {
	Download(*spider.Spider, *request.Request) *spider.Context
}

The Downloader interface. You can implement the interface by implement function Download. Function Download need to return Page instance pointer that has request result downloaded from Request.

type Surfer

type Surfer struct {
	// contains filtered or unexported fields
}

Surfer is a struct ...

func (*Surfer) Download

func (surfer *Surfer) Download(sp *spider.Spider, cReq *request.Request) *spider.Context

Download is a struct ...

Directories

Path Synopsis
Package surfer s a Go language prepared by the high concurrent web downloader, support GET/POST/HEAD Method and method http/https Protocol, while supporting the fixed UserAgent automatically save the cookie with a random large number of UserAgent disabled cookie two modes, a high degree of simulation of the browser behavior, enabling analog login and other functions.
Package surfer s a Go language prepared by the high concurrent web downloader, support GET/POST/HEAD Method and method http/https Protocol, while supporting the fixed UserAgent automatically save the cookie with a random large number of UserAgent disabled cookie two modes, a high degree of simulation of the browser behavior, enabling analog login and other functions.
agent
Package agent generates user agents strings for well known browsers and for custom browsers.
Package agent generates user agents strings for well known browsers and for custom browsers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL