baidu_spider

Documentation

Index

• Constants
• Variables
• func Filter(domain string, subdomains map[string]struct{}) []string
• type Agent
  • func (agent *Agent) Name() string
  • func (agent *Agent) Query(session *sources.Session, query *sources.Query) (chan sources.Result, error)

Constants

const (
	URL     = "https://www.baidu.com/s?wd=%s&pn=%d&rn=%d"
	URLInit = "https://www.baidu.com/"
	Source  = "baidu-spider"
)

Variables

var CommonSubNames = []string{"i", "w", "m", "en", "us", "zh", "w3", "app", "bbs",
	"web", "www", "job", "docs", "news", "blog", "data",
	"help", "live", "mall", "blogs", "files", "forum",
	"store", "mobile"}

Functions

func Filter

func Filter(domain string, subdomains map[string]struct{}) []string

Filter 生成搜索过滤语句 使用搜索引擎支持的-site:语法过滤掉搜索页面较多的子域以发现新域

Types

type Agent

type Agent struct {
	// contains filtered or unexported fields
}

func (*Agent) Name

func (agent *Agent) Name() string

func (*Agent) Query

func (agent *Agent) Query(session *sources.Session, query *sources.Query) (chan sources.Result, error)