Documentation ¶
Index ¶
- type Pipeline
- type PipelineX
- type Spider
- func (spi *Spider) LimitSybilCrawler(limit int)
- func (spi *Spider) Run(ctx context.Context, pipeline Pipeline) error
- func (spi *Spider) RunAndFetchTracker(ctx context.Context, pipeline PipelineX) error
- func (spi *Spider) Stimulate() bool
- func (spi *Spider) Use(crawler dht.Crawler)
- func (spi *Spider) UseSybilCrawler()
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type PipelineX ¶
type PipelineX interface { Pipeline DisposeMetaAndTracker(string, bt.RawMeta, []string) PullTrackerList(string) ([]string, bool) AppendTracker(string, []string) }
PipelineX used for spider.RunAndFetchTracker
type Spider ¶
type Spider struct {
// contains filtered or unexported fields
}
Spider can crawl Meta from dht.
func (*Spider) LimitSybilCrawler ¶
LimitSybilCrawler will use a dht.SybilCrawler
func (*Spider) RunAndFetchTracker ¶
RunAndFetchTracker run and fetch tracker use tex protocal
func (*Spider) Stimulate ¶
Stimulate will Stimulate the spider when Crawler is Stimulater and returns true. else do nothing and returns false.
func (*Spider) UseSybilCrawler ¶
func (spi *Spider) UseSybilCrawler()
UseSybilCrawler will use dht.SybilCrawler
Click to show internal directories.
Click to hide internal directories.