scraper

Documentation

Index

• type ScrapingManager
  • func NewManager(ctx context.Context, b *bus.Bus) *ScrapingManager
  • func (m ScrapingManager) Start(ctx context.Context)
  • func (m ScrapingManager) Stop(ctx context.Context)

Types

type ScrapingManager

type ScrapingManager struct {
	// contains filtered or unexported fields
}

ScraperManager used to handle the various scrapers

func NewManager

func NewManager(ctx context.Context, b *bus.Bus) *ScrapingManager

NewManager returns a configured instance of a ScraperManager

func (ScrapingManager) Start

func (m ScrapingManager) Start(ctx context.Context)

Start iterates through each scraper and starts them NOTE: a scraper should not have a blocking call to start

func (ScrapingManager) Stop

func (m ScrapingManager) Stop(ctx context.Context)

Stop iterates through scrapers and stops them