engine

Documentation

Index

• type Engine
  • func NewEngine(logger *slog.Logger, db *database.Database) (*Engine, error)
  • func (e *Engine) Register(f ScraperFactory)
  • func (e *Engine) ScrapeAll(query string) error
• type ScraperFactory

Types

type Engine

type Engine struct {
	Db *database.Database
	// contains filtered or unexported fields
}

The Engine

The engine is what orchestrates all of the different scrapers and acts
as the container for the database. The register method allows you to
extend the engine to utilise various scraper configs through [engine.ScraperFactory]

func NewEngine

func NewEngine(logger *slog.Logger, db *database.Database) (*Engine, error)

Create a new instance of an Engine

func (*Engine) Register

func (e *Engine) Register(f ScraperFactory)

Register a new scraper to the engine

func (*Engine) ScrapeAll

func (e *Engine) ScrapeAll(query string) error

Runs all registered scrapers

type ScraperFactory

type ScraperFactory func(*slog.Logger, *database.Database, string) *scrapers.Scraper