app

Documentation

Index

• func Start(c *Config)
• type Config
  • func (c *Config) Process()
  • func (c *Config) Scrape(URL string)

Functions

func Start

func Start(c *Config)

Types

type Config

type Config struct {
	SeedURL     string
	ScrapeLimit int
	GetEmails   bool
	Verbose     bool
	Debug       bool

	Client   http.Client
	Wg       *sync.WaitGroup
	DataChan chan string
	URLChan  chan string
	Logger   utils.Logger
	Regex    *regexp.Regexp
}

func (*Config) Process

func (c *Config) Process()

Scrapes new URLs and logs them

func (*Config) Scrape

func (c *Config) Scrape(URL string)

Extracts all links and emails from a page and sends them back to processor over channels