libgcwl

Documentation

Index

• func PrintBanner()
• type CrawlState
• type FlagConfig
  • func (c *FlagConfig) RunCrawler()

Functions

func PrintBanner

func PrintBanner()

PrintBanner outputs the welcome banner to users using the tool

Types

type CrawlState

type CrawlState struct {
	FoundURLs       map[string]struct{}
	UnprocessedURLs map[string]struct{}
	FoundWords      map[string]struct{}
}

CrawlState is used to track information we need during the crawl

type FlagConfig

type FlagConfig struct {
	SeedURLs []string `json:"seed_urls",xml:"seed_urls"`
	Depth    int      `json:"depth",xml:"depth"`
	Verbose  bool     `json:"verbose",xml:"verbose"`
}

FlagConfig is used to contain the different flag values that we receive from the user on the command line. Including things like help, verbose, depth, etc.

func (*FlagConfig) RunCrawler

func (c *FlagConfig) RunCrawler()

RunCrawler begins the crawling tool