gocrawl

README

Getting Set Up

Install all the dependencies by running

go get ./...

You can install the utility globally with

go install

Or you can use

go build

Followed by running the utility from the current directory

Running Tests

In the main directory run

go test ./...

Usage

For help run

gocrawl -h

To start crawling a website just run

gocrawl -s http://tomblomfield.com

This starts crawling the website concurrently so it will generate quite a bit of traffic.