crawl

README

crawl

Web crawler written in Google GO

Documentation

Overview

Package crawl is A web crawler which crawls a given url to find similar domain urls

Index

• func Crawl(sourceURL string) []string

Functions

func Crawl

func Crawl(sourceURL string) []string

Crawl takes the URL to be crawled and returns a slice URL strings We crawl the URL to find all the URL that starts with similar domain name and add into our channel These channels are read for data and further crawling is done on slice of URL strings