GitHub - ez90/website-crawler: A simple go script that crawl a given website and display usefull informations

#Simple crawler

Crawl all internal links for a specified URL

Command will display all crawled links with their responses status and a list of discovered internal links. It will never crawl the same URL twice.

Build

go build crawler.go

crawler --url <my_url>

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
htmlParser		htmlParser
.gitignore		.gitignore
README.md		README.md
crawler.go		crawler.go