8000 GitHub - hiitsu/miner: harvests files from website
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

hiitsu/miner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 

Repository files navigation

miner

Tiny dumb jquery selector based recursive web miner. It does not run scripts, works on the static html content received which might reduce the usefulness.

examples

To save all JPGs from front page' img tags: node miner.js --url=http://www.hs.fi/ or not using defaults: node miner.js --url=http://www.hs.fi/ --recurse=0 --tag=img --attr=src --extension=jpg --reset=true

To save all JPGs from front page and linked page. node miner.js --url=http://www.imdb.com/ --recurse=2

To save all MP3 files from front page and any link or link's link one it. node miner.js --url=http://www.imdb.com/ --recurse=2 --tag=a --attr=href --extension=mp3

To not reset data directory on every run use --reset=false

About

harvests files from website

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published
0