wget -mpck --html-extension --user-agent="" -e robots=off --wait 1 -P . www.foo.com- -m (Mirror) Turns on mirror-friendly settings like infinite recursion depth, timestamps, etc.
| { | |
| "version": "0.1.0", | |
| "name": "pump", | |
| "instructions": [ | |
| { | |
| "name": "initialize", | |
| "docs": [ | |
| "Creates the global state." | |
| ], | |
| "accounts": [ |
| From: http://www.labnol.org/software/wget-command-examples/28750/ | |
| How do I download an entire website for offline viewing? How do I save all the MP3s from a website to a folder on my computer? How do I download files that are behind a login page? How do I build a mini-version of Google? | |
| Wget is a free utility – available for Mac, Windows and Linux (included) – that can help you accomplish all this and more. What makes it different from most download managers is that wget can follow the HTML links on a web page and recursively download the files. It is the same tool that a soldier had used to download thousands of secret documents from the US army’s Intranet that were later published on the Wikileaks website. | |
| You mirror an entire website with wget | |
| Mirror an entire website with wget | |
| Spider Websites with Wget – 20 Practical Examples |