One Paragraph of project description goes here
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
| import requests | |
| import sys | |
| import json | |
| def waybackurls(host, with_subs): | |
| if with_subs: | |
| url = 'http://web.archive.org/cdx/search/cdx?url=*.%s/*&output=json&fl=original&collapse=urlkey' % host | |
| else: | |
| url = 'http://web.archive.org/cdx/search/cdx?url=%s/*&output=json&fl=original&collapse=urlkey' % host |
| import requests | |
| import re | |
| import sys | |
| from multiprocessing.dummy import Pool | |
| def robots(host): | |
| r = requests.get( | |
| 'https://web.archive.org/cdx/search/cdx\ | |
| ?url=%s/robots.txt&output=json&fl=timestamp,original&filter=statuscode:200&collapse=digest' % host) |
| " _ _ " | |
| " _ /|| . . ||\ _ " | |
| " ( } \||D ' ' ' C||/ { % " | |
| " | /\__,=_[_] ' . . ' [_]_=,__/\ |" | |
| " |_\_ |----| |----| _/_|" | |
| " | |/ | | | | \| |" | |
| " | /_ | | | | _\ |" | |
| It is all fun and games until someone gets hacked! |
| Filter | Description | Example |
|---|---|---|
| allintext | Searches for occurrences of all the keywords given. | allintext:"keyword" |
| intext | Searches for the occurrences of keywords all at once or one at a time. | intext:"keyword" |
| inurl | Searches for a URL matching one of the keywords. | inurl:"keyword" |
| allinurl | Searches for a URL matching all the keywords in the query. | allinurl:"keyword" |
| intitle | Searches for occurrences of keywords in title all or one. | intitle:"keyword" |