pathbuster 0.3.0

Description:

pathbuster 0.3.0

PathBuster - multiple hosts Web path scanner
This scanner is designed to check paths on multiple hosts at the same time.
One path is taken and checked multithreaded across all hosts, then the next path is taken, etc.
This gives us the following benefits:

there is no heavy load on one host (if we checked many paths in several threads on one host).
prevents a possible ban on the scanner by the WAF.
saving time, there is no need to run the scanner for each host separately.
a large number of results at once.

For convenience, the results are written to two files at once in the pathbuster-res folder:

file with hostname (all paths found for this host)
file with response code (file with all responses 200, 301, etc.)


Before starting scanning, the program checks the server's responses on random string and, if the response code is not excluded by the program settings, writes a sample (code and size) of the response for subsequent comparison.
This allows us to exclude a large number of false positives (for example, if the server responds to us 200 OK for all requests)
And it allows you to find answers that differ from the recorded samples, even if the code was the same.
Installation:
pip3 install -U pathbuster

Basic usage:
pathbuster -u /path/to/URLS_FILE -p /path/to/wordlist -srd pathbuster-res

Passive check with Nuclei
pathbuster -u /path/to/URLS_FILE -p /path/to/wordlist --store_response -srd pathbuster-res


options:
-h, --help show this help message and exit
-m HTTP_METHOD, --http_method HTTP_METHOD
HTTP method to use (default: GET)
-u URLS_FILE, --urls_file URLS_FILE
urls file (base url) (default: None)
-p PATHS_FILE, --paths_file PATHS_FILE
paths wordlist (default: None)
-e EXCLUDE_CODES, --exclude_codes EXCLUDE_CODES
Exclude status codes, separated by commas (Example: 404,403) (default: 404)
-x EXTENSIONS, --extensions EXTENSIONS
Extension list separated by commas (Example: php,asp) (default: )
-ac Automatically calibrate filtering options (default: False)
-sr, --store_response
Store finded HTTP responses (default: False)
-srd STORE_RESPONSE_DIR, --store_response_dir STORE_RESPONSE_DIR
Output directory (default: None)
-fe FILTER_REGEX, --filter-regex FILTER_REGEX
filter response with specified regex (-fe admin) (default: None)
-json store output in JSONL(ines) format (default: False)
-f, --follow_redirects
Follow HTTP redirects (same host only) (default: False)
-H HEADER, --header HEADER
Add custom HTTP request header, support multiple flags (Example: -H "Referer: example.com" -H "Accept: */*") (default: None)
--proxy PROXY proxy ip:port (default: None)
--max_response_size MAX_RESPONSE_SIZE
Maximum response size in bytes (default: 250000)
--max_errors MAX_ERRORS
Maximum errors before url exclude (default: 5)
-t THREADS, --threads THREADS
Number of threads (keep number of threads less than the number of hosts) (default: 10)
-ua USER_AGENT, --user_agent USER_AGENT
User agent (default: Mozilla/5.0 (compatible; pathbuster/0.1; +https://github.com/rivalsec/pathbuster))
--stats_interval STATS_INTERVAL
number of seconds to wait between showing a statistics update (default: 60)
-maxr MAX_REDIRECTS, --max_redirects MAX_REDIRECTS
Max number of redirects to follow (default: 5)