thetrove-downloader 1.1.0

Description:

thetrovedownloader 1.1.0

thetrove-downloader




This Python package allows to easily download any resource, be it folder or file,
from The Trove.is.
Installation and Requirements
The program requires Python 3.9 or above.
To install the program it is sufficient to use Python pip and get the package thetrove-downloader. The package will
add a new command thetrove-downloader.
python3 -m pip install thetrove-downloader

Updates should be performed via pip.
python3 -m pip install --upgrade thetrove-downloader

Usage
thetrove-downloader [-h] [-t, --target TARGET] [--target-url URL]
[-j, --json JSON] [-f, --folder FOLDER]
[-o, --output OUTPUT] [-b, --blacklist BLACKLIST]
[-w, --whitelist WHITELIST] [--no-download]
[--no-preserve-root]



-h, --help show help message


-t, --target <TARGET> download target (folder or file)
To download a specific resource, start the program with the --target command and supply a valid path (e.g., "
Books/White Star").


--target-url download target (full URL to folder or file)
Overrides --target argument with a full URL and sets the download target to the path of the URL (
e.g., "https://thetrove.is/Open%20Source/BIND%20RPG" to "Open Source/BIND RPG").


-j, --json <JSON> save/read instructions from a JSON file
The --json option allows to specify a file to save the target, folder, output, whitelist, and blacklist instructions
in JSON format. Launching the program with only the --json option will load the instructions previously saved in the
file and perform all of them in succession. This can be used to keep a local copy of specific resources up to date.


-f, --folder <FOLDER> destination folder
The --folder option allows to specify a folder in which the targeted resource will be downloaded.


-o, --output <OUTPUT> output name of download target
The --output option allows to override the folder/file name of the resource. Note that if the targeted resource is a
folder, the option will only apply to the top-level folder and leave all contents therein unaltered.


-b, --blacklist <BLACKLIST> regex blacklist for files/folders
The --whitelist option allows to specify a regex pattern that will be used to skip files and folder that do not
contain the supplied pattern. Overwrites blacklist options.


-w, --whitelist <WHITELIST> regex whitelist for files/folders, overrides blacklist
The --whitelist option allows to specify a regex pattern that will be used to skip files and folder that do not
contain the supplied pattern. Overwrites blacklist options.


--no-download list content without downloading
The --no-download flag allows to list the content under the supplied target without performing any download. The
instructions will still be saved in a JSON file if the --json options is used, but not the flag.


--no-preserve-root allows the download of root folders
The --no-preserve-root flag allows downloading entire root folders such as "/Books/". This flag is not preserved
with the --json option and must be passed manually every time.


NOTE: the program requires to use either --target, --json, or both --target and --json. Without one of
these combinations the program won't have a valid download instruction.
Notes on Crawling
The program performs all GET requests without throttling or delay and with no disallowed paths as per the instructions
at thetrove.is/robots.txt.
However, please do take care to not overuse this utility and risk overwhelming their connection. Thank you.
Root folders such as "/Books" are protected and cannot be downloaded without using the --no-preserve-root flag.
Contributing
All contributions and suggestions are welcome!
If you have suggestions for fixes or improvements, you can open an issue with your idea, see #Issues for
details.
Issues
If you encounter any problem during usage of the program, an issue can be opened on the project's page
on GitLab.
When opening an issue for a problem, please copy the error message – if any – and describe the operation in progress
when the error occurred.