GitLocker: The Coding Marketplace

Description:

aliyunoss2autoupload 0.1

Source:
https://github.com/tanbro/aliyunoss2-autoupload

Package:
https://pypi.org/project/aliyunoss2-autoupload/

aliyunoss2-autoupload
Monitor files by wildcard patterns, upload them to ALIYUN OSS, then move to backup directory.

Usage

Command Line
After the package installed, run the command in a terminator, show help messages
$ aliyunoss2-autoupload --help

usage: aliyunoss2-autoupload [-h] [--version] {run,echo_config_sample} ...

Watch files in a directory and upload them to Aliyun OSS on file writing
completed

optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit

subcommands:

{run,echo_config_sample}
<sub_command --help> Print the help of sub_commands
run Start to run then program. It will monitor and upload
files continuously.
echo_config_sample Echo configure file sample
$ aliyunoss2-autoupload run --help

usage: aliyunoss2-autoupload run [-h] [--only-once]
[--config-file CONFIG_FILE]
[--logging-config-file LOGGING_CONFIG_FILE]

optional arguments:
-h, --help show this help message and exit
--only-once, -o Upload only once, then exit. Will NOT monitor files.
(default=False)
--config-file CONFIG_FILE, -c CONFIG_FILE
The program configuration file. The program will first
try to load configuration file by environment variable
${ALIYUNOSS2_AUTOUPLOAD_CONF}. If the environment
variable not assigned, then try to load configuration
file "conf/aliyunoss2-autoupload.yml"
--logging-config-file LOGGING_CONFIG_FILE, -l LOGGING_CONFIG_FILE
The logging configuration file. The program will first
try to load logging configuration file by environment
variable ${ALIYUNOSS2_AUTOUPLOAD_LOG_CONF}. If the
environment variable not assigned, then try to load
configuration file
"conf/aliyunoss2-autoupload.log.yml"
$ aliyunoss2-autoupload echo_config_sample --help

usage: aliyunoss2-autoupload echo_config_sample [-h] {prog,log}

positional arguments:
{prog,log} Configure file to echo

optional arguments:
-h, --help show this help message and exit

Configuration File
The program will first try to load configuration file from environment variable ALIYUNOSS2_AUTOUPLOAD_CONF.
If the environment variable not assigned, then try to load configuration file “conf/aliyunoss2-autoupload.yml”.
The YAML file is like blow:
---

## Aliyun OSS configs
oss:
## Name of your Aliyun OSS bucket
name: "your_bucket_name"
## Endpoint URL of Aliyun OSS bucket
endpoint: "oss-xx-xxxxxx.aliyuncs.com"
## cname of the domain of Aliyun OSS bucket. Empty if no cname.
cname: ""
## Access Key ID of Aliyun OSS bucket
access_key_id: "your_access_key_id"
## Access Key Secret of Aliyun OSS bucket
access_key_secret: "your_access_key_secret"

## Directory name configs
dir:
## Calculate uploading file relative name by this local directory
rel_dir: ""
## Upload files to OSS in this dir
oss_dir: ""
## Move uploaded file the the directory. It MUST be a different dir from where the files are. If not, the file will be uploaded again and again.
bak_dir: ""

## watcher configs
watcher:
## The time interval(seconds) this program scan the directory
interval: 30
## If the interval between the current time and the file\'s modification time is greater than this value, the write is considered complete.
write_complete_time: 30
## Pattern of the files to watch and upload
patterns: "files/*.*"
## If find patterns recursively
recursive: false
## pool of at most max_workers threads to execute upload/backup tasks. If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5.
max_workers: ~
Also, the program will first try to load logging configuration file by environment variable ALIYUNOSS2_AUTOUPLOAD_LOG_CONF.
If the environment variable not assigned, then try to load configuration file “conf/aliyunoss2-autoupload.log.yml”.
Logging config file is also YAML.
Go to https://docs.python.org/3/library/logging.config.html for more information about Python logging config.

Install

Install by pip
pip install aliyunoss2-autoupload

Install from code
git clone https://github.com/tanbro/aliyunoss2-autoupload.git
cd aliyunoss2-autoupload
path/of/your/python setup.py install

Changelog

0.1

Added

Reload configuration every time before scanning files.
–only-once command line option.

Optimized

test case for main() function

0.1b2

Date:
2018-06-20

Adds

More detailed documents.

Bug Fixes:

Remove files in .gitignore, but tracked.

0.1b1

Date:
2018-06-19

Changes

Support old Python 2.7, Python 3.4
Config file name extension changed from ".yaml" to ".yml"
Default config file environment variable name
Default config file path if no environment variable

Adds

Some simple test cases
CircleCI
Codacy

0.1a3
The first ever-usable version.

Date:
2018-04-18

Changes

Now Python 3.5+ ONLY, because glob.iglob has no recursive argument and "**" wildcard in lower Python version

Bug Fixes

Backup directory errors
OSS exception

CONTRIBUTING
Contributions are welcome!
It’s advised to develop in venv
/path/of/your/python -m venv .venv
source .venv/bin/activate
git clone git@github.com:tanbro/aliyunoss2-autoupload.git
cd aliyunoss2-autoupload
python setup.py develop

Authors

liu xue yan <liu_xue_yan@foxmail.com>