BackuPy 1.10.1

Description:

BackuPy 1.10.1

BackuPy

Installation
Features
Design Goals
Usage Description
Command Line Interface
Configuration File
Building From Source

Installation

Install the latest release from PyPI (supports all platforms with Python and has no dependencies outside the standard library)

pip install backupy --upgrade

Features

Backup, Mirror, and Sync Modes
Compare files using attributes or CRCs
View changed file tree with curses
Detection and alerts of corrupted files
Detection and alerts of unexpected file modifications on destination outside of backups and mirrors, or sync conflicts (a file was modified on both sides since the last sync)
JSON formatted database for tracking files and CSV formatted logs
Filter files with regular expressions
Files are always safe by default, being moved to an identically structured archive directory before being deleted or overwritten

Design Goals

Backups should be future proof and verifiable, even without BackuPy

uses file-based increments and human readable database/log files that are also easy to parse


Code should be simple and easy to verify to ensure predicable and reliable operation

a callgraph is available in analysis/callgraph.svg
there are only three, easy to follow functions (under FileManager in backupy/fileman.py) that ever touch your files, no more, no less, three shall be the number of thou functions, and the number of the functions shall be three
use trustworthy dependencies


Follow the principle of least astonishment

clear backup behaviour between directories, the current status of files and how they will be handled upon execution should be perfectly obvious


Avoid feature creep and duplicating other programs

no delta-transfer (extend with another backend)
no network storage or FUSE support (these must be mounted by another program for BackuPy to see them)
no backup encryption (use encrypted storage)
no filesystem monitoring (this is not a continuous backup/sync program)


Easily extensible with other backends

all the low level functions used for file operations are under FileOps from backupy.utils for easy monkey patching (see example_extension.py)



Usage Description

Source and destination directories can be any directory accessible via the computer's file system

Destination can be empty or contain files from a previous backup (even one made without BackuPy), matching files on both sides will be skipped
Use the --posix flag if you plan on using BackuPy between Windows and any other OS


Main modes (how to handle new and deleted files)

Backup mode: copies files that are only in source to destination
Mirror mode: copies files that are only in source to destination and deletes files that are only in destination
Sync mode: copies files that are only in source to destination and copies files that are only in destination to source

you may also want to use --sync-delete to propagate deletions
see write_database_x2 in the configuration file if syncing more than two folders




Selection modes (which file to select in cases where different versions exist on both sides)

Source mode: copy source files to destination
Destination mode: copy destination files to source
Newer mode: copy newer files based on last modified time
None mode: don't copy either, differing files will only be logged for manual intervention


Compare modes (how to detect which files have changed)

Attribute mode: compare file attributes (size and last modified time)
Attribute+ mode: compare file attributes and calculate CRCs only for new and changed files for future verification
CRC mode: compare file attributes and CRC for every file, and checks previously stored CRCs to detect corruption

you may also want to use --verify to verify the CRC of files after they're copied




Test your options first with the --dry-run flag
See Command Line Interface and Configuration File below for all available options
By default, you will always be notified of any changes, unexpected modifications, sync conflicts, or file corruption before being prompted to continue, cancel, or skip selected files

it is recommended to use --qconflicts if using --noprompt, especially if also using --noarchive


By default, you will be prompted before any changes are made to your files and

overwritten files will be moved to <source|dest>/.backupy/Archives/yymmdd-HHMM/<original path>
deleted files will be moved to <source|dest>/.backupy/Trash/yymmdd-HHMM/<original path>


Symbolic links to folders are never followed and always copied verbatim
Symbolic links to files are followed by default, copying the referenced file, use --nofollow to copy symbolic links to files verbatim
To restore files, just copy them over from your destination to source (or swap source and destination in BackuPy)

Command Line Interface
usage: backupy [options] -- <source> <dest>
backupy <source> <dest> [options]
backupy <source> --load [-c mode] [--dbscan] [--dry-run]
backupy -h | --help | --version

positional arguments:
source Path to source
dest Path to destination

optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit

file mode options:

-m mode Main mode: for files that exist only on one side
MIRROR (default)
[source-only -> destination, delete destination-only]
BACKUP
[source-only -> destination, keep destination-only]
SYNC
[source-only -> destination, destination-only -> source]
-s mode Selection mode: for files that exist on both sides but differ
SOURCE (default)
[copy source to destination]
DEST
[copy destination to source]
NEW
[copy newer to opposite side]
NO
[do nothing]
-c mode Compare mode: for detecting which files differ
ATTR (default)
[compare file attributes: mod-time and size]
ATTR+
[compare file attributes and record CRC for changed files]
CRC
[compare file attributes and CRC for every file]

misc file options:

--sync-delete
Use the database to propagate deletions since the last sync
--fi regex [regex ...]
Filter: Only include files matching the regular expression(s)
(include all by default, searches file paths)
--fe regex [regex ...]
Filter: Exclude files matching the regular expression(s)
(exclude has priority over include, searches file paths)
--noarchive Disable archiving files before overwriting/deleting to:
<source|dest>/.backupy/Archives/yymmdd-HHMM/
<source|dest>/.backupy/Trash/yymmdd-HHMM/
--nofollow Do not follow symlinks when copying files
--nomoves Do not detect when files are moved or renamed

execution options:

--noprompt Complete run without prompting for confirmation
-d, --dbscan
Only scan files to check and update their database entries
-n, --dry-run
Perform a dry run with no changes made to your files
-q, --qconflicts
Quit if database conflicts are detected (always notified)
-> unexpected changes on destination (backup and mirror)
-> sync conflict (file modified on both sides since last sync)
-> file corruption (ATTR+ or CRC compare modes)
-v, --verify
Verify CRC of copied files

backend options (experimental):

--cold Do not scan files on destination and only use local databases
--rsync Use rsync for copying files

configuration options:

--nolog Disable writing log and file databases to:
<source>/.backupy/Logs/log-yymmdd-HHMM.csv
<source|dest>/.backupy/database.json
-p, --posix Force posix style paths on non-posix operating systems
-k, --save Save configuration to <source>/.backupy/config.json and exit
-l, --load Load configuration from <source>/.backupy/config.json and run

BackuPy is a simple backup program in python with an emphasis on data
integrity and transparent behaviour - https://github.com/elesiuta/backupy

BackuPy comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. See the GNU General
Public Licence for details.

Configuration File

The config file is saved to, and loaded from <source>/.backupy/config.json

it contains all the options from the command line interface along with some additional options
the only CLI options that can be used with --load and can override settings in config.json are -c mode, --dbscan, and --dry-run

the overrides can enable --dbscan or --dry-run but not disable


see backupy/config.py for where all the options and defaults are stored in code
below is a description of all the other options that are available


source_unique_id & dest_unique_id

unique id for each folder, used when write_database_x2 is enabled, each assigned a random string by default


archive_dir = ".backupy/Archive"

can be any subdirectory


config_dir = ".backupy"

can't be changed under normal operation


log_dir = ".backupy/Logs"

can be any subdirectory


trash_dir = ".backupy/Trash"

can be any subdirectory


cleanup_empty_dirs = True

delete directories when they become empty


root_alias_log = True

abbreviate absolute paths to source and dest with <source> and <dest> in logs


stdout_status_bar = True

show progress status bar


verbose = True

print list of differences between directories to stdout


write_database_x2 = False

write both source and destination databases to each side using their unique_id, useful for syncing groups of more than two folders or with the --sync-delete flag


write_log_dest = False

write a copy of the log to <dest>/<log_dir>/log-yymmdd-HHMM-dest.csv


write_log_summary = False

alternative log structure, written in addition to standard log


nocolour = False

disable colour when printing to stdout



Building From Source

Run tests with

python setup.py test


Building a python package

python setup.py sdist


Building an executable with the GUI (depends on Gooey) (deprecated, may rewrite with flutter)

pyinstaller build.spec


You can package the executable on Windows by running setup.iss with Inno Setup