shallow-backup 6.4

Description:

shallowbackup 6.4

shallow-backup




shallow-backup lets you easily create lightweight backups of installed packages, applications, fonts and dotfiles, and automatically push them to a remote Git repository.
I use it to manage my dotfiles.

Contents

Why?
Installation

Method 1: pipx
Method 2: Install From Source


Dependencies
Usage
Recipes

Maintain a separate repo for your dotfiles
Synchronize dotfiles on multiple computers
Reinstall dotfiles from a backup


What can I back up?
Configuration

Conditional Backup and Reinstallation


Git Integration

A Word of Caution
.gitignore


Output Structure
Want to Contribute?

Why?
I wanted a tool that allows you to:

Back up dotfiles from where they live on the system.
Back up files from any path on the system, not just $HOME.
Reinstall them from the backup directory idempotently.
Backup and reinstall files conditionally, so you can easily manage dotfiles across multiple systems.
Copy files on installation and backup, as opposed to symlinking them.
Backup package installations in a highly compressed manner
Not worry about accidentally doing something dangerous / destructive (is user-protective)

shallow-backup checks all of those boxes.
Installation


Warning
Be careful running this with elevated privileges. Code execution can be achieved with write permissions on the config file.

Method 1: pipx
$ pipx install shallow-backup

Method 2: Install From Source
$ git clone https://www.github.com/alichtman/shallow-backup.git
$ cd shallow-backup
$ pip3 install .

Dependencies


pre-commit
trufflehog

If you are missing the dependencies, you will be guided to install them.
Usage


To start the interactive program, run $ shallow-backup.
To backup your dotfiles, run $ shallow-backup --backup-dots.

shallow-backup was built with scripting in mind. Every feature that's supported in the interactive program is supported with command line arguments.
Usage: shallow-backup [OPTIONS]

Easily back up installed packages, dotfiles, and more.
You can edit which files are backed up in ~/.shallow-backup.

Written by Aaron Lichtman (@alichtman).

Options:

--add-dot TEXT Add a dotfile or dotfolder to config by path.
--backup-all Full back up.
--backup-configs Back up app config files.
--backup-dots Back up dotfiles.
--backup-fonts Back up installed fonts.
--backup-packages Back up package libraries.
--delete-config Delete config file.
--destroy-backup Delete backup directory.
--dry-run Don't backup or reinstall any files, just give
verbose output.

--new-path TEXT Input a new back up directory path.
--no-new-backup-path-prompt Skip setting new back up directory path prompt.
--no-splash Don't display splash screen.
--reinstall-all Full reinstallation.
--reinstall-configs Reinstall configs.
--reinstall-dots Reinstall dotfiles and dotfolders.
--reinstall-fonts Reinstall fonts.
--reinstall-packages Reinstall packages.
--remote TEXT Set remote URL for the git repo.

--show Display config file.
-v, --version Display version and author info.
-h, -help, --help Show this message and exit.

Recipes

Maintain a separate repo for your dotfiles
shallow-backup makes this easy! After making your first backup, cd into the dotfiles/ directory and run $ git init. Create a .gitignore, and a create / set up (link the upstream remote, etc) a new repo on your favorite version control platform. With operations involving the parent shallow-backup repo, shallow-backup will prompt you interactively to update the nested submodule. After that is taken care of, shallow-backup will move on to updating the parent. The dotfiles repo will be tracked as a submodule.
Synchronize dotfiles on multiple computers
Run shallow-backup --backup-dots on the first computer. Make a commit and push to the remote. Then pull these changes down on the second computer. Run shallow-backup --backup-dots on the second computer, and resolve the merge conflicts. Once you have a final version you're happy with, make a commit, push it, and run shallow-backup --reinstall-dots. On the first computer, pull the changes and run shallow-backup --reinstall-dots. Your changes are now sync'd across both computers and the remote repository.
Reinstall dotfiles from a backup
To reinstall your dotfiles, clone your dotfiles repo and make sure your shallow-backup config path can be found at either ~/.config/shallow-backup.conf or $XDG_CONFIG_HOME/.shallow_backup.conf. Set the backup-path key in the config to the path of your cloned dotfiles. Then run $ shallow-backup --reinstall-dots.
When reinstalling your dotfiles, the top level .git/, .gitignore, img/ and README.md files and directories are ignored.
zsh Completions
Available in zsh-users/zsh-completions. Follow the installation instructions here.
What can I back up?

By default, shallow-backup backs these up.


Dotfiles and dotfolders

.bashrc
.bash_profile
.gitconfig
.pypirc
.config/shallow-backup.json
.ssh/
.vim/
.zshrc



App Config Files

VSCode
Sublime Text 2/3
Terminal.app



Installed Packages

brew and cask
cargo
gem
pip
pip3
npm
macports
VSCode Extensions
Sublime Text 2/3 Packages
System Applications



User installed fonts.


Configuration
If you'd like to modify which files are backed up, you have to edit the JSON config file, located at ~/.config/shallow-backup.json. There are two ways to do this.

Select the appropriate option in the CLI and follow the prompts.
Open the file in a text editor and make your changes.

Editing the file in a text editor will give you more control and be faster.
Conditional Backup and Reinstallation

Warning
This feature allows code execution (by design). If untrusted users can write to your config, they can achieve code execution next time you invoke shallow-backup backup or reinstall functions. Starting in v5.2, the config file will have default permissions of 644, and a warning will be printed if others can write to the config.

Every key under dotfiles has two optional subkeys: backup_condition and reinstall_condition. Both of these accept expressions that will be evaluated with bash. An empty string ("") is the default value, and is considered to be True. If the return value of the expression is 0, this is considered True. Otherwise, it is False. This lets you do simple things like preventing backup with:
// Because `$ false` returns 1
"backup_condition": "false"

And also more complicated things like only backing up certain files if an environment variable is set:
"backup_condition": "[[ -n \"$ENV_VAR\" ]]"

Here's an example config based on my dotfiles:
{
"backup_path": "~/shallow-backup",
"lowest_supported_version": "5.0.0a",
"dotfiles": {
".config/agignore": {
"backup_condition": "uname -a | grep Darwin",
"reinstall_condition": "uname -a | grep Darwin"
},
".config/git/gitignore_global": { },
".config/jrnl/jrnl.yaml": { },
".config/kitty": { },
".config/nvim": { },
".config/pycodestyle": { },
...
".zshenv": { }
},
"root-gitignore": [
".DS_Store",
"dotfiles/.config/nvim/.netrwhist",
"dotfiles/.config/nvim/spell/en.utf-8.add",
"dotfiles/.config/ranger/plugins/ranger_devicons",
"dotfiles/.config/zsh/.zcompdump*",
"dotfiles/.pypirc",
"dotfiles/.ssh"
],
"dotfiles-gitignore": [
".DS_Store",
".config/nvim/.netrwhist",
".config/nvim/spell/en.utf-8.add*",
".config/ranger/plugins/*",
".config/zsh/.zcompdump*",
".config/zsh/.zinit",
".config/tmux/plugins",
".config/tmux/resurrect",
".pypirc",
".ssh/*"
],
"config_mapping": {
"/Users/alichtman/Library/Application Support/Sublime Text 2": "sublime2",
"/Users/alichtman/Library/Application Support/Sublime Text 3": "sublime3",
"/Users/alichtman/Library/Application Support/Code/User/settings.json": "vscode/settings",
"/Users/alichtman/Library/Application Support/Code/User/Snippets": "vscode/Snippets",
"/Users/alichtman/Library/Application Support/Code/User/keybindings.json": "vscode/keybindings",
"/Users/alichtman/Library/Preferences/com.apple.Terminal.plist": "terminal_plist"
}
}

Git Integration

A Word of Caution
This backup tool is git-integrated, meaning that you can easily store your backups remotely (on GitHub, for example.) Dotfiles and configuration files may contain sensitive information like API keys and ssh keys, and you don't want to make those public. To make sure no sensitive files are uploaded accidentally, shallow-backup creates a .gitignore file if it can't find one in the directory. It excludes .ssh/ and .pypirc by default. It's safe to remove these restrictions if you're pushing to a remote private repository, or you're only backing up locally. To do this, you should clear the .gitignore file without deleting it.
If you choose to back up to a public repository, look at every file you're backing up to make sure you want it to be public.

[!NOTE]
As of v6.2, trufflehog is run as a required precommit hook and will detect secrets.

.gitignore
As of v4.0, any .gitignore changes should be made in the shallow-backup config file. .gitignore changes that are meant to apply to all directories should be under the root-gitignore key. Dotfile specific gitignores should be placed under the dotfiles-gitignore key. The original default-gitignore key in the config is still supported for backwards compatibility, however, converting to the new config format is strongly encouraged.
Output Structure

shallow_backup/
├── configs
│   ├── plist
│   │   └── com.apple.Terminal.plist
│   ├── sublime_2
│   │   └── ...
│   └── sublime_3
│   └── ...
├── dotfiles
│ ├── .bash_profile
│ ├── .bashrc
│ ├── .gitconfig
│ ├── .pypirc
│ ├── ...
│ ├── shallow-backup.json
│   ├── .ssh/
│   │   └── known_hosts
│   ├── .vim/
│ └── .zshrc
├── fonts
│   ├── AllerDisplay.ttf
│   ├── Aller_Bd.ttf
│   ├── ...
│   ├── Ubuntu Mono derivative Powerline Italic.ttf
│   └── Ubuntu Mono derivative Powerline.ttf
└── packages
├── brew-cask_list.txt
├── brew_list.txt
├── cargo_list.txt
├── gem_list.txt
├── installed_apps_list.txt
├── npm_list.txt
├── macports_list.txt
├── pip_list.txt
└── sublime3_list.txt

Reinstalling Dotfiles

To reinstall your dotfiles, clone your dotfiles repo and make sure your shallow-backup config path can be found at either ~/.config/shallow-backup.json or $XDG_CONFIG_HOME/.shallow_backup.json. Set the backup-path key in the config to the path of your cloned dotfiles. Then run $ shallow-backup -reinstall-dots.
When reinstalling your dotfiles, the top level .git/, .gitignore, img/ and README.md files and directories are ignored.
Want to Contribute?

Check out CONTRIBUTING.md and the docs directory.