pearlcli 2.1.1

Description:

pearlcli 2.1.1

Pearl
Because only in the best Shells you will find a Pearl...






Project Status
Communication









Table of Contents

Description
Quickstart
Installation

Dependencies
Linux
OSX


Create your own Pearl package
Create your own Pearl repository
Troubleshooting
Contributing

Description
Pearl is a lightweight package manager for automating reproducible environments
between different systems (Linux and OSX).
It can be used for dotfiles, plugins, programs and any form of code
accessible via git.
As soon as a package gets installed, its content can be activated out of the box
according to certain events, like, for instance, a shell startup (Bash, Zsh or Fish) or
an editor startup (Vim or Emacs). This is possible via a smart and simple
hook mechanism
that integrates the package content within the Pearl ecosystem.
The main advantages on using Pearl are:

Create your own Pearl package in a very simple way.
Full control and sync of your dotfiles across different systems.
Automatic bootstrap of the package content whenever shells or editors get started.
Access to a wide range of existing packages via the OPH (Official Pearl Hub).
Allows to create your own shareable package repository.
Stable codebase with 100+ unit tests and exhaustive integration tests via Travis for Linux and OSX.
Small number of dependencies needed in order to ensure compatibility with most of the systems.

Comparison with similar solution: Ansible
You could achieve something similar from what Pearl provide by using
Ansible. Ansible is a powerful software for IT
automation which can be widely used for many use cases.
Despite of this, Ansible has few drawbacks when using it for lightweight forms of automation:

Pearl uses bash for writing simple scripts for automation:

it makes easier the integration with other programs in the system (without existing Playbooks may be hard and tedious to achieve this in Ansible);
bash is a powerful, accessible and well-known language;


Ansible requires way more dependencies than Pearl;
Ansible requires knowledge about how Ansible Playbooks works;
Pearl uses built-in functions and variables which heavily simplify construction of scripts for automation;
Pearl makes easier to remove packages and restore the system to an initial state;

Quickstart
The Pearl command allows to: create, list, search, install, update, emerge,
remove the Pearl packages defined according to the configuration located in
$XDG_CONFIG_HOME/pearl/pearl.conf (defaults to ~/.config/pearl/pearl.conf)

Create

Command create helps you create a new local Pearl package:

$ pearl create mydotfiles ~/dotfiles

This will create a directory pearl-config in ~/dotfiles containing all the templates to help you
start writing a Pearl package. ~/dotfiles does not need to be an empty directory.
Additionally, the local repository in $XDG_CONFIG_HOME/pearl/pearl.conf will be updated with
the new package entry called mydotfiles.
For more information about the pearl-config content look at the section below.
List

List all the available packages:

$ pearl list
...
pearl/dot-git
Awesome git dotfiles (https://github.com/pearl-hub/git)
pearl/sesaila [installed]
Awesome aliases for Bash, Zsh and Fish shells (https://github.com/pearl-hub/sesaila)
pearl/airline [installed]
Status/tabline for vim (https://github.com/vim-airline/vim-airline)
pearl/trash-cli [installed]
Command line interface to the freedesktop.org trashcan (https://github.com/pearl-hub/trash-cli)
...

Search

Search for vim Pearl packages:

$ pearl search vim
* Updating https://github.com/pearl-hub/repo.git repository
pearl/dot-vim
Awesome vim dotfiles (https://github.com/pearl-hub/vim)

Install

Install pearl/dot-vim package (as soon as the package is installed the package is ready out of the box in vim editor!):

$ pearl install dot-vim
* Updating https://github.com/pearl-hub/repo.git repository
* Installing pearl/dot-vim package


Install pearl/trash-cli package:

$ pearl install trash-cli
* Updating https://github.com/pearl-hub/repo.git repository
* Installing pearl/trash-cli package
$ trash -h
Usage: trash [OPTION]... FILE...

Put files in trash
...
...

Update

Update pearl/dot-vim package:

$ pearl update dot-vim
* Updating https://github.com/pearl-hub/repo.git repository
* Updating pearl/dot-vim package


Update Pearl and all its packages installed:

$ pearl update
...
* Updating https://github.com/pearl-hub/repo.git repository
* Updating Pearl script
* Updating pearl/dot-vim package
* Updating pearl/airline package
* Updating pearl/trash-cli package
* Updating pearl/caprica package
...

Emerge
Emerge is an idempotent command for either installing or updating a package
depending whether the package is already installed or not. This command turns
to be particularly useful for establishing dependencies between packages.
See the section below
for more details.
Remove

Remove pearl/dot-vim package:

$ pearl remove dot-vim
* Updating https://github.com/pearl-hub/repo.git repository
* Removing pearl/dot-vim package


Remove Pearl and all its packages installed:

$ pearl remove
...
Are you sure to REMOVE all the Pearl packages in $PEARL_HOME folder? (N/y)
* Updating https://github.com/pearl-hub/repo.git repository
* Removing pearl/dot-vim package
* Removing pearl/airline package
* Removing pearl/trash-cli package
* Removing pearl/caprica package
...

Recommended Pearl Hub packages to install:

cmd
kyrat
ranger
sesaila
trash-cli
txum

For dotfiles packages take a look here.
Check out the OPH (Official Pearl Hub)
for more packages you might be interested.
Installation
Dependencies
Before installing Pearl be sure that all dependencies are properly installed in your system.
The Pearl dependencies are the following:
Mandatory

python (>=3.5)
bash (>=4.1)
git (>=1.8)

Optional
The following are not mandatory dependencies but can be handy to have for the hook functions in Pearl package.
All the Linux distributions have these dependencies already installed.

GNU coreutils
grep
sed

Additional shells supported
Pearl supports also the following shells:

fish (>=2.2.0)
zsh (>=5.2)

Linux
Arch Linux
Pearl can be installed in Arch Linux through AUR.
The package is pearl-git.
For example, to install Pearl via yay AUR helper:
yay -S pearl-git

Any other AUR helpers can be found here.
Other Linux distributions
Assuming all Pearl dependencies are properly installed
in the system, to install Pearl you can use the pip command.
Unless there is a specific use case, it is not a good option to use virtual environments such as
virtualenv or conda because otherwise Pearl will be only visible within that environment.
It is recommended to use the system-wide pip which is generally locate in /usr/bin/pip.
The following will install the package in your $HOME directory (~/.local/):
/usr/bin/pip install --user pearlcli
export PATH="~/.local/bin:$PATH"

Pearl command will be located in ~/.local/bin/pearl
To create the $PEARL_HOME directory and the new pearl configuration file from template, run:
pearl init

OSX
In order to install all Pearl dependencies, you first need to install Homebrew.
To install all the needed dependencies via Homebrew:
brew update
brew install bash git coreutils grep gnu-sed python

The following will install the package under /usr/local:
/usr/local/bin/pip3 install pearlcli
# If the bin path is not already in $PATH:
export PATH="/usr/local/bin:$PATH"

Pearl command will be located in /usr/local/bin/pearl
To create the $PEARL_HOME directory and the new pearl configuration file from template, run:
pearl init

IMPORTANT NOTE: Pearl gets loaded through ~/.bashrc. The problem is that in OSX,
the terminal opens a login shell and only ~/.bash_profile will get executed.
Run the following only if ~/.bashrc is not loaded within ~/.bash_profile file:
echo "[[ -f ~/.bashrc ]] && source ~/.bashrc" >> ~/.bash_profile

This will make sure that ~/.bashrc will run at shell startup.
Create your own Pearl package
Any git repository is already a Pearl package. For instance, in order
to manage a dotfiles repository in Pearl, you just need to change
the Pearl configuration file located in $XDG_CONFIG_HOME/pearl/pearl.conf.
Add the following line to pearl.conf file:
PEARL_PACKAGES = {
"mydotfiles": {
"url": "https://github.com/user/mydotfiles.git",
"description": "My dotfiles"
},
}

In other words, update the PEARL_PACKAGES dictionary with a new entry containing the
name of the package (i.e. mydotfiles),
the git url (i.e. https://github.com/user/mydotfiles.git) and an optional description.
That's it! The package will be ready to be installed,
updated, emerged and removed via the Pearl system.
Structure of a Pearl package
Your own git repository can contain an optional directory
named pearl-config used by Pearl to integrate the package with the Pearl environment.
/ (package root)
│
├── pearl-config (optional directory)
│ │
│ ├── hooks.sh
│ ├── config.sh
│ ├── config.bash
│ ├── config.zsh
│ ├── config.fish
│ ├── config.vim
│ └── config.el
│
└── (additional package content)

The files inside pearl-config are also optional scripts:

hooks.sh - contains the hooks functions executed during the install, update and remove events.
config.sh - will be sourced whenever a new Bash/Zsh shell is starting up.
config.bash - will be sourced whenever a new Bash shell is starting up.
config.zsh - will be sourced whenever a new Zsh shell is starting up.
config.fish - will be sourced whenever a new Fish shell is starting up.
config.vim - will be executed whenever Vim editor is starting up.
config.el - will be sourced whenever Emacs editor is starting up.

The following variables can be used in any of the previous scripts:

PEARL_HOME - Pearl location ($XDG_DATA_HOME/pearl which by default is $HOME/.local/share/pearl)
PEARL_PKGDIR - Pearl package location
PEARL_PKGVARDIR - Pearl package location containing data needed for package
PEARL_PKGNAME - Pearl package name
PEARL_PKGREPONAME - Pearl package repo name (useful to detect and interact with packages within the same repo)

Additionally, the script hooks.sh can use the utility functions available in
Buava and Pearl utils directory that
make easier the integration with Pearl ecosystem.
Useful examples of Pearl packages can be checked in the
Official Pearl Hub.
The hooks.sh script
Hook functions

post_install - Called after an installation of the package occurs.
pre_update - Called before an update of the package occurs.
post_update - Called after an update of the package occurs.
pre_remove - Called before a removal of the package occurs.

An hooks.sh script example
post_install() {
warn "Remember to setup your config located in: ~/.dotfile"
# Do a smart backup before modifying the file
backup ${HOME}/.dotfile
"# New dotfile" > ${HOME}/.dotfile
link tmux "$PEARL_PKGDIR/mytmux.conf"

info "Awesome - new package installed!"
return 0
}
post_update() {
post_install
return 0
}
pre_remove() {
info "dotfiles package removed"
unlink tmux "$PEARL_PKGDIR/mytmux.conf"

# Do an idempotent delete
delete ${HOME}/.dotfile
return 0
}

The info and warn are functions that print a message
using different colors (namely cyan and yellow).
The link unlink are idempotent functions (the result will not change
if the function will be called multiple times) that are able
to link/unlink a config file in order to be loaded at startup by a certain program.
The backup keeps the last three backups of the file and do not perform backup
if the file has not been modified since the latest backup. The delete is a
function for idempotent remove (it will not raise an error if the file
no longer exist).
All these functions belong to the Buava package
in utils.sh
and to the Pearl utils.sh script. You can use them
inside the hooks.sh to any hook function.
Very important note: All the hook functions must be
idempotent
(the commands of each hook function must produce the same result even if
the command gets executed multiple times).
All buava commands are idempotent and this will help to write hook functions
very quickly.
Note: For OSX system, the GNU version sed and grep are automatically
imported in hooks.sh and can be directly used if needed.
Create a Pearl package from a local directory
Pearl package system will work even for local directories. This is particularly useful
whenever a Pearl package needs to be tested before pushing to a git repository.
For instance, the following lines in pearl.conf file will add a package located in
/home/joe/dotfiles:
PEARL_PACKAGES = {
"mydotfiles": {
"url": "/home/user/mydotfiles",
"description": "My dotfiles"
},
}

The directory path must be an absolute path.
The package will be ready to be installed, updated,
emerged and removed via the Pearl system.
The directory content can be structured in the exact way as described
in the section above.
Define dependencies between Pearl packages
Suppose you have a package mypack which depends on another package mydep,
you can update the mypack hooks.sh file in this way:
post_install() {
# Install/update the dependency here:
pearl emerge ${PEARL_PKGREPONAME}/mydep
}
post_update() {
post_install
}
pre_remove() {
# Uncomment below to strictly remove the dependency
# during the removal of the current package:
#pearl remove ${PEARL_PKGREPONAME}/mydep
}

The PEARL_PKGREPONAME variable will make sure to define dependencies only
between packages of the same repository.
To see a real example in Pearl Hub, take a look at the Kyrat hooks.sh.
Use third-party git repository not available in Pearl Hub
If you want to use a third-party git repository
that is not available in the Official Pearl Hub,
you can:

Create your own git repository and use the PEARL_PKGVARDIR directory (recommended)
Create your own git repository and use git submodule
Point directly to the third-party git repository

To see examples of Pearl packages from third-party git repos take a look at the
Official Pearl Hub.
Create your own git repository and use the PEARL_PKGVARDIR directory (recommended)
You can use the PEARL_PKGVARDIR directory during the installation phase to install the third-party git repository.
This is the best way to incorporate third-party project into Pearl ecosystem.
Here it is an example of hooks.sh file which install the ranger file manager into the directory ${PEARL_PKGVARDIR}/ranger:
function post_install(){
install_or_update_git_repo https://github.com/ranger/ranger.git "${PEARL_PKGVARDIR}/ranger" master
}

function post_update(){
post_install
}

function pre_remove(){
rm -rf ${PEARL_PKGVARDIR}/ranger
}

The function install_or_update_git_repo comes from the Buava
library in utils.sh
which is natively available in Pearl during the installation.
You can even use the functions install_git_repo or update_git_repo which respectively install or update the git repository.
For a full example take a look at the ranger Pearl Hub package.
Create your own git repository and use git submodule
Inside your git repository, you just need to add the third-party git repo as a
git submodule.
For instance, to add the powerline in your Pearl package,
you can introduce a submodule in the module directory:
git submodule add https://github.com/powerline/powerline.git module

The filesystem structure of the package will become something like this:
/ (package root)
│
├── pearl-config (optional directory)
├── module/ (contains third-party code)
└── (additional package content)

Then, you just need to modify the config scripts in order to integrate the third-party
project inside Pearl environment.
Point directly to the third-party git repository
Let's suppose you want to install the vim-rails plugin.
In your Pearl configuration ($XDG_CONFIG_HOME/pearl/pearl.conf), add your new Pearl package:
PEARL_PACKAGES = {
"vim-rails": {
"url": "https://github.com/tpope/vim-rails.git",
"description": "Ruby on Rails power tools"
},
}

Install the package:
pearl install vim-rails

Voila', your new vim plugin is ready to be used!
This approach is particularly useful whenever you do not need to specify
any pearl config to "enrich" the third-party project inside
the Pearl environment.
Create your own Pearl repository
A Pearl repository is just a git repository containing a file located in pearl-config/pearl.conf
with a list of packages. For instance, the OPH repository is available
here.
In order to use the new repository (i.e. "https://github.com/myrepo/pearl-repo.git"),
update the pearl.conf file by adding the following line:
PEARL_REPOS += ("https://github.com/myrepo/pearl-repo.git")

Troubleshooting
Corrupted Pearl Home directory

Q: What should I do if I accidentally removed files/packages in $PEARL_HOME?


A: You can recover the structure of the $PEARL_HOME by running:

$> pearl init


The command will create all the essential directories and symlinks in $PEARL_HOME.
It is harmless to run the init command multiple times since it is idempotent.

Corrupted package

Q: Why I can no longer update/remove a package?


A: This is probably because either one of the hook functions
is failing or the package content is corrupted. You can forcely remove the package:

$> pearl remove <packagename>


which bypass hook functions that are failing. If that does not even work,
you can delete a package by simply removing its directory:

$> rm -rf $PEARL_HOME/packages/pearl/<packagename>


After that, you can reinstall the package again.
The Pearl packages contain a dedicated directory var for storing
data needed for the package itself.
The var data are always managed by the package and they never gets deleted by Pearl
during the package removal.
If you want to delete the content in var package:

$> rm -rf $PEARL_HOME/var/pearl/<packagename>

Package shell variables/functions not visible in current shell after installation

Q: Why are not package's environment variables/functions visible in
my current shell after installing/updating the package?


A: After package install/update, the variables or
functions related to the current shell and defined in pearl-config/config.*
may not be available because a reload of Pearl configuration file is required.
You can fix this by simply run the function:

pearl-source


which reloads the configuration.
The use of such function is not always required but depends
whether the variables/functions involve the current shell where the
package install/update occurred (i.e. a new variable defined in config.sh
and the current shell is a bash or zsh). Alternatively, user can always
create a new shell and the package resources will be available as
expected.

Error during package install

Q: Why Do I get the following error:

Error on executing 'post_install' hook. Rolling back...


A: This occurs when the post_install hook function fails.
Pearl will attempt to roll back and force a removal of the package. In this way
you can attempt to install the package again once the hook function gets
fixed.

Contributing
You could help improving Pearl and the OPH in the following ways:

Reporting Bugs
Suggesting Enhancements
Writing Code

Change Log
2.1.1 - 2020-01-13

Replace install.sh with hooks.sh

install.sh will still be valid until next releases


Fail if no command is specified

2.1.0 - 2020-01-12

Add create command
Fix procedure to install Pearl in OSX
Fix ci to upload to PyPI

2.0.2 - 2020-01-11

Add instructions to install Pearl in Arch Linux
Fix bug for version option

2.0.1 - 2020-01-11

Manual intervention to switch to Pearl v2

Migration page


Codebase re-written in Python
Remove the post_remove
pearl.conf is not a python script. This requires manual intervention. Take a look at the pearl.conf.template file in codebase
pearl-metadata directory is finally deprecated
Add --no-confirm option
Add --force option

This option bypasses failures even during the hook function execution


Add --verbose option

-vv allows to enable xtrace in hook functions


Add --update-repos option
Shortcut commands (i.e. i to specify install command) are no longer available
Pearl file locations change drastically in order to be complaint with the
XDG Base Directory Specification

As of now, pearl.conf resides in $XDG_CONFIG_HOME/pearl (default ~/.config/pearl)
The new location for $PEARL_HOME is $XDG_DATA_HOME/pearl (default ~/.local/share/pearl)


Remove the variables $PEARL_ROOT and $PEARL_TEMPORARY

1.8.2 - 2019-10-13

Update README.md file

1.8.1 - 2019-07-13

Update buava:

New view action for setup_configuration helper function



1.8.0 - 2019-06-03

Add roll back mechanism during install package
Add grep and sed as optional dependencies since they may be used in hook functions
Add newer buava:

backup function
delete function
ideavim and gvim dotfiles for (un)link functions
install_or_update_vim_plugin_git_repo function
Add GNU sed and grep for OSX compat functions



1.7.2 - 2019-01-15

Fix variables PEARL_PKGNAME PEARL_PKGREPONAME for vim and emacs boot

1.7.1 - 2018-08-11

Deprecate pathogen
Ensure to cd when updating package pointing to local directory
Fix when package specified with full name does not exist
More log info when Git URL package change
Proceed even if install.sh is syntatically incorrect (prevent block for fixing the broken package)
Improve doc and add section about comparison with Ansible

1.7.0 - 2018-07-05

Add the idempotent emerge command which update/install packages.
No longer support the use of USR1 signal to source the Pearl config to the parent process. To explictly do that run pearl-source command instead.

1.6.3 - 2018-06-22

Location of the repo file is pearl-config/pearl.conf. Backward compatibility will be kept until 2.0.0
Make the branch name inferred from repo's HEAD rather than hardcode the branch with master

1.6.2 - 2018-06-09

Add variables PEARL_PKGNAME PEARL_PKGREPONAME
Fix boot vim for deprecating pearl-metadata
Update doc to use dynamic updates for third-party git repos
Add the buava git repo helpers

1.6.1 - 2018-02-04

Fix import osx-compat.sh
Fix update Pearl submodules during updates

1.6.0 - 2018-02-03

Change directory name to pearl-config. Pearl version 2.0.0 will deprecate pearl-metadata
link_to_path to customize symlink name
Changes in buava for Pearl configs:

osx_detect function to detect the OS platform
Improved choose function with indexes
Add ssh for [un]link function



1.5.6 - 2017-08-31

Fix Integ tests

1.5.5 - 2017-08-31

Update Buava:

Update download function
Add choose, input and contain_elements functions



1.5.4 - 2017-08-28

Update Buava:

Add download function
vimperator gtk2 programs for [un]link functions



1.5.3 - 2017-06-29

Add Pear test utils as new dependency
Add Bunit as new dependency
Add Buava as new dependency
Inform about the trap on USR1 signal

1.5.2 - 2017-01-07

Add support for new OSX image in Travis
Fallback to a default temp directory if tty does not work

1.5.1 - 2016-11-15

Fix git --no-parser log for missing newline

1.5.0 - 2016-11-13

Provide (un)link from/to in utils.sh
Provide list of last commits during add/update package
Fix unlink_from_path when source file is a symlink

1.4.5 - 2016-11-11

Provide (un)link from/to PATH variable in utils.sh

1.4.4 - 2016-09-26

Improving doc and add checkstyle

1.4.3 - 2016-05-25

Remove the requirement of updating the PATH on OSX

1.4.2 - 2016-05-10

Add support for OSX
Add check for existing PEARL_HOME variable for emacs/vim boot scripts
Add PEARL_HOME/bin directory to have symlinks for the Pearl packages executables
Avoid polluting PATH variable by introducing a check first

1.4.1 - 2016-04-30

Introduce $PEARL_PKGVARDIR on boot scripts
Packages do not need to have master as default branch
Change the installation process to avoid pipe bash problem
Ensure to get the most updated post_update function
Refactor unit tests in test-package.sh

1.4.0 - 2016-04-23

Add a dedicated directory $PEARL_PKGVARDIR for the Pearl packages in order to store data
needed during the execution of the package itself
Add warning in case of an old version of git or bash
Introduce $PEARL_PKGDIR environment variable for emacs and vim config files
Change the definition of public API

1.3.1 - 2016-04-21

Fix compatibility with Bash 4.1
Integration tests with fixed Bash and Git versions

1.3.0 - 2016-04-20

Provide the definition of public API
Add the emacs hook
Use a better approach to return values from bash functions
Change location of the boot files for pearl.fish and pearl.sh
Introduce the standard documentation for functions
Use try/catch approach to handle errors

1.2.0 - 2016-04-14

Fix the removal of packages by querying the local directory
Add (un)link functions for utils.sh

1.1.0 - 2016-04-09

Check if Git URL changed during updates
Introduce the template for new Pearl packages
Local directories can be used as Pearl packages
Add VERSION file

1.0.1 - 2016-04-08

Update docs
Add travis and integration tests
Introduce the installer

1.0.0 - 2016-04-03

Initial commit.