bio-pype 1.0.7

Description:

biopype 1.0.7

Simple and slim python framework to build, organize and standardize
bioinformatics analyses.

The system is built around the python
argparse
module, to provide command line interface to run the configured
analyses. The framework is heavily customizable and provides basic
method to access to Environment
Modules and implements various
queuing systems such as
moab/torque.


Install
From Pypi
pip install bio_pype
The development version from git
git clone https://bitbucket.org/ffavero/bio_pype
cd bio_pype
python setup.py test
python setup.py install
Installation with virtualenv in the `computerome
HPC <https://computerome.dtu.dk>`__ system:
(Or in any system where virtualenv is not installed at system level)
mkdir ~/venv_files

# Download virtualenv files (not installed in the python version used in computerome)
curl -L -o ~/venv_files/setuptools-20.2.2-py2.py3-none-any.whl https://github.com/pypa/virtualenv/raw/develop/virtualenv_support/setuptools-20.2.2-py2.py3-none-any.whl
curl -L -o ~/venv_files/pip-8.1.0-py2.py3-none-any.whl https://github.com/pypa/virtualenv/raw/develop/virtualenv_support/pip-8.1.0-py2.py3-none-any.whl
curl -L -o ~/venv_files/virtualenv.py https://github.com/pypa/virtualenv/raw/develop/virtualenv.py

# Create a virtual environment
python ~/venv_files/virtualenv.py --extra-search-dir=~/venv_files ~/venv_bio_pype
# Activate the environment
source ~/venv_bio_pype/bin/activate

# Install bio_pype as instructed above

# The pype command line utility will be available upon the "activation" of the environment or by specifying the full path, in this case:

# ~/venv_bio_pype/bin/pype

# To deactivate the virtual env:
deactivate


Basic usage
Access first level modules:
$ pype
usage: pype [-p PROFILE] {pipelines,profiles,repos,snippets} ...

Slim and simple framework to ease the managements of data, tools and pipelines in the computerome HPC

positional arguments:
pipelines Workflows built by assembling pipelines and snippets
profiles Set of databases softwares and meta information
repos Management of pype modules
snippets Single tasks implementations

optional arguments:
-p PROFILE, --profile PROFILE
Select the profile. This will define things like
databases, reference genomes paths,specific version of
programs to loads and other similar configurations.
Default: default

This is version 0.9.0 - Francesco Favero - 15 May 2017


Install Repos:
By default the repository of modules is empty.
To install a desired repository, list the available repository:
$ pype repos list -a
Install a selected repository:
$ pype repos install -f sequenza


More Details
It is possible to pass local modules for snippets and profiles via
environment variables, An example with the package test modules:
$ PYPE_SNIPPETS=test/data/snippets pype snippets
error: too few arguments
usage: pype snippets {reverse_fa,complement_fa,lower_fa,test} ...

positional arguments:
{reverse_fa,complement_fa,lower_fa,test}
reverse_fa reverse a fasta sequence
complement_fa lower case a fasta sequence
lower_fa lower case a fasta sequence
test test_snippets
Snippets
The snippets are python modules that perform a given task:
$ PYPE_SNIPPETS=test/data/snippets pype snippets test
error: argument --test is required
usage: pype snippets test --test TEST [-o OPT]

optional arguments:
--test TEST Test metavar
-o OPT test option
$ PYPE_SNIPPETS=test/data/snippets pype snippets test --test World
Hello World
$ PYPE_SNIPPETS=test/data/snippets pype snippets test --test mate -o Cheers
Cheers mate
Pipeline:
The pipelines are yaml files tbat group any snippets or other pipelines,
with the desired dependency order.
$ PYPE_SNIPPETS=test/data/snippets PYPE_PIPELINES=test/data/pipelines pype pipelines
error: too few arguments
usage: pype pipelines [--queue {echo,none}] {rev_compl_low_fa} ...

positional arguments:
{rev_compl_low_fa}
rev_compl_low_fa Reverse Complement Lower case a fasta

optional arguments:
--queue {msub,echo,none}
Select the queuing system to run the pipeline
--log LOG Path used to write the pipeline logs. Default working
directory.
The arguments for a pipeline are parsed on the fly from a YAML file,
which defines the tools and the flow of the execution:
info:
description: Reverse Complement Lower case a fasta
date: 06/07/2016
items:
- name: lower_fa
type: snippet
arguments:
-i: '%(complement_fa)s'
-o: '%(output)s'
dependencies:
items:
- name: complement_fa
type: snippet
arguments:
-i: '%(reverse_fa)s'
-o: '%(complement_fa)s'
dependencies:
items:
- name: reverse_fa
type: snippet
arguments:
-i: '%(input_fa)s'
-o: '%(reverse_fa)s'
resulting argparse interface:
$ PYPE_SNIPPETS=test/data/snippets PYPE_PIPELINES=test/data/pipelines pype pipelines rev_compl_low_fa
error: argument --complement_fa is required
usage: pype pipelines rev_compl_low_fa --complement_fa COMPLEMENT_FA --output
OUTPUT --reverse_fa REVERSE_FA
--input_fa INPUT_FA

optional arguments:
--complement_fa COMPLEMENT_FA
complement_fa type: str
--output OUTPUT output type: str
--reverse_fa REVERSE_FA
reverse_fa type: str
--input_fa INPUT_FA input_fa type: str
More details on read the docs
(slowly documenting the various features/changes).