psyrun 0.8.0

Description:

psyrun 0.8.0

Psyrun


Psyrun is a Python tool to define parameter spaces
and execute an evaluation function for each parameter assignment. In addition
Psyrun makes it easy to use serial farming, i.e. evaluating multiple parameter
assignments in parallel, on a multicore computers and high-performance clusters.

Documentation
The documentation can be found here.


Overview
Define parameter spaces and evaluate them:
from psyrun import map_pspace, Param

def objective(a, b, c):
return a * b + c

pspace = (Param(a=np.arange(1, 5))
* Param(b=np.linspace(0, 1, 10))
* Param(c=[1., 1.5, 10., 10.5]))
results = map_pspace(objective, pspace)
Or do it in parallel:
from psyrun import map_pspace_parallel
results = map_pspace_parallel(objective, pspace)
Define tasks by placing task_<name>.py files in the psy-tasks` directory:
from psyrun import Param

pspace = (Param(a=np.arange(1, 5))
* Param(b=np.linspace(0, 1, 10))
* Param(c=[1., 1.5, 10., 10.5]))

def execute(a, b, c):
return {'result': a * b + c}
and run them by typing psy run with support for serial farming on high
performance clusters.


Installation
pip install psyrun
To be able to use the NPZ store:
pip install numpy
pip install 'psyrun[npz]'
To be able to use the HDF5 store:
pip install numpy
pip install 'psyrun[h5]'


Requirements

Python >=2.7, >=3.4
six


Optional requirements
To have faulthandler activated for
jobs submitted with psy run in Python 2.7:

faulthandler

Python 3.4+ already includes the faulthandler module.
To use map_pspace_parallel:

joblib

To use NPZ files as store:

NumPy

To use HDF5 files as store:

NumPy
pytables

To run the unit tests:

joblib
NumPy
pytables
pytest >= 2.8

To build the documentation:

numpydoc





Changes

0.8.0 (June 26, 2018)
New features

Added the possibility to specify a setup function in task files that
will be executed once at the start of each worker process and can be used
to acquire resources and pass them to the processing function for individual
parameter sets.



0.7.1 (April 19, 2018)

Documentation improvements

Added documentation for the pool_size task attribute and included it in
the task template.



Bug fixes

Allow to continue tasks if results are missing even if the result file is
newer than the task file.




0.7.0 (February 18, 2018)
New features

Added support for the Slurm Workload Manager.



0.6.0
New features

Add psy new-task and psy kill commands.
Added AutodetectStore that determines the appropriate store from the
filename extension.
Added possibility to let psy merge custom stores if provided as
psyrun.stores entry point.
Added capability to set scheduler arguments based on the job name.



0.5.4
Bug fixes

Fix the psy run continue functionality.



0.5.3
Bug fixes

Fix psy status and
psyrun.backend.distribute.DistributeBackend.get_missing trying to read
incompatible data files in the output directory.
Fix psy status and
psyrun.backend.distribute.DistributeBackend.get_missing easily hitting
Python’s recursion depth limit.
Fix merging of npz files with missing integer values by converting them to
float where np.nan can be used.



0.5.2
Bug fixes

Fix incorrect psy status.
Fix psy run <task1> <task2> ... not running all tasks and run them in
order.



0.5.1

Bug fixes

Fix psy merge always assuming PickleStore.



Documentation improvements

Add recipe for converting data to Pandas data frame to documentation.




0.5

Initial release