rbfopt 4.2.6

Description:

rbfopt 4.2.6

Table of contents
This software is released under the Revised BSD License. By using
this software, you are implicitly accepting the terms of the license.
RBFOpt is a Python library for black-box optimization (also known as
derivative-free optimization). It is developed for Python 3 but
currently runs on Python 2.7 as well. This README contains
installation instructions and a brief overview. More details can be
found in the user manual.
Contents of this directory:

AUTHORS: Authors of the library.
CHANGELOG: Changelog.
LICENSE: Licensing information.
MANIFEST.in: List of additional files to be included in archives.
README.rst: This file.
VERSION: Version of the library.
manual.pdf: User manual.
requirements.txt: List of dependencies for this project.
setup.cfg: Configuration file for setup.py.
setup.py: Setup file.
tox.ini: Configuration file for tox.
bin/

rbfopt_cl_interface.py: Script for the command-line interface,
to run the library on a user-defined black-box function
implemented in a user-specified file.
rbfopt_test_interface.py: Script to test the library on a
global optimization test set.


src/

rbfopt/

rbfopt_black_box.py: Description of an abstract black-box
function.
rbfopt_algorithm.py: Main optimization algorithm, both
serial and parallel.
rbfopt_aux_problems.py: Interface for the auxiliary problems
solved during the optimization process.
rbfopt_degreeX_models.py: PyOmo models for the auxiliary
problems necessary for RBF functions with minimum required
polynomial degree X.
rbfopt_refinement: Routines for refinement phase.
rbfopt_settings.py: Global and algorithmic settings.
rbfopt_test_functions.py: Mathematical test functions.
rbfopt_user_black_box.py: A black-box class constructed from
user data.
rbfopt_utils.py: Utility routines.
doc/

conf.py: Configuration file for Sphinx.
Makefile: Makefile (for Linux/Mac) to build the
documentation.
make.bat: Batch file (for Windows) to build the
documentation.
*.rst: ReStructured Text files for the documentation.


examples/

rbfopt_black_box_example.py: Example of an implementation
of a simple black-box function.






tests/

context.py: Configuration file for nose.
test_functions.py: Global optimization test functions.
test_rbfopt_algorithm.py: Testing module for
rbfopt_algorithm.py (regular unit tests).
slow_test_rbfopt_algorithm.py: Testing module for
rbfopt_algorithm.py (additional, slow tests).
test_rbfopt_aux_problems.py: Testing module for
rbfopt_aux_problems.py.
test_rbfopt_degreeX_models.py: Testing module for
rbfopt_degreeX_models.py.
test_rbfopt_env.py: Environment variables for testing
environment.
test_rbfopt_mwe.py: Test the minimal working example given in the
documentation.
test_rbfopt_refinement: Testing module for rbfopt_refinement.py
test_rbfopt_settings.py: Testing module for rbfopt_settings.py.
test_rbfopt_utils.py: Testing module for rbfopt_utils.py.





Installation requirements
This package requires the following software:

Python version >= 3.7
NumPy version >= 1.11.0
SciPy version >= 0.17.0
Pyomo version >= 5.1.1

The software has been tested with the versions indicated above. It may
work with earlier version and should work with subsequent version, if
they are backward compatible. In particular, the software is known to
work with Pyomo version 4 and earlier versions of Scipy.
The code is developed for Python 3.7, but it currently also runs on
Python 2.7. Since Python 2.7 has reached end-of-life in January 2020,
we recommend using Python 3.7 or higher.
The easiest, and recommended, way to install the package is via the
Python module manager pip. The code is on PyPI, therefore it can be
installed from PyPI using:
pip install rbfopt
You can install from source, downloading an archive or cloning from
git (for example if you want to use a development version that is not
released on PyPI yet), using the command:
pip install .
You may need the -e switch to install in a virtual environment. To
build the documentation, you also need numpydoc:
pip install numpydoc
On Windows systems, we recommend WinPython, which comes with NumPy, SciPy
and pip already installed. After installing WinPython, it is typically
necessary to update the PATH environment variable. The above command
using pip to install missing libraries has been successfully tested on
a fresh WinPython installation.
RBFOpt requires the solution of convex and nonconvex nonlinear
programs (NLPs), as well as nonconvex mixed-integer nonlinear programs
(MINLPs) if some of the decision variables (design parameters) are
constrained to be integer. Solution of these subproblems is performed
through Pyomo, which in principle supports any solver with an AMPL
interface (.nl file format). The code is setup to employ Bonmin and
Ipopt, that are open-source, with a permissive license, and available
through the COIN-OR repository. The end-users are responsible for
checking that they have the right to use these solvers. To use
different solvers, a few lines of the source code have to be modified:
ask for help on GitHub or on the mailing list, see below.
To obtain pre-compiled binaries for Bonmin and Ipopt for several
platforms, we suggest having a look at the AMPL opensource solvers (also here) for static binaries. Note: These
binaries might be outdated: better performance can sometimes be
obtained compiling Bonmin from scratch (Bonmin contains Ipopt as
well), especially if compiling with a different solver for linear
systems rather than the default Mumps, e.g., ma27. Bonmin and Ipopt
must be compiled with ASL support.
In case any of the packages indicated above is missing, some features
may be disabled, not function properly, or the software may not run at
all.


Installation instructions and getting started

Install the package with pip as indicated above. This will install
the two executable Python scripts rbfopt_cl_interface.py and
rbfopt_test_interface.py in your bin/ directory (whatever is used
by pip for this purpose), as well as the module files in your
site-packages directory.
Make sure Bonmin and Ipopt are in your path; otherwise, use the
options minlp_solver_path and nlp_solver_path in RbfoptSettings to
indicate the full path to the solvers. If you use RBFOpt as a
library and create your own RbfoptSettings object, these options
can be given as:
import rbfopt
settings = rbfopt.RbfoptSettings(minlp_solver_path='full/path/to/bonmin', nlp_solver_path='full/path/to/ipopt')
If you use the command-line tools, you can simply provide the option preceded by double hyphen, as in:
rbfopt_test_interface.py --minlp_solver_path='full/path/to/bonmin' branin

Enjoy!
You can test the installation by running:
rbfopt_test_interface.py branin
See:
rbfopt_test_interface.py --help
for more details on command-line options for the testing tool.
Many more test functions, with different characteristics, are
implemented in the file rbfopt_test_functions.py. They can all be
used for testing.

Unit tests for the library can be executed by running:
nose2
or:
python setup.py test
from the current (main) directory. If some of the tests fail, the
library may or may not work correctly. Some of the test failures
are relatively harmless. You are advised to contact the mailing
list (see below) if you are unsure about some test failure.
Additional slow tests, that check if various parametrizations of
the optimization algorithm can solve some global optimization
problems, are found in the file slow_test_rbfopt_algorithm.py,
which is ignored by nose by default. To execute these tests, run:
nose2 tests.slow_test_rbfopt_algorithm




Minimal working example
After installation, the easiest way to optimize a function is to use
the RbfoptUserBlackBox class to define a black-box, and execute
RbfoptAlgorithm on it. This is a minimal example to optimize the
3-dimensional function defined below:
import rbfopt
import numpy as np
def obj_funct(x):
return x[0]*x[1] - x[2]

bb = rbfopt.RbfoptUserBlackBox(3, np.array([0] * 3), np.array([10] * 3),
np.array(['R', 'I', 'R']), obj_funct)
settings = rbfopt.RbfoptSettings(max_evaluations=50)
alg = rbfopt.RbfoptAlgorithm(settings, bb)
val, x, itercount, evalcount, fast_evalcount = alg.optimize()
Another possibility is to define your own class derived from
RbfoptBlackBox in a separate file, and execute the command-line
interface on the file. An example is provided under
src/rbfopt/examples, in the file rbfopt_black_box_example.py. This can
be executed with:
rbfopt_cl_interface.py src/rbfopt/examples/rbfopt_black_box_example.py


Parallel optimization
RBFOpt supports asynchronous parallel optimization using Python’s
multiprocessing library. This mode is enabled whenever the parameter
num_cpus is set to a value greater than 1. Black-box function
evaluations as well as some of the heaviest computatations carried out
by the algorithm will then be executed in parallel. Since the parallel
computations are asynchronous, determinism cannot be guaranteed: in
other words, if you execute the parallel optimizer twice in a row, you
may (and often will) get different results, even if you provide the
same random seed. This is because the order in which the computations
will be completed may change, and this may impact the course of the
algorithm.
The default parameters of the algorithm are optimized for the serial
optimization mode. For recommendations on what parameters to use with
the parallel optimizer, feel free to ask on the mailing list.
Note that the parallel optimizer is oblivious of the system-wide
settings for executing linear algebra routines (BLAS) in parallel. We
recommend setting the number of threads for BLAS to 1 when using the
parallel optimizer, see the next section.


Known issues with OpenBLAS
We are aware of an issue when launching multiple distinct processes
that use RBFOpt and the NumPy implementation is configured to use
OpenBLAS in parallel: in this case, on rare occasions we have observed
that some processes may get stuck forever when computing matrix-vector
multiplications. The problem can be fixed by setting the number of
threads for OpenBLAS to 1. We do not know if the same issue occurs
with other parallel implementations of BLAS.
For this reason, and because parallel BLAS uses resources suboptimally
when used in conjunction with the parallel optimizer of RBFOpt (if
BLAS runs in parallel, each thread of the parallel optimizer would
spawn multiple threads to run BLAS, therefore disregarding the option
num_cpus), RBFOpt attempts to set the number of BLAS threads to 1 at
run time.
All scripts (rbfopt_cl_interface.py and rbfopt_test_interface.py) set
the environment variables OMP_NUM_THREADS to 1. Furthermore, the
rbfopt module does the same when imported for the first time.
Note that these settings are only effective if the environment
variable is set before NumPy is imported; otherwise, they are
ignored. If you are facing the same issue, we recommend setting
environment variable OMP_NUM_THREADS to 1. In Python, this can be done
with:
import os
os.environ['OMP_NUM_THREADS'] = '1'


Documentation
The documentation for the code can be built using Sphinx with the
numpydoc extension. numpydoc can be installed with pip:
pip install numpydoc
After that, the directory src/rbfopt/doc/ contains a Makefile (on
Windows, use make.bat) and the Sphinx configuration file conf.py.
You can build the HTML documentation (recommended) with:
make html
The output will be located in _build/html/ and the index can be found
in _build/html/index.html.
A PDF version of the documentation (much less readable than the HTML
version) can be built using the command:
make latexpdf
An online version of the documentation for the latest master branch of
the code, and for the latest stable release, are available on
ReadTheDocs for the latest and stable version.


Citing RBFOpt
If you use RBFOpt in one of your projects or papers, please cite the
following papers (this is the only way in which the authors get
credit):

A. Costa and G. Nannicini. RBFOpt: an open-source library for
black-box optimization with costly function
evaluations. Mathematical Programming Computation,
10(4):597–629, 2018. (The paper can be downloaded as: Optimization
Online paper 4538)
G. Nannicini. On the implementation of a global optimization method
for mixed-variable problems. Open Journal of Mathematical
Optimization, 2(1), 2021. (Download link: OJMO)



RBFOpt for hyperparameter optimization
RBFOpt is used in IBM Watson Studio AutoAI. For a discussion
on the application of RBFOpt to hyperparameter optimization in machine
learning, besides the aforementioned paper published in OJMO, see
the paper:

G. I. Diaz, A. Fokoue-Nkoutche, G. Nannicini and H. Samulowitz. An
effective algorithm for hyperparameter optimization of neural
networks. IBM Journal of Research and Development 61, no. 4/5
(2017): 9-1. (Download link: IBM Journal of R&D)



Support
If you believe there is a bug or an issue, please open an issue on
GitHub. If you have a general question, please use GitHub’s
“Discussions” feature (the tab can be opened at the top of the page).