basalt 0.2.9

Description:

basalt 0.2.9

Basalt - Graph Storage API for C++ and Python
Basalt is a graph storage API powered by RocksDB persistent
key-value store for fast storage like NVMe technologies.


Documentation is currently hosted on GitHub: [https://bluebrain.github.io/basalt]
Development stage
The C++ and Python APIs of Basalt are stable already, but substantial additions might come in the future. Thus this library development status is still beta.
Usage
Python
Graph Topology API
class PLInfluences(basalt.GraphTopology):
"""A directed graph where vertices are programming languages.
"""
directed(True)

class Vertex(Enum):
LANGUAGE = 1

# Declare a vertex type
vertex("language", Vertex.LANGUAGE)
# Declare a directed edge between 2 programming languages
# to represent how they relate.
edge(Vertex.LANGUAGE, Vertex.LANGUAGE, name="influenced", plural="influenced")

@classmethod
def load_from_dbpedia(cls):
# [...]

g = PLInfluences.load_from_dbpedia("/path/on/disk")
# Iterate over all vertices of type "languages"
for language in g.languages:
print(language.id, language.data())
# Iterate over all vertices connected to vertex `language`
# through the `influenced` edge type.
for influenced in language.influenced:
print(" ", influenced.data())

Low-level Python bindings
# Load or initialize a graph on disk
g = basalt.UndirectedGraph("/path/on/disk")
# Add one vertex of type 0 and identifier 1
g.vertices.add((0, 1))
# Insert 10 vertices at once
# (10, 0), (10, 1), ... (10, 10)
g.vertices.add(numpy.full((10,), 1, dtype=numpy.int32), # types
numpy.arange(10, dtype=numpy.int64)) # ids
# Connect 2 vertices
g.edges.add((0, 1), (1, 0))
# Connect vertex (0, 1) to several vertices at once
# (0,1)->(1,0), (0,1)->(1,1), ... (0,1)->(1,9)
g.edges.add((0, 1),
numpy.full((9,), 1, dtype=numpy.int32),
numpy.arange(9, dtype=numpy.int64)
# Commit changes on disk
g.commit()

C++ API
// Load or initialize a graph on disk
basalt::UndirectedGraph g("/path/on/disk");
// Add one vertex of type 0 and identifier 1
g.vertices().insert({0, 1});
// Add one vertex of type 0 and identifier 2
g.vertices().insert({0, 2});
// Iterate over vertices
for (const auto& vertex: g.vertices()) {
std::clog << vertex << '\n';
}
// Connect both vertices
g.edges().insert({0, 1}, {0, 2}));
for (const auto& edge: g.edges()) {
std::clog << edge.first << " -> " << edge.second << '\n';
}
// Commit changes on disk
g.commit();

Installation
C++ API
Conan package
This repository provides a Conan package to ease integration into your existing projects.
CMake
It is also possible to build and install the library using CMake, see build section below.
Python API
Pypi
Python bindings of Basalt are available on Pypi.
Blue Brain 5 supercomputer
Basalt is currently released as module on Blue Brain 5 supercomputer:
$ module purge
$ . /gpfs/bbp.cscs.ch/apps/hpc/jenkins/config/modules.sh
$ module load py-basalt
$ python3
Python 3.6.3 (default, Oct 3 2017, 07:47:49)
[GCC 6.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import basalt
>>> basalt.__version__
'0.2.2'
>>> basalt.__rocksdb_version__
'5.17.2'

Manual build and installation instructions
Requirements

CMake build system, version 3.5.1 or higher.
RocksDB, a persistent key-value store,
version 4.1.1 or higher.
Python 3, version 3.5 or higher.

Getting the code
This repository grabs a few third-party libraries as git modules.
To clone them when you clone basalt, use git clone --recursive option.
If you have already cloned basalt, you can get the git submodules with
the command:
git submodule update --recursive --init
Building the library 101
C++ Library only
To build the basalt C++ shared library and run the tests:
cd /path/to/basalt
mkdir build
pushd build
cmake ..
CTEST_OUTPUT_ON_FAILURE=1 make all test

To install the library:
pushd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local .
make all install

Python 3 bindings
To build and run the tests:
cd /path/to/basalt
python3 setup.py test

To install the package:

with pip: pip3 install -U .
with distutils: python3 setup.py install
to create binary tarballs:

most simple: python3 setup.py bdist
wheel: pip3 install wheel; python3 setup.py bdist_wheel
relocatable archive: python3 setup.py bdist_dumb --relative



CMake variables and targets
Main CMake variables:

Basalt_FORMATTING:BOOL: provide the build target clang-format to check C++ code formatting
Basalt_STATIC_ANALYSIS:BOOL: provide the build target clang-tidy to perform static analysis of the C++ code
Basalt_ARCH: value given to the -m compiler option. "native" for instance
Basalt_PRECOMMIT:BOOL: Enable automatic checks before git commits
Basalt_CXX_OPTIMIZE:BOOL: Compile C++ with optimization
Basalt_CXX_SYMBOLS:BOOL: Compile C++ with debug symbols
Basalt_CXX_WARNINGS:BOOL=ON: Compile C++ with warnings

For a more detailed list, please refer to file CMakeCache.txt in CMake build directory.
CMake targets:

basalt: build the pure C++ library (without Python bindings)
_basalt: build the C++ library with Python bindings
unit-tests: build a C++ executable testing the C++ pure library
all: build the 3 targets above
test: execute the tests. It is recommended to execute the command ctest --output-on-failure -VV instead
install: install the pure C++ library and the CMake configuration required to easily use basalt
in another CMake project

Python setuptools commands
Here are the main Python setuptools commands available.

build: build native library
test: build and test the package. It also executes the C++ unit-tests as well as the code snippets in the Sphinx documentation.
install: install the Python package
doctest: execute the code snippets in the Sphinx documentation
build_sphinx: build the Sphinx documentation

For instance: python3 setup.py build_sphinx
Files Layout
├── basalt ................... python code of the package
├── cmake
│   └── hpc-coding-conventions git module for C++ code guidelines
├── dev ...................... development related scripts
├── doc ...................... sphinx documentation source code
├── include
│   └── basalt ............... public headers of the C++ library
├── README.md ................ that's me!
├── src
│   ├── basalt ............... C++ library implementation
│   └── third_party .......... C++ libraries (mostly as git modules)
└── tests
├── benchmarks ........... scripts to execute before creating a git tag
├── py ................... python unit-tests
└── unit ................. C++ unit-tests using Catch2

Embedded third-parties
External libraries are including either by copy/paste or git submodules
in src/third_party directory.

Catch2:
modern, C++-native, header-only, test framework for unit-tests, TDD
and BDD unit-test library.
fmt: A modern formatting library
(not part of CMake build yet)
pybind11: Seamless operability between C++11 and Python
SpdLog: Fast C++ logging library.

Contributing
If you want to improve the project or you see any issue, every contribution is welcome.
Please check contribution guidelines for more information.