GitLocker: The Coding Marketplace

Description:

gbrl 1.0.3

Gradient Boosting Reinforcement Learning (GBRL)
GBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as XGBoost, CatBoost, but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries.

Overview
GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:

GBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
Key Features:

GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.
Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.
Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 GBRL_SB3.

Performance
The following results, obtained using the GBRL_SB3 repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:

Getting started
Prerequisites

Python 3.9 or higher
LLVM and OpenMP (macOS).

Installation
To install GBRL via pip, use the following command:
pip install gbrl

For further installation details and dependencies see the documentation.
Usage Example
For a detailed usage example, see tutorial.ipynb
Current Supported Features
Tree Fitting

Greedy (Depth-wise) tree building - (CPU/GPU)
Oblivious (Symmetric) tree building - (CPU/GPU)
L2 split score - (CPU/GPU)
Cosine split score - (CPU/GPU)
Uniform based candidate generation - (CPU/GPU)
Quantile based candidate generation - (CPU/GPU)
Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)

MultiRMSE loss (only)

Categorical inputs
Input feature weights - (CPU/GPU)

GBT Inference

SGD optimizer - (CPU/GPU)
ADAM optimizer - (CPU only)
Control Variates (gradient variance reduction technique) - (CPU only)
Shared Tree for policy and value function - (CPU/GPU)
Linear and constant learning rate scheduler - (CPU/GPU only constant)
Support for up to two different optimizers (e.g, policy/value) - **(CPU/GPU if both are SGD)
SHAP value calculation

Documentation
For comprehensive documentation, visit the GBRL documentation.
Citation
@article{gbrl,
title={Gradient Boosting Reinforcement Learning},
author={Benjamin Fuhrer, Chen Tessler, Gal Dalal},
year={2024},
eprint={2407.08250},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2407.08250},
}

Licenses
Copyright © 2024, NVIDIA Corporation. All rights reserved.
This work is made available under the NVIDIA Source Code License-NC. Click here. to view a copy of this license.