Description:

cfrainbow 0.1.1

CFRainbow
Implementations Of Counterfactual Regret Minimization In Its Many Shapes & Colors


CFRainbow provides implementations of the basic CFR algorithm and some of the improved variants of CFR
for computing Nash Equilibria in the 2-player 0-sum case as well as Correlated Equilibria in the general-sum case.
The pacakge is loosely built for modularity and general applicability by building on the Openspiel framework by Deepmind. CFRainbow is not built
for performance and will not scale well to large game implementations. Most algorithms are implemented with basic python data structures.
The package is in the early WIP phase.
Usage
The easiest way to use CFRainbow is to import the desired algorithm from the package, a regret minimizer, and the run method with a pyspiel game object or its OpenSpiel name as input.
For example, to use the Vanilla CFR algorithm:
import cfrainbow
from cfrainbow import cfr, rm

cfrainbow.run(
cfr.VanillaCFR,
n_iter=1000,
game="kuhn_poker",
regret_minimizer=rm.RegretMatcher
)

This will run the algorithm for the number of iterations given and compute the exploitability.
You can also use the solver object directly and call iterate on it after instantiating it with the correct arguments.
For example:
from cfrainbow import cfr, rm
from cfrainbow.utils import load_game, KuhnPolicyPrinter, normalize_policy_profile

root_state = load_game("kuhn_poker").new_initial_states()

solver = cfr.VanillaCFR(
root_state,
regret_minimizer_type=rm.RegretMatcher,
alternating=True, # whether to do alternating or simultaneous updates
)
# iterate for a given number of iterations.
for i in range(1000):
solver.iterate()

avg_policy = normalize_policy_profile(solver.average_policy())

print(
KuhnPolicyPrinter(digits=2).print_profile(avg_policy)
)

Output (Infostate --> Action Policy):
PLAYER 0:
P1: Jack | P2: ? | cb --> ['check: 1.00', 'bet: 0.00']
P1: Jack | P2: ? --> ['check: 0.81', 'bet: 0.19']
P1: Queen | P2: ? | cb --> ['check: 0.48', 'bet: 0.52']
P1: Queen | P2: ? --> ['check: 0.99', 'bet: 0.01']
P1: King | P2: ? | cb --> ['check: 0.00', 'bet: 1.00']
P1: King | P2: ? --> ['check: 0.42', 'bet: 0.58']
PLAYER 1:
P1: ? | P2: Queen | c --> ['check: 0.99', 'bet: 0.01']
P1: ? | P2: Queen | b --> ['check: 0.66', 'bet: 0.34']
P1: ? | P2: King | c --> ['check: 0.00', 'bet: 1.00']
P1: ? | P2: King | b --> ['check: 0.00', 'bet: 1.00']
P1: ? | P2: Jack | c --> ['check: 0.67', 'bet: 0.33']
P1: ? | P2: Jack | b --> ['check: 1.00', 'bet: 0.00']

Note that in alternating updates each iteration is a single player's policy update.
In simultaneous updates both players' policy updates constitute one iteration.
Available Algorithms
The following list shows the available algorithms that have been implemented (✔️), those that are still work in progress (🔨👷‍♂️),
and those that are planned to be implemented in the future (📅):



Algorithm
Status
Convergence Tests
Paper Results Reproducing




Best-Response CFR
✔️
✔️
🔨👷‍


Discounted CFR
✔️
✔️
🔨👷‍


Exponential CFR
✔️
✔️
❌


Internal CFR
🔨👷‍♂️
🔨👷‍
🔨👷‍


Joint-Reconstruction CFR
🔨👷‍♂️
🔨👷‍
🔨👷‍


Chance Sampling Monte Carlo CFR
✔️
✔️
🔨👷‍


External Sampling Monte Carlo CFR
✔️
✔️
🔨👷‍


Outcome Sampling Monte Carlo CFR
✔️
✔️
🔨👷‍


Predictive Plus CFR
✔️
✔️
🔨👷‍


Pure CFR
✔️
✔️
🔨👷‍


Sampling CFR
✔️
🔨👷‍
🔨👷‍


Vanilla CFR
✔️
✔️
🔨👷‍


Lazy CFR
📅
📅
📅



Installation
Web Install
Pip
To install CFRainbow from PyPi using pip, run the following command:
pip install cfrainbow

Poetry
If you're using Poetry to manage your Python packages, you can add CFRainbow to your project by adding the following to your pyproject.toml file:
[tool.poetry.dependencies]
cfrainbow = "*"

Then run:
poetry install

to install your own package. Poetry will then install cfrainbow as dependency alongside your own package.
Source Install
To install CFRainbow from master, please follow these steps:
git clone https://github.com/maichmueller/cfrainbow.git
cd cfrainbow

and install the package with pip
pip install .

or poetry
poetry install

use the option --no-dev to ensure a non-editable installation.

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter