Description:
evalica 0.2.0
Evalica, your favourite evaluation toolkit
Evalica is a Python library that transforms pairwise comparisons into ranked lists of items. It offers convenient high-performant Rust implementations of the corresponding methods via PyO3, and additionally provides naïve Python code for most of them. Evalica is fully compatible with NumPy arrays and pandas data frames.
Tutorial (and Tutorial.ipynb)
Chatbot-Arena.ipynb
Pair2Rank
The logo was created using Recraft.
Installation
pip: pip install evalica
Anaconda: conda install conda-forge::evalica
Usage
Imagine that we would like to rank the different meals and have the following dataset of three comparisons produced by food experts.
Item X
Item Y
Winner
pizza
burger
x
burger
sushi
y
pizza
sushi
tie
Given this hypothetical example, Evalica takes these three columns and computes the outcome of the given pairwise comparison according to the chosen model. Note that the first argument is the column Item X, the second argument is the column Item Y, and the third argument corresponds to the column Winner.
>>> from evalica import elo, Winner
>>> result = elo(
... ['pizza', 'burger', 'pizza'],
... ['burger', 'sushi', 'sushi'],
... [Winner.X, Winner.Y, Winner.Draw],
... )
>>> result.scores
pizza 1014.972058
burger 970.647200
sushi 1014.380742
Name: elo, dtype: float64
As a result, we obtain Elo scores of our items. In this example, pizza was the most favoured item, sushi was the runner-up, and burger was the least preferred item.
Item
Score
pizza
1014.97
burger
970.65
sushi
1014.38
Command-Line Interface
Evalica also provides a simple command-line interface, allowing the use of these methods in shell scripts and for prototyping.
$ evalica -i food.csv bradley-terry
item,score,rank
Tacos,0.43428827947351706,1
Sushi,0.19060105855071743,2
Burger,0.14797720376982199,3
Pasta,0.12815347866987045,4
Pizza,0.0989799795360731,5
Refer to the food.csv file as an input example.
Implemented Methods
Method
In Python
In Rust
Counting
✅
✅
Average Win Rate
✅
✅
Bradley–Terry
✅
✅
Elo
✅
✅
Eigenvalue
✅
✅
PageRank
✅
✅
Newman
✅
✅
Copyright
Copyright (c) 2024 Dmitry Ustalov. See LICENSE for details.
License
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
Customer Reviews
There are no reviews.
