Description:

pgbinny 0.0.3

pg_binny

Discretize a whole dataframe into ≤N bins, using Top N categories.

%nbdev_hide

The discretize function handles discrete & continuous columns:

Continuous columns are cut into N bins using supplied cutting function (defaults to qcut for quantile cuts.
Categorical columns: take the Top N-1, with the rest tossed into "Other"

TODO: Describe and show the plot helpers too.
Install
conda install pg_binny
-or-
pip install pg_binny
-or (locally)-
pip install -e . (That's "pip install -e dot")
How to use
Make a sample dataframe.
import pandas as pd
import pg_binny as binny


dataset = 'car_crashes'
try:
import seaborn as sns
df = sns.load_dataset(dataset)
except ModuleNotFoundError:
df = pd.read_csv(f'https://raw.githubusercontent.com/mwaskom/seaborn-data/master/{dataset}.csv')
df.sample(5)


<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}

.dataframe thead th {
text-align: right;
}

</style>




total
speeding
alcohol
not_distracted
no_previous
ins_premium
ins_losses
abbrev




19
15.1
5.738
4.530
13.137
12.684
661.88
96.57
ME


15
15.7
2.669
3.925
15.229
13.659
649.06
114.47
IA


35
14.1
3.948
4.794
13.959
11.562
697.73
133.52
OH


50
17.4
7.308
5.568
14.094
15.660
791.14
122.04
WY


43
19.4
7.760
7.372
17.654
16.878
1004.75
156.83
TX




Discretize with default bins
dfd = binny.discretize(df)
dfd.sample(5)

---------------------------------------------------------------------------

AttributeError Traceback (most recent call last)

<ipython-input-2-41b3e27056d4> in <module>
----> 1 dfd = binny.discretize(df)
2 dfd.sample(5)
3


AttributeError: module 'pg_binny' has no attribute 'discretize'

dfd['speeding'].dtype

CategoricalDtype(categories=[(1.7910000000000001, 2.413], (2.413, 3.496], (3.496, 3.948], (3.948, 4.095], (4.095, 4.608], (4.608, 5.032], (5.032, 6.014], (6.014, 6.923], (6.923, 7.76], (7.76, 9.45]],
, ordered=True)

dfd['total'].dtype

CategoricalDtype(categories=[(5.899, 11.1], (11.1, 12.3], (12.3, 13.6], (13.6, 14.5], (14.5, 15.6], (15.6, 17.4], (17.4, 18.1], (18.1, 19.4], (19.4, 21.4], (21.4, 23.9]],
, ordered=True)

You can set the #bins and the cutting function (defaults to quantile cut, but you may prefer plain-old cut, or something else.
?binny.discretize

Signature:
binny.discretize(
 df,
 nbins=10,
 cut=<function qcut at 0x7fae29d843b0>,
 verbose=2,
 drop_useless=True,
)
Docstring:
Discretize columns in {df} to have at most {nbins} categories.
* Categorical columns: take the Top n-1 plus "Other"
* Continuous columns: cut into {nbins} using {cut}.

Returns a new discretized dataframe with the same column names.
Promotes discrete columns to categories.

Parameters
-----------
df: Dataframe to discretize
nbins: Max number of bins to use. May return fewer.
cut: Cutting method. Default `pd.qcut`. Consider pd.cut, or write your own.
verbose: 0: silent, 1: colnames, 2: (Default) top N for each column
drop_useless: Removes columns that have < 2 unique values.

Replaces numerical NA values with 'NA'.
File: /Volumes/Peregrine/binny/pg_binny/core.py
Type: function

Other functions
[x for x in dir(binny) if x[:2] not in ['__', 'pa', 'pd', 'rc']]

['autolabel',
'clean_category',
'discretize',
'drop_singletons',
'is_numeric',
'isnum']

?binny.autolabel

Signature: binny.autolabel(ax, border=False) -> None
Docstring:
Label bars in a barplot {ax} with their height.
Thanks to matplotlib, composition.ai, and jsoma/chart.py.

TODO: how to label with their legend labels?
File: /Volumes/Peregrine/binny/pg_binny/core.py
Type: function

?binny.clean_category

Signature: binny.clean_category(df, col: str) -> None
Docstring:
Remove unused categories from df.col, inplace.
If not a category, do nothing.
File: /Volumes/Peregrine/binny/pg_binny/core.py
Type: function

?binny.is_numeric

Signature: binny.is_numeric(col: str)
Docstring:
Returns True iff already numeric, or can be coerced.
Usage: df.apply(is_numeric)
Usage: is_numeric(df['colname'])

Returns Boolean series.

From:
https://stackoverflow.com/questions/54426845/how-to-check-if-a-pandas-dataframe-contains-only-numeric-column-wise
File: /Volumes/Peregrine/binny/pg_binny/core.py
Type: function

History
pg_binny is an example extracting some frequently copy/pasted routines into a general purpose nbdev project.
Originally called binny because it bins things, that was already taken on PyPi (for... a project that bins things). The prefix pg is short for the project we were working on.
The routines and text are completely general.

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter