pandacell 2020.9.19

Description:

pandacell 2020.9.19

pandacell
Author: Eirik B. Stavestrand
Introduces a %df (or %%df) magic which can be used in Jupyter notebooks and the IPython console.
The magic executes the contents of a cell on a Pandas DataFrame.
Description
Pandas is great and all, but writing Pandas code can be tedious. For example when simply making summing two columns:
In [1]: df["a"] + df["b"]

It might not look like such a big deal, but all those brackets and quotation marks add up.
Using pandacell, the above syntax can be written as:
In [2]: %df a + b

Under the hoods, this is accomplished simply by passing the cell contents as a string to Pandas' df.eval function.
This isn't very complex, but it does provide a fair deal of functionality and adds a whole lot of readability.
If you wish to store the results to a new column, use regular assignment along with the -i (or --inplace) flag:
In [3]: %df -i c = a + b

It also works with multiple assignments:
In [4]: %%df -i
...: c = a + b
...: f = c - a

You can use Pandas' various accessors and series method calls:
In [5]: %%df -i
...: name_upper = name.str.upper()
...: yr = timestamp.dt.year
...: lower_cased = species.where(cond=species.str[0].str.islower(), other=None)

Since variable names are assumed to be columns in the dataframe, regular variables in the local/global namespace can be accessed by prefixing with @
In [6]: a = 1
...: %df a = @a + 1

In [7]: def myfunc(row):
...: return row + 43
...: %df b = a.apply(@myfunc)

By default, pandacell operates on any dataframe named df. This can be overridden with the -n (or --name) flag:
In [8]: %df -n=df_in c = a + b

You can also print subset a dataframe with the -q (or --query) flag:
In [9]: %df -q species == "setosa"
Out[9]:
sepal_length sepal_width petal_length petal_width species a
0 5.1 3.5 1.4 0.2 setosa 0
1 4.9 3.0 1.4 0.2 setosa 0


In [10]: %df -q species.isna() #check for missing values
Out[10]:
Empty DataFrame
Columns: [sepal_length, sepal_width, petal_length, petal_width, species]
Index: []

This can be combined with the -i flag to subset the dataframe in-place:
In [10]: %df -q -i species == "setosa"

Pandacell even supports comments
In [11]: %%df -i
...: # Line comment
...: c = a + b # Comment at end of line

Inspired by
https://github.com/catherinedevlin/ipython-sql
Development
https://github.com/eirki/pandacell