betterdicts 0.5.0

Description:

betterdicts 0.5.0

betterdicts
A collection of dictionary and dictionary-like utility classes.
The base type is betterdict, which only seeks to improve on the built-in
dict type by providing some extra functionality.
On top of this several other useful dictionaries have been built:

stack_dict: a multi-levelled dict whose state can be pushed and popped like
a stack.
attr_dict and jsdict: ergonomic dictionaries where attribute access
(xs.key) can be used in place of the usual square bracket syntax
(xs["key"]).
persistent_dict: a dict which automatically saves itself to disk with every
change, so it retains its state between script execution.
dynamic_dict and cache_dict: dictionaries that represent or act like a
memoized function.
number_dict: a dict on which arithmetic can be performed. Think of it as a
simple dict version of a NumPy array.

betterdict
from betterdicts import betterdict

Intended to work just like a dict, but with extra functionality and convenience.
Most of these things can be achieved with simple generator expressions or
similar, but having need of them constantly it got to a boiling point.
Increased Functional Friendliness
obj.update() and obj.clear() now return obj instead of None, so they are
no longer as useless.
>>> d = betterdict()
>>> d.update(a=1).update(zip('abc', [7,4,2]))
{'a': 7, 'b': 4, 'c': 2}

There's also betterdict.insert(key, value, [missing]) as an alternative to
obj[key] = value. It returns the previous value (if present).
>>> d.insert('b', 69)
4
>>> d.insert('d', 0)
>>> d.insert('e', 0, 'default return value goes here')
'default return value goes here'
>>> d
{'a': 7, 'b': 69, 'c': 2, 'd': 0, 'e': 0}

Python being a statement-oriented language rather than expression-oriented can
sometimes get in the way when you want to do simple operations inside of a list
comprehension or test. These methods are intended to temper this problem.
>>> d = betterdict()
>>> [d.insert(len(w), w) for w in 'fare thee well great heart'.split()]
[None, 'fare', 'thee', None, 'great']
>>> d
{4: 'well', 5: 'heart'}

Note that del obj[x] (a good example of Python's statement fanaticism) already
has an expression alternative in dict.pop(x, [default]).
The annoyance of statements led to the infamous walrus wart^Woperator being
added after several bikeshed bonfires, finally admitting that
expression-oriented language design is objectively superior. [TODO: add
tongue-in-cheek emoji here.]
Combining and Collecting
Often you're collecting things into dictionaries where the key you're interested
in is not unique. Or simiarly, if given a stream of keyed data, you usually need
some extra logic to handle identical keys.
One way to handle this is to write out some loop and add some
tedious-but-mandatory if branch in the loop. These loops start to feel like
Chinese water torture when they number in the thousands.
The default behavior of dict if you just give it an iterator is to discard
(overwrite). That default behavior has been kept, but alternatives have been
provided.

betterdict.combine(func, it, [initial=...])

This works like the built-in reduce(func, it, [initial]). It works well when
you're dealing with simple data like numbers and you just want a single answer.
>>> betterdict.combine(int.__add__, [('a',1), ('a',5), ('b',2), ('a',7)])
{'a': 13, 'b': 2}

It's short-hand for two kinds of loops:
# with a specified initial value
for k, v in source:
d[k] = func(d.get(k, initial), v)

# without
for k, v in source:
if k in d:
d[k] = func(d[k], v)
else:
d[k] = v

This can be used to put values into collections like in a functional language, as the following example:
>>> betterdict.combine(lambda x, y: x + (y,), [('a',1), ('a',5), ('b',2), ('a',7)], initial=())
{'a': (1, 5, 7), 'b': (2,)}

But it's very inefficient and painful to use with mutable containers like lists.
This is where collect() comes in:

betterdict.collect(type, it, [add_action=...])

This will return a dictionary-of-collections, where the collection is created
with type and added with add_action (which defaults to being auto-detected
as something like list.append, set.add, etc.).
>>> betterdict.collect(list, [('a',1), ('a',5), ('b',2), ('a',7)])
{'a': [1, 5, 7], 'b': [2]}

A more realistic example:
>>> some_stream_of_words = open('README.md', 'r', encoding='utf-8').read().split()
>>> l2w = betterdict.collect(set, ((len(w), w) for w in some_stream_of_words))
>>> l2w[2]
{'{}', 'in', 'of', 'be', 'up', '(a', '-1', 'is', 'no', '4,', '6}', 'to', 'if', 'as', 'v)', '1,', '38', '"\'', '6:', 'As', '-c', '2,', 'w)', '8:', 'or', 'v:', '2}', '2:', 'on', '10', '==', 'x,', '7,', '0}', '-l', '4}', "',", 'k:', 'do', 'So', 'it', '4:', 'd:', '9,', '3)', 'at', '7:', 'Or', '3:', '40', '5:', '##', 'ad', '^D', 'an', '20', '9:', 'ls', "T'", 'p2', 'y:', '1}', 'It', 'To', 'so', '1)', '17', '0)', 'by', 'k,', '3,', '//', 'x:', '0,', '26', '5,', '+=', 'p1', 'me', '1:'}

These functions are magic in that they can work both as class methods and as
instance methods. The instance methods obj.collect(...) and obj.combine(...)
works kind of like a combining or collecting obj.update(). The only exception
is that obj.combine() works immutably by returning a new dictionary as if it
was a binary operator.
See help(betterdict.combine) and help(betterdict.collect) for more
information.
Filtering and Mapping
.map_keys(f), .map_values(f), .map_pairs(f), .filter_keys(p),
.filter_values(p), .filter_pairs(p) all do the somewhat obvious thing.
>>> q = betterdict(enumerate("I'm nothing but heart"))
>>> q.filter_keys(lambda x: 4 < x < 10)
{5: 'o', 6: 't', 7: 'h', 8: 'i', 9: 'n'}
>>> q.filter_values(str.isupper)
{0: 'I'}

Though it should be noted that unlike .update they do not modify the
dictionary in-place as they're targeting more functional programming.
map_pairs(f) and filter_pairs(f) take a function of two arguments.
>>> q.map_pairs(lambda x, y: (x, x * y))
{0: '', 1: "'", 2: 'mm', 3: ' ', 4: 'nnnn', 5: '', 6: 't', 7: 'hh', 8: 'iii', 9: 'nnnn', 10: '', 11: ' ', 12: 'bb', 13: 'uuu', 14: 'tttt', 15: '', 16: 'h', 17: 'ee', 18: 'aaa', 19: 'rrrr', 20: ''}

Since map_keys() and map_pairs() might map two values to the same key, they
also take a function and initial value as optional arguments to do a reduction,
similar to how the built-in reduce() works. (Refer to .combine().)
>>> q.map_keys(lambda x: x%2, str.__add__)
{0: 'Imntigbthat', 1: "' ohn u er"}

There's also .filter([keys=f], [values=g]) and .map([keys=f], [values=g])
which can be used to filter/map both keys and values with two different
functions in one step:
>>> q.filter(lambda k: 4 < k < 10, lambda v: v in 'aeiouy')
{5: 'o', 8: 'i'}

Inversion
Flipping those arrows.
>>> q = betterdict(enumerate('divebar'))
>>> q
{0: 'd', 1: 'i', 2: 'v', 3: 'e', 4: 'b', 5: 'a', 6: 'r'}
>>> q.invert()
{'d': 0, 'i': 1, 'v': 2, 'e': 3, 'b': 4, 'a': 5, 'r': 6}
>>> q == q.invert().invert()

Note that q.invert() does not modify q, but returns a new dictionary.
(Functional friendliness where it makes sense.)
Most of the time the map is not expected to be injective (one-to-one) though,
and there are two ways of handling that:

you can either collect values into a container like a list, or

>>> q = betterdict(enumerate('syzygy'))
>>> q.invert()
{'s': 0, 'y': 5, 'z': 2, 'g': 4}
>>> q.invert_and_collect(list)
{'s': [0], 'y': [1, 3, 5], 'z': [2], 'g': [4]}
>>> q.invert_and_collect(set)
{'s': {0}, 'y': {1, 3, 5}, 'z': {2}, 'g': {4}}


you can combine values repeatedly until you get a single answer.

>>> q.invert_and_combine(int.__mul__)
{'s': 0, 'y': 15, 'z': 2, 'g': 4}
>>> q.invert_and_combine(lambda x, y: x+[y], [])
{'s': [0], 'y': [1, 3, 5], 'z': [2], 'g': [4]}

As seen above, collecting and combining are sometimes just special cases of a
more general operation (folding, repeated applicaitino of a monoid, etc). More
functional languages usually only has one operation for this, but I chose to
separate them because they have very a different feel in in Python, with very
different performance characteristics.
jsdict (ad hoc use only)
from betterdicts import jsdict, njsdict, rjsdict

jsdict is a betterdicts which works like JavaScript object, where keys and
attributes are the same. This is accomplished with zero overhead.
>>> d = jsdict()
>>> d['hello'] = 1
>>> d.filepath = '/'
>>> d
{'hello': 1, 'filepath': '/'}
>>> d.hello, d['filepath']
(1, '/')

For obvious reasons, this is a touch insane. JavaScript was never accused of
good design, and bringing it to Python where conventions are different will lead
to awful things:
>>> d = jsdict(clear=0)
>>> d.clear()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable

Thus these are for ad hoc use only, i.e. in one-off scripts or when doging
work directly in the REPL.
Refer to attr_dict below for a slightly safer alternative.
But sometimes this kind of convenience is just too good to pass up. You'll know
this in your heart too, if you've ever dealt with loading configuration from
some source that gives you a dict, and then having to type out ['...'] so
many times you actually get angry.
>>> import json
>>> config = json.loads('{"lazy": "sarah", "active": false}', object_pairs_hook=jsdict)
>>> config.active
False
>>> config
{'lazy': 'sarah', 'active': False}

I get equally angry whenever a source gives me some object of a godforsaken
abstract ConfigurationProxyHelper class which is just a glorified dict
without any of the functionality or compatibility.

rjsdict is a jsdict where obj.key for a missing key automatically
returns and inserts a new rjsdict instance. Think of it as a recursive
jsdict. It's useful for when you're building some hierarchical structure:

>>> config = rjsdict()
>>> config.lights = 'off'
>>> config.player.health = 40
>>> config.player.name = 'Mr. T'
>>> config
{'lights': 'off', 'player': {'health': 40, 'name': 'Mr. T'}}


njsdict is a jsdict where obj.key evaluates to None instead of raising
AttributeError. (obj is not modified.)

attr_dict
from betterdicts import attr_dict

Inspired by jsdict above, it works in much the same way (though different
mechanism). It is provided as a slightly safer alternative, because it doesn't
overwrite extant attributes:
>>> q = attr_dict(a=10)
>>> q.a
10
>>> q.b = 20
>>> q
{'a': 10, 'b': 20}
>>> q.pop = -1
>>> q
{'a': 10, 'b': 20, 'pop': -1}
>>> q.pop
<built-in method pop of attr_dict object at 0x7fc85a580c40>

But note it is still has the glow of insanity, which will inspire nothing but
anger, fear, and frustration.
dynamic_dict
from betterdicts import dynamic_dict, cache_dict

How many times have you been frustrated by the fact that the standard
collections.defaultdict() calls its factory function without providing the key
context?
Yeah, me too.
dynamic_dict(f) is equivalent to a betterdict but if a missing key is requested
it is first created with f(key) and inserted.
>>> d = dynamic_dict(hex)
>>> d[16]
'0x10'
>>> d[100]
'0x64'
>>> d
{16: '0x10', 100: '0x64'}

cache_dict
How many times have you wanted concrete access to the function cache when using
something like functools.cache?
Yeah, me too.
Intended to be used as a decorator on a function, this will turn the function
into a callable dictionary. The dictionary is its own cache and can freely be
inspected, modified, etc.
@cache_dict
def heavy_bite(n):
"Calculates a heavy bite."
if n < 1: return 1
print(f'calculating heavy bite {n}...')
return heavy_bite(n // 3) * heavy_bite(n - 3 + (-n % 3)) + heavy_bite(n - 1)

>>> heavy_bite(10)
calculating heavy bite 10...
calculating heavy bite 3...
calculating heavy bite 1...
calculating heavy bite 2...
calculating heavy bite 9...
calculating heavy bite 6...
calculating heavy bite 5...
calculating heavy bite 4...
calculating heavy bite 8...
calculating heavy bite 7...
2880
>>> heavy_bite(10)
2880
>>> del heavy_bite[10]
>>> heavy_bite(10)
calculating heavy bite 10...
2880
>>> heavy_bite.__doc__
'Calculates a heavy bite.'

NOTE: per now it only caches on its first argument. The rest are just passed
through (in case of a cache miss). Ideally I want a way to specify an argument
signature with indication of which ones constitutes the key.
number_dict
Acts like a collections.Counter() with arithmetic support like a number.
This is sort of a poor man's numpy-dict.
>>> q = number_dict(range(5))
>>> q
{0: 1, 1: 1, 2: 1, 3: 1, 4: 1}
>>> q[1] += 5
>>> q[4] += 1
>>> q[9]
0
>>> q[9] = 9
>>> (q+1)**2
{0: 4, 1: 49, 2: 4, 3: 4, 4: 9, 9: 100}
>>> 1 / q
{0: 1.0, 1: 0.16666666666666666, 2: 1.0, 3: 1.0, 4: 0.5, 9: 0.1111111111111111}

persistent_dict
The simplest possible persistent state exposed as a betterdict. Also
intended for ad hoc use.
This is for when you need something really simple and magic to store some flat
data between script invocations, without the extra management of a database or
file formats.
[franksh@moso ~]$ python -iq -c 'from betterdicts import persistent_dict'
>>> p = persistent_dict()
>>> p
{}
>>> p['foo'] = 17
>>> p
{'foo': 17}
>>> persistent_dict()
{'foo': 17}
>>> p['bar'] = [1,2]
>>> persistent_dict()
{'foo': 17, 'bar': [1, 2]}
>>> ^D
[franksh@moso ~]$ ls -l cache.pickle
-rw-r--r-- 1 franksh franksh 38 Aug 26 17:18 cache.pickle
[franksh@moso ~]$ python -iq -c 'from betterdicts import persistent_dict'
>>> persistent_dict()
{'foo': 17, 'bar': [1, 2]}
>>>

It defaults to loading and saving from ./cache.pickle in whatever the current
working directory happens to be, as seen above.
Any change made directly[^1] to the dictionary causes it to save itself to disk
as a pickle file. So obviously if you're building up some initial data quickly,
you want to use another dictionary first, and then convert it later with
persistent_dict([filename], data). This will automatically both load and save.
To use a custom filename use persisent_dict([filename]):
[franksh@moso ~]$ python -iq -c 'from betterdicts import persistent_dict'
>>> p1 = persistent_dict('foo.p', dict(a=1,b=22,c=333))
>>> p2 = persistent_dict('bar.p', p1.invert())
not pers ({1: 'a', 22: 'b', 333: 'c'},) {}
>>> persistent_dict('foo.p')
{'a': 1, 'b': 22, 'c': 333}
>>> persistent_dict('bar.p')
{1: 'a', 22: 'b', 333: 'c'}
>>> del p2[22]
>>> p2 == persistent_dict('bar.p')
True

Warning: Two separate persistent_dict() objects bound to the same file will
not automatically stay in sync, instead they will keep overwriting each other's
data!
stack_dict
Stack dicts emulate how scopes or namespaces work. It allows you to repeatedly
save the state of the dictonary (push_stack()) and later retore it
(pop_stack()).
>>> from betterdicts import stack_dict
>>> q=stack_dict(a=1,b=2)
>>> q
{'a': 1, 'b': 2}
>>> q.push_stack()
>>> q
{'a': 1, 'b': 2}
>>> q['c'] = 7
>>> del q['a']
>>> q
{'b': 2, 'c': 7}
>>> q.pop_stack()
>>> q
{'a': 1, 'b': 2}
>>> q.push_stack(hello=0, world=-1) # push_stack works like update()
>>> q.push_stack(hello=1000, world=1000)
>>> q
{'a': 1, 'b': 2, 'hello': 1000, 'world': 1000}
>>> q.pop_stack(); q
{'a': 1, 'b': 2, 'hello': 0, 'world': -1}
>>> q.pop_stack(); q
{'a': 1, 'b': 2}

Stack dicts can also be used with with-blocks:
>>> from betterdicts import stack_dict
>>> q = stack_dict({'a': 1, 'b': 2})
>>> with q:
... q['c'] = 10
... print(q)
...
{'a': 1, 'b': 2, 'c': 10}
>>> q # q is reset to its previous state after the `with`.
{'a': 1, 'b': 2}

[^1]: "deep" changes, like modifying a mutable object in the dictionary are not detected