GitLocker: The Coding Marketplace

Description:

iodict 0.0.8

IODict
iodict is a thread safe object store which is writting in pure python.
The dictionary implementation follows the Dict API, but stores
items using their birthtime allowing users to treat this datastore as a
file system backed OrderedDict.

Items in the object store use file system attributes, when available to
store key and birthtime information. File system attributes enhance the
capability of the object store; however, they're not required. In the
event xattrs are not available, file stat is used for file creation time.
While stat works, in many cases, item ordering is not guarenteed.

Dictionary Usage
import iodict
data = iodict.IODict(path='/tmp/iodict') # Could be any path on the file system
data["key"] = "value"
data
{'key': "value"}

dir(data)
['__class__',
'__delattr__',
'__delitem__',
'__dict__',
'__dir__',
'__doc__',
'__enter__',
'__eq__',
'__exit__',
'__format__',
'__ge__',
'__getattribute__',
'__getitem__',
'__gt__',
'__hash__',
'__init__',
'__init_subclass__',
'__iter__',
'__le__',
'__len__',
'__lt__',
'__module__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__setattr__',
'__setitem__',
'__sizeof__',
'__str__',
'__subclasshook__',
'__weakref__',
'_db_path',
'_encoder',
'_lock',
'clear',
'copy',
'fromkeys',
'get',
'items',
'keys',
'pop',
'popitem',
'setdefault',
'update',
'values']

When running in a multiprocessing / threading application, a lock is required
to be passed into the iodict class.
import threading

import iodict
data = iodict.IODict(path='/tmp/iodict', lock=threading.Lock)

By default, if no lock is provided, a multiprocessing lock will be created.

The lock object allows the iodict to respect the locking paradigm of the
executing application.
Durable Queue Usage
The DurableQueue class is used to create a disk-backed queue which implements
the standarad queue.Queue API.
import iodict
q = iodict.DurableQueue(path='/tmp/iodict') # Could be any path on the file system
q.put("test")
data = q.get()
data
'test'

Flushing Capable Queue Usage
The FlushQueue class is used to extend the capabilities of a standard queue
object by providing an extension which can be used to flush the objects within
queue to a disk. This is useful in situation when the application needs to halt
or otherwise stop working, but the inflight processes need to be saved and
resumed at a later time.
import queue

import iodict

class NewQueue(queue.Queue, iodict.FlushQueue):
def __init__(self, path, lock=None, semaphore=None):
super().__init__()
self.path = path
self.lock = lock
self.semaphore = semaphore

q = NewQueue(path='/tmp/iodict') # Could be any path on the file system
q.put("test")
q.qsize()
1
q.flush()
q.qsize()
0
q.ingest()
q.qsize()
1
q.get()
'test'