GitLocker: The Coding Marketplace

Description:

sifr 0.0.4

… and with the sign 0 … any number may be written
—Fibionacci

Count things in various time based windows using in-memory, redis or riak
storage.

Installation
Install the basic package:
pip install sifr
Install sifr with redis dependencies:
pip install 'sifr[redis]'
Install sifr with riak dependencies:
pip install 'sifr[riak]'
Install sifr with sifrd service dependencies:
pip install 'sifr[daemon]'

Examples

Using sifr with direct storage
import datetime
import redis, riak

from sifr.span import Year, Month, Day, Hour, Minute, get_time_spans
from sifr.storage import MemoryStorage, RedisStorage, RiakStorage

redis_client = redis.Redis()
redis_store = RedisStorage(redis_client)

riak_client = riak.RiakClient()
riak_store = RiakStorage(riak_client)

memory_store = MemoryStorage()

stores = [memory_store, redis_store, riak_store]

now = datetime.datetime.now()
user_id = 1
page = "index.html"

# construct the windows. These are the resolutions that will be tracked.
spans = [
span(now, ["views", "user", user_id])
for span in [Year, Month, Day, Hour, Minute]
]
# incr a counter for all resolutions
[store.incr_multi(spans) for store in stores]

# incr a unique counter
[store.incr_unique_multi(spans, page) for store in stores]
[store.incr_unique_multi(spans, page) for store in stores]

# track the page view
[store.track_multi(spans, page) for store in stores]
[store.track_multi(spans, page) for store in stores]

# get the counts/uniques for a single year window
for store in stores:
assert 1 == store.count(Year(now, ["views", "user", 1]))
assert 1 == store.cardinality(Year(now, ["views", "user", 1]))
assert set(["index.html"]) == store.uniques(Year(now, ["views", "user", 1]))

# get the counts/uniques for a range
start = now - datetime.timedelta(minutes=1)
end = now + datetime.timedelta(minutes=1)

span_range = get_time_spans(start, end, ["views", "user", 1], [Minute])
for store in stores:
assert [1] == [store.count(span) for span in span_range]
assert [1] == [store.cardinality(span) for span in span_range]
assert [set(["index.html"])] == [store.uniques(span) for span in span_range]

Using sifr via rpc
sifr.yml (using a redis backend)
storage: redis
redis_url: redis://localhost:6379/1
host: localhost
port: 6000
sifr.yml (using a riak backend)
storage: riak
riak_nodes:
- host: localhost
pb_port: 8087
host: localhost
port: 6000
Run the server
sifrd msgpack_server --config=sifr.yml
Interact with the server
from sifr import RPCClient
client = RPCCient(host='localhost', port=6000, resolutions=["year", "month", "day"])
client.incr("views:user:1")
client.incr_unique("views:user:1", "index.html")
client.incr_unique("views:user:1", "index.html")
client.track("views:user:1", "index.html")
client.track("views:user:1", "index.html")

assert 1 == client.count("views:user:1", datetime.datetime.now(), "day")
assert 1 == client.cardinality("views:user:1", datetime.datetime.now(), "day")
assert set(["index.html"]) == client.uniques("views:user:1", datetime.datetime.now(), "day")

References

Minuteman
Zero

Changelog

0.0.4 2015-06-16

Removed transactional pipelines from redis storage.

0.0.3 2015-06-10

Initial release