0 purchases
Description:
fnvhashc 0.1.4
fnvhash-c
Python FNV hash implementation based on cython, to give you an alternative choice when you need a high speed hash in python, could reduce the latency of a single call to tens of nanoseconds.
Because of deprecation, we selectively did not implement FNV-0.
With code structure referenced to https://github.com/znerol/py-fnvhash
Install
pip install fnvhash-c
Feature
Implementation in pure C, you can expect a 30x to 50x speedup.
Hash latency reduce to under 100ns.
A simple bloom filter built inside , which uses a slightly different parameter than the default value to keep result diverse. Ultra fast to use for distributed blacklist scenarios.
CityHash included, since the original repo is no longer maintained, it is difficult to compile directly on windows.
Documentation
https://fnvhash-c.readthedocs.io
Example
# example.py
import fnvhash_c
print(fnvhash_c.fnv1_32(b'Hello world!'))
print(fnvhash_c.fnv1a_32(b'Hello world!'))
print(fnvhash_c.fnv1_64(b'Hello world!'))
print(fnvhash_c.fnv1a_64(b'Hello world!'))
center = fnvhash_c.BloomFilter(capability = 4096)
# To reduce runtime overhead, The default capacity is set to a constant compiled in the program.
# If you wish to change it, you need to compile the libs yourself to make sure the program work fine.
# Generally speaking, depending on the conversion time between Python and C,
# a shorter capability usually helps to make filter run faster.
import random
random_char_generator = lambda : f"{random.randint(1000000000,9999999999)}+salt".encode()
test_time = 1000000
hit = 0
miss = 0
for _ in range(test_time):
if center.hit(random_char_generator()):
hit += 1
else:
miss += 1
# Since the filter is total blank ,the hit rate should be very low
assert (hit * 100 / test_time) <= 0.01
print(f"Empty filter hit rate: {round(hit * 100 / test_time)}%")
# Now we put something into the filter list.
black_list = [random_char_generator() for _ in range(10000)]
for char in black_list:
center.update(char)
hit = 0
for _ in range(test_time):
if center.hit(random_char_generator()):
hit += 1
print(f"Filter with 10k elements hit rate: {round(hit * 100 / test_time,2)}%")
hit = 0
for char in black_list:
if center.hit(char):
hit += 1
print(f"Items in the blacklist hit rate: {round(hit * 100 / len(black_list),2)}%", ) # should be 100%
License
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
Files:
Customer Reviews
There are no reviews.
