Geometric Filter: A Space and Time Efficient Lookup Table with Bounded Error

Lookup tables are frequently used in many applications to store and retrieve keyvalue pairs. Designing efficient lookup tables can be challenging with constraints placed on storage, query response time and/or result accuracy. This thesis proposes Geometric filter, a lookup table with a space requir...

Full description

Bibliographic Details
Main Author: Zhao, Yang
Other Authors: Rafiei, Davood (Computing Science)
Format: Others
Language:en
Published: 2009
Subjects:
Online Access:http://hdl.handle.net/10048/585
Description
Summary:Lookup tables are frequently used in many applications to store and retrieve keyvalue pairs. Designing efficient lookup tables can be challenging with constraints placed on storage, query response time and/or result accuracy. This thesis proposes Geometric filter, a lookup table with a space requirement close to the theoretical lower bound, efficient construction, fast querying speed, and guaranteed accuracy. Geometric filter consists of a sequence of hash tables, the sizes of which form a descending geometric series. Compared with its predecessor, Bloomier filter, its encoding runs two times faster, uses less memory, and it allows updates after encoding. We analyze the efficiency of the proposed lookup table in terms of its storage requirement and error bound, and run experiments on Web 1TB 5-gram dataset to evaluate its effectiveness.