Usage Of Dedicated Data Structures For Url Databases In A Large-Scale Crawling
The article discuss usage of Berkeley DB data structures such as hash tables and b-trees forimplementation of a high performance URL database. The article presents a formal model fora data structures oriented URL database, which can be used as an alternative for a relationaloriented URL database.
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AGH University of Science and Technology Press
2009-01-01
|
| Series: | Computer Science |
| Subjects: | |
| Online Access: | http://journals.agh.edu.pl/csci/article/download/124/72 |