The scalable hyperlink store

This paper describes the Scalable Hyperlink Store, a distributed in-memory “database” for storing large portions of the web graph. SHS is an enabler for research on structural properties of the web graph as well as new link-based ranking algorithms. Previous work on specialized hyperlink databases focused on finding efficient compression algorithms for web graphs. By contrast, this work focuses on the systems issues of building such a database. Specifically, it describes how to build a hyperlink database that is fast, scalable, fault-tolerant, and incrementally updateable. Categories and Subject Descriptors H.2.4 [Database Management]: Systems—distributed databases General Terms Performance, Reliability Keywords Web graph, hyperlink database, scalability

• Type: Conference
• Year: 2009
• Where: HT
• Authors: Marc Najork
• Keywords: HT 2009, Hyperlink Database, Internet Technology, Scalable Hyperlink Store, Web Graph