Optimizing positional index structures for versioned document collections

Versioned document collections are collections that contain multiple versions of each document. Important examples are Web archives, Wikipedia and other wikis, or source code and documents maintained in revision control systems. Versioned document collections can become very large, due to the need to retain past versions, but there is also a lot of redundancy between versions that can be exploited. Thus, versioned document collections are usually stored using special differential (delta) compression techniques, and a number of researchers have recently studied how to exploit this redundancy to obtain more succinct full-text index structures. In this paper, we study index organization and compression techniques for such versioned full-text index structures. In particular, we focus on the case of positional index structures, while most previous work has focused on the non-positional case. Building on earlier work in [32], we propose a framework for indexing and querying in versioned doc...

• Type: Journal
• Year: 2012
• Where: SIGIR
• Authors: Jinru He, Torsten Suel
• Keywords: Delta Compression, Information Storage And Retrieval, Information Technology, Redundancy Elimination, SIGIR 2012