Better external memory suffix array construction

Suffix arrays are a simple and powerful data structure for text processing that can be used for full text indexes, data compression, and many other applications in particular in bioinformatics. However, so far it has looked prohibitive to build suffix arrays for huge inputs that do not fit into main memory. This paper presents design, analysis, implementation, and experimental evaluation of several new and improved algorithms for suffix array construction. The algorithms are asymptotically optimal in the worst case or on the average. Our implementation can construct suffix arrays for inputs of up to 4GBytes in hours on a low cost machine. As a tool of possible independent interest we present a systematic way to design, analyze, and implement pipelined algorithms.

• Type: Journal
• Year: 2008
• Where: JEA
• Authors: Roman Dementiev, Juha Kärkkäinen, Jens Mehnert, Peter Sanders
• Keywords: JEA 2008, Powerful Data Structure, Suffix Array, Suffix Array Construction