Indexing and searching tera-scale Grid-Based Digital Libraries

— The University of California, Berkeley and the University of Liverpool in conjunction with the San Diego Supercomputer Center, are developing a framework for GridBased Digital Library systems and Information Retrieval Services (Cheshire3) that operates in both single-processor and distributed computing environments. In this paper we discuss some results of testing Grid-based parallel approaches in indexing and retrieval for a variety of information resources, ranging from small test collections like the TREC and INEX collections, to mediumscale metadata collections like Medline and a test version of University of California Online Union Catalog, MELVYL (with 15 million and 9.5 million records respectively) ranging up to large-scale collections like the US National Records and Archives Administration (NARA) Preservation Prototype. This paper examines our approaches to indexing and retrieving from these collections and the architecture of the system that supports them.

• Type: Conference
• Year: 2006
• Where: INFOSCALE
• Authors: Robert Sanderson, Ray R. Larson
• Keywords: INFOSCALE 2006, Mediumscale Metadata Collections, San Diego Supercomputer Center, Small Test Collections