Supporting high performance bioinformatics flat-file data processing using indices

As an essential part of in vitro analysis, biological database query has become more and more important in the research process. A few challenges that are specific to bioinformatics applications are data heterogeneity, large data volume and exponential data growth, constant appearance of new data types and data formats. We have developed an integration system that processes data in their flat file formats. Its advantages include the reduction of overhead and programming efforts. In the paper, we discuss the usage of indicing techniques on top of this flat file query system. Besides the advantage of processing flat files directly, the system also improves its performance and functionality by using indexes. Experiments based on real life queries are used to test the integration system.

• Type: Conference
• Year: 2008
• Where: IPPS
• Authors: Xuan Zhang, Gagan Agrawal
• Keywords: Distributed And Parallel Computing, Exponential Data Growth, IPPS 2008, Large Data Volume, flat files