Building efficient multi-threaded search nodes

Search nodes are single-purpose components of large Web search engines and their efficient implementation is critical to sustain thousands of queries per second and guarantee individual query response times within a fraction of a second. Current technology trends indicate that search nodes ought to be implemented as multithreaded multi-core systems. The straightforward solution that system designers can apply in this case is simply to follow standard practice by deploying one asynchronous thread per active query in the node and attaching each thread to a different core. Each concurrent thread is responsible for sequentially processing a single query at a time. The only potential source of read/write conflicts among threads are the accesses to the different application caches present in the search node. However, new Web applications pose much more demanding requirements in terms of read/write conflicts than recent past applications since now data updates must take place concurrently wi...

• Type: Journal
• Year: 2010
• Where: CIKM
• Authors: Carolina Bonacic, Carlos García, Mauricio Marín, Manuel Prieto-Matías, Francisco Tirado
• Keywords: CIKM 2010, Individual Query Response, Information Technology, Query Processing, Web Search Engines