Implementing a high-volume, low-latency market data processing system on commodity hardware using IBM middleware

A stock market data processing system that can handle high data volumes at low latencies is critical to market makers. Such systems play a critical role in algorithmic trading, risk analysis, market surveillance, and many other related areas. We show that such a system can be built with generalpurpose middleware and run on commodity hardware. The middleware we use is IBM System S, which has been augmented with transport technology from IBM WebSphere MQ Low Latency Messaging. Using eight commodity x86 blades connected with Ethernet and Infiniband, this system can achieve 80 µsec average latency at 3 times the February 2008 options market data rate and 206 µsec average latency at 15 times the February 2008 rate. Categories and Subject Descriptors C.3 [Performance of Systems]: Performance Attributes General Terms Design, Implementation, Performance Keywords Market Data Processing, Commodity Hardware, IBM Middleware

• Type: Conference
• Year: 2009
• Where: SC
• Authors: Xiaolan J. Zhang, Henrique Andrade, Bugra Gedik, Richard King, John F. Morar, Senthil Nathan, Yoonho Park, Raju Pavuluri, Edward Pring, Randall Schnier, Philippe Selo, Michael Spicer, Volkmar Uhlig, Chitra Venkatramani
• Keywords: Applied Computing, Commodity Hardware, Market Data Processing, SC 2009, µsec Average Latency