Carnegie Mellon's CyDAT: Harnessing a Wide Array of Telemetry Data to Enhance Distributed System Diagnostics

The number and complexity of distributed applications has exploded, and to-date, each has had to create its own method for providing diagnostic tools and performance metrics. These distributed services have become increasingly dependent, not only on the system and network infrastructures upon which they are built, but also each other. The effectiveness of a diagnostician is seriously hindered by the difficulty in accessing diagnostic data. However, even when access can be gained, it exposes the daunting challenge of correlating a myriad of different data formats and an incredible amount of data (both in static files and real time streams). To say that diagnosis of distributed systems is a complex and difficult is a vast understatement; and the task is getting tougher every day. There is a paucity of tools, data mining methods and logfile standards that has been worsening for years. Researchers face the same difficulty in gaining access to data for purposes of experimentation. Respondi...

• Type: Conference
• Year: 2008
• Where: OSDI
• Authors: Chas DiFatta, Mark Poepping, Daniel V. Klein
• Keywords: Data Mining Methods, Operating System, OSDI 2008, Real Time Streams, Transport Telemetry Data