Flight Data Recorder: Monitoring Persistent-State Interactions to Improve Systems Management

Mismanagement of the persistent state of a system--all the executable files, configuration settings and other data that govern how a system functions--causes reliability problems, security vulnerabilities, and drives up operation costs. Recent research traces persistent state interactions--how state is read, modified, etc.--to help troubleshooting, change management and malware mitigation, but has been limited by the difficulty of collecting, storing, and analyzing the 10s to 100s of millions of daily events that occur on a single machine, much less the 1000s or more machines in many computing environments. We present the Flight Data Recorder (FDR) that enables always-on tracing, storage and analysis of persistent state interactions. FDR uses a domain-specific log format, tailored to observed file system workloads and common systems management queries. Our lossless log format compresses logs to only 0.47-0.91 bytes/interaction. In this log format, 1000 machine-days of logs--over 25 bi...

• Type: Conference
• Year: 2006
• Where: OSDI
• Authors: Chad Verbowski, Emre Kiciman, Arunvijay Kumar, Brad Daniels, Shan Lu, Juhan Lee, Yi-Min Wang, Roussi Roussev
• Keywords: Log Format, Operating System, OSDI 2006, Persistent State, Persistent State Interactions