Architecture of the internet archive

The Internet Archive is a live production system supporting close to a petabyte of data and delivering an average of 2.3Gb/sec of data to Internet users. We describe the architecture of this system with an emphasis on its robustness and how it is managed by a very small team of systems personnel. Notably, the current system does not employ a cache. We analyze the reasons for this decision and show that an effective cache could not be built until now. However, new solid state disk technology may offer promising new cache implementations. Categories and Subject Descriptors D.2 [Software]: Software Engineering; D.2.8 [Software Engineering]: Metrics—complexity measures, performance measures

• Type: Conference
• Year: 2009
• Where: SYSTOR
• Authors: Elliot Jaffe, Scott Kirkpatrick
• Keywords: Control Systems, Effective Cache, Software Engineering, Solid State Disk, SYSTOR 2009