The Sloan Digital Sky Survey Data Archive Server

The Sloan Digital Sky Survey (SDSS) Data Archive Server (DAS) provides public access to over 12Tb of data in 17 million files produced by the SDSS data reduction pipeline. Many tasks which seem trivial when serving smaller, less complex data sets present challenges when serving data of this volume and technical complexity. The included output files should be chosen to support as much science as possible from publicly released data, and only publicly released data. Users must have the resources needed to read and interpret the data correctly. Server administrators must generate new data releases at regular intervals, monitor usage, quickly recover from hardware failures, and monitor the data served by the DAS both for contents and corruption. We discuss these challenges, describe tools we use to administer and support the DAS, and discuss future development plans.

• Type: Journal
• Year: 2008
• Where: CSE
• Authors: Eric H. Neilsen Jr.
• Keywords: CSE 2008, Data Archive Server, SDSS Data, Sloan Digital Sky Survey, Theoretical Computer Science