Databases that preserve a historical record of activities and data offer the important benefit of system accountability: past events can be analyzed to detect breaches and maintain data quality. But the retention of history can also pose a threat to privacy. System designers need to carefully balance the need for privacy and accountability by controlling how and when data is retained by the system and who will be able to recover and analyze it. This paper describes the technical challenges faced in enhancing database systems so that they can securely manage history. These include: first, assessing the unintended retention of data in existing database systems that can threaten privacy; second, redesigning system components to avoid this unintended retention; and third, developing new system features to support accountability when it is desired.