Responding to natural or man-made disasters in a timely and effective manner can reduce deaths and injuries, contain or prevent secondary disasters, and reduce the resulting economic losses and social disruption. Appropriate IT solutions can improve this response. However, exhaustive and realistic validation of these IT solutions is difficult; proofs are not available, simulations lack realism, and drills are expensive and cannot be reproduced. This paper presents DrillSim: a simulation environment that plays out the activities of a crisis response (e.g., evacuation). It has capabilities to integrate real-life drills into a simulated response activity using an instrumented environment with sensing and communication capabilities. IT solutions can be plugged in the simulation system to study their effectiveness in disaster management and response. This way, by using a simulation coupled with an on-going drill, IT solutions can be tested in a less expensive but realistic scenario.