We present a novel method for information-theoretic exploration, leveraging recent work on mapping and localization. We describe exploration as the constrained optimization problem of computing a trajectory to minimize posterior map error, subject to the constraints of traveling through a set of sensing locations to ensure map coverage. This trajectory is found by reducing the map to a skeleton graph and searching for a minimum entropy tour through the graph. We describe how a specific factorization of the map covariance allows the reuse of EKF updates during the optimization, giving an efficient gradient ascent search for the maximum information gain tour through sensing locations, where each tour naturally incorporates revisiting well-known map regions. By generating incrementally larger tours as the exploration finds new regions of the environment, we demonstrate that our approach can perform autonomous exploration with improved accuracy.