We present an efficient multigrid algorithm that is adequate to solve a heavy linear system given in cloth simulation. Although multigrid has been successfully applied to the Poisson problems, it is hard to migrate to complicated cloth deformations due to its lack of physical meaning in level construction. We addresses this problem by developing a physically faithful technique ensuring the conservation of all physical quantities across levels. The performance of our approach is demonstrated on a number of garment simulations implemented by the state of the art techniques: the implicit integration, the triangle-based in-plane energy model, and the curvature-based bending energy model. Our multigrid algorithm is about four times faster than the preconditioned Conjugate Gradient method for a garment with 20K particles.