We consider the class of constant depth AND/OR circuits augmented with a layer of modular counting gates at the bottom layer, i.e AC0 ◦MODm circuits. We show that the following holds for several types of gates G: by adding a gate of type G at the output, it is possible to obtain an equivalent probabilistic depth 2 circuit of quasipolynomial size consisting of a gate of type G at the output and a layer of modular counting gates, i.e G ◦ MODm circuits. The types of gates G we consider are modular counting gates and threshold-style gates. For all of these, strong lower bounds are known for (deterministic) G ◦ MODm circuits.