In this paper we give the first construction of a pseudorandom generator, with seed length O(log n), for CC0[p], the class of constant-depth circuits with unbounded fan-in MODp gates, for some prime p. More accurately, the seed length of our generator is O(log n) for any constant error > 0. In fact, we obtain our generator by fooling distributions generated by low degree polynomials, over Fp, when evaluated on the Boolean cube. This result significantly extends previous constructions that either required a long seed [LVW93] or that could only fool the distribution generated by linear functions over Fp, when evaluated on the Boolean cube [LRTV09, MZ09]. Enroute of constructing our PRG, we prove two structural results for low degree polynomials over finite fields that can be of independent interest.