In this paper, we explore fast algorithms for computing on encrypted polynomials. More specifically, we describe efficient algorithms for computing the Discrete Fourier Transform, multiplication, division, and multipoint evaluation on encrypted polynomials. The encryption scheme we use needs to be additively homomorphic, with a plaintext domain that contains appropriate primitive roots of unity. We show that some modifications to the key generation setups and working with variants of the original hardness assumptions one can adapt the existing homomorphic encryption schemes to work in our algorithms. The above set of algorithms on encrypted polynomials are useful building blocks for the design of secure computation protocols. We demonstrate their usefulness by utilizing them to solve two problems from the literature, namely the oblivious polynomial evaluation (OPE) and the private set intersection but expect the techniques to be applicable to other problems as well.