In this paper we consider a finite element discretization of the Oldroyd-B model of viscoelastic flows. The method uses standard continuous polynomial finite element spaces for velocities, pressures and stresses. Inf-sup stability and stability for convection-dominated flows are obtained by adding a term penalizing the jump of the solution gradient over element faces. To increase robustness when the Deborah number is high we add a non-linear artificial viscosity of shock-capturing type. The method is analyzed on a linear model problem, optimal a priori error estimates are proven that are independent of the solvent viscosity s. Finally we demonstrate the performance of the method on some known benchmark cases. AMS subject classifications. 65N12, 65N30, 76A10, 76M10 Key words. finite element methods, stabilized methods, continuous interior penalty, viscoelastic flows, Oldroyd-B