We present algebraic conditions on constraint languages Γ that ensure the hardness of the constraint satisfaction problem CSP(Γ) for complexity classes L, NL, P, NP and ModpL. These criteria also give non-expressibility results for various restrictions of Datalog. Furthermore, we show that if CSP(Γ) is not first-order definable then it is L-hard. Our proofs rely on tame congruence theory and on a fine-grain analysis of the complexity of reductions used in the algebraic study of CSP. The results pave the way for a refinement of the dichotomy conjecture stating that each CSP(Γ) lies in P or is NP-complete and they match the recent classification of [2] for Boolean CSP. We also infer a partial classification theorem for the complexity of CSP(Γ) when the associated algebra of Γ is the full idempotent reduct of a preprimal algebra. Constraint satisfaction problems (CSP) provide a unifying framework to study various computational problems arising naturally in artificial intellig...