Averaging techniques are popular tools in adaptive finite element methods since they provide efficient a posteriori error estimates by a simple postprocessing. In the second paper of our analysis of their reliability, we consider conforming h-FEM of higher (i.e., not of lowest) order in two or three space dimensions. In this paper, reliablility is shown for conforming higher order finite element methods in a model situation, the Laplace equation with mixed boundary conditions. Emphasis is on possibly unstructured grids, nonsmoothness of exact solutions, and a wide class of local averaging techniques. Theoretical and numerical evidence supports that the reliability is up to the smoothness of given right-hand sides.