It has been observed from image denoising experiments that translation invariant (TI) wavelet transforms often outperform orthogonal wavelet transforms. This paper compares the two transforms from the viewpoint of approximation theory, extending previous results based on Haar wavelets. The advantages of the TI expansion over orthogonal expansion are twofold: the TI expansion produces smaller approximation error when approximating a smooth function, and it mitigates Gibbs artifacts when approximating a discontinuous function.