This paper presents a new kernel method for appearance-based object recognition, highly robust to noise and occlusion. It consists of a fully connected Markov Random Field that integrates results of Spin Glass theory with Gibbs probability distributions via nonlinear kernel mapping. We call the resulting model Spin Glass-Markov Random Field. We present theoretical analysis and several experiments that show its effectiveness and robustness to noise and occlusion. We obtain in both cases excellent results. Particularly, we achieve a recognition rate above 93 % with just 40 % of visible portion of the object.