The purpose of this paper is to develop a spectral analysis of the Hessenberg matrix obtained by the GMRES algorithm used for solving a linear system with a singular matrix. We prove that the singularity of the Hessenberg matrix depends on the nature of A and some others criteria like the zero eigenvalue multiplicity and the projection of the initial residual on particular subspaces. We also introduce some new results about the distinct kinds of breakdown which may occur in the algorithm when the system is singular. Key words: GMRES, Krylov subspace, singular system, Hessenberg matrix, geometric multiplicities, breakdowns