The physical properties of color are usually described by their spectra, eigenvector expansions or low-dimensional descriptors such as RGB or CIE-Lab. In the first part of the paper we show that many of the traditional methods can be unified in a framework where color spectra are elements of an infinite-dimensional Hilbert space that are described by projections onto low-dimensional spaces. We derive some fundamental geometrical properties of the subset of the Hilbert space formed by all color spectra. We describe the projection operators that map the elements of the Hilbert space to elements in a finite dimensional vector space. This leads to a generalization of the concepts of spectral locus and purple line. It will be shown that for geometrical reasons the color space is topologically equivalent to a cone. In the second part of the paper we illustrate the theoretical concepts with four large databases of spectra from color systems and a series of multi-spectral images of natural sc...