Interestingness is an important aesthetic property, which literally means something that arouses curiosity and is a precursor to attention. Aesthetics is becoming more important as multimedia systems become more human and content centric as opposed to technology centric. In this paper, we use insights from cognitive science, neurophysiology of the early visual system and a mix of human experiments and computational modeling for the purpose of investigating interestingness. Categories in image interestingness and their computational realization are explored through a nontrivial dataset and a real-world problem.