In this paper a framework for Image Quality Assessment (IQA) is introduced based on the properties of Receptive Fields (RFs) which are the primary mechanism for detection of visual patterns in the Human Visual System (HVS). The proposed framework offers a probabilistic approach to the perceptual IQA, based on the probability of detecting discrepancies (distortion) between the corresponding features of a test and a reference image. The proposed Probabilistic Perceptual Image Quality (PPIQ) framework facilitates defining specific perceptual metrics for specific applications. To give an example on how the PPIQ framework can be utilized to define an Image Quality Metric (IQM), a sample IQM is introduced based on the properties of simple RFs of the early vision in the HVS. The sample IQM, based on the PPIQ framework, exhibits comparable accuracy to that of the legacy methods in terms of predicting the outcome of subjective image quality experiments.