Contextual information and a priori knowledge play important roles in image segmentation based on neural networks. This paper proposed a method for including contextual information in a model-based neural network (MBNN) that has the advantage of combining a priori knowledge. This is achieved by including Markov random field (MRF) into the MBNN and this novel neural network is termed as MRF-MBNN. Then the proposed method is applied to segmenting the images. Experimental results indicate the MRF-MBNN is superior to the MBNN in image segmentation. This study is a successful attempt of incorporating contextual information and a prior knowledge into neural networks to segment images.