The paper describes a new approach using a Conditional Random Fields (CRFs) to extract physical and logical layouts in unconstrained handwritten letters such as those sent by individuals to companies. In this approach, the extraction of the layouts is considered as a labeling task consisting in assigning a label to each pixel of the document image. This label is chosen among a set of labels depicting the layout elements. The CRF-based method models two stochastic processes : the first one corresponds to the association between pixels and labels, the second one to the relationship of one label with respect to its neighboring labels. The CRF model gives access to the global conditional probability of a given labeling of the image according to image features and some prior knowledge about the structure of the document. This global probability is computed by means of local conditional probabilities at each pixel. To find the best label field, a key point of our model is the implementation...