Local binary pattern (LBP) is widely used to extract image features in various visual recognition tasks. LBP is formulated in quite a simple form and thus enables us to extract effective image features with a low computational cost. There, however, are some limitations mainly regarding sensitivity to noise and loss of image contrast information. In this paper, we propose a novel LBP-based image feature to remedy those drawbacks without degrading the simplicity of the original LBP formulation. Encoding local pixel intensities into binary patterns can be regarded as separating them into two modes (clusters). We introduce Fisher discriminant criterion to optimize the LBP coding for exploiting binary patterns stably and discriminatively with robustness to noise. Besides, image contrast information is incorporated in a unified way by leveraging the discriminant score as a weight on the corresponding binary pattern; thereby, the prominent patterns are emphasized. In the experiments on pede...