This paper proposes a novel method called FLGP to construct a classifier device of capability in feature selection and feature extraction. FLGP is developed with layered genetic programming that is a kind of the multiple-population genetic programming. Populations advance to an optimal discriminant function to divide data into two classes. Two methods of feature selection are proposed. New features extracted by certain layer are used to be the training set of next layer's populations. Experiments on several well-known datasets are made to demonstrate performance of FLGP.