In this paper we discuss problems of constructing classifiers from imbalanced data. We describe a new approach to selective preprocessing of imbalanced data which combines local over-sampling of the minority class with filtering difficult examples from the majority classes. In experiments focused on rule-based and tree-based classifiers we compare our approach with two other related pre-processing methods