In this paper we introduce the idea of model inference assisted fuzzing aimed to cost effectively improve software security. We experimented with several model inference techniques and applied fuzzing to the inferred models in order to generate robustness attacks. We proved our prototypes against real life software, namely anti-virus and archival software solutions. Several critical vulnerabilities were found in multiple file formats in multiple products. Based on the discovered vulnerabilities and the positive impact on the security we argue that our approach strikes a practical balance between completely random and manually designed model-based test case generation techniques.