In this paper we developed an Inductive Logic Programming (ILP) based framework ExOpaque that is able to extract a set of Horn clauses from an arbitrary opaque machine learning model, to describe the behavior of the opaque model with high fidelity while maintaining the simplicity of the Horn clauses for human interpretations. In addition, traditional ILP systems often utilize a set of background knowledge to make accurate predictions. We also proposed a new method to use generated artificial training examples and high-accuracy opaque models to boost the prediction accuracy of an ILP system when background knowledge is absent and hard to obtain.