Analysis of the IJCNN 2007 agnostic learning vs. prior knowledge challenge

We organized a challenge for IJCNN 2007 to assess the added value of prior domain knowledge in machine learning. Most commercial data mining programs accept data pre-formatted in the form of a table, with each example being encoded as a linear feature vector. Is it worth spending time incorporating domain knowledge in feature construction or algorithm design or can off-the-shelf programs working directly on simple low-level features do better than skilled data analysts? To answer these questions, we formatted five datasets using two data representations. The participants to the "prior knowledge" track used the raw data, with full knowledge of the meaning of the data representation. Conversely, the participants to the "agnostic learning" track used a pre-formatted data table, with no knowledge of the identity of the features. The results indicate that black-box methods using relatively unsophisticated features work quite well and rapidly approach the best attainable...

• Type: Journal
• Year: 2008
• Where: NN
• Authors: Isabelle Guyon, Amir Saffari, Gideon Dror, Gavin C. Cawley
• Keywords: Data Representation, Domain Knowledge, Low-level Features, Neural Networks, NN 2008