Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations

In transfer learning we aim to solve new problems using fewer examples using information gained from solving related problems. Transfer learning has been successful in practice, and extensive PAC analysis of these methods has been developed. However it is not yet clear how to define relatedness between tasks. This is considered as a major problem as it is conceptually troubling and it makes it unclear how much information to transfer and when and how to transfer it. In this paper we propose to measure the amount of information one task contains about another using conditional Kolmogorov complexity between the tasks. We show how existing theory neatly solves the problem of measuring relatedness and transferring the ‘right’ amount of information in sequential transfer learning in a Bayesian setting. The theory also suggests that, in a very formal and precise sense, no other reasonable transfer method can do much better than our Kolmogorov Complexity theoretic transfer method, and t...

• Type: Conference
• Year: 2007
• Where: NIPS
• Authors: M. M. Mahmud, Sylvian R. Ray
• Keywords: Information Technology, NIPS 2007, Sequential Transfer, Transfer Learning, Transfer Method