Information Theory, Inference, and Learning Algorithms

This book is aimed at senior undergraduates and graduate students in Engineering, Science, Mathematics, and Computing. It expects familiarity with calculus, probability theory, and linear algebra as taught in a 1st- or second-year undergraduate course on mathematics for scientists and engineers. Conventional courses on information theory cover not only the beautiful theoretical ideas of Shannon, but also practical solutions to communication problems. This book goes further, bringing in Bayesian data modelling, Monte Carlo methods, variational methods, clustering algorithms, and neural networks.

• Authors: David J. C. MacKay
• Keywords: Data Compression, Information Theory, Machine Learning, Neural Networks, Noisy-Channel Coding, Probabilities And Inference, Sparse Graph Codes