Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981 You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.
Information Theory, Inference, and Learning Algorithms
David J.C. MacKay
Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981 You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.
Information Theory, Inference, and Learning Algorithms
David J.C. MacKay
[email protected] c
1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003
Version 6.0 (as published) June 26, 2003
Please send feedback on this book via http://www.inference.phy.cam.ac.uk/mackay/itila/ This book will be published by C.U.P. in September 2003. It will remain viewable on-screen on the above website, in postscript, djvu, and pdf formats.
(C.U.P. replace this page with their own page ii.)
Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981 You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.
Contents
. . . .
v 3 22 48
Data Compression . . . . . . . . . . . . . . . . . . . . . . . .
65
1 2 3
I
4 5 6 7
II
III
IV
Preface . . . . . . . . . . . . . . . . . . . Introduction to Information Theory . . . Probability, Entropy, and Inference . . . . More about Inference . . . . . . . . . . .
The Source Coding Theorem . . S