Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981 You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links. Information Theory, Inference, and Learning Algorithms David J.C. MacKay Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981 You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links. Information Theory, Inference, and Learning Algorithms David J.C. MacKay
[email protected] c 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003 Version 6.0 (as published) June 26, 2003 Please send feedback on this book via http://www.inference.phy.cam.ac.uk/mackay/itila/ This book will be published by C.U.P. in September 2003. It will remain viewable on-screen on the above website, in postscript, djvu, and pdf formats. (C.U.P. replace this page with their own page ii.) Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981 You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links. Contents . . . . v 3 22 48 Data Compression . . . . . . . . . . . . . . . . . . . . . . . . 65 1 2 3 I 4 5 6 7 II III IV Preface . . . . . . . . . . . . . . . . . . . Introduction to Information Theory . . . Probability, Entropy, and Inference . . . . More about Inference . . . . . . . . . . . The Source Coding Theorem . . S