Probability And Statistics By Example: Volume 2, Markov Chains: A Primer In Random Processes And Their Applications (v. 2)

This page intentionally left blank Probability and Statistics by Example: II Probability and statistics are as much about intuition and problem solving, as they are about theorem proving. Because of this, students can find it very difficult to make a successful transition from lectures to examinations to practice, since the problems involved can vary so much in nature. Since the subject is critical in many modern applications such as mathematical finance, quantitative management, telecommunications, signal processing, bioinformatics, as well as traditional ones such as insurance, social science and engineering, the authors have rectified deficiencies in traditional lecture-based methods by collecting together a wealth of exercises for which they have supplied complete solutions. These solutions are adapted to needs and skills of students. Following on from the success of Probability and Statistics by Example: Basic Probability and Statistics, the authors here concentrate on random processes, particularly Markov processes, emphasising models rather than general constructions. Basic mathematical facts are supplied as and when they are needed and historical information is sprinkled throughout. Probability and Statistics by Example: II Markov Chains: a Primer in Random Processes and their Applications Yuri Suhov University of Cambridge Mark Kelbert University of Wales–Swansea CAMBRIDGE UNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK Published in the United States of America by Cambridge University Press, New York www.cambridge.org Information on this title: www.cambridge.org/9780521847674 © Y. Suhov and M. Kelbert 2008 This publication is in copyright. Subject to statutory exception and to the provision of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published in print format 2008 ISBN-13 978-0-511-45517-9 eBook (EBL) ISBN-13 978-0-521-84767-4 hardback ISBN-13 978-0-521-61234-0 paperback Cambridge University Press has no responsibility for the persistence or accuracy of urls for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. Contents Preface page vii 1 Discrete-time Markov chains 1.1 The Markov property and its immediate consequences 1.2 Class division 1.3 Hitting times and probabilities 1.4 Strong Markov property 1.5 Recurrence an