Markov Processes for Stochastic Modeling  book cover
1st Edition

Markov Processes for Stochastic Modeling

  • This format is currently out of stock.
ISBN 9780412606601
Published January 1, 1997 by Chapman and Hall/CRC
341 Pages

USD $180.00

Prices & shipping based on shipping country


Book Description

This book presents an algebraic development of the theory of countable state space Markov chains with discrete and continuous time parameters.

Table of Contents

Introduction: Stochastic processes; the Markov property; some examples; transition probabilities; the strong Markov property; exercises. Discrete-time Markov chains: First time passages; classification of states; recurrent Markov chains; finite Markov chains; time-reversable Markov chains; the rate of convergence to stationary; absorbing Markov chanins and their applications; Lossy Markov chains; exercises. Monotone Markov chains: Preliminaries; distribution classes of interest; stochastic ordering relations; monotone Markov chains; unimodality of transition probabilities; first-passage-time distributions; bounds for quasi-stationary distributions; renewal processes in discrete time; comparability of Markov chains; exercises. Continuous-time Markov chains: transition probability functions; finite Markov chains in continuous time; uniformization; more on finite Markov chains; absorbing Markov chains in continuous time; calculation of transition probability functions; stochastic monotonicity; semi-Markov processes; exercises. Birth-death processes: Boundary classifications; birth-death polynomials; finite birth-death processes; the Karlin-McGregor representation theorem; asymptotics of birth-death polynomials; quasi-stationary distributions; the decay parameter; the M/M/1 queue; exercises. Appendix A Review of matrix theory: Nonnegative matrices; ML-matrices; infinite matrices. Appendix B Generating functions and Laplace transforms: Generating functions; Laplace transforms. Appendix C Total positivity: TP functions; the variation-diminishing property.

View More



Kijima, Masaaki