Preface
1. Review of Probability
Short History
Review of Basic Probability Definitions
Some Common Probability Distributions
Bernoulli Distribution
Binomial Distribution
Geometric Distribution
Negative Binomial Distribution
Poisson Distribution
Properties of a Probability Distribution
Conditional Probability
Independent Events
Random Variables
Expected Value of a Random Variable
Properties of the Expected Value
Expected Value of a Random Variable with Common Distributions
Bernoulli Distribution
Binomial Distribution
Geometric Distribution
Negative Binomial Distribution
Poisson Distribution
Functions of a Random Variable
Joint Distributions
Generating Functions
Probability Generating Functions
Moment Generating Functions
Exercises
2. Discrete-Time, Finite-State Markov Chains
Introduction
Notation
Transition Matrices
Directed Graphs: Examples of Markov Chains
Random Walk with Reflecting Boundaries
Gambler’s Ruin
Ehrenfest Model
Central Problem of Markov Chains
Condition to Ensure a Unique Equilibrium State
Finding the Equilibrium State
Transient and Recurrent States
Indicator Functions
Perron–Frobenius Theorem
Absorbing Markov Chains
Mean First Passage Time
Mean Recurrence Time and the Equilibrium State
Fundamental Matrix for Regular Markov Chains
Dividing a Markov Chain into Equivalence Classes
Periodic Markov Chains
Reducible Markov Chains
Summary
Exercises
3. Discrete-Time, Infinite-State Markov Chains
Renewal Processes
Delayed Renewal Processes
Equilibrium State for Countable Markov Chains
Physical Interpretation of the Equilibrium State
Null Recurrent versus Positive Recurrent States
Difference Equations
Branching Processes
Random Walk in Zd
Exercises
4. Exponential Distribution and Poisson Process
Continuous Random Variables
Cumulative Distribution Function (Continuous Case)
Exponential Distribution
o(h) Functions
Exponential Distribution as a Model for Arrivals
Memoryless Random Variables
Poisson Process
Poisson Processes with Occurrences of Two Types
Exercises
5. Continuous Time Markov Chains
Introduction
Generators of Continuous Markov Chains: The Kolmogorov
Forward and Backward Equations
Connections of the Infinitesimal Generator, the Embedded
Markov Chain, Transition Rates, and the Stationary Distribution
Connection between the Steady State of a Continuous Markov
Chain and the Steady State of the Embedded Matrix
Explosions
Birth and Birth–Death Processes
Birth Process (Forward Equation)
Birth Process (Backward Equation)
Birth and Death Processes
Forward Equations for Birth–Death Processes
Birth–Death Processes: Backward Equations
Recurrence and Transience in Birth–Death Processes
6. Queuing Models and Detailed Balance Equations
M/M/1 Model
M/M/1/K Queue: One Server (Size of the Queue Is Limited)
Detailed Balance Equations
M/M/K Model
M/M/c/K Queue: c Servers (Size of the Queue Is Limited to K)
M/M/∞ Model
Exercises
7. Reversible Markov Chains
Random Walks on Weighted Graphs
Discrete-Time Birth–Death Process as a Reversible Markov Chain
Continuous-Time Reversible Markov Chains
Exercises
8.Digraphs
Exercises
Glossary of Terms
Bibliography
Index
Biography
James R. Kirkwood holds a Ph.D. from the University of Virginia. He has had ten mathematics textbooks published on various topics including calculus, real analysis, mathematical biology and mathematical physics. His original research was in mathematical physics, and he co-authored the seminal paper in a topic now called Kirkwood-Thomas Theory in mathematical physics. During the summer, he teaches real analysis to graduate students at the University of Virginia. He has been awarded several National Science Foundation grants. Dr. Kirkwood’s books for CRC Press include, An Introduction to Analysis, third edition ©2024; A Transition to Advanced Mathematics (with Raina S. Robeva) ©2024; Linear Algebra (with Bessie H. Kirkwood) ©2024; Elementary Linear Algebra (with Bessie H. Kirkwood) ©2023.
