Modeling and Analysis of Stochastic Systems: 3rd Edition (Hardback) book cover

Modeling and Analysis of Stochastic Systems

3rd Edition

By Vidyadhar G. Kulkarni

Chapman and Hall/CRC

584 pages | 59 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781498756617
pub: 2016-10-07
SAVE ~$26.00
eBook (VitalSource) : 9781315367910
pub: 2016-11-18
from $65.00

FREE Standard Shipping!


Building on the author’s more than 35 years of teaching experience, Modeling and Analysis of Stochastic Systems, Third Edition, covers the most important classes of stochastic processes used in the modeling of diverse systems. For each class of stochastic process, the text includes its definition, characterization, applications, transient and limiting behavior, first passage times, and cost/reward models.

The third edition has been updated with several new applications, including the Google search algorithm in discrete time Markov chains, several examples from health care and finance in continuous time Markov chains, and square root staffing rule in Queuing models. More than 50 new exercises have been added to enhance its use as a course text or for self-study. The sequence of chapters and exercises has been maintained between editions, to enable those now teaching from the second edition to use the third edition.

Rather than offer special tricks that work in specific problems, this book provides thorough coverage of general tools that enable the solution and analysis of stochastic models. After mastering the material in the text, readers will be well-equipped to build and analyze useful stochastic models for real-life situations.


"The third edition of Modeling and Analysis of Stochastic Systems remains an excellent book for a graduate-level study of stochastic processes. The aim of the book is modeling with stochastic elements in practical settings and analysis of the resulting stochastic model. The target audience is quantitative disciplines such as operations research, statistics, computer science, economics, and others where the book is well positioned, since it is application-driven and does not require measure theoretic probability. … The numerous exercises, separated into modeling, computational, and conceptual classes, are a strength of this text. The author also notes that most come from his own homework and exams, making them a valuable resource."

—James M. Flegal, University of California, Riverside, in Journal of the American Statistical Association, January 2018

Table of Contents


What in the World is a Stochastic Process?

How to Characterize a Stochastic Process

What Do We Do with a Stochastic Process?

Discrete-Time Markov Chains: Transient Behaviour

Definition and Characterization


DTMCs in Other Fields

Marginal Distributions

Occupancy Times

Computation of Matrix Powers

Modeling Exercises

Computational Exercises

Conceptual Exercises

Discrete-Time Markov Chains: First Passage Times


Cumulative Distribution Function of T

Absorption Probabilities

Expectation of T

Generating Function and Higher Moments of T

Computational Exercises

Conceptual Exercises

Discrete-Time Markox Chains: Limiting Behaviour

Exploring the Limiting Behaviour by Examples

Classification of States

Determining Recurrence and Transience: Finite DTMCs

Determining Recurrence and Transience: Infinite DTMSc

Limiting Behaviour of Irreducible DTMCs

Examples: Limiting Behaviour of Infinite State-Space Irreducible DTMCs

Limiting Behaviour of Reducible DTMCs

DTMCs with Costs and Rewards


Computational Exercises

Conceptual Exercises

Poisson Processes

Exponential Distributions

Poisson Process: Definitions

Event Times in a Poisson Process

Superposition and Splitting of Poisson Processes

Non-Homogeneous Poisson Process

Compound Poisson Process

Computational Exercises

Conceptual Exercises

Continuous-Time Markov Chains

Definitions and Sample Path Properties


CTMCs in Other Fields

Transient Behaviour: Marginal Distribution

Transient Behaviour: Occupancy Times

Computation of P(t): Finite State-Space

Computation of P(t): Infinite State-Space

First-Passage Times

Exploring the Limiting Behaviour by Examples

Classification of States

Limiting Behaviour of Irreducible CTMCs

Limiting Behaviour of Reducible CTMCs

CTMCs with Costs and Rewards

Phase Type Distributions


Modeling Exercises

Computational Exercises

Conceptual Exercises

Queueing Models


Properties of General Queueing Systems

Birth and Death Queues

Open Queueing Networks

Closed Queueing Networks

Single Server Queues

Retrial Queue

Infinite Server Queue

Modeling Exercises

Computational Exercises

Renewal Processes


Properties of N(t)

The Renewal Function

Renewal-Type Equation

Key Renewal Theorem

Recurrence Times

Delayed Renewal Processes

Semi-Markov Processes

Renewal Processes with Costs/Rewards

Regenerative Processes

Computational Exercises

Conceptual Exercises

Markov Regenerative Processes

Definitions and Examples

Markov Renewal Process and Markov Renewal Function

Key Renewal Theorem for MRPs

Semi-Markov Processes: Further Results

Markov Regenerative Processes

Applications to Queues

Modeling Exercises

Computational Exercises

Conceptual Exercises

Diffusion Process

Brownian Motion

Sample Path Properties of BM

Kolmogorov Equations for Standard Brownian Motion

First Passage Times

Reflected SBM

Reflected BM and Limiting Distributions

BM and Martingales

Cost/Reward Models

Stochastic Integration

Stochastic Differential Equations and Ito's Formula

Applications to Finance

Computational Exercises

Conceptual Exercises


Probability of Events


Univariate Random Variables

Multivariate Random Variables

Generating Functions

Laplace-Stieltjes Transforms

Laplace Transforms

Modes of Convergence

Results from Analysis

Difference and Differential Equations

Answers to Selected Problems



About the Series

Chapman & Hall/CRC Texts in Statistical Science

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
BUSINESS & ECONOMICS / Operations Research
MATHEMATICS / Probability & Statistics / General