Markov Chains and Decision Processes for Engineers and Managers: 1st Edition (Hardback) book cover

Markov Chains and Decision Processes for Engineers and Managers

1st Edition

By Theodore J. Sheskin

CRC Press

492 pages | 24 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781420051117
pub: 2010-11-08
$130.00
x
eBook (VitalSource) : 9780429137143
pub: 2016-04-19
from $28.98


FREE Standard Shipping!

Description

Recognized as a powerful tool for dealing with uncertainty, Markov modeling can enhance your ability to analyze complex production and service systems. However, most books on Markov chains or decision processes are often either highly theoretical, with few examples, or highly prescriptive, with little justification for the steps of the algorithms used to solve Markov models. Providing a unified treatment of Markov chains and Markov decision processes in a single volume, Markov Chains and Decision Processes for Engineers and Managers supplies a highly detailed description of the construction and solution of Markov models that facilitates their application to diverse processes.

Organized around Markov chain structure, the book begins with descriptions of Markov chain states, transitions, structure, and models, and then discusses steady state distributions and passage to a target state in a regular Markov chain. The author treats canonical forms and passage to target states or to classes of target states for reducible Markov chains. He adds an economic dimension by associating rewards with states, thereby linking a Markov chain to a Markov decision process, and then adds decisions to create a Markov decision process, enabling an analyst to choose among alternative Markov chains with rewards so as to maximize expected rewards. An introduction to state reduction and hidden Markov chains rounds out the coverage.

In a presentation that balances algorithms and applications, the author provides explanations of the logical relationships that underpin the formulas or algorithms through informal derivations, and devotes considerable attention to the construction of Markov models. He constructs simplified Markov models for a wide assortment of processes such as the weather, gambling, diffusion of gases, a waiting line, inventory, component replacement, machine maintenance, selling a stock, a charge account, a career path, patient flow in a hospital, marketing, and a production line. This treatment helps you harness the power of Markov modeling and apply it to your organization’s processes.

Solutions manual is available upon qualifying course adoption.

Table of Contents

Markov Chain Structure and Models

Historical Note

States and Transitions

Model of the Weather

Random Walks

Estimating Transition Probabilities

Multiple-Step Transition Probabilities

State Probabilities after Multiple Steps

Classification of States

Markov Chain Structure

Markov Chain Models

Problems

References

Regular Markov Chains

Steady State Probabilities

First Passage to a Target State

Problems

References

Reducible Markov Chains

Canonical Form of the Transition Matrix

The Fundamental Matrix

Passage to a Target State

Eventual Passage to a Closed Set Within a Reducible Multichain

Limiting Transition Probability Matrix

Problems

References

A Markov Chain with Rewards (MCR)

Rewards

Undiscounted Rewards

Discounted Rewards

Problems

References

A Markov Decision Process (MDP)

An Undiscounted MDP

A Discounted MDP

Problems

References

Special Topics: State Reduction and Hidden Markov Chains

State Reduction

An Introduction to Hidden Markov

Problems

References

Index

About the Originator

Subject Categories

BISAC Subject Codes/Headings:
MAT029010
MATHEMATICS / Probability & Statistics / Bayesian Analysis
TEC016000
TECHNOLOGY & ENGINEERING / Industrial Design / General
TEC029000
TECHNOLOGY & ENGINEERING / Operations Research