1st Edition

Stochastic Processes From Applications to Theory

By Pierre Del Moral, Spiridon Penev Copyright 2014
    916 Pages 124 Color Illustrations
    by Chapman & Hall

    916 Pages 124 Color Illustrations
    by Chapman & Hall

    Unlike traditional books presenting stochastic processes in an academic way, this book includes concrete applications that students will find interesting such as gambling, finance, physics, signal processing, statistics, fractals, and biology. Written with an important illustrated guide in the beginning, it contains many illustrations, photos and pictures, along with several website links. Computational tools such as simulation and Monte Carlo methods are included as well as complete toolboxes for both traditional and new computational techniques.


    An illustrated guide

    Motivating examples

    Lost in the Great Sloan Wall

    Meeting Alice in Wonderland

    The lucky MIT Blackjack team

    The Kruskal's magic trap card

    The magic fern from Daisetsuzan

    The Kepler-22b Eve

    Poisson's typos


    Selected topics

    Stabilizing populations

    The traps of Reinforcement

    Casino roulette

    Surfing Google's waves

    Pinging hackers


    Computational & theoretical aspects

    From Monte Carlo to Los Alamos

    Signal processing & Population dynamics

    The lost equation

    Towards a general theory

    The theory of speculation


    Stochastic simulation

    Simulation toolbox

    Inversion technique

    Change of variables

    Rejection techniques

    Sampling probabilities

    Bayesian inference

    Laplace's rule of successions

    Fragmentation and coagulation

    Conditional probabilities

    Bayes' formula

    The regression formula

    Gaussian updates

    Conjugate priors

    Spatial Poisson point processes

    Some preliminary results

    Conditioning principles

    Poisson-Gaussian clusters


    Monte Carlo integration

    Law of large numbers

    Importance sampling

    Twisted distributions

    Sequential Monte Carlo

    Tails distributions


    Some illustrations

    Stochastic processes

    Markov chain models

    Black-box type models

    Boltzmann-Gibbs measures

    The Ising model

    The Sherrington-Kirkpatrick model

    The traveling salesman model

    Filtering & Statistical learning

    The Bayes formula

    The Singer's radar model


    Discrete time processes

    Markov chains

    Description of the models

    Elementary transitions

    Markov integral operators

    Equilibrium measures

    Stochastic matrices

    Random dynamical systems

    Linear Markov chain model

    Two states Markov models

    Transition diagrams

    The tree of outcomes

    General state space models

    Nonlinear Markov chains

    Self-interacting processes

    Mean field particle models

    McKean-Vlasov diffusions

    Interacting jump processes


    Analysis toolbox

    Linear algebra

    Diagonalisation type techniques

    The Perron Frobenius theorem

    Functional analysis

    Spectral decompositions

    Total variation norms

    Contraction inequalities

    The Poisson equation


    Geometric drift conditions

    V -norm contractions

    Stochastic analysis

    Coupling techniques

    The total variation distance

    The Wasserstein metric

    Stopping times and coupling

    Strong stationary times

    Some illustrations

    Minorization condition and coupling

    Markov chains on complete graphs

    Kruskal random walk


    Some preliminaries

    Applications to Markov chains

    Martingales with Fixed terminal values

    A Doeblin-Ito formula

    Occupation measures

    Optional stopping theorems

    A gambling model

    Fair games

    Unfair games

    Maximal inequalities

    Limit theorems

    Topological aspects

    Irreducibility and aperiodicity

    Recurrent and transient states

    Continuous state spaces

    Path space models


    Computational toolbox

    A weak ergodic theorem

    Some illustrations

    Parameter estimation

    A Gaussian subset shaker

    Exploration of the unit disk

    Markov Chain Monte Carlo methods


    Metropolis and Hastings models

    Gibbs-Glauber dynamics

    The Propp and Wilson sampler

    Time inhomogeneous MCMC models

    Simulated annealing algorithm

    A perfect sampling algorithm

    Feynman-Kac path integration

    Weighted Markov chains

    Evolution equations

    Particle absorption models

    Doob h-processes

    Quasi-invariant measures

    Cauchy problems with terminal conditions

    Dirichlet-Poisson problems

    Cauchy-Dirichlet-Poisson problems

    Feynman-Kac particle methodology

    Mean field genetic type particle models

    Path space models

    Backward integration

    A random particle matrix model

    A conditional formula for ancestral trees

    Particle Markov Chain Monte Carlo methods

    Many-body Feynman-Kac measures

    A particle Metropolis-Hastings model

    Duality formulae for many-body models

    A couple of particle Gibbs samplers

    Quenched and annealed measures

    Feynman-Kac models

    Particle Gibbs models

    Particle Metropolis-Hastings models

    Some application domains

    Interacting MCMC algorithms

    Nonlinear Filtering models

    Markov chain restrictions

    Self-avoiding walks

    Importance twisted measures

    Kalman-Bucy Filters

    Forward Filters

    Backward Filters

    Ensemble Kalman Filters

    Interacting Kalman Filters


    Continuous time processes

    Poisson processes

    A counting process

    Memoryless property

    Uniform random times

    The Doeblin-Ito formula

    The Bernoulli process

    Time inhomogeneous models

    Description of the models

    Poisson thinning simulation

    Geometric random clocks


    Markov chain embeddings

    Homogeneous embeddings

    Description of the models

    Semigroup evolution equations

    Some illustrations

    A two states Markov process

    Matrix valued equations

    Discrete Laplacian

    Spatially inhomogeneous models

    Explosion phenomenon

    Finite state space models

    Time in homogenous models

    Description of the models

    Poisson thinning models

    Exponential and geometric clocks


    Jump processes

    A class of pure jump models

    Semigroup evolution equations

    Approximation schemes

    Sum of generators

    Doob-Meyer decompositions

    Discrete time models

    Continuous time martingales

    Optional stopping theorems

    Doeblin-Ito-Taylor formulae

    Stability properties

    Invariant measures

    Dobrushin contraction properties


    Piecewise deterministic processes

    Dynamical systems basics

    Semigroup and flow maps

    Time discretization schemes

    Piecewise deterministic jump models

    Excursion valued Markov chains

    Evolution semigroups

    Infinitesimal generators

    The Fokker-Planck equation

    A time discretization scheme

    Doeblin-Ito-Taylor formulae

    Stability properties

    Switching processes

    Invariant measures

    An application to Internet architectures

    The Transmission Control Protocol

    Regularity and stability properties

    The limiting distribution


    Diffusion processes

    Brownian motion

    Discrete vs continuous time models

    Evolution semigroups

    The heat equation

    A Doeblin-Ito-Taylor formula

    Stochastic differential equations

    Diffusion processes

    The Doeblin-Ito differential calculus

    Evolution equations

    The Fokker-Planck equation

    Weak approximation processes

    A backward stochastic differential equation

    Multidimensional diffusions

    Multidimensional stochastic differential equations

    An integration by parts formula

    Laplacian and Orthogonal transformations

    The Fokker-Planck equation


    Jump diffusion processes

    Piecewise diffusion processes

    Evolution semigroups

    The Doeblin-Ito formula

    The Fokker-Planck equation

    An abstract class of stochastic processes

    Generators and carré du champ operators

    Perturbation formulae

    Jump-diffusion processes with killing

    Feynman-Kac semigroups

    Cauchy problems with terminal conditions

    Dirichlet-Poisson problems

    Cauchy-Dirichlet-Poisson problems

    Some illustrations

    1-dimensional Dirichlet-Poisson problems

    A backward stochastic differential equation


    Nonlinear jump diffusion processes

    Nonlinear Markov processes

    Pure diffusion models

    The Burgers equation

    Feynman-Kac jump type models

    A jump type Langevin model

    Mean field particle models

    Some application domains

    Fouque-Sun systemic risk model

    Burgers equation

    A Langevin-McKean-Vlasov model

    The Dyson equation


    Stochastic analysis toolbox

    Time changes

    Stability properties

    Some illustrations

    Gradient flow processes

    1-dimensional diffusions

    Foster-Lyapunov techniques

    Contraction inequalities

    Minorization properties

    Some applications

    Ornstein-Uhlenbeck processes

    Stochastic gradient processes

    Langevin diffusions

    Spectral analysis

    Hilbert spaces and Schauder bases

    Spectral decompositions

    Poincaré inequality


    Path space measures

    Pure jump models

    Likelihood functionals

    Girsanov's transformations

    Exponential martingales

    Diffusion models

    The Wiener measure

    Path space diffusions

    Girsanov transformations

    Exponential change twisted measures

    Diffusion processes

    Pure jump processes

    Some illustrations

    Risk neutral Financial markets

    Poisson markets

    Diffusion markets

    Elliptic diffusions

    Nonlinear filtering

    Diffusion observations

    Duncan-Zakai equation

    Kushner-Stratonovitch equation

    Kalman-Bucy Filters

    Nonlinear diffusion and Ensemble Kalman-Bucy Filters

    Robust Filtering equations

    Poisson observations


    Processes on manifolds

    A review of differential geometry

    Projection operators

    Covariant derivatives of vector fields

    First order derivatives

    Second order derivatives

    Divergence and mean curvature

    Lie brackets and commutation formulae

    Inner product derivation formulae

    Second order derivatives and some trace formulae

    The Laplacian operator

    Ricci curvature

    Bochner-Lichnerowicz formula


    Stochastic differential calculus on manifolds

    Embedded manifolds

    Brownian motion on manifolds

    A diffusion model in the ambient space

    The infinitesimal generator

    Monte Carlo simulation

    Stratonovitch differential calculus

    Projected diffusions on manifolds

    Brownian motion on orbifolds


    Parameterizations and charts

    Differentiable manifolds and charts

    Orthogonal projection operators

    Riemannian structures

    First order covariant derivatives

    Pushed forward functions

    Pushed forward vector fields

    Directional derivatives

    Second order covariant derivative

    Tangent basis functions

    Composition formulae

    Hessian operators

    Bochner-Lichnerowicz formula


    Stochastic calculus in chart spaces

    Brownian motion on Riemannian manifolds

    Diffusions on chart spaces

    Brownian motion on spheres

    The unit circle S = S1 _ R2

    The unit sphere S = S2 _ R3

    Brownian motion on the Torus

    Diffusions on the simplex


    Some analytical aspects

    Geodesics and the exponential map

    A Taylor expansion

    Integration on manifolds

    The volume measure on the manifold

    Wedge product and volume forms

    The divergence theorem

    Gradient flow models

    Steepest descent model

    Euclidian state spaces

    Drift changes and irreversible Langevin diffusions

    Langevin diffusions on closed manifolds

    Riemannian Langevin diffusions

    Metropolis-adjusted Langevin models

    Stability and some functional inequalities


    Some illustrations

    Prototype manifolds

    The Circle

    The 2-Sphere

    The Torus

    Information theory

    Nash embedding theorem

    Distribution manifolds

    Bayesian statistical manifolds

    The Cramer-Rao lower bound

    Some illustrations

    Boltzmann-Gibbs measures

    Multivariate normal distributions

    Some application areas

    Simple random walks

    Random walk on lattices


    Dimension 1

    Dimension 2

    Dimension d > 3

    Random walks on graphs

    Simple exclusion process

    Random walks on the circle

    Markov chain on cycles

    Markov chain on the circle

    Spectral decomposition

    Random walk on hypercubes


    A macroscopic model

    A lazy random walk

    Urn processes

    Ehrenfest model

    Pólya urn model


    Iterated random functions


    A motivating example

    Uniform selection

    An ancestral type evolution model

    An absorbed Markov chain

    Shuffling cards


    The top-in-at random shuffle

    The random transposition shuffle

    The riffle shuffle

    Fractal models

    Exploration of Cantor's discontinuum

    Some fractal images


    Computational & Statistical physics

    Molecular dynamics simulation

    Newton's second law of motion

    Langevin diffusion processes

    The Schrödinger equation

    A physical derivation

    A Feynman-Kac formulation

    Bra-kets and path integral formalism

    Spectral decompositions

    The harmonic oscillator

    Diffusion Monte Carlo models

    Interacting particle systems


    Contact process

    Voter process

    Exclusion process


    Dynamic population models

    Discrete time birth and death models

    Continuous time models

    Birth and death generators

    Logistic processes

    Epidemic model with immunity

    Lotka-Volterra predator-prey stochastic model

    The Moran genetic model

    Genetic evolution models

    Branching processes

    Birth and death models with linear rates

    Discrete time branching

    Continuous time branching processes

    Absorption - death process

    Birth type branching process

    Birth and death branching processes

    Kolmogorov-Petrovskii-Piskunov equations


    Gambling, ranking and control

    The Google page rank

    Gambling betting systems

    Martingale systems

    St. Petersburg martingales

    Conditional gains and losses

    Conditional gains

    Conditional losses

    Bankroll managements

    The Grand Martingale

    The D'Alembert Martingale

    The Whittacker Martingale

    Stochastic optimal control

    Bellman equations

    Control dependent value functions

    Continuous time models

    Optimal stopping

    Games with Fixed terminal condition

    Snell envelope

    Continuous time models


    Mathematical finance

    Stock price models

    Up and down martingales

    Cox-Ross-Rubinstein model

    Black-Scholes-Merton model

    European option pricing

    Call and Put options

    Self-financing portfolios

    Binomial pricing technique

    Black-Scholes-Merton pricing model

    The Black-Scholes partial differential equation

    Replicating portfolios

    Option price and hedging computations

    A numerical illustration





    Pierre Del Moral and Spiridon Penev are professors in the School of Mathematics and Statistics at the University of New South Wales.

    "The title itself suggests that the reader should expect something different, applications to theory and not theory to applications. The title is correct, and that is the main theme of the book. Start with some general applications, and then build the theory around them. The range of applications and the depth of the discussions are impressive." (Igor Cialenco, Illinois Institute of Technology)

    "This is a great reference… It lays out a lot of calculations in simple and direct ways. If you go through this book as a first-year grad student, you will understand lots of material and be prepared for many things." (Richard Sowers, University of Illinois at Urbana-Champaign)

    "(This book makes) theoretical tools developed in the stochastic analysis/probability community available to a significant community of applied mathematicians. As such, it should be highly successful, as it is well written and clear." (John Fricks, The Pennsylvania State University)