1st Edition

# Stochastic Processes From Applications to Theory

**Also available as eBook on:**

Unlike traditional books presenting stochastic processes in an academic way, this book includes concrete applications that students will find interesting such as gambling, finance, physics, signal processing, statistics, fractals, and biology. Written with an important illustrated guide in the beginning, it contains many illustrations, photos and pictures, along with several website links. Computational tools such as simulation and Monte Carlo methods are included as well as complete toolboxes for both traditional and new computational techniques.

**An illustrated guide**

*Motivating examples*

Lost in the Great Sloan Wall

Meeting Alice in Wonderland

The lucky MIT Blackjack team

The Kruskal's magic trap card

The magic fern from Daisetsuzan

The Kepler-22b Eve

Poisson's typos

Exercises

*Selected topics*

Stabilizing populations

The traps of Reinforcement

Casino roulette

Surfing Google's waves

Pinging hackers

Exercises

*Computational & theoretical aspects*

From Monte Carlo to Los Alamos

Signal processing & Population dynamics

The lost equation

Towards a general theory

The theory of speculation

Exercises

**Stochastic simulation **

*Simulation toolbox*

Inversion technique

Change of variables

Rejection techniques

Sampling probabilities

Bayesian inference

Laplace's rule of successions

Fragmentation and coagulation

Conditional probabilities

Bayes' formula

The regression formula

Gaussian updates

Conjugate priors

Spatial Poisson point processes

Some preliminary results

Conditioning principles

Poisson-Gaussian clusters

Exercises

*Monte Carlo integration*

Law of large numbers

Importance sampling

Twisted distributions

Sequential Monte Carlo

Tails distributions

Exercises

*Some illustrations*

Stochastic processes

Markov chain models

Black-box type models

Boltzmann-Gibbs measures

The Ising model

The Sherrington-Kirkpatrick model

The traveling salesman model

Filtering & Statistical learning

The Bayes formula

The Singer's radar model

Exercises

**Discrete time processes **

*Markov chains*

Description of the models

Elementary transitions

Markov integral operators

Equilibrium measures

Stochastic matrices

Random dynamical systems

Linear Markov chain model

Two states Markov models

Transition diagrams

The tree of outcomes

General state space models

Nonlinear Markov chains

Self-interacting processes

Mean field particle models

McKean-Vlasov diffusions

Interacting jump processes

Exercises

**Analysis toolbox**

Linear algebra

Diagonalisation type techniques

The Perron Frobenius theorem

Functional analysis

Spectral decompositions

Total variation norms

Contraction inequalities

The Poisson equation

V-norms

Geometric drift conditions

V -norm contractions

Stochastic analysis

Coupling techniques

The total variation distance

The Wasserstein metric

Stopping times and coupling

Strong stationary times

Some illustrations

Minorization condition and coupling

Markov chains on complete graphs

Kruskal random walk

Martingales

Some preliminaries

Applications to Markov chains

Martingales with Fixed terminal values

A Doeblin-Ito formula

Occupation measures

Optional stopping theorems

A gambling model

Fair games

Unfair games

Maximal inequalities

Limit theorems

Topological aspects

Irreducibility and aperiodicity

Recurrent and transient states

Continuous state spaces

Path space models

Exercises

*Computational toolbox*

A weak ergodic theorem

Some illustrations

Parameter estimation

A Gaussian subset shaker

Exploration of the unit disk

Markov Chain Monte Carlo methods

Introduction

Metropolis and Hastings models

Gibbs-Glauber dynamics

The Propp and Wilson sampler

Time inhomogeneous MCMC models

Simulated annealing algorithm

A perfect sampling algorithm

Feynman-Kac path integration

Weighted Markov chains

Evolution equations

Particle absorption models

Doob h-processes

Quasi-invariant measures

Cauchy problems with terminal conditions

Dirichlet-Poisson problems

Cauchy-Dirichlet-Poisson problems

Feynman-Kac particle methodology

Mean field genetic type particle models

Path space models

Backward integration

A random particle matrix model

A conditional formula for ancestral trees

Particle Markov Chain Monte Carlo methods

Many-body Feynman-Kac measures

A particle Metropolis-Hastings model

Duality formulae for many-body models

A couple of particle Gibbs samplers

Quenched and annealed measures

Feynman-Kac models

Particle Gibbs models

Particle Metropolis-Hastings models

Some application domains

Interacting MCMC algorithms

Nonlinear Filtering models

Markov chain restrictions

Self-avoiding walks

Importance twisted measures

Kalman-Bucy Filters

Forward Filters

Backward Filters

Ensemble Kalman Filters

Interacting Kalman Filters

Exercises

**Continuous time processes** *Poisson processes*

A counting process

Memoryless property

Uniform random times

The Doeblin-Ito formula

The Bernoulli process

Time inhomogeneous models

Description of the models

Poisson thinning simulation

Geometric random clocks

Exercises

*Markov chain embeddings*

Homogeneous embeddings

Description of the models

Semigroup evolution equations

Some illustrations

A two states Markov process

Matrix valued equations

Discrete Laplacian

Spatially inhomogeneous models

Explosion phenomenon

Finite state space models

Time in homogenous models

Description of the models

Poisson thinning models

Exponential and geometric clocks

Exercises

*Jump processes*

A class of pure jump models

Semigroup evolution equations

Approximation schemes

Sum of generators

Doob-Meyer decompositions

Discrete time models

Continuous time martingales

Optional stopping theorems

Doeblin-Ito-Taylor formulae

Stability properties

Invariant measures

Dobrushin contraction properties

Exercises

*Piecewise deterministic processes*

Dynamical systems basics

Semigroup and flow maps

Time discretization schemes

Piecewise deterministic jump models

Excursion valued Markov chains

Evolution semigroups

Infinitesimal generators

The Fokker-Planck equation

A time discretization scheme

Doeblin-Ito-Taylor formulae

Stability properties

Switching processes

Invariant measures

An application to Internet architectures

The Transmission Control Protocol

Regularity and stability properties

The limiting distribution

Exercises

*Diffusion processes*

Brownian motion

Discrete vs continuous time models

Evolution semigroups

The heat equation

A Doeblin-Ito-Taylor formula

Stochastic differential equations

Diffusion processes

The Doeblin-Ito differential calculus

Evolution equations

The Fokker-Planck equation

Weak approximation processes

A backward stochastic differential equation

Multidimensional diffusions

Multidimensional stochastic differential equations

An integration by parts formula

Laplacian and Orthogonal transformations

The Fokker-Planck equation

Exercises

*Jump diffusion processes*

Piecewise diffusion processes

Evolution semigroups

The Doeblin-Ito formula

The Fokker-Planck equation

An abstract class of stochastic processes

Generators and carré du champ operators

Perturbation formulae

Jump-diffusion processes with killing

Feynman-Kac semigroups

Cauchy problems with terminal conditions

Dirichlet-Poisson problems

Cauchy-Dirichlet-Poisson problems

Some illustrations

1-dimensional Dirichlet-Poisson problems

A backward stochastic differential equation

Exercises

*Nonlinear jump diffusion processes*

Nonlinear Markov processes

Pure diffusion models

The Burgers equation

Feynman-Kac jump type models

A jump type Langevin model

Mean field particle models

Some application domains

Fouque-Sun systemic risk model

Burgers equation

A Langevin-McKean-Vlasov model

The Dyson equation

Exercises

*Stochastic analysis toolbox*

Time changes

Stability properties

Some illustrations

Gradient flow processes

1-dimensional diffusions

Foster-Lyapunov techniques

Contraction inequalities

Minorization properties

Some applications

Ornstein-Uhlenbeck processes

Stochastic gradient processes

Langevin diffusions

Spectral analysis

Hilbert spaces and Schauder bases

Spectral decompositions

Poincaré inequality

Exercises

*Path space measures*

Pure jump models

Likelihood functionals

Girsanov's transformations

Exponential martingales

Diffusion models

The Wiener measure

Path space diffusions

Girsanov transformations

Exponential change twisted measures

Diffusion processes

Pure jump processes

Some illustrations

Risk neutral Financial markets

Poisson markets

Diffusion markets

Elliptic diffusions

Nonlinear filtering

Diffusion observations

Duncan-Zakai equation

Kushner-Stratonovitch equation

Kalman-Bucy Filters

Nonlinear diffusion and Ensemble Kalman-Bucy Filters

Robust Filtering equations

Poisson observations

Exercises

**Processes on manifolds**

*A review of differential geometry*

Projection operators

Covariant derivatives of vector fields

First order derivatives

Second order derivatives

Divergence and mean curvature

Lie brackets and commutation formulae

Inner product derivation formulae

Second order derivatives and some trace formulae

The Laplacian operator

Ricci curvature

Bochner-Lichnerowicz formula

Exercises

*Stochastic differential calculus on manifolds*

Embedded manifolds

Brownian motion on manifolds

A diffusion model in the ambient space

The infinitesimal generator

Monte Carlo simulation

Stratonovitch differential calculus

Projected diffusions on manifolds

Brownian motion on orbifolds

Exercises

*Parameterizations and charts*

Differentiable manifolds and charts

Orthogonal projection operators

Riemannian structures

First order covariant derivatives

Pushed forward functions

Pushed forward vector fields

Directional derivatives

Second order covariant derivative

Tangent basis functions

Composition formulae

Hessian operators

Bochner-Lichnerowicz formula

Exercises

*Stochastic calculus in chart spaces*

Brownian motion on Riemannian manifolds

Diffusions on chart spaces

Brownian motion on spheres

The unit circle S = S1 _ R2

The unit sphere S = S2 _ R3

Brownian motion on the Torus

Diffusions on the simplex

Exercises

*Some analytical aspects*

Geodesics and the exponential map

A Taylor expansion

Integration on manifolds

The volume measure on the manifold

Wedge product and volume forms

The divergence theorem

Gradient flow models

Steepest descent model

Euclidian state spaces

Drift changes and irreversible Langevin diffusions

Langevin diffusions on closed manifolds

Riemannian Langevin diffusions

Metropolis-adjusted Langevin models

Stability and some functional inequalities

Exercises

*Some illustrations*

Prototype manifolds

The Circle

The 2-Sphere

The Torus

Information theory

Nash embedding theorem

Distribution manifolds

Bayesian statistical manifolds

The Cramer-Rao lower bound

Some illustrations

Boltzmann-Gibbs measures

Multivariate normal distributions

**Some application areas**

**Simple random walks**

Random walk on lattices

Description

Dimension 1

Dimension 2

Dimension d __>__ 3

Random walks on graphs

Simple exclusion process

Random walks on the circle

Markov chain on cycles

Markov chain on the circle

Spectral decomposition

Random walk on hypercubes

Description

A macroscopic model

A lazy random walk

Urn processes

Ehrenfest model

Pólya urn model

Exercises

*Iterated random functions*

Description

A motivating example

Uniform selection

An ancestral type evolution model

An absorbed Markov chain

Shuffling cards

Introduction

The top-in-at random shuffle

The random transposition shuffle

The riffle shuffle

Fractal models

Exploration of Cantor's discontinuum

Some fractal images

Exercises

*Computational & Statistical physics*

Molecular dynamics simulation

Newton's second law of motion

Langevin diffusion processes

The Schrödinger equation

A physical derivation

A Feynman-Kac formulation

Bra-kets and path integral formalism

Spectral decompositions

The harmonic oscillator

Diffusion Monte Carlo models

Interacting particle systems

Introduction

Contact process

Voter process

Exclusion process

Exercises

*Dynamic population models*

Discrete time birth and death models

Continuous time models

Birth and death generators

Logistic processes

Epidemic model with immunity

Lotka-Volterra predator-prey stochastic model

The Moran genetic model

Genetic evolution models

Branching processes

Birth and death models with linear rates

Discrete time branching

Continuous time branching processes

Absorption - death process

Birth type branching process

Birth and death branching processes

Kolmogorov-Petrovskii-Piskunov equations

Exercises

*Gambling, ranking and control*

The Google page rank

Gambling betting systems

Martingale systems

St. Petersburg martingales

Conditional gains and losses

Conditional gains

Conditional losses

Bankroll managements

The Grand Martingale

The D'Alembert Martingale

The Whittacker Martingale

Stochastic optimal control

Bellman equations

Control dependent value functions

Continuous time models

Optimal stopping

Games with Fixed terminal condition

Snell envelope

Continuous time models

Exercises

*Mathematical finance*

Stock price models

Up and down martingales

Cox-Ross-Rubinstein model

Black-Scholes-Merton model

European option pricing

Call and Put options

Self-financing portfolios

Binomial pricing technique

Black-Scholes-Merton pricing model

The Black-Scholes partial differential equation

Replicating portfolios

Option price and hedging computations

A numerical illustration

Exercises

**Bibliography **

**Index **

### Biography

**Pierre Del Moral** and **Spiridon Penev** are professors in the School of Mathematics and Statistics at the University of New South Wales.

"The title itself suggests that the reader should expect something different, applications to theory and not theory to applications. The title is correct, and that is the main theme of the book. Start with some general applications, and then build the theory around them. The range of applications and the depth of the discussions are impressive." (Igor Cialenco, Illinois Institute of Technology)

"This is a great reference… It lays out a lot of calculations in simple and direct ways. If you go through this book as a first-year grad student, you will understand lots of material and be prepared for many things." (Richard Sowers, University of Illinois at Urbana-Champaign)

"(This book makes) theoretical tools developed in the stochastic analysis/probability community available to a significant community of applied mathematicians. As such, it should be highly successful, as it is well written and clear." (John Fricks, The Pennsylvania State University)