Principles of Uncertainty

By Joseph B. Kadane

© 2011 – Chapman and Hall/CRC

504 pages | 28 B/W Illus.

Purchasing Options:
Hardback: 9781439861615
pub: 2011-05-17
US Dollars$93.95

e–Inspection Copy

About the Book

An intuitive and mathematical introduction to subjective probability and Bayesian statistics.

An accessible, comprehensive guide to the theory of Bayesian statistics, Principles of Uncertainty presents the subjective Bayesian approach, which has played a pivotal role in game theory, economics, and the recent boom in Markov Chain Monte Carlo methods. Both rigorous and friendly, the book contains:

  • Introductory chapters examining each new concept or assumption
  • Just-in-time mathematics – the presentation of ideas just before they are applied
  • Summary and exercises at the end of each chapter
  • Discussion of maximization of expected utility
  • The basics of Markov Chain Monte Carlo computing techniques
  • Problems involving more than one decision-maker

Written in an appealing, inviting style, and packed with interesting examples, Principles of Uncertainty introduces the most compelling parts of mathematics, computing, and philosophy as they bear on statistics. Although many books present the computation of a variety of statistics and algorithms while barely skimming the philosophical ramifications of subjective probability, this book takes a different tack. By addressing how to think about uncertainty, this book gives readers the intuition and understanding required to choose a particular method for a particular purpose.


"… it is a book about Bayesian probability, statistics, and decision making. … an excellent choice. … The exercises at the end of each chapter are well conceived and in this way most useful for checking understanding of the text. Typos and errata are rare in this book. … recommended to statisticians (without restriction of any kind), statistics students at a graduate or PhD level, and empirical researchers in general … a sound basis for a course in Bayesian probability, statistics, and decision making."

Biometrical Journal, 2013

This book mainly focuses on the use of Bayesian statistics. It is written using stories and many examples to which readers can relate, and is thus an engaging and appealing text on what is generally a very dry mathematical subject.

—John J. Shea, IEEE Electrical Insulation Magazine, May/June 2013, Vol. 29, No. 3

This text provides a unique blend of theory, methods, philosophy and applications that is suitable for a course in Bayesian probability and statistics. … provides thought-provoking material for teaching. …

—Erkki P. Liski, International Statistical Review, 2012

In this remarkable book, Kadane begins at the most rudimentary level, develops all the needed mathematics on the fly, and still manages to flesh out at least the core of the whole story, slowly, thoughtfully, and rigorously, right up to graduate level. Major theorems all proved in detail appear here, but not for their own sake; the author always carefully selects them to clarify the basic meaning of the subject and his own views concerning the pitfalls and subtleties of its proper application. Summing Up: Highly recommended.

—D.V. Feldman, CHOICE, February 2012

Principles of Uncertainty is a profound and mesmerising book on the foundations and principles of subjectivist or behaviouristic Bayesian analysis. … the book is a pleasure to read. And highly recommended for teaching as it can be used at many different levels. … A must-read for sure!

—Christian Robert, The Statistics Forum/CHANCE, October 2011

It's a lovely book, one that I hope will be widely adopted as a course textbook.

—Michael Jordan, University of California, Berkeley, USA

A careful, complete, and lovingly written exposition of the subjective Bayesian viewpoint by one of its most eloquent and staunch defenders. Summarizes a lifetime of theory, methods, and application developments for the Bayesian inferential engine. A must-read for anyone looking for a deep understanding of the foundations of Bayesian methods and what they offer modern statistical practice.

—Bradley P. Carlin, Professor and Head of Division of Biostatistics, University of Minnesota, Minneapolis, USA

Table of Contents


Avoiding being a sure loser

Disjoint events

Events not necessarily disjoint

Random variables, also known as uncertain quantities

Finite number of values

Other properties of expectation

Coherence implies not a sure loser

Expectations and limits

Conditional Probability and Bayes Theorem

Conditional probability

The Birthday Problem

Simpson's Paradox

Bayes Theorem

Independence of events

The Monty Hall problem

Gambler's Ruin problem

Iterated Expectations and Independence

The binomial and multinomial distributions

Sampling without replacement

Variance and covariance

A short introduction to multivariate thinking

Tchebychev's inequality

Discrete Random Variables

Countably many possible values

Finite additivity

Countable Additivity

Properties of countable additivity

Dynamic sure loss

Probability generating functions

Geometric random variables

The negative binomial random variable

The Poisson random variable

Cumulative distribution function

Dominated and bounded convergence

Continuous Random Variables


Joint distributions

Conditional distributions and independence

Existence and properties of expectations


An interesting relationship between cdf's and expectations of continuous random variables

Chapter retrospective so far

Bounded and dominated convergence

The Riemann-Stieltjes integral

The McShane-Stieltjes Integral

The road from here

The strong law of large numbers



Discrete Random Variables

Univariate Continuous Distributions

Linear spaces


Number systems; DeMoivre's formula


Eigenvalues, eigenvectors and decompositions

Non-linear transformations

The Borel-Kolmogorov paradox

Normal Distribution


Moment generating functions

Characteristic functions

Trigonometric Polynomials

A Weierstrass approximation theorem

Uniqueness of characteristic functions

Characteristic function and moments

Continuity Theorem

The Normal distribution

Multivariate normal distributions

Limit theorems

Making Decisions


An example

In greater generality

The St. Petersburg Paradox

Risk aversion

Log (fortune) as utility

Decisions after seeing data

The expected value of sample information

An example

Randomized decisions

Sequential decisions

Conjugate Analysis

A simple normal-normal case

A multivariate normal case, known precision

The normal linear model with known precision

The gamma distribution

Uncertain Mean and Precision

The normal linear model, uncertain precision

The Wishart distribution

Both mean and precision matrix uncertain

The beta and Dirichlet distributions

The exponential family

Large sample theory for Bayesians

Some general perspective

Hierarchical Structuring of a Model


Missing data


Model uncertainty/model choice

Graphical Hierarchical Models


Markov Chain Monte Carlo



The Metropolis Hasting Algorithm

Extensions and special cases

Practical considerations

Variable dimensions: Reversible jumps

Multiparty Problems

A simple three-stage game

Private information

Design for another's analysis

Optimal Bayesian Randomization

Simultaneous moves

The Allais and Ellsberg paradoxes

Forming a Bayesian group

Exploration of Old Ideas



Confidence intervals and sets


Choosing among models

Goodness of fit

Sampling theory statistics

Objective" Bayesian Methods

Epilogue: Applications


A final thought

About the Author

Joseph B. Kadane, Department of Statistics, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

About the Series

Chapman & Hall/CRC Texts in Statistical Science

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
MATHEMATICS / Probability & Statistics / General
MATHEMATICS / Probability & Statistics / Bayesian Analysis