# Principles of Uncertainty

## Preview

## Book Description

**An intuitive and mathematical introduction to subjective probability and Bayesian statistics.**

An accessible, comprehensive guide to the theory of Bayesian statistics, **Principles of Uncertainty** presents the subjective Bayesian approach, which has played a pivotal role in game theory, economics, and the recent boom in Markov Chain Monte Carlo methods. Both rigorous and friendly, the book contains:

- Introductory chapters examining each new concept or assumption
- Just-in-time mathematics – the presentation of ideas just before they are applied
- Summary and exercises at the end of each chapter
- Discussion of maximization of expected utility
- The basics of Markov Chain Monte Carlo computing techniques
- Problems involving more than one decision-maker

Written in an appealing, inviting style, and packed with interesting examples, **Principles of Uncertainty** introduces the most compelling parts of mathematics, computing, and philosophy as they bear on statistics. Although many books present the computation of a variety of statistics and algorithms while barely skimming the philosophical ramifications of subjective probability, this book takes a different tack. By addressing how to think about uncertainty, this book gives readers the intuition and understanding required to choose a particular method for a particular purpose.

## Table of Contents

**Probability**Avoiding being a sure loser

Disjoint events

Events not necessarily disjoint

Random variables, also known as uncertain quantities

Finite number of values

Other properties of expectation

Coherence implies not a sure loser

Expectations and limits

**Conditional Probability and Bayes Theorem**

Conditional probability

The Birthday Problem

Simpson's Paradox

Bayes Theorem

Independence of events

The Monty Hall problem

Gambler's Ruin problem

Iterated Expectations and Independence

The binomial and multinomial distributions

Sampling without replacement

Variance and covariance

A short introduction to multivariate thinking

Tchebychev's inequality

**Discrete Random Variables**

Countably many possible values

Finite additivity

Countable Additivity

Properties of countable additivity

Dynamic sure loss

**Probability generating functions**

Geometric random variables

The negative binomial random variable

The Poisson random variable

Cumulative distribution function

Dominated and bounded convergence

**Continuous Random Variables**

Introduction

Joint distributions

Conditional distributions and independence

Existence and properties of expectations

Extensions

An interesting relationship between cdf's and expectations of continuous random variables

Chapter retrospective so far

Bounded and dominated convergence

The Riemann-Stieltjes integral

The McShane-Stieltjes Integral

The road from here

The strong law of large numbers

**Transformations**

Introduction

Discrete Random Variables

Univariate Continuous Distributions

Linear spaces

Permutations

Number systems; DeMoivre's formula

Determinants

Eigenvalues, eigenvectors and decompositions

Non-linear transformations

The Borel-Kolmogorov paradox

**Normal Distribution**

Introduction

Moment generating functions

Characteristic functions

Trigonometric Polynomials

A Weierstrass approximation theorem

Uniqueness of characteristic functions

Characteristic function and moments

Continuity Theorem

The Normal distribution

Multivariate normal distributions

Limit theorems

**Making Decisions**

Introduction

An example

In greater generality

The St. Petersburg Paradox

Risk aversion

Log (fortune) as utility

Decisions after seeing data

The expected value of sample information

An example

Randomized decisions

Sequential decisions

**Conjugate Analysis**

A simple normal-normal case

A multivariate normal case, known precision

The normal linear model with known precision

The gamma distribution

Uncertain Mean and Precision

The normal linear model, uncertain precision

The Wishart distribution

Both mean and precision matrix uncertain

The beta and Dirichlet distributions

The exponential family

Large sample theory for Bayesians

Some general perspective

**Hierarchical Structuring of a Model**

Introduction

Missing data

Meta-analysis

Model uncertainty/model choice

Graphical Hierarchical Models

Causation

**Markov Chain Monte Carlo**

Introduction

Simulation

The Metropolis Hasting Algorithm

Extensions and special cases

Practical considerations

Variable dimensions: Reversible jumps

**Multiparty Problems**

A simple three-stage game

Private information

Design for another's analysis

Optimal Bayesian Randomization

Simultaneous moves

The Allais and Ellsberg paradoxes

Forming a Bayesian group

**Exploration of Old Ideas**

Introduction

Testing

Confidence intervals and sets

Estimation

Choosing among models

Goodness of fit

Sampling theory statistics

Objective" Bayesian Methods

**Epilogue: Applications**

Computation

A final thought

## Author(s)

### Biography

Joseph B. Kadane, Department of Statistics, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

## Reviews

"… it is a book about Bayesian probability, statistics, and decision making. … an excellent choice. … The exercises at the end of each chapter are well conceived and in this way most useful for checking understanding of the text. Typos and errata are rare in this book. … recommended to statisticians (without restriction of any kind), statistics students at a graduate or PhD level, and empirical researchers in general … a sound basis for a course in Bayesian probability, statistics, and decision making."

—Biometrical Journal, 2013This book mainly focuses on the use of Bayesian statistics. It is written using stories and many examples to which readers can relate, and is thus an engaging and appealing text on what is generally a very dry mathematical subject.

—John J. Shea,IEEE Electrical Insulation Magazine, May/June 2013, Vol. 29, No. 3This text provides a unique blend of theory, methods, philosophy and applications that is suitable for a course in Bayesian probability and statistics. … provides thought-provoking material for teaching. …

—Erkki P. Liski,International Statistical Review, 2012In this remarkable book, Kadane begins at the most rudimentary level, develops all the needed mathematics on the fly, and still manages to flesh out at least the core of the whole story, slowly, thoughtfully, and rigorously, right up to graduate level. Major theorems all proved in detail appear here, but not for their own sake; the author always carefully selects them to clarify the basic meaning of the subject and his own views concerning the pitfalls and subtleties of its proper application. Summing Up: Highly recommended.

—D.V. Feldman,CHOICE, February 2012

Principles of Uncertaintyis a profound and mesmerising book on the foundations and principles of subjectivist or behaviouristic Bayesian analysis. … the book is a pleasure to read. And highly recommended for teaching as it can be used at many different levels. … A must-read for sure!

—Christian Robert, The Statistics Forum/CHANCE, October 2011It's a lovely book, one that I hope will be widely adopted as a course textbook.

—Michael Jordan, University of California, Berkeley, USAA careful, complete, and lovingly written exposition of the subjective Bayesian viewpoint by one of its most eloquent and staunch defenders. Summarizes a lifetime of theory, methods, and application developments for the Bayesian inferential engine. A must-read for anyone looking for a deep understanding of the foundations of Bayesian methods and what they offer modern statistical practice.

—Bradley P. Carlin, Professor and Head of Division of Biostatistics, University of Minnesota, Minneapolis, USA