Bayesian Programming: 1st Edition (Hardback) book cover

Bayesian Programming

1st Edition

By Pierre Bessiere, Emmanuel Mazer, Juan Manuel Ahuactzin, Kamel Mekhnacha

Chapman and Hall/CRC

380 pages | 61 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781439880326
pub: 2013-12-20
SAVE ~$20.00
eBook (VitalSource) : 9780429066627
pub: 2013-12-20
from $50.00

FREE Standard Shipping!


Probability as an Alternative to Boolean Logic

While logic is the mathematical foundation of rational reasoning and the fundamental principle of computing, it is restricted to problems where information is both complete and certain. However, many real-world problems, from financial investments to email filtering, are incomplete or uncertain in nature. Probability theory and Bayesian computing together provide an alternative framework to deal with incomplete and uncertain data.

Decision-Making Tools and Methods for Incomplete and Uncertain Data

Emphasizing probability as an alternative to Boolean logic, Bayesian Programming covers new methods to build probabilistic programs for real-world applications. Written by the team who designed and implemented an efficient probabilistic inference engine to interpret Bayesian programs, the book offers many Python examples that are also available on a supplementary website together with an interpreter that allows readers to experiment with this new approach to programming.

Principles and Modeling

Only requiring a basic foundation in mathematics, the first two parts of the book present a new methodology for building subjective probabilistic models. The authors introduce the principles of Bayesian programming and discuss good practices for probabilistic modeling. Numerous simple examples highlight the application of Bayesian modeling in different fields.

Formalism and Algorithms

The third part synthesizes existing work on Bayesian inference algorithms since an efficient Bayesian inference engine is needed to automate the probabilistic calculus in Bayesian programs. Many bibliographic references are included for readers who would like more details on the formalism of Bayesian programming, the main probabilistic models, general purpose algorithms for Bayesian inference, and learning problems.


Along with a glossary, the fourth part contains answers to frequently asked questions. The authors compare Bayesian programming and possibility theories, discuss the computational complexity of Bayesian inference, cover the irreducibility of incompleteness, and address the subjectivist versus objectivist epistemology of probability.

The First Steps toward a Bayesian Computer

A new modeling methodology, new inference algorithms, new programming languages, and new hardware are all needed to create a complete Bayesian computing framework. Focusing on the methodology and algorithms, this book describes the first steps toward reaching that goal. It encourages readers to explore emerging areas, such as bio-inspired computing, and develop new programming languages and hardware architectures.


"Bayesian Programming comprises a methodology, a programming language, and a set of tools for developing and applying … complex models. … The approach is described in great detail, with many worked examples backed up by an online code repository. Unlike other books that tend to focus almost entirely on mathematics, this one gives equal time to conceptual and methodological guidance for the model-builder. It grapples with the knotty problems that arise in practice, some of which do not yet have clear solutions."

—From the Foreword by Stuart Russell, University of California, Berkeley

"The book has many worked examples backed up by an online code repository. The book provides a contibution on conceptual and methodological guidelines for model-builders. The authors discuss the problem how to build a Bayesian computer. The book has an excellent bibliography."

—Nirode C. Mohanty, in Zentralblatt MATH 1281

Table of Contents


Probability an alternative to logic

A need for a new computing paradigm

A need for a new modeling methodology

A need for new inference algorithms

A need for a new programming language and new hardware

A place for numerous controversies

Running real programs as exercises

Bayesian Programming Principles

Basic Concepts



The normalization postulate

Conditional probability

Variable conjunction

The conjunction postulate (Bayes theorem)


The marginalization rule

Joint distribution and questions


Parametric forms


Specification = variables + decomposition + parametric forms

Description = specification + identification


Bayesian program = description + question


Incompleteness and Uncertainty

Observing a water treatment unit

Lessons, comments, and notes

Description = Specification + Identification

Pushing objects and following contours

Description of a water treatment unit

Lessons, comments, and notes

The Importance of Conditional Independence

Water treatment center Bayesian model (continuation)

Description of the water treatment center

Lessons, comments, and notes

Bayesian Program = Description + Question

Water treatment center Bayesian model (end)

Forward simulation of a single unit

Forward simulation of the water treatment center

Control of the water treatment center


Lessons, comments, and notes

Bayesian Programming Cookbook

Information Fusion

"Naive" Bayes sensor fusion

Relaxing the conditional independence fundamental hypothesis


Ancillary clues

Sensor fusion with false alarm

Inverse programming

Bayesian Programming with Coherence Variables

Basic example with Boolean variables

Basic example with discrete variables

Checking the semantic of Λ

Information fusion revisited using coherence variables

Reasoning with soft evidence



Bayesian Programming Subroutines

The sprinkler model

Calling subroutines conditioned by values

Water treatment center revisited (final)

Fusion of subroutines


Bayesian Programming Conditional Statement

Bayesian if-then-else

Behavior recognition

Mixture of models and model recognition

Bayesian Programming Iteration

Generic iteration

Generic Bayesian filters

Markov localization

Bayesian Programming Formalism and Algorithms

Bayesian Programming Formalism

Logical propositions

Probability of a proposition

Normalization and conjunction postulates

Disjunction rule for propositions

Discrete variables

Variable conjunction

Probability on variables

Conjunction rule for variables

Normalization rule for variables

Marginalization rule

Bayesian program





Bayesian Models Revisited

General purpose probabilistic models

Engineering-oriented probabilistic models

Cognitive-oriented probabilistic models

Bayesian Inference Algorithms Revisited

Stating the problem

Symbolic computation

Numerical computation: General sampling algorithms for approximate Bayesian inference

Approximate inference in ProBT

Bayesian Learning Revisited

Parameter identification

Expectation-Maximization (EM)

Learning structure of Bayesian networks

Frequently Asked Questions and Frequently Argued Matter

Frequently Asked Question and Frequently Argued Matter

Alternative Bayesian inference engines

Bayesian programming applications

Bayesian programming vs. Bayesian networks

Bayesian programming vs. Bayesian modeling

Bayesian programming vs. possibility theories

Bayesian programming vs. probabilistic programming

Computational complexity of Bayesian inference

Cox theorem

Discrete vs. continuous variables

Incompleteness irreducibility

Maximum entropy principle justifications

Noise or ignorance?

Objectivism vs. subjectivism controversy and the "mind projection fallacy"

Unknown distribution


Bayesian filter

Bayesian inference

Bayesian network

Bayesian program

Coherence variable

Conditional statement







Preliminary knowledge






About the Authors

Pierre Bessiere is with CNRS, the French National Centre for Scientific Research. Juan-Manuel Ahuactzin, Kamel Mekhnacha, and Emmanuel Mazer are with Probayes Inc., France.

About the Series

Chapman & Hall/CRC Machine Learning & Pattern Recognition

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Machine Theory
MATHEMATICS / Probability & Statistics / General