Bayesian Programming  book cover
1st Edition

Bayesian Programming

ISBN 9781439880326
Published December 20, 2013 by Chapman and Hall/CRC
380 Pages 61 B/W Illustrations

FREE Standard Shipping
USD $160.00

Prices & shipping based on shipping country


Book Description

Probability as an Alternative to Boolean Logic
While logic is the mathematical foundation of rational reasoning and the fundamental principle of computing, it is restricted to problems where information is both complete and certain. However, many real-world problems, from financial investments to email filtering, are incomplete or uncertain in nature. Probability theory and Bayesian computing together provide an alternative framework to deal with incomplete and uncertain data.

Decision-Making Tools and Methods for Incomplete and Uncertain Data
Emphasizing probability as an alternative to Boolean logic, Bayesian Programming covers new methods to build probabilistic programs for real-world applications. Written by the team who designed and implemented an efficient probabilistic inference engine to interpret Bayesian programs, the book offers many Python examples that are also available on a supplementary website together with an interpreter that allows readers to experiment with this new approach to programming.

Principles and Modeling
Only requiring a basic foundation in mathematics, the first two parts of the book present a new methodology for building subjective probabilistic models. The authors introduce the principles of Bayesian programming and discuss good practices for probabilistic modeling. Numerous simple examples highlight the application of Bayesian modeling in different fields.

Formalism and Algorithms
The third part synthesizes existing work on Bayesian inference algorithms since an efficient Bayesian inference engine is needed to automate the probabilistic calculus in Bayesian programs. Many bibliographic references are included for readers who would like more details on the formalism of Bayesian programming, the main probabilistic models, general purpose algorithms for Bayesian inference, and learning problems.

Along with a glossary, the fourth part contains answers to frequently asked questions. The authors compare Bayesian programming and possibility theories, discuss the computational complexity of Bayesian inference, cover the irreducibility of incompleteness, and address the subjectivist versus objectivist epistemology of probability.

The First Steps toward a Bayesian Computer
A new modeling methodology, new inference algorithms, new programming languages, and new hardware are all needed to create a complete Bayesian computing framework. Focusing on the methodology and algorithms, this book describes the first steps toward reaching that goal. It encourages readers to explore emerging areas, such as bio-inspired computing, and develop new programming languages and hardware architectures.

Table of Contents

Probability an alternative to logic
A need for a new computing paradigm
A need for a new modeling methodology
A need for new inference algorithms
A need for a new programming language and new hardware
A place for numerous controversies
Running real programs as exercises

Bayesian Programming Principles
Basic Concepts
The normalization postulate
Conditional probability
Variable conjunction
The conjunction postulate (Bayes theorem)
The marginalization rule
Joint distribution and questions
Parametric forms
Specification = variables + decomposition + parametric forms
Description = specification + identification
Bayesian program = description + question

Incompleteness and Uncertainty
Observing a water treatment unit
Lessons, comments, and notes

Description = Specification + Identification
Pushing objects and following contours
Description of a water treatment unit
Lessons, comments, and notes

The Importance of Conditional Independence
Water treatment center Bayesian model (continuation)
Description of the water treatment center
Lessons, comments, and notes

Bayesian Program = Description + Question
Water treatment center Bayesian model (end)
Forward simulation of a single unit
Forward simulation of the water treatment center
Control of the water treatment center
Lessons, comments, and notes

Bayesian Programming Cookbook
Information Fusion
"Naive" Bayes sensor fusion
Relaxing the conditional independence fundamental hypothesis
Ancillary clues
Sensor fusion with false alarm
Inverse programming

Bayesian Programming with Coherence Variables
Basic example with Boolean variables
Basic example with discrete variables
Checking the semantic of Λ
Information fusion revisited using coherence variables
Reasoning with soft evidence

Bayesian Programming Subroutines
The sprinkler model
Calling subroutines conditioned by values
Water treatment center revisited (final)
Fusion of subroutines

Bayesian Programming Conditional Statement
Bayesian if-then-else
Behavior recognition
Mixture of models and model recognition

Bayesian Programming Iteration
Generic iteration
Generic Bayesian filters
Markov localization

Bayesian Programming Formalism and Algorithms
Bayesian Programming Formalism

Logical propositions
Probability of a proposition
Normalization and conjunction postulates
Disjunction rule for propositions
Discrete variables
Variable conjunction
Probability on variables
Conjunction rule for variables
Normalization rule for variables
Marginalization rule
Bayesian program

Bayesian Models Revisited
General purpose probabilistic models
Engineering-oriented probabilistic models
Cognitive-oriented probabilistic models

Bayesian Inference Algorithms Revisited
Stating the problem
Symbolic computation
Numerical computation: General sampling algorithms for approximate Bayesian inference
Approximate inference in ProBT

Bayesian Learning Revisited
Parameter identification
Expectation-Maximization (EM)
Learning structure of Bayesian networks

Frequently Asked Questions and Frequently Argued Matter
Frequently Asked Question and Frequently Argued Matter
Alternative Bayesian inference engines
Bayesian programming applications
Bayesian programming vs. Bayesian networks
Bayesian programming vs. Bayesian modeling
Bayesian programming vs. possibility theories
Bayesian programming vs. probabilistic programming
Computational complexity of Bayesian inference
Cox theorem
Discrete vs. continuous variables
Incompleteness irreducibility
Maximum entropy principle justifications
Noise or ignorance?
Objectivism vs. subjectivism controversy and the "mind projection fallacy"
Unknown distribution

Bayesian filter
Bayesian inference
Bayesian network
Bayesian program
Coherence variable
Conditional statement
Preliminary knowledge


View More



Pierre Bessiere is with CNRS, the French National Centre for Scientific Research. Juan-Manuel Ahuactzin, Kamel Mekhnacha, and Emmanuel Mazer are with Probayes Inc., France.


"Bayesian Programming comprises a methodology, a programming language, and a set of tools for developing and applying … complex models. … The approach is described in great detail, with many worked examples backed up by an online code repository. Unlike other books that tend to focus almost entirely on mathematics, this one gives equal time to conceptual and methodological guidance for the model-builder. It grapples with the knotty problems that arise in practice, some of which do not yet have clear solutions."
—From the Foreword by Stuart Russell, University of California, Berkeley

"The book has many worked examples backed up by an online code repository. The book provides a contibution on conceptual and methodological guidelines for model-builders. The authors discuss the problem how to build a Bayesian computer. The book has an excellent bibliography."
—Nirode C. Mohanty, in Zentralblatt MATH 1281