A First Step toward a Unified Theory of Richly Parameterized Linear Models
Using mixed linear models to analyze data often leads to results that are mysterious, inconvenient, or wrong. Further compounding the problem, statisticians lack a cohesive resource to acquire a systematic, theory-based understanding of models with random effects.
Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects takes a first step in developing a full theory of richly parameterized models, which would allow statisticians to better understand their analysis results. The author examines what is known and unknown about mixed linear models and identifies research opportunities.
The first two parts of the book cover an existing syntax for unifying models with random effects. The text explains how richly parameterized models can be expressed as mixed linear models and analyzed using conventional and Bayesian methods.
In the last two parts, the author discusses oddities that can arise when analyzing data using these models. He presents ways to detect problems and, when possible, shows how to mitigate or avoid them. The book adapts ideas from linear model theory and then goes beyond that theory by examining the information in the data about the mixed linear model’s covariance matrices.
Each chapter ends with two sets of exercises. Conventional problems encourage readers to practice with the algebraic methods and open questions motivate readers to research further. Supporting materials, including datasets for most of the examples analyzed, are available on the author’s website.
Table of Contents
Mixed Linear Models: Syntax, Theory, and Methods
An Opinionated Survey of Methods for Mixed Linear Models
Mixed linear models in the standard formulation
Conventional analysis of the mixed linear model
Bayesian analysis of the mixed linear model
Conventional and Bayesian approaches compared
A few words about computing
Two More Tools: Alternative Formulation, Measures of Complexity
Alternative formulation: The "constraint-case" formulation
Measuring the complexity of a mixed linear model fit
Richly Parameterized Models as Mixed Linear Models
Penalized Splines as Mixed Linear Models
Penalized splines: Basis, knots, and penalty
More on basis, knots, and penalty
Mixed linear model representation
Additive Models and Models with Interactions
Additive models as mixed linear models
Models with interactions
Spatial Models as Mixed Linear Models
Models for areal data
Two-dimensional penalized splines
Time-Series Models as Mixed Linear Models
Example: Linear growth model
Dynamic linear models in some generality
Example of a multi-component DLM
Two Other Syntaxes for Richly Parameterized Models
Schematic comparison of the syntaxes
Gaussian Markov random fields
Likelihood inference for models with unobservables
From Linear Models to Richly Parameterized Models: Mean Structure
Adapting Diagnostics from Linear Models
Added variable plots
Puzzles from Analyzing Real Datasets
Overview of the next three chapters
A Random Effect Competing with a Fixed Effect
Slovenia data: Spatial confounding
Kids and crowns: Informative cluster size
The simplified model and an overview of the results
Details of derivations
Conclusion: What might cause differential shrinkage?
Competition between Random Effects
Collinearity between random effects in three simpler models
Testing hypotheses on the optical-imaging data and DLM models
Random Effects Old and New
Old-style random effects
New-style random effects
Beyond Linear Models: Variance Structure
Mysterious, Inconvenient, or Wrong Results from Real Datasets
Periodontal data and the ICAR model
Periodontal data and the ICAR with two classes of neighbor pairs
Two very different smooths of the same data
Misleading zero variance estimates
Multiple maxima in posteriors and restricted likelihoods
Overview of the remaining chapters
Re-Expressing the Restricted Likelihood: Two-Variance Models
A tentative collection of tools
Exploring the Restricted Likelihood for Two-Variance Models
Which vj tell us about which variance?
Two mysteries explained
Extending the Re-Expressed Restricted Likelihood
Restricted likelihoods that can and can’t be re-expressed
Expedients for restricted likelihoods that can’t be re-expressed
Zero Variance Estimates
Some observations about zero variance estimates
Some thoughts about tools
Multiple Maxima in the Restricted Likelihood and Posterior
Restricted likelihoods with multiple local maxima
Posteriors with multiple modes
James S. Hodges
"Richly Parameterized Linear Models is a strong addition to the literature on mixed models, offering a much more unified treatment and broader scope than many existing texts. … The book also provides an excellent treatment of diagnostics for mixed models. … The literature on mixed models has expanded in recent decades, often in disparate ways. Richly Parameterized Linear Models provides a step toward unifying this growing area of research and serves as an excellent resource for applied researchers with experience and interest in mixed models."
—Journal of the American Statistical Association, Vol. 110, 2015
"This is a thought-provoking book that stimulates discussion and further research. … It is a refreshing idea to concentrate on poorly understood ‘mysteries’ in the class of linear mixed models. … If you have ever encountered similar issues in linear mixed models, you will appreciate this book. Furthermore, if you are looking for new scientific challenges, then you will find a large resource of still open questions and useful references. Graduate students will develop a deeper understanding of the linear mixed model and critical thinking in the application of this model class. … We recommend this book, as it will stimulate further research, leading to deeper insight into richly parametrized linear models."
—Biometrical Journal, 2015
"Hodges’ book is a really recommendable reference for mixed models users. Use of random effects models has exponentially grown during the last few decades mainly due to the availability of software making the fit of these models possible. Such software has made mixed models available to a wide community of users, thus complex analyses with complex covariance structures have filled both applied and theoretical journals. Nevertheless, as masterfully described within this book, the hypotheses underneath these models, and the corresponding covariance structures, are not harmless at all but, on the contrary, they may have a large impact on their fit. Hodges’ monograph explores and highlights the repercussion of many aspects involved in the definition of most mixed models that are usually unknown or, even worse, ignored. Hodges makes us aware of these issues that should be kept very in mind by any mixed models user and illustrates them with lots of real case studies.
This is an extremely clarifying tool for those who usually work with GLMMs in general. Mathematical details underlying GLMMs are usually overwhelming … Hodges makes a journey to the grounds of linear mixed models, where mathematical detail is more accessible, providing a deep insight also of use in GLMMs. The approach described in this book yields some intuition about what can be happening underneath the complex GLMMs that are usually used in practice."
—Miguel Martinez Beneito, Foundation for the Promotion of Health and Biomedical Research in the Valencian Region
"I recommend this text to any student/researcher who is interested in mixed models. The book is written in an enthralling and engaging style and is overflowing with interesting observations, has a unique spin, and is very thought provoking. Over the past 20 years there has been a tendency toward the fitting of more and more complex models, with the potential negative implications of this endeavor (which I will loosely term ‘overfitting’) being lost amid the enthusiasm for bigger and allegedly ‘more realistic’ models. Those with such tendencies would definitely benefit from studying this book in order to gain insight into the unexpected consequences that some mixed model choices can have. All of the model types appearing in the title are covered in great detail, including coverage of diagnostics for mixed models, a sorely under-examined topic. Real data analyses are prominent in the discussions, with strange behavior of fitting being highlighted; for example, there is a section entitled, ‘Mysterious, inconvenient, or wrong results from real datasets.’ The author is not afraid to state an opinion, which is very refreshing; see, for example, the critique/gentle destruction of h-likelihood in Section 7.3. Open questions are also advertised and PhD students looking for research problems could do a lot worse than read this book. … an excellent, inspiring, and entertaining statistics book …"
—Jon Wakefield, Professor of Statistics, University of Washington
"This new book by James S. Hodges is different. Whereas others describe a polished version of the theme of the book, Hodges takes another approach. In a very personal writing style, Hodges explores what we do and do not understand about mixed linear models. Why does his approach work so well? It makes you think!!! By also discussing mysterious, inconvenient, or plainly wrong results, we simply gain more insight and understanding. This works for me; I have never gained so much (hard to get) insight in so short a time from any other book I have read. I highly recommend it!"
—Håvard Rue, Norwegian University of Science and Technology
"This book is a masterpiece, destined to become a classic. Linear mixed models are widely used by statisticians and analysts in many fields, often under other names: hierarchical, longitudinal, dynamic, random effects, multi-level, and others. Statistical packages routinely fit these models. Sometimes the packages return sensible results; sometimes not. Sometimes the user can tell whether the results are sensible; sometimes not. Oddities include ‘zero variance estimates, multiple maxima, counterintuitive outlier effects, odd fits (e.g., a wiggly smooth with one smoother but not with another apparently similar smoother), big changes in fit from apparently modest changes in the model or data, and unpredictable convergence of numerical routines,’ to quote from the book. There is not presently a unified theory, like that for linear regression, to explain how, why, and when our numerical routines give results that should be questioned, or at least examined further. Even so, this book does the best job I have seen of explaining what can go wrong and what the state of the art is.
The subject is important; the writing is excellent; and the examples are compelling. I am excited by the prospect of teaching a course from this book. Its clarity of thought and presentation are exemplary. I recommend it for anyone who fits complicated models."
—Michael Lavine, University of Massachusetts Amherst