1st Edition

Linear Model Methodology

By Andre I. Khuri Copyright 2009
    562 Pages 13 B/W Illustrations
    by Chapman & Hall

    Given the importance of linear models in statistical theory and experimental research, a good understanding of their fundamental principles and theory is essential. Supported by a large number of examples, Linear Model Methodology provides a strong foundation in the theory of linear models and explores the latest developments in data analysis.

    After presenting the historical evolution of certain methods and techniques used in linear models, the book reviews vector spaces and linear transformations and discusses the basic concepts and results of matrix algebra that are relevant to the study of linear models. Although mainly focused on classical linear models, the next several chapters also explore recent techniques for solving well-known problems that pertain to the distribution and independence of quadratic forms, the analysis of estimable linear functions and contrasts, and the general treatment of balanced random and mixed-effects models. The author then covers more contemporary topics in linear models, including the adequacy of Satterthwaite’s approximation, unbalanced fixed- and mixed-effects models, heteroscedastic linear models, response surface models with random effects, and linear multiresponse models. The final chapter introduces generalized linear models, which represent an extension of classical linear models.

    Linear models provide the groundwork for analysis of variance, regression analysis, response surface methodology, variance components analysis, and more, making it necessary to understand the theory behind linear modeling. Reflecting advances made in the last thirty years, this book offers a rigorous development of the theory underlying linear models.

    Linear Models: Some Historical Perspectives

    The Invention of Least Squares

    The Gauss–Markov Theorem


    Maximum Likelihood Estimation

    Analysis of Variance (ANOVA)

    Quadratic Forms and Craig’s Theorem

    The Role of Matrix Algebra

    The Geometric Approach

    Basic Elements of Linear Algebra


    Vector Spaces

    Vector Subspaces

    Bases and Dimensions of Vector Spaces

    Linear Transformations

    Basic Concepts in Matrix Algebra

    Introduction and Notation

    Some Particular Types of Matrices

    Basic Matrix Operations

    Partitioned Matrices


    The Rank of a Matrix

    The Inverse of a Matrix

    Eigenvalues and Eigenvectors

    Idempotent and Orthogonal Matrices

    Quadratic Forms

    Decomposition Theorems

    Some Matrix Inequalities

    Function of Matrices

    Matrix Differentiation

    The Multivariate Normal Distribution

    History of the Normal Distribution

    The Univariate Normal Distribution

    The Multivariate Normal Distribution

    The Moment Generating Function

    Conditional Distribution

    The Singular Multivariate Normal Distribution

    Related Distributions

    Examples and Additional Results

    Quadratic Forms in Normal Variables

    The Moment Generating Function

    Distribution of Quadratic Forms

    Independence of Quadratic Forms

    Independence of Linear and Quadratic Forms

    Independence and Chi-Squaredness of Several Quadratic Forms

    Computing the Distribution of Quadratic Forms


    Full-Rank Linear Models

    Least-Squares Estimation

    Properties of Ordinary Least-Squares Estimation

    Generalized Least-Squares Estimation

    Least-Squares Estimation under Linear Restrictions on β

    Maximum Likelihood Estimation

    Inference Concerning β

    Examples and Applications

    Less-Than-Full-Rank Linear Models

    Parameter Estimation

    Some Distributional Properties

    Reparameterized Model

    Estimable Linear Functions

    Simultaneous Confidence Intervals on Estimable Linear Functions

    Simultaneous Confidence Intervals on All Contrasts among the Means with Heterogeneous Group Variances

    Further Results Concerning Contrasts and Estimable Linear Functions

    Balanced Linear Models

    Notation and Definitions

    The General Balanced Linear Model

    Properties of Balanced Models

    Balanced Mixed Models

    Complete and Sufficient Statistics

    ANOVA Estimation of Variance Components

    Confidence Intervals on Continuous Functions of the Variance Components

    Confidence Intervals on Ratios of Variance Components

    The Adequacy of Satterthwaite’s Approximation

    Satterthwaite’s Approximation

    Adequacy of Satterthwaite’s Approximation

    Measuring the Closeness of Satterthwaite’s Approximation



    Unbalanced Fixed-Effects Models

    The R-Notation

    Two-Way Models without Interaction

    Two-Way Models with Interaction

    Higher-Order Models

    A Numerical Example

    The Method of Unweighted Means

    Unbalanced Random and Mixed Models

    Estimation of Variance Components

    Estimation of Estimable Linear Functions

    Inference Concerning the Random One-Way Model

    Inference Concerning the Random Two-Way Model

    Exact Tests for Random Higher-Order Models

    Inference Concerning the Mixed Two-Way Model

    Inference Concerning the Random Two-Fold Nested Model

    Inference Concerning the Mixed Two-Fold Nested Model

    Inference Concerning the General Mixed Linear Model


    Additional Topics in Linear Models

    Heteroscedastic Linear Models

    The Random One-Way Model with Heterogeneous Error Variances

    A Mixed Two-Fold Nested Model with Heteroscedastic Random Effects

    Response Surface Models

    Response Surface Models with Random Block Effects

    Linear Multiresponse Models

    Generalized Linear Models


    The Exponential Family

    Estimation of Parameters

    Goodness of Fit

    Hypothesis Testing

    Confidence Intervals

    Gamma-Distributed Response



    Exercises appear at the end of each chapter, except for Chapter 1.


    André I. Khuri is a Professor Emeritus in the Department of Statistics at the University of Florida in Gainesville.

    This is a comprehensive and up-to-date textbook on the theory of linear models. … Every chapter besides the first historical one contains many exercises … . There is also a huge bibliography. The textbook represents an important source for all researchers and lectures in linear models.
    —Hilmar Drygas, Zentralblatt MATH, 2012

    The outstanding book, written by a prominent researcher and author, presents a wealth of materials on linear models in Chapters 1 though 12 and includes materials on generalized linear models in the last chapter. The material on linear models is an amazing collection of important topics that would benefit researchers, teachers, students, and practitioners and has added value to the book. Many illustrative examples are presented with SAS codes. The examples are practically important and thoughtfully chosen. Exercises at the end of Chapters 2-13 are excellent, and some are valuable resources for the researchers in this area. … The author has to be commended for his success in executing this so elegantly.
    —Subir Ghosh, Journal of Quality Technology, Vol. 44, January 2012

    This book provides a very well-written and rigorous account of the theory of linear models. … In sum, this is a carefully written and reliable book that reflects the experience of the author in teaching graduate level courses on linear models. I will certainly add it to the list of reference textbooks for the graduate one-quarter course on linear model theory taught at UC Santa Cruz.
    —Raquel Prado, Journal of the American Statistical Association, September 2011, Vol. 106

    This text is a possible choice for a second course in linear model theory.
    —David J. Olive, Technometrics, May 2011

    The material is well chosen and well organized, and includes many results that are not found in other textbooks. … Throughout the book, the presentation is very clear and well organized, with a focus on mathematical developments. Most results are stated with proofs, some material is based on the author’s own contributions to the field. Generally, many important special cases are treated in detail, which will make the book also highly useful as a reference. There are also many worked-out examples from different subject areas to illustrate the methods. Later chapters also include some instructions on how to use the methods in SAS. Furthermore, there are lots of exercises at the end of each chapter. … The book is very accessible and encompassing … the book will be an excellent choice both as a text and as a reference book.
    —T. Mildenberger, Statistical Papers, April 2011

    The material on which this book is based has been taught in a couple of courses at the University of Florida for about 20 years and the author’s skills and experience in doing this are superbly represented in this fine text. … there are numerous exercises that reinforce both the theoretical and the practical aspects of regression… This is an excellent, reliable, and comprehensive text.
    International Statistical Review (2010), 78

    This book provides a thorough overview which is similar to other available texts but in a very different way. The choice of topics covered, their organization and presentation are the unique features that distinguish this book. … This book is well structured as a textbook as well as a reference with every chapter explaining the definitions, principles and methods of the subject matter illustrated by data-based examples with the details on use of SAS software, wherever possible. … the topics that are covered in Chapters 7–12 are not generally found in a single book. … The book would make an excellent textbook for a course on linear models at masters and graduate levels. Moreover, some parts of the book can also be a part of a course on analysis of variance. Overall, the book is a valuable reference for those involved in research and teaching in this area.
    Journal of the Royal Statistical Society, Series A, 2010