# Linear Model Methodology

## Preview

## Book Description

Given the importance of linear models in statistical theory and experimental research, a good understanding of their fundamental principles and theory is essential. Supported by a large number of examples, **Linear Model Methodology** provides a strong foundation in the theory of linear models and explores the latest developments in data analysis.

After presenting the historical evolution of certain methods and techniques used in linear models, the book reviews vector spaces and linear transformations and discusses the basic concepts and results of matrix algebra that are relevant to the study of linear models. Although mainly focused on classical linear models, the next several chapters also explore recent techniques for solving well-known problems that pertain to the distribution and independence of quadratic forms, the analysis of estimable linear functions and contrasts, and the general treatment of balanced random and mixed-effects models. The author then covers more contemporary topics in linear models, including the adequacy of Satterthwaite’s approximation, unbalanced fixed- and mixed-effects models, heteroscedastic linear models, response surface models with random effects, and linear multiresponse models. The final chapter introduces generalized linear models, which represent an extension of classical linear models.

Linear models provide the groundwork for analysis of variance, regression analysis, response surface methodology, variance components analysis, and more, making it necessary to understand the theory behind linear modeling. Reflecting advances made in the last thirty years, this book offers a rigorous development of the theory underlying linear models.

## Table of Contents

**Linear Models: Some Historical Perspectives**

The Invention of Least Squares

The Gauss–Markov Theorem

Estimability

Maximum Likelihood Estimation

Analysis of Variance (ANOVA)

Quadratic Forms and Craig’s Theorem

The Role of Matrix Algebra

The Geometric Approach

**Basic Elements of Linear Algebra**

Introduction

Vector Spaces

Vector Subspaces

Bases and Dimensions of Vector Spaces

Linear Transformations

**Basic Concepts in Matrix Algebra**

Introduction and Notation

Some Particular Types of Matrices

Basic Matrix Operations

Partitioned Matrices

Determinants

The Rank of a Matrix

The Inverse of a Matrix

Eigenvalues and Eigenvectors

Idempotent and Orthogonal Matrices

Quadratic Forms

Decomposition Theorems

Some Matrix Inequalities

Function of Matrices

Matrix Differentiation

**The Multivariate Normal Distribution**

History of the Normal Distribution

The Univariate Normal Distribution

The Multivariate Normal Distribution

The Moment Generating Function

Conditional Distribution

The Singular Multivariate Normal Distribution

Related Distributions

Examples and Additional Results

**Quadratic Forms in Normal Variables**

The Moment Generating Function

Distribution of Quadratic Forms

Independence of Quadratic Forms

Independence of Linear and Quadratic Forms

Independence and Chi-Squaredness of Several Quadratic Forms

Computing the Distribution of Quadratic Forms

Appendix

**Full-Rank Linear Models**

Least-Squares Estimation

Properties of Ordinary Least-Squares Estimation

Generalized Least-Squares Estimation

Least-Squares Estimation under Linear Restrictions on β

Maximum Likelihood Estimation

Inference Concerning β

Examples and Applications

**Less-Than-Full-Rank Linear Models**

Parameter Estimation

Some Distributional Properties

Reparameterized Model

Estimable Linear Functions

Simultaneous Confidence Intervals on Estimable Linear Functions

Simultaneous Confidence Intervals on All Contrasts among the Means with Heterogeneous Group Variances

Further Results Concerning Contrasts and Estimable Linear Functions

**Balanced Linear Models**

Notation and Definitions

The General Balanced Linear Model

Properties of Balanced Models

Balanced Mixed Models

Complete and Sufficient Statistics

ANOVA Estimation of Variance Components

Confidence Intervals on Continuous Functions of the Variance Components

Confidence Intervals on Ratios of Variance Components

**The Adequacy of Satterthwaite’s Approximation**

Satterthwaite’s Approximation

Adequacy of Satterthwaite’s Approximation

Measuring the Closeness of Satterthwaite’s Approximation

Examples

Appendix

**Unbalanced Fixed-Effects Models**

The *R*-Notation

Two-Way Models without Interaction

Two-Way Models with Interaction

Higher-Order Models

A Numerical Example

The Method of Unweighted Means

**Unbalanced Random and Mixed Models**

Estimation of Variance Components

Estimation of Estimable Linear Functions

Inference Concerning the Random One-Way Model

Inference Concerning the Random Two-Way Model

Exact Tests for Random Higher-Order Models

Inference Concerning the Mixed Two-Way Model

Inference Concerning the Random Two-Fold Nested Model

Inference Concerning the Mixed Two-Fold Nested Model

Inference Concerning the General Mixed Linear Model

Appendix

**Additional Topics in Linear Models**

Heteroscedastic Linear Models

The Random One-Way Model with Heterogeneous Error Variances

A Mixed Two-Fold Nested Model with Heteroscedastic Random Effects

Response Surface Models

Response Surface Models with Random Block Effects

Linear Multiresponse Models

**Generalized Linear Models**

Introduction

The Exponential Family

Estimation of Parameters

Goodness of Fit

Hypothesis Testing

Confidence Intervals

Gamma-Distributed Response

**Bibliography**

**Index**

*Exercises appear at the end of each chapter, except for Chapter 1.*

## Author(s)

### Biography

**André I. Khuri** is a Professor Emeritus in the Department of Statistics at the University of Florida in Gainesville.

## Reviews

This is a comprehensive and up-to-date textbook on the theory of linear models. … Every chapter besides the first historical one contains many exercises … . There is also a huge bibliography. The textbook represents an important source for all researchers and lectures in linear models.

—Hilmar Drygas,Zentralblatt MATH, 2012The outstanding book, written by a prominent researcher and author, presents a wealth of materials on linear models in Chapters 1 though 12 and includes materials on generalized linear models in the last chapter. The material on linear models is an amazing collection of important topics that would benefit researchers, teachers, students, and practitioners and has added value to the book. Many illustrative examples are presented with SAS codes. The examples are practically important and thoughtfully chosen. Exercises at the end of Chapters 2-13 are excellent, and some are valuable resources for the researchers in this area. … The author has to be commended for his success in executing this so elegantly.

—Subir Ghosh,Journal of Quality Technology, Vol. 44, January 2012This book provides a very well-written and rigorous account of the theory of linear models. … In sum, this is a carefully written and reliable book that reflects the experience of the author in teaching graduate level courses on linear models. I will certainly add it to the list of reference textbooks for the graduate one-quarter course on linear model theory taught at UC Santa Cruz.

—Raquel Prado,Journal of the American Statistical Association, September 2011, Vol. 106This text is a possible choice for a second course in linear model theory.

—David J. Olive,Technometrics, May 2011The material is well chosen and well organized, and includes many results that are not found in other textbooks. … Throughout the book, the presentation is very clear and well organized, with a focus on mathematical developments. Most results are stated with proofs, some material is based on the author’s own contributions to the field. Generally, many important special cases are treated in detail, which will make the book also highly useful as a reference. There are also many worked-out examples from different subject areas to illustrate the methods. Later chapters also include some instructions on how to use the methods in SAS. Furthermore, there are lots of exercises at the end of each chapter. … The book is very accessible and encompassing … the book will be an excellent choice both as a text and as a reference book.

—T. Mildenberger,Statistical Papers, April 2011The material on which this book is based has been taught in a couple of courses at the University of Florida for about 20 years and the author’s skills and experience in doing this are superbly represented in this fine text. … there are numerous exercises that reinforce both the theoretical and the practical aspects of regression… This is an excellent, reliable, and comprehensive text.

—International Statistical Review(2010), 78This book provides a thorough overview which is similar to other available texts but in a very different way. The choice of topics covered, their organization and presentation are the unique features that distinguish this book. … This book is well structured as a textbook as well as a reference with every chapter explaining the definitions, principles and methods of the subject matter illustrated by data-based examples with the details on use of SAS software, wherever possible. … the topics that are covered in Chapters 7–12 are not generally found in a single book. … The book would make an excellent textbook for a course on linear models at masters and graduate levels. Moreover, some parts of the book can also be a part of a course on analysis of variance. Overall, the book is a valuable reference for those involved in research and teaching in this area.

—Journal of the Royal Statistical Society, Series A, 2010