Preface
1 Distribution Theory
1.1 Introduction
1.2 Probability Measures
1.3 Some Important Theorems of Probability
1.4 Commonly Used Distributions
1.5 Stochastic Order Relations
1.6 Quantiles
1.7 Inversion of the CDF
1.8 Transformations of Random Variables
1.9 Moment Generating Functions
1.10 Moments and Cumulants
1.11 Problems
2 Multivariate Distributions
2.1 Introduction
2.2 Parametric Classes of Multivariate Distributions
2.3 Multivariate Transformations
2.4 Order Statistics
2.5 Quadratic Forms, Idempotent Matrices and Cochran’s Theorem
2.6 MGF and CGF of Independent Sums
2.7 Multivariate Extensions of the MGF
2.8 Problems
3 Statistical Models
3.1 Introduction
3.2 Parametric Families for Statistical Inference
3.3 Location-Scale Parameter Models
3.4 Regular Families
3.5 Fisher Information
3.6 Exponential Families
3.7 Sufficiency
3.8 Complete and Ancillary Statistics
3.9 Conditional Models and Contingency Tables
3.10 Bayesian Models
3.11 Indifference, Invariance and Bayesian Prior Distributions
3.12 Nuisance Parameters
3.13 Principles of Inference
3.14 Problems
4 Methods of Estimation
4.1 Introduction
4.2 Unbiased Estimators
4.3 Method of Moments Estimators
4.4 Sample Quantiles and Percentiles
4.5 Maximum Likelihood Estimation
4.6 Confidence Sets
4.7 Equivariant Versus Shrinkage Estimation
4.8 Bayesian Estimation
4.9 Problems
5 Hypothesis Testing
5.1 Introduction
5.2 Basic Definitions
5.3 Principles of Hypothesis Tests
5.4 The Observed Level of Significance (P-Values)
5.5 One and Two Sided Tests
5.6 Hypothesis Tests and Pivots
5.7 Likelihood Ratio Tests
5.8 Similar Tests
5.9 Problems
6 Linear Models
6.1 Introduction
6.2 Linear Models - Definition
6.3 Best Linear Unbiased Estimators (BLUE)
6.4 Least-squares Estimators, BLUEs and Projection Matrices
6.5 Ordinary and Generalized Least-Squares Estimators
6.6 ANOVA Decomposition and the F Test for Linear Models
6.7 The F Test for One-Way ANOVA
6.8 Simultaneous Confidence Intervals
6.9 Multiple Linear Regression
6.10 Problems
7 Decision Theory
7.1 Introduction
7.2 Ranking Estimators by MSE
7.3 Prediction
7.4 The Structure of Decision Theoretic Inference
7.5 Loss and Risk
7.6 Uniformly Minimum Risk Estimators (The Location-Scale Model
7.7 Some Principles of Admissibility
7.8 Admissibility for Exponential Families (Karlin’s Theorem)
7.9 Bayes Decision Rules
7.10 Admissibility and Optimality
7.11 Problems
8 Uniformly Minimum Variance Unbiased (UMVU) Estimation
8.1 Introduction
8.2 Definition of UMVUE’s
8.3 UMVUE’s and Sufficiency
8.4 Methods of Deriving UMVUEs
8.5 Nonparametric Estimation and U-statistics
8.6 Rank Based Measures of Correlation
8.7 Problems
9 Group Structure and Invariant Inference
9.1 Introduction
9.2 MRE Estimators for Location Parameters
9.3 MRE Estimators for Scale Parameters
9.4 Invariant Density Families
9.5 Some Applications of Invariance
9.6 Invariant Hypothesis Tests
9.7 Problems
10 The Neyman-Pearson Lemma
10.1 Introduction
10.2 Hypothesis Test as Decision Rules
10.3 Neyman-Pearson (NP) Tests
10.4 Monotone Likelihood Ratios (MLR)
10.5 The Generalized Neyman-Pearson Lemma
10.6 Invariant Hypothesis Tests
10.7 Permutation Invariant Tests
10.8 Problems
11 Limit Theorems
11.1 Introduction
11.2 Limits of Sequences of Random Variables
11.3 Limits of Expected Values
11.4 Uniform Integrability
11.5 The Law of Large Numbers
11.6 Weak Convergence
11.7 Multivariate Extensions of Limit Theorems
11.8 The Continuous Mapping Theorem
11.9 MGFs, CGFs and Weak Convergence
11.10 The Central Limit Theorem for Triangular Arrays
11.11 Weak Convergence of Random Vectors
11.12 Problems
12 Large Sample Estimation - Basic Principles
12.1 Introduction
12.2 The _-Method
12.3 Variance Stabilizing Transformations
12.4 The _-Method and Higher Order Approximations
12.5 The Multivariate _-Method
12.6 Approximating the Distributions of Sample Quantiles: The Bahadur Representation Theorem
12.7 A Central Limit Theorem for U-statistics
12.8 The Information Inequality
12.9 Asymptotic Efficiency
12.10 Problems
13 Asymptotic Theory for Estimating Equations
13.1 Introduction
13.2 Consistency and Asymptotic Normality of M-estimators
13.3 Asymptotic Theory of MLEs
13.4 A General Form for Regression Models
13.5 Nonlinear Regression
13.6 Generalized Linear Models (GLMs)
13.7 Generalized Estimating Equations (GEE)
13.8 Consistency of M-estimators
13.9 Asymptotic Distribution of ˆ_n
13.10 Regularity Conditions for Estimating Equations
13.11 Problems
14 Large Sample Hypothesis Testing
14.1 Introduction
14.2 Model Assumptions
14.3 Large Sample Tests for Simple Hypotheses
14.4 Nuisance Parameters and Composite Null Hypotheses
14.5 A Comparison of the LR, Wald and Score Tests
14.6 Pearson’s _2 Test for Independence in Contingency Tables
14.7 Estimating Power for Approximate _2 Tests
14.8 Problems
A Parametric Classes of Densities
B Topics in Linear Algebra
B.1 Numbers
B.2 Equivalence Relations
B.3 Vector Spaces
B.4 Matrices
B.5 Dimension of a Subset of Rd
C Topics in Real Analysis and Measure Theory
C.1 Metric spaces
C.2 Measure Theory
C.3 Integration
C.4 Exchange of Integration and Differentiation
C.5 The Gamma and Beta Functions
C.6 Stirling’s Approximation of the Factorial
C.7 The Gradient Vector and the Hessian Matrix
C.8 Normed Vector Spaces
C.9 Taylor’s Theorem
D Group Theory
D.1 Definition of a Group
D.2 Subgroups
D.3 Group Homomorphisms
D.4 Transformation Groups
D.5 Orbits and Maximal Invariants
Bibliography
Index
Biography
Anthony Almudevar is an Associate Professor of Biostatistics and Computational Biology at the University of Rochester. His research interests include statistical methodology, graphical models, bioinformatics, optimization and control theory. Other published volumes include Almudevar A (2014) Approximate Iterative Algorithms, CRC Press, and Statistical Modeling for Biological Systems: In Memory of Andrei Yakovlev, Anthony Almudevar, David Oakes and Jack Hall, editors (2020), Springer.
"The large variety of topics covered in the current version allows students at all level, be it bachelor, master or Ph.D. to find topics suitable for their own study. On one hand, the book contains fairly advanced topics to help researchers in their own research areas, and on the other hand, it provides enough materials for lecturers to prepare lecture notes and design an entire course on theoretical statistics. Overall, I am extremely excited about Theory of Statistical Inference and eagerly waiting for numerous students, researchers, lecturers and even working professionals to enjoy the immense benefit coming out of its publication."
Somabha Mukherjee, National University of Singapore, Singapore, Journal of the American Statistical Association, February 2024.
