# Understanding Advanced Statistical Methods

## Preview

## Book Description

Providing a much-needed bridge between elementary statistics courses and advanced research methods courses, **Understanding Advanced Statistical Methods** helps students grasp the fundamental assumptions and machinery behind sophisticated statistical topics, such as logistic regression, maximum likelihood, bootstrapping, nonparametrics, and Bayesian methods. The book teaches students how to properly model, think critically, and design their own studies to avoid common errors. It leads them to think differently not only about math and statistics but also about general research and the scientific method.

With a focus on statistical models as *producers* of data, the book enables students to more easily understand the machinery of advanced statistics. It also downplays the "population" interpretation of statistical models and presents Bayesian methods before frequentist ones. Requiring no prior calculus experience, the text employs a "just-in-time" approach that introduces mathematical topics, including calculus, where needed. Formulas throughout the text are used to explain why calculus and probability are essential in statistical modeling. The authors also intuitively explain the theory and logic behind real data analysis, incorporating a range of application examples from the social, economic, biological, medical, physical, and engineering sciences.

Enabling your students to answer the *why* behind statistical methods, this text teaches them how to successfully draw conclusions when the premises are flawed. It empowers them to use advanced statistical methods with confidence and develop their own statistical recipes. Ancillary materials are available on the book’s website.

## Table of Contents

**Introduction: Probability, Statistics, and Science**Reality, Nature, Science, and Models

Statistical Processes: Nature, Design and Measurement, and Data

Models

Deterministic Models

Variability

Parameters

Purely Probabilistic Statistical Models

Statistical Models with Both Deterministic and Probabilistic Components

Statistical Inference

Good and Bad Models

Uses of Probability Models

**Random Variables and Their Probability Distributions**

Introduction

Types of Random Variables: Nominal, Ordinal, and Continuous

Discrete Probability Distribution Functions

Continuous Probability Distribution Functions

Some Calculus–Derivatives and Least Squares

More Calculus–Integrals and Cumulative Distribution Functions

**Probability Calculation and Simulation**

Introduction

Analytic Calculations, Discrete and Continuous Cases

Simulation-Based Approximation

Generating Random Numbers

**Identifying Distributions**

Introduction

Identifying Distributions from Theory Alone

Using Data: Estimating Distributions via the Histogram

Quantiles: Theoretical and Data-Based Estimates

Using Data: Comparing Distributions via the Quantile–Quantile Plot

Effect of Randomness on Histograms and *q*–*q *Plots

**Conditional Distributions and Independence**

Introduction

Conditional Discrete Distributions

Estimating Conditional Discrete Distributions

Conditional Continuous Distributions

Estimating Conditional Continuous Distributions

Independence

**Marginal Distributions, Joint Distributions, Independence, and Bayes’ Theorem**Introduction

Joint and Marginal Distributions

Estimating and Visualizing Joint Distributions

Conditional Distributions from Joint Distributions

Joint Distributions When Variables Are Independent

Bayes’ Theorem

**Sampling from Populations and Processes**Introduction

Sampling from Populations

Critique of the Population Interpretation of Probability Models

The Process Model versus the Population Model

Independent and Identically Distributed Random Variables and Other Models

Checking the iid Assumption

**Expected Value and the Law of Large Numbers**

Introduction

Discrete Case

Continuous Case

Law of Large Numbers

Law of Large Numbers for the Bernoulli Distribution

Keeping the Terminology Straight: Mean, Average, Sample Mean, Sample Average, and Expected Value

Bootstrap Distribution and the Plug-In Principle

**Functions of Random Variables: Their Distributions and Expected Values**

Introduction

Distributions of Functions: The Discrete Case

Distributions of Functions: The Continuous Case

Expected Values of Functions and the Law of the Unconscious Statistician

Linearity and Additivity Properties

Nonlinear Functions and Jensen’s Inequality

Variance

Standard Deviation, Mean Absolute Deviation, and Chebyshev’s Inequality

Linearity Property of Variance

Skewness and Kurtosis

**Distributions of Totals**

Introduction

Additivity Property of Variance

Covariance and Correlation

Central Limit Theorem

**Estimation: Unbiasedness, Consistency, and Efficiency**

Introduction

Biased and Unbiased Estimators

Bias of the Plug-In Estimator of Variance

Removing the Bias of the Plug-In Estimator of Variance

The Joke Is on Us: The Standard Deviation Estimator Is Biased after All

Consistency of Estimators

Efficiency of Estimators

**Likelihood Function and Maximum Likelihood Estimates**

Introduction

Likelihood Function

Maximum Likelihood Estimates

Wald Standard Error

**Bayesian Statistics**

Introduction: Play a Game with Hans!

Prior Information and Posterior Knowledge

Case of the Unknown Survey

Bayesian Statistics: The Overview

Bayesian Analysis of the Bernoulli Parameter

Bayesian Analysis Using Simulation

What Good Is Bayes?

**Frequentist Statistical Methods**

Introduction

Large-Sample Approximate Frequentist Confidence Interval for the Process Mean

What Does *Approximate *Really Mean for an Interval Range?

Comparing the Bayesian and Frequentist Paradigms

**Are Your Results Explainable by Chance Alone?**

Introduction

What Does *by Chance Alone *Mean?

The *p*-Value

The Extremely Ugly "*pv *≤ 0.05" Rule of Thumb

**Chi-Squared, Student’s t, and F-Distributions, with Applications**

Introduction

Linearity and Additivity Properties of the Normal Distribution

Effect of Using an Estimate of

*s*

Chi-Squared Distribution

Frequentist Confidence Interval for

*s*

Student’s

*t*-Distribution

Comparing Two Independent Samples Using a Confidence Interval

Comparing Two Independent Homoscedastic Normal Samples via Hypothesis Testing

*F*-Distribution and ANOVA Test

*F*-Distribution and Comparing Variances of Two Independent Groups

**Likelihood Ratio Tests**

Introduction

Likelihood Ratio Method for Constructing Test Statistics

Evaluating the Statistical Significance of Likelihood Ratio Test Statistics

Likelihood Ratio Goodness-of-Fit Tests

Cross-Classification Frequency Tables and Tests of Independence

Comparing Non-Nested Models via the AIC Statistic

**Sample Size and Power**

Introduction

Choosing a Sample Size for a Prespecified Accuracy Margin

Power

Noncentral Distributions

Choosing a Sample Size for Prespecified Power

Post Hoc Power: A Useless Statistic

**Robustness and Nonparametric Methods**

Introduction

Nonparametric Tests Based on the Rank Transformation

Randomization Tests

Level and Power Robustness

Bootstrap Percentile-*t *Confidence Interval

Final Words

Index

*Vocabulary, Formula Summaries, and Exercises appear at the end of each chapter.*

## Author(s)

### Biography

**Peter H. Westfall** is the Paul Whitfield Horn Professor of Statistics and James Niver Professor of Information Systems and Quantitative Sciences at Texas Tech University. A Fellow of the ASA and the AAAS, Dr. Westfall has published several books and over 100 papers on statistical theory and methods. He also has won several teaching awards and is the former editor of *The American Statistician*. He earned a PhD in statistics from the University of California, Davis.

**Kevin S.S. Henning **is a clinical assistant professor of business analysis in the Department of Economics and International Business at Sam Houston State University, where he teaches business statistics and forecasting. He earned a PhD in business statistics from Texas Tech University.

## Reviews

"This nicely written textbook fills the gap between elementary statistics courses and more advanced research methods courses. The book helps one to grasp the key assumptions and machinery behind advanced statistical topics … Each chapter ends with useful exercises."

—Mathematical Reviews, August 2014"… full of interesting insights and excellent examples and explanations for essential basic statistical concepts. The use of thought experiments; the detailed algebraic developments of proofs; and the explanations of frequentist and Bayesian statistics, confidence intervals, hypothesis testing, and so on, are all first rate. … a solid teaching resource."

—Australian & New Zealand Journal of Statistics, 2014"… useful as a prerequisite for advanced study of statistical analysis, such as regression, experimental design, survival analysis, and categorical data analysis … examples in this book seem very useful and may help expand the view of newcomers to statistics."

—Biometrics, June 2014"This book contains just as many formulas as other statistics texts, but with intuitive, engaging, insightful, and irreverent explanations … the authors strive mightily to part the curtain that hides the fundamentals of statistical thinking from most students. … The book has 20 chapters that cover the usual topics, and more, in an undergraduate/graduate math stat text; it is suitable for a fast-paced semester course offered to serious students. The ‘and more’ refers to the strong emphasis throughout the book on thoughtful applications in a wide variety of disciplines. … The coverage of mathematical statistics is extensive and benefits from a substantial effort by the authors to explain the intuition motivating the procedures and the correct interpretation of specific results. … A companion Web site has a wealth of material useful for the instructor and students. … the text represents a successful effort by the authors to advance and improve the statistics education paradigm for courses offered to upper-level undergraduate and graduate students."

—The American Statistician, May 2014"There is a gap between elementary statistics courses and advanced research techniques. This gap is reflected by difficulties in linking statistical theory with its application in the real world. This book is an ideal way to overcome this problem. …

The main advantage of this book is the possibility to achieve advanced research skills. The theory behind data analysis is well explained, using plenty of real examples from social, economic, medical, physical and engineering sciences. The theory and application are well balanced and very well linked. All examples are illustrated in MS Excel.

This book helps to teach students to explore statistics more deeply, avoiding the typical trap of students learning little about the applications of what they are studying and why they are doing it. I think this book will be very useful in the sense that students will be forced to think differently about things, not only about math and statistics, but also about research and the scientific method.

The reviewer enjoyed reading the book and it is worth emphasising its usefulness for teachers, students and researchers."

—Božidar V. Popović,Journal of Applied Statistics, 2014"The book covers the content of a typical undergraduate math stat text, but with much more thought to application than a typical text. It appears to be close to Rice’s text (

Mathematical Statistics and Data Analysis) in spirit and level, but perhaps comes closer to that spirit than Rice’s. It would be worth considering for a course using Rice. I also recommend it as a reference for anyone teaching applied statistics."

—Martha K. Smith, Professor Emerita of Mathematics, University of Texas at Austin"I work with scientists who are pioneers in their fields and their ignorance of statistical concepts never ceases to amaze me. I believe most of this can be traced to the way we teach statistics to non-statisticians: as a bag of tools rather than a systematic way to think about data collection and analysis. This book is unique in the way it approaches this topic. It does not subscribe to the cookbook template of teaching statistics but focuses instead on understanding the distinction between the observed data and the mechanisms that generated it. This focus allows a better distinction between models, parameters, and estimates and should help pave a way to instill statistical thinking to undergraduate students."

—Mithat Gönen, Memorial Sloan-Kettering Cancer Center"

Understanding Advanced Statistical Methodsis an excellent source for the curious student. The book introduces a novel approach to learning statistics by providing comprehensive coverage of concepts in a captivating framework. Students are not only encouraged to understand the intuition and structure behind the concepts, but also motivated to think seriously about the pertinent questions before they ask. Therefore, the book strives to build a solid background in fundamental concepts and to equip students with the necessary skills so that they can expand their toolbox in their future endeavors. The book will no doubt be the standard reference in advanced statistics courses and bring about profound changes in how statistics should be taught."

—Ozzy Akay, Assistant Professor, Texas Tech University"Don't let the authors' exuberant and iconoclastic style fool you into thinking that this book is not a serious text. It definitely is. The style has a purpose—to romp around the field's sacred cows and show the reader as quickly as possible the real working principles behind how statistical methods are developed and some of the methods’ most important applications. In that sense, the subject of the book truly is theoretical statistics, but both the motivation and the presentation are so thoroughly grounded in practice that many readers will see it as a practical guide. But the authors don’t intend for it to be a statistical cheat sheet: each of their many engaging and illuminating examples points forward to more that could be studied, and invites readers to pursue those studies. This isn’t the last statistics textbook students will ever need, but it should be the first."

—Randy Tobias, Director, Linear Models R&D, SAS Institute Inc.