Observer Performance Methods for Diagnostic Imaging: Foundations, Modeling, and Applications with R-Based Examples, 1st Edition (Hardback) book cover

Observer Performance Methods for Diagnostic Imaging

Foundations, Modeling, and Applications with R-Based Examples, 1st Edition

By Dev P. Chakraborty

CRC Press

542 pages | 95 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781482214840
pub: 2017-12-21
SAVE ~$37.00
$185.00
$148.00
x


FREE Standard Shipping!

Description

"This book presents the technology evaluation methodology from the point of view of radiological physics and contrasts the purely physical evaluation of image quality with the determination of diagnostic outcome through the study of observer performance. The reader is taken through the arguments with concrete examples illustrated by code in R, an open source statistical language."

– from the Foreword by Prof. Harold L. Kundel, Department of Radiology, Perelman School of Medicine, University of Pennsylvania

"This book will benefit individuals interested in observer performance evaluations in diagnostic medical imaging and provide additional insights to those that have worked in the field for many years."

– Prof. Gary T. Barnes, Department of Radiology, University of Alabama at Birmingham

This book provides a complete introductory overview of this growing field and its applications in medical imaging, utilizing worked examples and exercises to demystify statistics for readers of any background. It includes a tutorial on the use of the open source, widely used R software, as well as basic statistical background, before addressing localization tasks common in medical imaging. The coverage includes a discussion of study design basics and the use of the techniques in imaging system optimization, memory effects in clinical interpretations, predictions of clinical task performance, alternatives to ROC analysis, and non-medical applications.

Dev P. Chakraborty, PhD, is a clinical diagnostic imaging physicist, certified by the American Board of Radiology in Diagnostic Radiological Physics and Medical Nuclear Physics. He has held faculty positions at the University of Alabama at Birmingham, University of Pennsylvania, and most recently at the University of Pittsburgh.

Reviews

"This book will benefit individuals interested in observer performance evaluations in diagnostic medical imaging and provide additional insights to those that have worked in the field for many years."

— from the Foreword by Gary T. Barnes, Professor Emeritus, Department of Radiology, University of Alabama Birmingham

"As opposed to most of the books with a primary statistical orientation, this book presents the technology evaluation methodology from the point of view of radiological physics and contrasts the purely physical evaluation of image quality with the determination of diagnostic outcome through the study of observer performance. The reader is taken through the arguments with concrete examples illustrated by code in R, an open source statistical language."

— Harold L. Kundel, Emeritus Professor, Department of Radiology, Perelman School of Medicine, University of Pennsylvania

Table of Contents

Preliminaries

Introduction

Clinical tasks

Imaging device development and its clinical deployment

Image quality vs. task performance

Why physical measures of image quality are not enough

Model observers

Measuring observer performance: four paradigms

Hierarchy of assessment methods

Overview of the book and how to use it

PART I

The binary paradigm

Introduction

Decision vs. truth: the fundamental 2x2 table of ROC analysis

Sensitivity and specificity

Reasons for the names sensitivity and specificity

Estimating sensitivity and specificity

Disease prevalence

Accuracy

Positive and negative predictive values

Example: calculation of PPV and NPV

PPV and NPV are irrelevant to laboratory tasks

Modeling the binary task

Introduction

Decision variable and decision threshold

Changing the decision threshold: Example I

Changing the decision threshold: Example II

The equal-variance binormal model

The normal distribution

Demonstration of the concepts of sensitivity and specificity

Inverse variation of sensitivity and specificity

The ROC curve

Assigning confidence intervals to an operating point

Variability in sensitivity and specificity: the Beam et al study

The ratings paradigm

Introduction

The ROC counts table

Operating points from counts table

Relation between ratings paradigm and the binary task

Ratings are not numerical values

A single "clinical" operating point from ratings data

The forced choice paradigm

Observer performance studies as laboratory simulations of clinical tasks

Discrete vs. continuous ratings: the Miller study

The BIRADS ratings scale and ROC studies

The controversy

Empirical AUC

Introduction

The empirical ROC plot

Empirical operating points from ratings data

AUC under the empirical ROC plot

The Wilcoxon statistic

Bamber’s theorem

The importance of Bamber’s theorem

Appendix 5.A: Details of Wilcoxon theorem

Binormal model

Introduction

The binormal model

Least-squares estimation

Maximum likelihood estimation (MLE)

Expression for area under ROC curve

Appendix 6.A: Expressions for partial and full area under the binormal ROC

Sources of variability affecting AUC

Introduction

Three sources of variability

Dependence of AUC on the case sample

Estimating case-sampling variability using the DeLong method

Estimating case-sampling variability of AUC using the bootstrap method

Estimating case-sampling variability of AUC using the jackknife method

Estimating case-sampling variability of AUC using a calibrated simulator

Dependence of AUC on the reader’s expertise

Dependence of AUC on the modality

Effect on empirical AUC of variations in thresholds and numbers of bins

Empirical vs. fitted AUCs

PART II

Hypothesis Testing

Introduction

Hypothesis testing for a single-modality single-reader ROC study

Type-I errors

One-sided vs. two sided tests

Statistical power

Some comments on the code

Why is alpha chosen to be 5%

Dorfman-Berbaum-Metz-Hillis (DBMH) analysis

Co-author: Xuetong Zhai, MS

Introduction

Random and fixed factors

Reader and case populations and data correlations

Three types of analyses

General approach

The Dorfman-Berbaum-Metz (DBM) method

Random-reader random-case (RRRC) analysis

Fixed-reader random-case (FRRC) analysis

Random-reader fixed-case (RRFC) analysis

DBMH Analysis: Example 1

DBMH Analysis: Example 2

Validation of DBMH analysis

Meaning of pseudovalues

Obuchowski-Rockette-Hillis (ORH) analysis

Co-author: Xuetong Zhai, MS

Introduction

The single reader multiple treatment model

The multiple reader multiple treatment model ORH model

Special cases: fixed-reader and fixed-case analyses

Example of ORH analysis

Comparison of ORH and DBMH methods

Sample size estimation for ROC studies

Introduction

Statistical power

Sample size estimation

Dependence of statistical power on estimates of model parameters

Formulae for random-reader random-case (RRRC) sample size estimation

Formulae for fixed-reader random-case (FRRC) sample size estimation

Formulae for random-reader fixed-case (RRFC) sample size estimation

Example 1

Example 2

Details of the sample size estimation process

Cautionary notes: the Kraemer et al paper

Prediction accuracy of sample size estimation method

On the unit of effect-size: a proposal

PART III

The FROC paradigm

Introduction

Location specific paradigms

The FROC paradigm as a search task

A pioneering FROC study in medical imaging

Population and binned FROC plots

The "solar" analogy: search vs. classification performance

Empirical operating characteristics derivable from FROC data

Introduction

Latent vs. actual marks

Formalism: the FROC plot

Formalism: the alternative FROC (AFROC) plot

The EFROC plot

Formalism: the inferred ROC plot

Formalism: the weighted-AFROC (wAFROC) plot

Formalism: the AFROC1 plot

Formalism: the weighted-AFROC1 (wAFROC1) plot

Example: "raw" FROC plots

Confusion about location-level "true-negatives"

Example: binned FROC plots

Example: "raw" FROC / AFROC plots

Example: binned AFROC plots

Example: binned FROC/AFROC/ROC plots

Recommendations

Computation and meanings of empirical FROC FOM-statistics and AUC measures

Introduction

Empirical AFROC FOM-statistic

Empirical weighted-AFROC FOM-statistic

Two Theorems

Understanding the AFROC and wAFROC empirical plots

Physical interpretation of AFROC-based FOMs

Visual Search Paradigms

Introduction

Grouping and labeling ROIs in an image

Recognition/Finding vs. detection

Two visual search paradigms

Determining where the radiologist is looking

The Nodine - Kundel search model

Analyzing simultaneously acquired eye-tracking & FROC data

The radiological search model (RSM)

Introduction

The radiological search model (RSM)

Physical interpretation of RSM parameters

Model re-parameterization

Equation Chapter 16 Section 1

Predictions of the RSM

Introduction

Inferred integer ROC ratings

Constrained end-point property of the RSM-predicted ROC curve

The RSM-predicted ROC curve

Example: RSM-predicted ROC/pdf curves

The RSM-predicted FROC curve

The RSM-predicted AFROC curve

Quantifying search performance

Quantifying lesion-classification performance

The FROC curve is a poor descriptor of search performance

Evidence for the RSM

Fitting RSM to FROC/ROC data and key findings

Introduction

FROC Likelihood function

IDCA Likelihood function

ROC Likelihood function

RSM vs. PROPROC and CBM, and a serendipitous finding

Reason for serendipitous finding

Analyzing FROC data and sample size estimation

Introduction

Example analysis of a FROC dataset

Plotting wAFROC curves

Single fixed-factor Analysis

Crossed treatment analysis

Sample size estimation: wAFROC FOM

PART IV

Proper ROC models

Introduction

Theorem: slope of ROC equals likelihood ratio

Theorem: likelihood ratio observer maximizes AUC

Proper vs. improper ROC curves

Degenerate datasets

The likelihood ratio observer

PROPROC formalism

The contaminated binormal model

The bigamma model

 

The bivariate binormal model and its software implementation (CORROC2)

Introduction

The bivariate binormal model

The multivariate probability density function

Visualizing the bivariate probability density function

Estimating bivariate binormal model parameters

CORROC2 software

Application to a real dataset

Comparing performance of standalone CAD to a group of radiologists interpreting the same cases

Introduction

The Hupse-Karssemeijer et al. study

Extending the analysis to random cases

Ambiguity in interpreting a point-based FOM

Results using full-area measures

Design and calibration of a single-modality multiple-reader decision variable simulator and using it to validate the proposed CAD analysis method

Co-author: Xuetong Zhai, MS

Introduction

Bivariate contaminated binormal model (BCBM)

Single-modality multiple-reader decision variable simulator

Calibration, validation of simulator and testing its NH behavior

About the Author

Dev P. Chakraborty received his PhD in physics in 1977 from the University of Rochester, NY. Following postdoctoral fellowships at the University of Pennsylvania (UPENN) and the University of Alabama at Birmingham (UAB), since 1982 he has worked as a clinical diagnostic imaging physicist. He is American Board of Radiology certified in Diagnostic Radiological Physics and Medical Nuclear Physics (1987). He has held faculty positions at UAB (1982 - 1988), UPENN (1988-2002) and the University of Pittsburgh (2002-2016). At UPENN he supervised hospital imaging equipment quality control, resident physics instruction and conducted independent research. He is an author on 78 peer-reviewed publications, the majority of which are first-authored. He has received research funding from the Whittaker Foundation, the Office of Women's Health, the FDA, the DOD, and has served as principal investigator on several NIH RO1 grants.

About the Series

Imaging in Medical Diagnosis and Therapy

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
MED080000
MEDICAL / Radiology & Nuclear Medicine
SCI055000
SCIENCE / Physics
TEC059000
TECHNOLOGY & ENGINEERING / Biomedical