Computerized Multistage Testing: Theory and Applications, 1st Edition (Hardback) book cover

Computerized Multistage Testing

Theory and Applications, 1st Edition

Edited by Duanli Yan, Alina A. von Davier, Charles Lewis

Chapman and Hall/CRC

546 pages | 90 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781466505773
pub: 2014-05-07
SAVE ~$19.59
eBook (VitalSource) : 9780429096358
pub: 2016-04-19
from $46.98

FREE Standard Shipping!


Devising tests that evaluate a nation’s educational standing and implement efficacious educational reforms requires a careful balance among the contributions of technology, psychometrics, test design, and the learning sciences. Unlike other forms of adaptive testing, multistage testing (MST) is highly suitable for testing educational achievement because it can be adapted to educational surveys and student testing. Computerized Multistage Testing: Theory and Applications covers the methodologies, underlying technology, and implementation aspects of this type of test design.

The book discusses current scientific perspectives and practical considerations for each step involved in setting up an MST program. It covers the history of MST, test design and implementation for various purposes, item pool development and maintenance, IRT-based and classical test theory-based methodologies for test assembly, routing and scoring, equating, test security, and existing software. It also explores current research, existing operational programs, and innovative future assessments using MST.

Intended for psychologists, social scientists, and educational measurement scientists, this volume provides the first unified source of information on the design, psychometrics, implementation, and operational use of MST. It shows how to apply theoretical statistical tools to testing in novel and useful ways. It also explains how to explicitly tie the assumptions made by each model to observable (or at least inferable) data conditions.

Winner of the 2016 AERA Award for Significant Contribution to Educational Measurement and Research Methodology

The 2016 American Education Research Association (AERA) Div. D award committee for Significant Contributions to Educational Measurement and Research Methodology has recognized unanimously this collaborative work advancing the theory and applications of computerized MST. This annual award recognizes published research judged to represent a significant conceptual advancement in the theory and practice of educational measurement and/or educational research methodology. The 2016 award was made under the heading: Measurement, Psychometrics, and Assessment. This collective work, published in 2014 as an edited volume titled Computerized Multistage Testing: Theory and Applications, was cited by the committee both for the originality of the conceptual foundations presented in support of multistage testing and for arguing persuasively for its potential impact on the practice of educational measurement.


"… this is a terrific book and the editors should be congratulated."

Psychometrika, Vol. 80, No. 1, March 2015

Table of Contents

Test Design, Item Pool, and Maintenance

Overview of Computerized Multistage Tests Duanli Yan, Charles Lewis, and Alina A. von Davier

Linear Tests and Computerized Adaptive Tests (CATs)

Multistage Tests (MSTs)

MST Designs for Different Purposes

Implementation Schemes

Designing MST

Content Balance and Assembly

Exposure Control


Scoring, Linking, and Equating

Reliability, Validity, Fairness, and Test Security

Current and Future Applications

Logistic Challenges


Multistage Test Designs: Moving Research Results into Practice April L. Zenisky and Ronald K. Hambleton

The MST Design Structure

The State of Research: MST Development and Design Considerations

Conclusions and Next Steps

Item Pool Design and Maintenance for Multistage Testing Bernard P. Veldkamp

Designing an Item Pool Blueprint

Applications in Item Writing



Mixed-Format Tests: Issues and Methods Jiseon Kim and Barbara G. Dodd

Literature Review on Design Components in Mixed-Format MST

Comparing Other Testing Approaches

Issues and Future Research Suggestions for Mixed-Format MST


Design and Implementation of Large-Scale Multistage Testing Systems Richard Luecht

MST Design and Implementation Considerations

Conclusions: A Research Agenda

Test Assembly

Overview of Test Assembly Methods in Multistage Testing Yi Zheng, Chun Wang, Michael J. Culbertson, and Hua-Hua Chang

MST Framework

MST Assembly Design

Automated Assembly for MST

Setting Difficulty Anchors and Information Targets for Modules

"On-the-Fly" MST (OMST) Assembly Paradigm

MST, CAT, and Other Designs—Which Way to Go?

Using a Universal Shadow-Test Assembler with Multistage Testing Wim J. van der Linden and Qi Diao

Solving Shadow-Test Assembly Problems

Basic Design Parameters

Different Testing Formats

Relative Efficiency of Formats

Empirical Study

Concluding Comments

Multistage Testing by Shaping Modules on the Fly Kyung (Chris) T. Han and Fanmin Guo

MST by Shaping

MST-S versus MST-R versus CAT

Discussion and Conclusion

Optimizing the Test Assembly and Routing for Multistage Testing Angela Verschoor and Theo Eggen

Optimizing MST Assembly: A Nonexhaustive Search

Limited Item Pools, Two- and Three-Parameter Models


Routing, Scoring, and Equating

IRT-Based Multistage Testing Alexander Weissman


Motivation for Tailored Testing

Routing Rules

Scoring and Classification Methodologies

Final Comments

A Tree-Based Approach for Multistage Testing Duanli Yan, Charles Lewis, and Alina A. von Davier

Regression Trees

Tree-Based Computerized Adaptive Tests

Tree-Based Multistage Testing


An Application


Limitations and Future Research

Multistage Testing for Categorical Decisions Robert Smith and Charles Lewis

Computer-Mastery Methods

Information Targeted at Cut versus at Ability

Influence of Multiple Cut Scores

Factors That Can Reduce Optimal Solutions

Example Based on Smith and Lewis (1995)

Adaptive Mastery Multistage Testing Using a Multidimensional IRT Model C.A.W. Glas


Definition of the Decision Problem

Computation of Expected Loss and Risk Using Backward Induction

Selection of Items and Testlets

Simulation Studies

Conclusions and Further Research

Multistage Testing Using Diagnostic Models Matthias von Davier and Ying (Alison) Cheng

The DINA Model and the General Diagnostic Model

Experience with CD-CATs



Considerations on Parameter Estimation, Scoring, and Linking in Multistage Testing Shelby J. Haberman and Alina A. von Davier


The Item Response Model

The Test Score

Approaches to Parameter Estimation


Test Reliability, Validity, Fairness, and Security

Reliability of Multistage Tests Using Item Response Theory Peter W. van Rijn

Test Reliability

Application: IRT Reliability for MST in NAEP


Multistage Test Reliability Estimated via Classical Test Theory Samuel A. Livingston and Sooyeon Kim

The Estimation Procedure

Testing the Accuracy of the Estimation Procedure

How Accurate Were the Estimates?

Evaluating Validity, Fairness, and Differential Item Functioning in Multistage Testing Rebecca Zwick and Brent Bridgeman

Content Balancing

Opportunities for Item Review and Answer Changing

Skipping Strategies

MST Routing Algorithms

The Digital Divide

Comparability of Computer Platforms

Accommodations for Students with Disabilities and English Language Learners

Differential Item Functioning Analysis in MSTs

Application of the Empirical Bayes DIF Approach to Simulated MST Data


Test Security and Quality Control for Multistage Tests Yi-Hsuan Lee, Charles Lewis, and Alina A. von Davier

An Overview of a Three-Component Procedure

Tools to Evaluate Test Security and Quality Control

A Simulation Study Using CUSUM Statistics to Monitor Item Performance


Applications in Large-Scale Assessments

Multistage Test Design and Scoring with Small Samples Duanli Yan, Charles Lewis, and Alina A. von Davier

Small Data Sample

Item Pool

Various MST Module Designs

Routing and Scoring

Comparisons of the Six MST Designs


Limitations and Future Research

The Multistage Test Implementation of the GRE Revised General Test Frédéric Robin, Manfred Steffen, and Longjuan Liang

From CAT to MST

MST Design




The Multistage Testing Approach to the AICPA Uniform Certified Public Accounting Examinations Krista J. Breithaupt, Oliver Y. Zhang, and Donovan R. Hare

Research on Multistage Testing

Item Bank Development for MST

Content Security Monitoring for MST

Inventory Exposure Planning for MST


Transitioning a K–12 Assessment from Linear to Multistage Tests Carolyn Wentzel, Christine M. Mills, and Kevin C. Meara

Administering CTP Items Online

Creating a New MST Scale Using IRT

Multistage-Adaptive Test Development

Score Reporting


A Multistage Testing Approach to Group-Score Assessments Andreas Oranje, John Mazzeo, Xueli Xu, and Edward Kulick

Targeted Testing

Goals of the Study




Controlling Multistage Testing Exposure Rates in International Large-Scale Assessments Haiwen Chen, Kentaro Yamamoto, and Matthias von Davier

Item Exposure Rate Control for Multistage Adaptive Assessments

Method: How to Compute and Adjust the Item Exposure Rates




Software Tools for Multistage Testing Simulations Kyung (Chris) T. Han and Michal Kosinski




Closing Remarks

Past and Future of Multistage Testing in Educational Reform Isaac I. Bejar

Future of MST

A Model-Based Three-Stage Design

Item Generation and Automated Scoring and Broadly Accessible Test Content

Summary and Conclusions


About the Series

Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
MATHEMATICS / Probability & Statistics / General