1st Edition
Invariant Measurement with Raters and Rating Scales Rasch Models for Rater-Mediated Assessments
The purpose of this book is to present methods for developing, evaluating and maintaining rater-mediated assessment systems. Rater-mediated assessments involve ratings that are assigned by raters to persons responding to constructed-response items (e.g., written essays and teacher portfolios) and other types of performance assessments.
This book addresses the following topics: (1) introduction to the principles of invariant measurement, (2) application of the principles of invariant measurement to rater-mediated assessments, (3) description of the lens model for rater judgments, (4) integration of principles of invariant measurement with the lens model of cognitive processes of raters, (5) illustration of substantive and psychometric issues related to rater-mediated assessments in terms of validity, reliability, and fairness, and (6) discussion of theoretical and practical issues related to rater-mediated assessment systems.
Invariant measurement is fast becoming the dominant paradigm for assessment systems around the world, and this book provides an invaluable resource for graduate students, measurement practitioners, substantive theorists in the human sciences, and other individuals interested in invariant measurement when judgments are obtained with rating scales.
Preface.
I. Introduction
1. Introduction and Overview
2. Progress in the Social Sciences: An Historical and Philosophical Perspective
II. Theories of Measurement and Judgment for Rating Scales
3. Measurement Models for Rater-Mediated Assessments: A Tale of Two Research Traditions
4. Lens Models of Human Judgment for Rater-Mediated Assessments
III. Foundational Areas for Rating Scales
5. Validity, Invariant Measurement, and Rater-Mediated Assessments
6. Reliability, Precision, and Errors of Measurement for Ratings
7. Fairness in Rater-Mediated Assessment: Appropriate Interpretation and Use of Ratings
8. Case Study: Evidence for the Validity, Reliability, and Fairness of Ratings on a Middle Grades Writing Assessment
IV. Technical Issues and IRT Models for Ratings
9. Models for Ratings Based on Item Response Theory
10. Parameter Estimation for the Polytomous Rasch Model
V. Practical Issues
11. Model-Data Fit for Polytomous Rating-Scale Models
12. Designing Rater-Mediated Assessment Systems
13. Examining Rating Scale Functioning
VI. Final Word
14. Invariant Measurement with Raters and Rating Scales: Summary and Discussion
Biography
George Engelhard, Jr. is a professor at The University of Georgia, USA, and a professor emeritus at Emory University, USA.
Stefanie A. Wind is an assistant professor at The University of Alabama, USA.
"No author can match George Engelhard’s oeuvre in the use of Rasch models for rater-mediated performance assessments. This latest volume, with Stefanie Wind, draws on a broad background of measurement theory, philosophy and test standards to focus invariant measurement – the crucial next step in measurement practice. This will be my go-to reference text." - Trevor G. Bond, James Cook University, Australia
"This is an accessible and comprehensive treatment on the use of ratings for measurement purposes. The authors are trusted sources who are active in the field and that practical knowledge and experience shows in their writing." – Jonathan Templin, University of Kansas, USA
