1st Edition

Invariant Measurement with Raters and Rating Scales Rasch Models for Rater-Mediated Assessments

By George Engelhard Jr., Stefanie Wind Copyright 2018
    368 Pages
    by Routledge

    368 Pages
    by Routledge

    The purpose of this book is to present methods for developing, evaluating and maintaining rater-mediated assessment systems. Rater-mediated assessments involve ratings that are assigned by raters to persons responding to constructed-response items (e.g., written essays and teacher portfolios) and other types of performance assessments. 

    This book addresses the following topics: (1) introduction to the principles of invariant measurement, (2) application of the principles of invariant measurement to rater-mediated assessments, (3) description of the lens model for rater judgments, (4) integration of principles of invariant measurement with the lens model of cognitive processes of raters, (5) illustration of substantive and psychometric issues related to rater-mediated assessments in terms of validity, reliability, and fairness, and (6) discussion of theoretical and practical issues related to rater-mediated assessment systems.

    Invariant measurement is fast becoming the dominant paradigm for assessment systems around the world, and this book provides an invaluable resource for graduate students, measurement practitioners, substantive theorists in the human sciences, and other individuals interested in invariant measurement when judgments are obtained with rating scales.


    I. Introduction

    1. Introduction and Overview

    2. Progress in the Social Sciences: An Historical and Philosophical Perspective

    II. Theories of Measurement and Judgment for Rating Scales

    3. Measurement Models for Rater-Mediated Assessments: A Tale of Two Research Traditions

    4. Lens Models of Human Judgment for Rater-Mediated Assessments

    III. Foundational Areas for Rating Scales

    5. Validity, Invariant Measurement, and Rater-Mediated Assessments

    6. Reliability, Precision, and Errors of Measurement for Ratings

    7. Fairness in Rater-Mediated Assessment: Appropriate Interpretation and Use of Ratings

    8. Case Study: Evidence for the Validity, Reliability, and Fairness of Ratings on a Middle Grades Writing Assessment

    IV. Technical Issues and IRT Models for Ratings

    9. Models for Ratings Based on Item Response Theory

    10. Parameter Estimation for the Polytomous Rasch Model

    V. Practical Issues

    11. Model-Data Fit for Polytomous Rating-Scale Models

    12. Designing Rater-Mediated Assessment Systems

    13. Examining Rating Scale Functioning

    VI. Final Word

    14. Invariant Measurement with Raters and Rating Scales: Summary and Discussion


    George Engelhard, Jr. is a professor at The University of Georgia, USA, and a professor emeritus at Emory University, USA.

    Stefanie A. Wind is an assistant professor at The University of Alabama, USA.  

     "No author can match George Engelhard’s oeuvre in the use of Rasch models for rater-mediated performance assessments. This latest volume, with Stefanie Wind, draws on a broad background of measurement theory, philosophy and test standards to focus invariant measurement – the crucial next step in measurement practice. This will be my go-to reference text." - Trevor G. Bond, James Cook University, Australia

    "This is an accessible and comprehensive treatment on the use of ratings for measurement purposes. The authors are trusted sources who are active in the field and that practical knowledge and experience shows in their writing." Jonathan Templin, University of Kansas, USA