1st Edition

Fundamental Considerations in Technology Mediated Language Assessment

Edited By Karim Sadeghi, Dan Douglas Copyright 2023
    302 Pages 20 B/W Illustrations
    by Routledge

    302 Pages 20 B/W Illustrations
    by Routledge

    Fundamental Considerations in Technology Mediated Language Assessment aims to address issues such as how the forced integration of technology into second language assessment has shaped our understanding of key traditional concepts like validity, reliability, washback, authenticity, ethics, fairness, test security, and more.

    Although computer-assisted language testing has been around for more than two decades in the context of high-stakes proficiency testing, much of language testing worldwide has shifted to “at-home” mode, and relies heavily on the mediation of digital technology, making its widespread application in classroom settings in response to the COVID-19 outbreak unprecedented. Integration of technology into language assessment has brought with it countless affordances and at the same time challenges, both theoretically and practically. One major theoretical consideration requiring attention is the way technology has contributed to a reconceptualization of major assessment concepts/constructs. There is very limited literature available on the theoretical underpinnings of technology mediated language assessment. This book aims to fill this gap.

    This book will appeal to academic specialists, practitioners, or professionals in the field of language assessment, advanced and/or graduate students, and a range of scholars or professionals in disciplines like educational technology, applied linguistics, and teaching English to speakers of other languages (TESOL).


    List of contributors

    Foreword: Lyle F Bachman

    Chapter 1: Technology mediated language assessment: Key considerations

    Karim Sadeghi, Dan Douglas

    Section I: Validity Concerns in Technology Mediated Language Assessment

    Chapter 2: Validation in remote L2 assessment: Appraising models and frameworks

    Scott Walters

    Chapter 3: Exploring the Role of Self-Regulation in Young Learners’ Writing Assessment and Intervention using BalanceAI Automated Diagnostic Feedback

    Eunice Eunhee Jang, Melissa Hunte, Christie Barron, Liam Hannah

    Chapter 4:  Balancing construct coverage and efficiency: Test design and validation considerations for a remote-proctored online language test

    Larry Davis, John M. Norris, Spiros Papageorgiou, Shoko Sasayama

    Chapter 5: From pen-and-paper trials to computer-based test: Impact on validity

    Ivy Chen, Ute Knoch

    Chapter 6: Evaluating Finnish 9th graders’ learning outcomes in English during the pandemic: How valid are the conclusions drawn?

    Marita Härmälä

    Section II: Reliability Issues and Machine Scoring

    Chapter 7: Exploring rater behaviors on handwritten and typed reading-to-write essays using FACETS

    Sun-Young Shin, Senyung Lee, Yena Park

    Chapter 8: Validating an AI-driven Scoring System: the Model Card approach

    Barry O’Sullivan, Trevor Breakspear, William Bayliss

    Chapter 9: Web-based testing and automated scoring: construct conceptualization and improving reliability

    Nathan Carr

    Section III: Impact, Security and Ethical Considerations

    Chapter 10: Testing young multilingual learners remotely: prioritizing equity, fairness, and social justice

    Mark Chapman, Jason A. Kemp, Ahyoung Alicia Kim, David MacGregor, Fabiana MacMillan,

    Chapter 11: Improving the impact of technology on diagnostic language assessment

    Ari Huhta

    Chapter 12: Proctoring remote language assessments

    Erik Voss

    Section IV: Options and Issues in Technology Mediated Classroom Assessment

    Chapter 13: Interdisciplinary collaboration in building the future of Learning Oriented Language Assessment

    Nick Saville, Paula Buttery

    Chapter 14: Interactional competence in video-mediated speaking assessment in Vietnam

    Noriko Iwashita, Van Trao Nguyen, Hong Giang Nguyen 

    Chapter 15: Test-taking strategies in technology-assisted language assessment

    Andrew Cohen, Teymour Rahmati, Karim Sadeghi

    Chapter 16: Researching technology-mediated classroom language assessment practices: Methodological challenges and opportunities

    Keith Menary, Luke Harding

    Chapter 17: Conclusion

    Dan Douglas, Karim Sadeghi


    Karim Sadeghi is a Professor of TESOL at Urmia University. He is the founding Editor-in-Chief of the Iranian Journal of Language Teaching Research (Scopus Q1 Journal). His recent publications include Assessing Second Language Reading (2021), Talking about Second Language Acquisition (2022), and Theory and Practice in Second Language Teacher Identity (2022, co-edited with Dr Farah Ghaderi).

    Dan Douglas is a Professor Emeritus in the Applied Linguistics and Technology program at Iowa State University, specializing in assessing language ability in specific academic and professional contexts. His books include Understanding Language Testing (2010) and Assessing Languages for Specific Purposes (2000). He was President of the International Language Testing Association from 2005 to 2006 and again from 2013 to 2015, and Editor of the journal Language Testing from 2002 to 2007. In 2019 he received the Cambridge/ILTA Distinguished Achievement Award.

    "Technology has taken over language assessment and we are all becoming familiar with the latest generation of products, taking assessments at home on mobile devices, interacting with avatars and having our performance scored by a machine. In this pioneering collection, Sadeghi and Douglas have brought together some of the leading voices in the field to help us to get to grips with the profound practical and theoretical questions raised by the shift to technology mediated assessment. This book is indispensable reading for everyone who makes or studies language assessments in the digital age."

    Tony Green, University of Bedfordshire, UK


    "This book presents in depth discussions by some forty language testing scholars from all over the world on the use of technology and AI in language assessment. Most chapters recognize that Covid-19 has driven learning, teaching and assessment to the internet and observe how the negative impact of the epidemic on students’ participation in schooling is paired with positive impulses for technological developments, thereby offering new opportunities for remote assessment and for timely and adaptive feedback to students’ learning. The issue of validity is recurrent throughout the book. Yet as long as the language testing field judges the performance of machines by their capability to generate outcomes similar to those produced by humans, automated processes will never reach their full potential."

    John H. A. L. De Jong, Language Testing Services 


    "Educators have been utilizing technology steadily and increasingly in different areas of education for almost half a century. The assessment field is no exception. However, the COVID -19 Pandemic forced education into an almost complete online system where the infrastructures were not adequate in many places. Further, although some educational organizations offered online courses before the pandemic, many teachers did not have enough knowledge or could not receive training in time to operate in virtual contexts. More importantly, the critical role of assessment in making valid and reliable decisions required more secure technology than those in other fields. In such a critical context, practitioners needed guidelines from specialists and policymakers to cope with the crisis. This book is a pleasant contribution to addressing their needs. The selection of the topics demonstrates the editors’ informed taste and deep concern about the use of technology in language assessment. The variety of the issues addressed in the book will make it helpful to a wide range of practitioners and scholars. An important point that adds to the value of this book is that it will serve as a useful reference even after the COVID-19 pandemic when technology continues to dominate education in general and language assessment in particular."

    Hosssein Farhady, Professor of Applied Linguistics, Yeditepe University, Istanbul, Turkey