1st Edition
Fundamental Considerations in Technology Mediated Language Assessment
Fundamental Considerations in Technology Mediated Language Assessment aims to address issues such as how the forced integration of technology into second language assessment has shaped our understanding of key traditional concepts like validity, reliability, washback, authenticity, ethics, fairness, test security, and more.
Although computer-assisted language testing has been around for more than two decades in the context of high-stakes proficiency testing, much of language testing worldwide has shifted to “at-home” mode, and relies heavily on the mediation of digital technology, making its widespread application in classroom settings in response to the COVID-19 outbreak unprecedented. Integration of technology into language assessment has brought with it countless affordances and at the same time challenges, both theoretically and practically. One major theoretical consideration requiring attention is the way technology has contributed to a reconceptualization of major assessment concepts/constructs. There is very limited literature available on the theoretical underpinnings of technology mediated language assessment. This book aims to fill this gap.
This book will appeal to academic specialists, practitioners, or professionals in the field of language assessment, advanced and/or graduate students, and a range of scholars or professionals in disciplines like educational technology, applied linguistics, and teaching English to speakers of other languages (TESOL).
Acknowledgements
List of contributors
Foreword: Lyle F Bachman
Chapter 1: Technology mediated language assessment: Key considerations
Karim Sadeghi, Dan Douglas
Section I: Validity Concerns in Technology Mediated Language Assessment
Chapter 2: Validation in remote L2 assessment: Appraising models and frameworks
Scott Walters
Chapter 3: Exploring the Role of Self-Regulation in Young Learners’ Writing Assessment and Intervention using BalanceAI Automated Diagnostic Feedback
Eunice Eunhee Jang, Melissa Hunte, Christie Barron, Liam Hannah
Chapter 4: Balancing construct coverage and efficiency: Test design and validation considerations for a remote-proctored online language test
Larry Davis, John M. Norris, Spiros Papageorgiou, Shoko Sasayama
Chapter 5: From pen-and-paper trials to computer-based test: Impact on validity
Ivy Chen, Ute Knoch
Chapter 6: Evaluating Finnish 9th graders’ learning outcomes in English during the pandemic: How valid are the conclusions drawn?
Marita Härmälä
Section II: Reliability Issues and Machine Scoring
Chapter 7: Exploring rater behaviors on handwritten and typed reading-to-write essays using FACETS
Sun-Young Shin, Senyung Lee, Yena Park
Chapter 8: Validating an AI-driven Scoring System: the Model Card approach
Barry O’Sullivan, Trevor Breakspear, William Bayliss
Chapter 9: Web-based testing and automated scoring: construct conceptualization and improving reliability
Nathan Carr
Section III: Impact, Security and Ethical Considerations
Chapter 10: Testing young multilingual learners remotely: prioritizing equity, fairness, and social justice
Mark Chapman, Jason A. Kemp, Ahyoung Alicia Kim, David MacGregor, Fabiana MacMillan,
Chapter 11: Improving the impact of technology on diagnostic language assessment
Ari Huhta
Chapter 12: Proctoring remote language assessments
Erik Voss
Section IV: Options and Issues in Technology Mediated Classroom Assessment
Chapter 13: Interdisciplinary collaboration in building the future of Learning Oriented Language Assessment
Nick Saville, Paula Buttery
Chapter 14: Interactional competence in video-mediated speaking assessment in Vietnam
Noriko Iwashita, Van Trao Nguyen, Hong Giang Nguyen
Chapter 15: Test-taking strategies in technology-assisted language assessment
Andrew Cohen, Teymour Rahmati, Karim Sadeghi
Chapter 16: Researching technology-mediated classroom language assessment practices: Methodological challenges and opportunities
Keith Menary, Luke Harding
Chapter 17: Conclusion
Dan Douglas, Karim Sadeghi
Biography
Karim Sadeghi is a Professor of TESOL at Urmia University. He is the founding Editor-in-Chief of the Iranian Journal of Language Teaching Research (Scopus Q1 Journal). His recent publications include Assessing Second Language Reading (2021), Talking about Second Language Acquisition (2022), and Theory and Practice in Second Language Teacher Identity (2022, co-edited with Dr Farah Ghaderi).
Dan Douglas is a Professor Emeritus in the Applied Linguistics and Technology program at Iowa State University, specializing in assessing language ability in specific academic and professional contexts. His books include Understanding Language Testing (2010) and Assessing Languages for Specific Purposes (2000). He was President of the International Language Testing Association from 2005 to 2006 and again from 2013 to 2015, and Editor of the journal Language Testing from 2002 to 2007. In 2019 he received the Cambridge/ILTA Distinguished Achievement Award.
"Technology has taken over language assessment and we are all becoming familiar with the latest generation of products, taking assessments at home on mobile devices, interacting with avatars and having our performance scored by a machine. In this pioneering collection, Sadeghi and Douglas have brought together some of the leading voices in the field to help us to get to grips with the profound practical and theoretical questions raised by the shift to technology mediated assessment. This book is indispensable reading for everyone who makes or studies language assessments in the digital age."
Tony Green, University of Bedfordshire, UK
"This book presents in depth discussions by some forty language testing scholars from all over the world on the use of technology and AI in language assessment. Most chapters recognize that Covid-19 has driven learning, teaching and assessment to the internet and observe how the negative impact of the epidemic on students’ participation in schooling is paired with positive impulses for technological developments, thereby offering new opportunities for remote assessment and for timely and adaptive feedback to students’ learning. The issue of validity is recurrent throughout the book. Yet as long as the language testing field judges the performance of machines by their capability to generate outcomes similar to those produced by humans, automated processes will never reach their full potential."
John H. A. L. De Jong, Language Testing Services
"Educators have been utilizing technology steadily and increasingly in different areas of education for almost half a century. The assessment field is no exception. However, the COVID -19 Pandemic forced education into an almost complete online system where the infrastructures were not adequate in many places. Further, although some educational organizations offered online courses before the pandemic, many teachers did not have enough knowledge or could not receive training in time to operate in virtual contexts. More importantly, the critical role of assessment in making valid and reliable decisions required more secure technology than those in other fields. In such a critical context, practitioners needed guidelines from specialists and policymakers to cope with the crisis. This book is a pleasant contribution to addressing their needs. The selection of the topics demonstrates the editors’ informed taste and deep concern about the use of technology in language assessment. The variety of the issues addressed in the book will make it helpful to a wide range of practitioners and scholars. An important point that adds to the value of this book is that it will serve as a useful reference even after the COVID-19 pandemic when technology continues to dominate education in general and language assessment in particular."
Hosssein Farhady, Professor of Applied Linguistics, Yeditepe University, Istanbul, Turkey
