1st Edition
Education Research On Trial Policy Reform and the Call for Scientific Rigor
Read the author's commentary for the Teachers College Record here: http://www.tcrecord.org/Content.asp?ContentID=15915
It is not an exaggeration to say that the field of education has been under attack. Many, particularly in Washington, D.C., have proclaimed the research to be shoddy. They have called for new "scientific" standards for research. Randomized control trials have been promoted. In many of these discussions, the only criterion is making a more rational and scientific approach to education research. Since the federal government plays a leadership role in defining the terms of education debates, this critique is important. It stands to radically reshape research and possibly school priorities in the future.
The essays in this book take up this important topic. They offer critical insight into how this debate came to flourish. Some of the authors take issue with core assertions of the debate; other are sympathetic. Taken together, they help to broaden and deepen our understanding of the efforts to revamp the field of education research and, ultimately education. The chapters also discuss the factors that facilitate, and impede, research from having an impact on policy.
Teaching and Learning Goals Include:
-- helps illuminate the relationship between education research and policy
--critically examines key assumptions of federal legislation particularly the call for scientific rigor in the No Child Left Behind Legislation
--helps students understand the broader intellectual context of this crisis in education
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Pamela Barnhouse Walters and Annette Lareau
Part I: The Call for Rigor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
The first set of reprinted documents below traces the development of the charges that education research is low quality and of limited usefulness. The second set highlights the key elements of the federal reforms to improve education research. The final set provides examples of some of the major responses from the education research community.
- The Problem
- Remedies for Improvement
- Reactions from the Education Research Community . . . . . . . . . . . . . . . . . . . . . . 53
- The Politics of Science: Battles for Scientific Authority in the Field of
- A History of Efforts to Improve the Quality of Federal Education
- Assessing Quality in Educational Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
- Can Non-Randomized Studies Provide Evidence of Causal Effects?
- Blending Quality and Utility: Lessons Learned From the Quality Debates . . . . 260
- Narrow Questions, Narrow Answers: The Need to Broaden the
- A Quixotic Quest? Philosophical Issues in Assessing the Quality of
Carl F. Kaestle, "The Awful Reputation of Education Research." . . . . . . . . . . . . . . . . 23
Grover J. (Russ) Whitehurst, "New Wine, New Bottles." Address at the
annual meeting of the American Educational Research Association. . . . . . . . . . . . . . 30
Selection from Request for Proposals for Predoctoral Interdisciplinary
Research Training Programs in the Education Sciences, Issued by the
Institute of Education Sciences in 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
The Definition of "Scientifically Based Research" in the No Child Left
Behind Act of 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
The Definitions of Scientific Validity in the Education Sciences
Reform Act of 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Mission and Functions of the Institute of Education Sciences, as Detailed
in Education Sciences Reform Act of 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
The What Works Clearinghouse Standards for Evaluating Existing
Research Studies of Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
A selection from the 2002 report from the National Research Council,
Scientific Research in Education. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Margaret Eisenhart and Lisa Towne, "Contestation and Change in
National Policy on ‘Scientifically Based’ Education Research." . . . . . . . . . . . . . . . . 64
The American Educational Research Association’s "Standards for
Reporting on Empirical Social Science Research in AERA Publications." . . . . . . . . 72
Part II: The Politics of Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Education Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Pamela Barnhouse Walters
Walters argues that the current debates about the quality of education research and the best ways to improve it do not turn only on issues of the scientific merits of competing positions. The debates are part of political and social struggles between groups of scientific experts, and between policymakers and scientific experts, over who gets to decide what counts as science and to claim scientific legitimacy within the research field.
Research: From Gardner’s Task Force to the Institute of Education
Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Maris Vinovskis
Vinovskis shows that the current critique of the quality of education research is related in important ways to recurring dissatisfaction on the part of federal lawmakers and bureaucrats with the decisions and priorities of the federal agencies that provide the bulk of federal funding for education research.
Part III: Seeking Rigor; Finding Rigor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Barbara Schneider
Schneider addresses the question of whether the quality of education research is as bad as its critics charge by comparing the scientific standards and processes in place at major education journals with the standards and processes in place in journals in other fields generally considered to be more scientific. She finds the education journals to be comparably rigorous.
A Case Study Using the Regression Discontinuity Design . . . . . . . . . . . . . . . . . . 228
Larry V. Hedges and Jennifer Hanis
While in sympathy with the call to make education research more rigorous, Hedges and Hanis show that randomized controlled trials are not the only way to rigorously assess causal relationships about education. They illustrate the usefulness of regression discontinuity models for assessing causality in conditions in which random assignment is not possible.
Sheri Ranis
Ranis shows that the debates about the quality of education research have been propelled by and conflated with debates about the utility of education research in ways often unacknowledged. She demonstrates that research utility became a "resonant problematic" that provided a powerful justification for the movement to improve the quality of education research.
Part IV: Toward a More Comprehensive Understanding of Science . . . . . . . . . . . . . . 286
Methodological Scope of Education Research . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Annette Lareau
Lareau argues that the education sciences movement has misapplied the medical model to education research. She suggests there is a need for more attention to a broader array of questions about meaning, process, and interactional dynamics and greater attention to issues of implementation.
Education Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Denis C. Phillips
Philips demonstrates that the current drive to establish a single model of scientific research in education takes an overly-simplistic view of the nature of "science," in the process ignoring the complexities inherent in studying the intrinsically social and cultural dynamics of schooling. He calls the search for a single model of scientific research a "quixotic quest."
Biography
Pamela B. Walters, Annette Lareau, Sheri Ranis
" As a whole, this book is insightful, thorough, and occasionally feisty." -- The Canadian Journal of Sociology, 2009
