1st Edition
Never Waste a Good Crisis Lessons Learned from Data Fraud and Questionable Research Practices
This book covers statistical consequences of breaches of research integrity such as fabrication and falsification of data, and researcher glitches summarized as questionable research practices. It is unique in that it discusses how unwarranted data manipulation harms research results and that questionable research practices are often caused by researchers’ inadequate mastery of the statistical methods and procedures they use for their data analysis. The author’s solution to prevent problems concerning the trustworthiness of research results, no matter how they originated, is to publish data in publicly available repositories and encourage researchers not trained as statisticians not to overestimate their statistical skills and resort to professional support from statisticians or methodologists.
The author discusses some of his experiences concerning mutual trust, fear of repercussions, and the bystander effect as conditions limiting revelation of colleagues’ possible integrity breaches. He explains why people are unable to mimic real data and why data fabrication using statistical models stills falls short of credibility. Confirmatory and exploratory research and the usefulness of preregistration, and the counter-intuitivenature of statistics, are discussed.
The author questions the usefulness of statistical advice concerning frequentist hypothesis testing, Bayes-factor use, alternative statistics education, and reduction of situational disturbances like performance pressure, as stand-alone means to reduce questionable research practices when researchers lack experience with statistics.
An interview with the author can be found here: https://www.tilburguniversity.edu/magazine/overview/former-rector-sijtsma-turn-statistician-fight-fraud-and-sloppiness.
1. Why this book? 2. Fraud and Questionable Research Practices 3. Learning from Data Fraud 4. Investigating Data Fabrication and Falsification 5. Confirmation and Exploration 6. Causes of Questionable Research Practices 7. Reducing Questionable Research Practices
Biography
Klaas Sijtsma is an emeritus professor of methods and techniques of psychological research at Tilburg University. He is also a former dean of a school suffering from a huge data fraud affair by one of its eminent professors, which he had to cope with during his term. Being both an applied statistician and an experienced administrator having to deal with serious breaches of scientific integrity gave him a unique perspective on the problems discussed in this book. He gave many lectures on the topic and authored several publications. This book summarizes his views. In addition, he has published more than 200 papers and book chapters on statistical topics and coauthored three books on measurement of psychological attributes such as intelligence, personality traits, and attitudes. At present, he is co-chair of the Committee on Research Integrity of Erasmus University Rotterdam.
"This excellent book is written by the dean replacing his predecessor Diederik Stapel, who had been unmasked as large scale data fabricator. The title of the book says it all: Klaas Sijtsma didn’t waste the good crisis his university was in. The book contains an engaging mixture of the personal story of having to control the damage done by the Stapel case, Sijtsma’s encounters with questionable research questions during his career in applied statistics and psychometrics, and a strong plea to improve research quality. That mix works well. I especially like the insightful explanations about why statistics is difficult for most researchers and how easily they’re let astray by their intuition. It’s also explained clearly why transparency about the research methods used and the data obtained is essential for research to be trustworthy. Taking everything together the book is a convincing plea to engage in open methods and open data, and to involve methodologists and statisticians in research more intensively. Also leaders of research institutes can learn a lot from the book about how to make their organization less vulnerable to research fraud and methodological errors."
- Lex Bouter, Professor Emeritus of Methodology and Integrity, Vrije Universiteit Amsterdam
