An Introduction to Metascience The Discipline of Evaluating the Creation and Dissemination of Research

Chapter One - An introduction to metascience
1.1. What is metascience?
1.2. The intended audience of this book
1.3. Pedagogical features in this book
1.4. Summary of the upcoming chapters
1.4.1. Chapter Two - Mitigating biases during the production and dissemination of research
1.4.2. Chapter Three - Journalology: The science of publishing
1.4.3. Chapter Four - The impact of funding agencies on the production of research
1.4.4. Chapter Five - Improving the culture in research workplaces
1.4.5. Chapter Six - Understanding and addressing questionable research practices
1.4.6. Chapter Seven - Addressing the reproducibility crisis
1.4.7. Chapter Eight - Ethics and metascience
1.5. References
Chapter Two - Mitigating biases during the production and dissemination of research
2.1. Introduction
2.2. Overview of biases in research
2.3. Biases whilst designing a study
2.3.1. Ethnocentric bias
2.3.1.1. Defining ethnocentric bias
2.3.1.2. Strategies that can reduce ethnocentric bias
2.3.1.2.1. Mandating the reporting of sample characteristics
2.3.1.2.2. Cultural diversity badges
2.3.1.2.3. Diversity targets
2.3.1.2.4. Increasing ethnic diversity on editorial boards
2.3.2. Gender bias
2.3.2.1. Defining gender bias
2.3.2.2. Strategies to reduce gender bias
2.3.2.2.1. The Sex and Gender Equity in Research (SAGER) guidelines
2.3.2.2.2. Gender diversity badges
2.4. Biases during data collection and analysis
2.4.1. Cognitive bias
2.4.2. Confirmation bias
2.4.3. Selection bias
2.4.4. Data collection bias
2.4.5. Measurement bias
2.4.6. Search engine bias
2.5. Biases during the reporting and dissemination of research
2.5.1. Time-lag bias
2.5.2. Place of publication bias
2.5.3. Citation bias
2.5.4. Checklists to detect bias in manuscripts
2.6. Biases in artificial intelligence
2.6.1. The role of artificial intelligence in the creation of literature reviews
2.6.2. Limitations of artificial intelligence
2.7. Conclusion
2.8. Additional readings
2.9. References
Appendices
Chapter Three - Journalology: The science of publishing
3.1. Introduction
3.2. A description and historical origins of journalology
3.3. The peer review process
3.4. Issues and potential improvements of the peer review process
3.4.1. Making peer review a teamwork effort
3.4.1.1. Segmented peer review
3.4.1.2. Expert collaboration
3.4.2. Avoiding publication bias and increasing transparency
3.4.2.1. Open peer review
3.4.2.2. Transparent peer review
3.4.2.3. Withholding results from the peer review process
3.4.3. Improving the accuracy of the peer review process
3.4.3.1. Training and opportunities to peer review manuscripts
3.4.4. Reducing the duration of the peer review process
3.4.4.1. Early screening of manuscripts
3.4.4.2. Using incentives to entice peer reviewers
3.4.5. Other suggestions to improve the peer review process
3.4.5.1. Removing multiple examinations in the peer review process
3.4.5.2. Publishing manuscripts about the peer review process
3.4.5.3. Improving the professionalism of reports by peer reviewers
3.5. Preprinted articles
3.6. The creation of predatory publishers and Beall’s list
3.7. Consequences of predatory journals
3.7.1. Corrupting research
3.7.2. Undermining the training of scholars
3.7.3. Increased email correspondence to academics
3.8. Checklists and flow diagrams to identify predatory journals
3.9. Conclusion
3.10. Additional readings
3.11. References
Appendices
Chapter Four - Impact of funding agencies on the production of research
4.1. Introduction
4.2. Improving the grant application process
4.2.1. Implementing a two-stage application process
4.2.2. Overcoming the ‘incumbency advantage’
4.2.3. Multiple application opportunities to reduce application burden and stress
4.2.4. Improving the quality of feedback that unsuccessful applicants receive
4.3. How funding agencies can improve the quality of research
4.3.1. Improving the evaluation of knowledge translation in research proposals
4.3.2. Addressing sex and gender bias in research
4.3.3. Preventing inappropriate influence by funding agencies
4.3.4. Making research reproducible
4.4. Conclusion
4.5. Additional readings
4.6. References
Appendix
Chapter Five - Improving equity, diversity, and inclusion in academia
5.1. Introduction
5.2. Academic bullying
5.2.1. Overview of academic bullying
5.2.2. Factors that can cause and exacerbate academic bullying
5.2.2.1. Citation metrics
5.2.3. Strategies to reduce academic bullying
5.2.3.1. Legislative changes
5.3. Racism in research workplaces
5.4. Women in academia
5.4.1. Women’s participation in academia
5.4.1.1. Representation of women in senior academic roles
5.4.1.2. Representation of women on editorial boards
5.4.2. Factors that inhibit the inclusion and promotion of women in academia
5.4.2.1. Wage disparities based on gender
5.4.2.2. Networking opportunities
5.4.3. Improving the inclusion, retention, and promotion of women in academia
5.5. General recommendations for improving equity, diversity, and inclusion in academia
5.5.1. Changes to publishing policies
5.5.1.1. Adopt a journal diversity statement with clear, actionable steps to achieve it
5.5.1.2. Promote the use of inclusive and bias-free language
5.5.1.3. Appoint a journal’s equity, diversity, and inclusion director or lead
5.5.1.4. Establish a mentoring approach
5.5.1.5. Monitor adherence to equity, diversity, and inclusion principles
5.5.1.6. Publish reports on equity, diversity, and inclusion
5.5.2. The role of universities and academic institutions
5.5.3. The role of funding agencies
5.6. Conclusion
5.7. Additional readings
5.8. References
Appendix
Chapter Six - Understanding and addressing questionable research practices
6.1. Introduction
6.2. Defining questionable research practices
6.2.1. Cherry picking
6.2.2. P-hacking
6.2.3. Hypothesising After Results are Known
6.3. Occurrence of questionable research practices
6.4. Strategies to reduce questionable research practices
6.4.1. Using evidence-based language
6.4.2. Justifying specific tests for p-values
6.4.3. Pre-registering a study’s design
6.4.4. Reforming grant awarding agencies
6.4.5. Educating scholars about questionable research practices
6.4.6. Creating reporting procedures
6.4.7. Reforming the ‘publish or perish’ culture
6.4.8. Removing any financial incentives for academic publishing
6.4.9. Creating an independent research integrity agency
6.4.10. Making researchers pledge an oath to uphold research integrity
6.4.11. Developing a confidential reporting system
6.4.12. Aubert Bonn and colleagues’ suggestions about improving research integrity
6.5. Conclusion
6.6. Additional readings
6.7. References
Appendix
Chapter Seven - Addressing the reproducibility crisis
7.1. Introduction
7.2. Defining reproducibility
7.3. Consequences of irreproducible research
7.4. Strategies to increase reproducible research
7.4.1. Publishing datasets
7.4.1.1. Conventional practices about publishing and archiving datasets
7.4.1.2. Researchers’ attitudes about disseminating datasets
7.4.1.3. Teaching academics data management processes
7.4.1.4. Improving international data sharing agreements
7.4.1.5. Journal requesting datasets from authors
7.4.2. Establishing journals that only publish replication studies
7.4.3. Teaching academic staff about reproducibility
7.4.4. Open Science Badges
7.4.5. Incorporating reproducibility requirements into the criteria for research funding
7.4.6. Reforming academic hiring practices to promote reproducible research
7.4.7. Pre-registering studies
7.4.8. Improving the readability of a study’s methodology
7.4.9. Improving the clarity of conference presentations
7.4.10. Requiring researchers to self-examine their previous research
7.5. Conclusion
7.6. Additional readings
7.7. References
Appendices
Chapter Eight - Human Research Ethics Committees and metascience
8.1. Introduction
8.2. The creation of Human Research Ethics Committees
8.3. Operational issues with Human Research Ethics Committees
8.3.1 Providing ethics training to applicants
8.3.2 Educating members of Human Research Ethics Committees to examine ethics applications
8.3.3 Interactive ethics presentations
8.3.4 Retrospective ethics reviews
8.3.5 Participant feedback to ethics committees
8.3.6 Creating consistent policies for Health Research Ethics Committees
8.3.7 Reducing HREC application rejection rates
8.4 Conclusion
8.5 Additional readings
8.6 References
Appendix
Chapter Nine - Final remarks

Biography

Dr Gabriel Bennett, the pen name for Dr Matthew Bennett, holds a PhD in Disability Studies from Flinders University, Australia. He has lectured in Disability Studies at Griffith University, Queensland. He has also advised the Australian Government’s AutismCRC and has published articles for the Journal of Autism and Developmental Disorders. He is actively involved in supporting autistics to achieve their potential in society by disseminating his knowledge about the autism spectrum via lectures, conference presentations, and publications.

Dr Emma Goodall is an adjunct research fellow at the University of Southern Queensland, holds a PhD in Education, focused on teaching students on the autism spectrum. She is an executive member of the Australian Society for Autism Research, independent researcher through Healthy Possibilities, a published author and keynote speaker in the areas of autism, sexuality and relationships, education, and interoception.