From Algorithms to Evidence Using GenAI in Evaluation Practice

List of Contributors

Acknowledgements

Foreword

Frans L. Leeuw

1. The Transformation of Evaluation Practice: From Algorithms to Evidence

Steffen Bohni Nielsen, Kerry Bruce, and Valentine J Gandhi

2. Threading the Needle: Making Evaluation Affordable and Participatory for Grassroots Nonprofits

Sadie Paschke

3. Scoping evaluation with GenAI: A case study of the partnership principle in Poland

Marlena Stępień

4. From Supporting Evaluators to Evaluating AI: A Human-Centered Approach to Developing EvalAssist, an AI-for-Good Augmentation Tool for Evaluation Design

Lauren Damme

5. Designing with Generative AI: Using GPT-4 to Build a Monitoring, Evaluation and Learning Framework

Geordie Fung

6. From Ritual to Reflex: Reimagining Results Measurement with Generative AI

Lisandro Martin, Stephen Porter, Vasundhara Jolly, Alexander Lehner, Federico Hernan Tiberti, Rijak Grover, Jonathan Stöterau, and Medha Nair

7. Playing to Plan: Using a GenAI-Powered Chatbot Game to Support Logic Model Development in a City Government Context

Xinru Yan

8. Harnessing Generative AI for Data Structuring and Geospatial Analysis in Evaluation

María Alejandra Lucero Manzano

9. Balancing Innovation and Rigor at World Bank Independent Evaluation Group: Thoughtful Integration of Artificial Intelligence for Evaluation

Harsh Anuj, Estelle Raimondo, and Hannah Den Boer

10. Integrating Generative AI into Thematic Analysis: A Structured Approach for Large-Scale Document Review in Evaluation

Lauren Ropp and Azra K. Nurkic

11. Bridging Complexity with Intelligence: The Role of Generative AI in a Mixed-Methods Evaluation of SDG1 in Sub-Saharan Africa

Hailemichael Taye Beyene

12. Using AI for Interviewing in a Challenging Context

Silva Ferretti and Steve Powell

13. How to Train a Work Horse.  Using Generative AI for Labelling Data

Steffen Bohni Nielsen, Amanda Sutter, Isabelle Bourgeois, Bianca Montrosse-Moorhead, and Sebastian Lemire

14. Sifting for Gold: Mining Relevant Content in a Large Document Collection

Joachim Sandnes, Ove Haugland Jakobsen, and Tom Næss

15. Harnessing Generative AI for Geospatial Analysis in Environmental Evaluations

Anupam Ananda and Geeta Batra

16. Automating Partnership Principle Assessment: Using Generative AI to Analyze Stakeholder Participation in European Operational Program Governance

Igor Lyubashenko

17. Elevating Local Voices: Qualitative Analysis Reimagined

Trevor Smith and Kathryn Kraft

18. Generative AI for Protocol-Dirven Structured Analysis in Evaluations

Indran A. Naidoo, Anoop Sharma, Steven Jonckheere, and Hannah den Boer

19. Experimenting with AI Tools in CGIAR Science Group Evaluations: A Candid Assessment of Adoption by Learning

Lea Corsetti, Edwin Asare and Diana Cekova

20. AIDA - Assessment for Development Analytics

Richard Jones, Davor Djosan, and Anish Lal Pradhan

21. ChatGPT x GBA+ - Using GenAI to Apply Evaluative Criteria: Assessing Reports Against Canada's GBA+ Framework

Savita Bailur

22. Human in the Loop: AI-supported Evaluation in Hungary

Diana Kun

23. Reviewer 0: Using Generative AI as the Evaluator’s First Line of Feedback

Steffen Bohni Nielsen

24. Beyond the Radar: Addressing Undeclared AI Use in Evaluation Practice

Steve Jacob

25. Enhancing Evaluation Dissemination & Reporting with Generative AI

Nishank Vashney

26. Leveraging Self-Hosted AI Solutions for Data-Secured Research and Evaluation

Salvador Bustamante Aragonés

27. Leveraging Generative AI to Transform Evaluation Reports into Engaging Multi-Media Products

Jade Lamb, Meredith Feenstra, Kerry Bruce and Shereen Bahn

28. Podcasting Findings. Using Generative Artificial Intelligence to Tailor Dissemination to Diverse Stakeholders

Niels Brandt Petersen, Rikke Thoning Solaas, Sara Kabel Pedersen and Steffen Bohni Nielsen

29. Bridging Evaluation and Official Statistics Through GenAI: The EQUA Platform

Francesco Mazzeo Rinaldi, Giovanni Giuffrida, and Vincenzo Miracula

30. Where Methodological Automation Ends and Judgement Begins: Evaluation in the Generative Artificial Intelligence Era

Bianca Montrosse-Moorhead and Tahirah David

31. Artificial Intelligence in Evaluation Practice: Uses, Good Practice, and Emerging Competencies

Tessie Catsambas

32. Artificial Intelligence and the Ethics and Practice of Evaluation

Ananda Millard and Tom Ling

33. FRAME: A Practical Framework for Responsible AI in Monitoring and Evaluation

Valentine J Gandhi, Kerry Bruce and Steffen Bohni Nielsen

34. GenAI and the Evolution of Evidence in Evaluation

Kerry Bruce, Valentine J Gandhi and Steffen Bohni Nielsen

Biography

Kerry Bruce (DrPH) is a global health and evaluation practitioner working to support both practical monitoring, evaluation and learning projects and training. She is the CEO of ClearUp Consulting and is a member of INTEVAL.

Valentine J Gandhi (PhD) wears multi-disciplinary hats as a Policy Advisor, MEL,  Data Science and Cybersecurity expert.  He is the founder of the think tank The Development CAFE, and the host of the EvalEdge Podcast for European Evaluation Society. He has worked in over 40 countries including in conflict zones.

Steffen Bohni Nielsen (PhD) is Director General at the Danish National Research Centre for the Working Environment (NFA) and a member of the Danish National Research and Innovation Council and chairman of the Partnership for European Research in Occupational Safety and Health (PERSOH). He is a member of INTEVAL and has published extensively in the field of evaluation.