Macrocognition Metrics and Scenarios: Design and Evaluation for Real-World Teams translates advances by scientific leaders in the relatively new area of macrocognition into a format that will support immediate use by members of the software testing and evaluation community for large-scale systems as well as trainers of real-world teams. Macrocognition is defined as how activity in real-world teams is adapted to the complex demands of a setting with high consequences for failure. The primary distinction between macrocognition and prior research is that the primary unit for measurement is a real-world team coordinating their activity, rather than individuals processing information, the predominant model for cognition for decades. This book provides an overview of the theoretical foundations of macrocognition, describes a set of exciting new macrocognitive metrics, and provides guidance on using the metrics in the context of different approaches to evaluation and measurement of real-world teams.
'Over 100 years of research, focused on measuring and understanding highly constrained human behavior and performance, has broken out of the laboratory and given way to a new paradigm for measuring, predicting and harmonizing the capabilities of humans in complex, unconstrained environments. The shift in paradigms has been roiling and emerging over more than several decades creating the intellectual underpinnings for Cognitive Systems Engineering, Naturalistic Decision making and now Macrocognition, defined by Schraagen, Klein, and Hoffman (2008) as the study of cognitive adaptation to complexity. The revolution in IT, beginning in the late eighties, energized this movement by spawning a stunning growth in complexity of work environments and systems and by stimulating a marked shift towards cognitive work with emphasis on thinking, decision making and problem solving, The failure of the old paradigm of laboratory behavioral research to deal with the challenges and vulnerabilities arising from complexity motivated the critical need for new theory, methods, measures and applications that minimize negative emergent effects and maximize resilience, agility, and real-time and evolutionary adaptivity. This work fills a very important niche in the growing literature of Macrocognition. Its unique contribution is its balanced focus on theory and the state of the art in measurement and evaluation of meta-cognition prepared by an impressive array of current thought-leaders in the field. The book is clearly written and should be required reading for every serious student, scholar, and/or practitioner concerned with harmonizing the capabilities of humans and the demands of work and work environments. I was also very pleased to discover some very provocative original thinking that, in turn, stimulated an epiphany that I expect will pay dividends in my work. I endorse and will recommend this work without reservation.' Kenneth R Boff, Tennenbaum Institute, Georgia Institute of Technology and former Chief Scientist, Human Effectiveness, Air Force Research Laboratory, USA 'Patterson and Miller have synthesised a provocative set of perspectives on the measurement of cognitive processes in team-based work environments. With an authoritative line-up of contributors, this volume provides a wealth of new material on methods of task decomposition for cognitive data gathering in complex team settings. A notable feature is the blend of critical thinking on principles of evaluation with a serious appreciation of real world applications for the emergent techniques.' Rhona Flin, University of Aberdeen, UK