1st Edition
Dance Data, Cognition, and Multimodal Communication
Dance Data, Cognition, and Multimodal Communication is the result of a collaborative and transdisciplinary effort towards a first definition of "dance data", with its complexities and contradictions, in a time where cognitive science is growing in parallel to the need of a renewed awareness of the body’s agency in our manyfold interactions with the world.
It is a reflection on the observation of bodily movements in artistic settings, and one that views human social interactions, multimodal communication, and cognitive processes through a different lens—that of the close collaboration between performing artists, designers, and scholars.
This collection focuses simultaneously on methods and technologies for creating, documenting, or representing dance data. The editors highlight works focusing on the dancers’ embodied minds, including research using neural, cognitive, behavioural, and linguistic data in the context of dance composition processes. Each chapter deals with dance data from an interdisciplinary perspective, presenting theoretical and methodological discussions emerging from empirical studies, as well as more experimental ones.
The book, which includes digital Support Material on the volume's Routledge website, will be of great interest to students and scholars in contemporary dance, neuro-cognitive science, intangible cultural heritage, performing arts, cognitive linguistics, embodiment, design, new media, and creativity studies.
List of Illustrations
List of Contributors
Introduction
CARLA FERNANDES
Part I
Performance-as-Research: Dance data from the artists’ perspectives
1 Negotiating deliberate choice-making: Insights from an interdisciplinary and multimodal encounter during the making of a New Contemporary Dance
SYLVIA RIJMER
2 Dance | Data | Storytelling
STEPHAN JÜRGENS
3 Enabling multimodal interaction in mixed-abled dance: Insights into creating highly accessible teaching tools for inclusive cultural work
SUSANNE QUINTEN AND MIA SOPHIA BILITZA
Part II
Dance documentation and dance scores
4 Recording "Effect": A case study in technical, practical, and critical perspectives on dance data creation
DAVID RITTERSHAUS, ANTON KOCH, SCOTT DELAHUNTA, AND FLORIAN JENETT
5 Digital-born artworks and interactive experience: Documentation and archiving
PAULA VARANDA
6 Dance scoring and en-action as a creative tool for dance documentation
BERTHA BERMÚDEZ-PASCUAL
7 Terpsicore – dance and performing arts archive
DANIEL TÉRCIO, CATARINA CANELAS, AND ANA LUÍSA VALDEIRA
Part III
Computational dance data: Between the real and the virtual
8 Augmented seeing and sensing
ANGUS G. FORBES
9 Motion capture and the digital dance aesthetic: Using inertial sensor motion tracking for devising and producing contemporary dance performance
DANIEL STRUTT
10 Capturing and visualizing 3D dance data: Challenges and lessons learnt
CLÁUDIA RIBEIRO, RAFAEL KUFFNER, AND CARLA FERNANDES
Part IV
The brain’s experience of dance
11 The embodied neuroaesthetics of watching dance
EMILY S. CROSS AND REBECCA SMITH
12 Dancing neurons: Common brain activity fMRI analysis of the cerebral phenomena behind dance perception
SOFIA AMARAL MARTINS AND FRANK POLLICK
13 "I see something, and I like it": Unveiling a choreographer’s decision-making process using quantitative and qualitative methods
ANA RITA FONSECA, RODRIGO ABRIL-DE-ABREU, AND CARLA FERNANDES
Part V
Dance expertise and cognition
14 Dance expertise, embodied cognition, and the body in the brain
BETTINA BLÄSING
15 What makes dancers extraordinary? Insights from a cognitive science perspective
CARLA FERNANDES, VITO EVOLA, AND JOANNA SKUBISZ
16 The role of dance experience, visual processing strategies, and quantitative movement features in recognition of emotion from whole-body movements
REBECCA SMITH AND FRANK POLLICK
Part VI
Cognitive metaphor and gestures in dance and theatre
17 Unpeeling meaning: An analogy and metaphor identification and analysis tool for modern and post-modern dance, and beyond
VICKY J . FISHER
18 Understanding non-verbal metaphor: A cognitive approach to metaphor in dance
LACEY OKONSKI, JULIE MADDEN, AND KAITLIN TOTHPAL
19 Study on hand movements accompanied during the description of dance appreciation
ZI HYUN KIM AND HEDDA LAUSBERG
20 Reduction of gesticulation and information patterning strategies in acted speech
GIORGINA CANTALINI AND MASSIMO MONEGLIA
21 Lines of experience: Towards a research method
MICHAEL O’CONNOR
Note about Funding
Index
Contributors
Rodrigo Abril-de-Abreu is an Invited Researcher at BlackBox, Faculdade de
Ciências Sociais e Humanas, Universidade Nova de Lisboa. He holds a PhD
in Neuroscience, with a specialization in Social Behaviour. His main
research interests lie in the topics of cooperation, empathy, and social
learning in human and non-human animals.
Bertha Bermúdez-Pascual is an Artist, Independent Researcher, and an
External PhD student at the Faculty of Humanities, Capaciteitsgroep Media &
Cultuur, Amsterdam University, Amsterdam, The Netherlands. Her research
interests focus on dance transmission, documentation, archiving, and
dissemination using digital media. She has coordinated transdisciplinary
research projects (Capturing) Intention, Inside Movement Knowledge, and
Pre-choreographic Elements between 2005 and 2014. Currently she is
finishing her PhD Performing Archives at the Amsterdam University while
working as independent advisor and coordinator for dance projects and artistic
processes.
Mia Sophia Bilitza is a Research Assistant at the faculty of Rehabilitation
Science in Music and Movement in Rehabilitation and Pedagogy in
Disability, TU Dortmund University, Germany. She is currently obtaining
her PhD about respect behaviour in dance contexts and its possibilities for
change in heterogenous societies. Her research interests are respect, dance,
integration, and inclusion. She is also a choreographer and dance manager
and works internationally in the field of community dance.
Bettina Bläsing is a Lecturer in Rehabilitation Science at the Technical
University Dortmund, Germany. After completing her PhD in Biology,
she worked as a Postdoctoral Researcher at the Max-Planck-Institute for
Evolutionary Anthropology at Leipzig and at the Center of Excellence
Cognitive Interaction Technology at Bielefeld. In 2019, she received the
venia legendi in Sport Science at Bielefeld University for completing her
habilitation on memory, learning, and expertise in dance.
Catarina Canelas is a Junior Researcher at Instituto de Etnomusicologia –
Centro De Estudos Em Música e Dança at the Faculty of Human Kinetics of
the University of Lisbon (FMH-UL). She has been working on Terpsicore
(dance and performing arts archive) since 2017. This is her first article to be
published. She has both degrees in Dance from FMH-UL and in Social
Policy from Instituto Superior de Ciências Sociais e Políticas, Lisbon
University.
Giorgina Cantalini is a Didactic Manager at the Civica Scuola di Teatro Paolo
Grassi and Professor of Italian and Communication at the Civica Scuola
Interpreti e Traduttori Altiero Spinelli, both in Milan, Italy. She obtained her
PhD in Linguistics at Roma Tre University in 2018 with a dissertation on
Gesture/Prosody Synchronization in Acting and Spontaneous Speech. She is
an actress and a drama professor as well, and she has developed a technique for
exploiting body movements for expressive reading.
Emily S. Cross is a Professor of Social Robotics within the Institute of
Neuroscience and Psychology at the University of Glasgow in Scotland and
Professor of Human Neuroscience within the Department of Cognitive
Science at Macquarie University in Australia. She obtained her PhD in
Cognitive Neuroscience from Dartmouth College in 2008 and has danced
and toured with several contemporary dance companies in the USA, UK,
and NZ. She currently directs the Social Brain in Action laboratory, which
explores how different kinds of experience shape brain and behaviour.
Scott deLahunta is a Professor of Dance, Centre for Dance Research,
Coventry University (UK) and co-directing (with Florian Jenett) Motion
Bank, Hochschule Mainz – University of Applied Sciences, Germany. He
has worked as writer, researcher, and organiser on a range of international
projects, bringing performing arts with a focus on choreography into
conjunction with other disciplines and practices. http://www.sdela.dds.nl.
Vito Evola is currently a Researcher in Cognitive Linguistics and Multimodal
Communication at the Universidade Nova de Lisboa, having previously
lectured and conducted research at universities in Palermo, Berkeley,
Cleveland, Aachen, and Geneva. His research lies at the intersection of
language, culture, and cognition, and analyses data from both common
and more specialized contexts, such as patient–doctor interactions,
psychotherapy and forensic interviews, religious discourse, and the
performing arts.
Carla Fernandes is Principal Investigator, Head of ICNOVA’s Research
Group on Performance & Cognition, and Professor at FCSH – Universidade
NOVA de Lisboa, where she directs the ERC-funded “BlackBox LAB
Arts&Cognition.” Her current research focus is in the intersection of
Performing Arts and Cognitive Science, Multimodal Communication,
Intangible Cultural Heritage and New Media, fascinated by the complexity
of the human mind and non-verbal behavior in creativity settings. She
holds a PhD in Linguistics, supervises numerous PhD and MA theses, and is
author in international indexed journals and books.
Vicky J. Fisher is an Affiliated Researcher with the Multimodal Language
and Cognition Group, Radboud University/Max Planck Institute for
Psycholinguistics, Nijmegen, The Netherlands. Originally from England,
Vicky is a participatory dance practitioner, founder member of CandoCo
Dance Company, and taught dance theory and practice for over 20 years.
Her research focus is on dance as a form of embodied analogy/metaphor,
integrating approaches from dance practice, multimodal linguistics, and
cognitive psychology.
Ana Rita Fonseca is a Postdoctoral Researcher at BlackBox, Faculdade de
Ciências Sociais e Humanas, Universidade Nova de Lisboa. She obtained
her PhD in Neuroscience from Universidade Nova de Lisboa in 2016. Her
research interests include Dance-making, Movement Creation, and
Creativity.
Angus G. Forbes is an Associate Professor of Computational Media at the
University of California, Santa Cruz, where he directs the Creative Coding
Lab, an interdisciplinary research group that investigates topics in
visualization, graphics, human-computer interaction, and machine
learning. More information about the lab can be found at https://
creativecoding.soe.ucsc.edu/.
Florian Jenett is a Professor of Computer Science, Department of Design,
Hochschule Mainz – University of Applied Sciences, Germany. He has
been running his own art studio since 2007, has shown works in various
exhibitions, and is a long-time contributor to Processing, an international
Creative Coding open-source project. From 2010 onwards, he has been
researching for the Motion Bank that he co-directs alongside Scott
deLahunta since 2014.
Stephan Jürgens is a Postdoc Researcher at ITI-LARSyS, University of
Madeira, Portugal. He holds a PhD in Contemporary Choreography and
New Media Technologies. His research interests concentrate on the design
of creative strategies for live performance involving interactive systems.
Recently, Stephan has been involved in several 3D media productions
exploring dance in VR.
Zi Hyun Kim obtained her PhD in Sports Science from German Sport
University Cologne in 2016. At present, she is a Research Scholar at the
World Ethnic Dance Institute of Korea National University of Arts in
Seoul, Korea, and interested in cross-cultural comparison studies on hand
movement in connection with dance appreciation.
Anton Koch is a Research Assistant with Motion Bank, Hochschule Mainz –
University of Applied Sciences, Germany. He is the lead engineer and
technological strategist, focusing on data modelling, experimental
application infrastructures and deployment strategies for research projects
in the humanities.
Rafael Kuffner dos Anjos is a Postdoctoral Researcher at CMIC, Victoria
University of Wellington, Wellington, New Zealand. He obtained his PhD
in Information Systems and Computer Engineering at the University of
Lisbon in 2018. His research interests include Computer Graphics, Imagebased
rendering, and XR.
Hedda Lausberg is a Full Professor in the Department of Neurology,
Psychosomatic Medicine and Psychiatry at German Sport University
Cologne, Germany. She is a neurologist, specialist in psychosomatic
medicine and psychotherapy, and psychiatrist.
Julie Madden is a Lecturer at the University of Tennessee at Chattanooga.
She completed her undergraduate training at the University of California,
Santa Cruz, and received her PhD in Cognitive Psychology from Florida
State University. Dr. Madden is currently, in collaboration with Dr. Mike
Kaschak, putting the final touches on a book chapter on best practices for
replication science, specifically focusing on research in embodied cognition.
Sofia Amaral Martins is a Dance Science Teacher at Neuroscience of
Dance, Portugal. Upon attending the University of Glasgow in 2017, she
decided to join her two passions by doing her research thesis on the cerebral
phenomena beyond dance. She went on to develop the Substancia Nigra
dance project, where she teaches neuroscience of dance workshops
worldwide. The project aims to break the science art division between
dancers and neuroscience and instigate the use of dance as medicine.
Massimo Moneglia is a Professor of Linguistics at the Department of Letters
and Philosophy of the University of Florence, Italy, where he currently
directs the LABLITA research unit devoted to the collection and
annotation of multilingual and multimodal language resources. Among
his main achievements are the C-ORAL-ROM Multilingual Corpus of
Spontaneous Speech and the IMAGACT Cross-linguistic Ontology of
Action.
Michael O’Connor is currently a PhD Candidate at the Vrije Universiteit in
Amsterdam, NL. He is a choreographer and dancer, whose work is
influenced by cognitive science, social, and philosophical theories.
Lacey Okonski holds a PhD in Psychology. Previously a Fulbright Fellow
and Visiting Professor in Brazil, she is currently at Umeå University,
Sweden. Her research interests include psycholinguistics, metaphor, and
dance. Her work has been featured in various publications, such as the
Journal of Pragmatics, Metaphor and the Social World, and The American Journal
of Psychology. Dr. Okonski has ballet training and was a performing member
of Mambo Romero Latin Dance Company in San Francisco.
Frank Pollick is a Professor of Psychology at the University of Glasgow, UK.
He is interested in how we perceive human movement and how this
movement is interpreted within brain mechanisms of emotion and higher
order cognition. He has previously worked with the Jang Seon-Hee Ballet
Company and the Rosie Kay Dance Company on dance neuroscience
projects. He was also a member of the AHRC-funded Watching Dance
Project.
Susanne Quinten is a Substitute Professor at the faculty of Rehabilitation
Sciences, TU Dortmund University, Germany. Her research interests
include arts and disability, and arts education with particular regard to
mixed-abled dance. She is the editor of several books and publishing
journals.
Cláudia Ribeiro is a Postdoctoral Researcher at FCUL, in the research group
LASIGE. She obtained her PhD in Information Systems and Computer
Engineering at the University of Lisbon. Her research interests include
machine learning, deep learning (focusing on emotion recognition in
multimedia content), and interactive systems.
Sylvia Rijmer is a Professor of Dance at the Escola Superior de Dança (IPL)
and holds a BFA in Dance, and a Master’s Degree in Dance Teaching. She
is also an Independent Dance-Maker and Researcher based in Lisbon,
Portugal. Formally trained (Nationale Balletacademie, Elmhurst, Juilliard),
she has danced professionally in various repertoire and freelance dance
companies (Ballet Gulbenkian, Stadttheater Bern Ballet, Drift among
others). Her current choreographic research takes an experimental
approach, which seeks to bridge the gap between Arts and Sciences to
encounter alternative possibilities for making and experiencing dance.
David Rittershaus is a Dance and Theatre Scholar with Motion Bank,
Hochschule Mainz – University of Applied Sciences, Germany. He is
working on his dissertation on the digital documentation and study of
choreographic processes in cooperation with the University of Giessen
(Institute for Applied Theatre Studies). Since 2020, he has been teaching
regularly at the Department of Theatre Studies at Goethe University Frankfurt.
Joanna Skubisz is a PhD Candidate in General Linguistics at the
Universidade Nova de Lisboa, Portugal, and a former young Researcher
in the BlackBox project (2014–2019). Research on head gestures marks the
beginning of her early scientific work. Her research interest includes
reporting quantitative methods in gesture studies. Currently, in her PhD
dissertation, she investigates the impact of vocalizations and touch in dance
practice.
Rebecca Smith is a PhD Student within the Institute of Neuroscience and
Psychology at the University of Glasgow in Scotland. As a classically trained
ballet dancer, she brings considerable dance expertise to bear on her PhD
work, which sits at the intersection of social psychology, dance, and
artificial intelligence/robotics.
Daniel Strutt is a Lecturer in the Department of Media, Communications
and Cultural Studies at Goldsmiths, University of London, teaching film
theory and analysis alongside social, cultural, and economic theory. His
book The Digital Image and Reality: Affect, Metaphysics and Post-Cinema
was published with Amsterdam University Press in 2019. With previous
research work on CREATe and the AHRC Creative Economy
Programme, he also does collaborative work with digital audiovisual
artists. https://www.gold.ac.uk/media-communications/staff/strutt-dan/
Daniel Tércio is an Associated Professor at the University of Lisbon, Faculty
of Human Kinetics, and Researcher at INET-MD. He has published more
than 20 articles in specialized journals, such as Performance Research (UK)
and Repertório (Brazil). He has authored several book chapters, some of
which appear in volumes published by prestigious publishing houses, such
as Peter Lang and SAGE. As a critic, his dance reviews appear regularly in
the Portuguese press since 2004.
Kaitlin Tothpal received her Bachelor of Arts degree in psychology and
education from the University of California, Santa Cruz. She began her
dance training (ballet, tap, jazz, etc.) at the age of three in Los Angeles
County. Kaitlin has experience of teaching dance as well as dancing
professionally. She also has experience in the field of behavior therapy for
autistic children to young adults.
Ana Luísa Valdeira is a Researcher at Instituto de Etnomusicologia – Centro
de Estudos em Música e Danca, and Editor at Imprensa da Universidade de
Lisboa (University of Lisbon Press). She obtained her PhD in Arts from
University of Lisbon in 2020.
Paula Varanda obtained her PhD from Middlesex University in Media and
Performing Arts in 2016, London, UK. She was a Dance Critic for
PÚBLICO newspaper for 12 years in Portugal and published books and
articles about dance, performance, technology, and community practices.
Her research interests are on aesthetics, arts education and audience
development, the body and new media in contemporary societies, and
cultural policies that foster access and sustainability to the arts.
Biography
Carla Fernandes is Principal Investigator, Head of ICNOVA’s Research Group on Performance & Cognition, and Professor at FCSH – Universidade Nova de Lisboa, where she directs the ERC-funded "BlackBox LAB Arts&Cognition." Her current research focus is in the intersection of Performing Arts and Cognitive Science, Multimodal Communication, Intangible Cultural Heritage, and New Media. She is fascinated by the complexity of the human mind and non-verbal behavior in creativity settings. She holds a PhD in Linguistics, supervises numerous PhD and MA theses, and is author in international indexed peer-reviewed journals and books.
Vito Evola is currently a Researcher in Cognitive Linguistics and Multimodal Communication at the Universidade Nova de Lisboa, having previously lectured and conducted research at universities in Palermo, Berkeley, Cleveland, Aachen, and Geneva. His research lies at the intersection of language, culture, and cognition, and analyses data from both common and more specialized contexts, such as patient–doctor interactions, psychotherapy and forensic interviews, religious discourse, and the performing arts.
Cláudia Ribeiro is a Postdoctoral Researcher at FCUL, in the research group LASIGE. She obtained her PhD in Information Systems and Computer Engineering at the University of Lisbon. Her research interests include machine learning, deep learning (focusing on emotion recognition in multimedia content), and interactive systems.
