1st Edition

Dance Data, Cognition, and Multimodal Communication

Edited By Carla Fernandes, Vito Evola, Cláudia Ribeiro Copyright 2023
    456 Pages 82 B/W Illustrations
    by Routledge

    Dance Data, Cognition, and Multimodal Communication is the result of a collaborative and transdisciplinary effort towards a first definition of "dance data", with its complexities and contradictions, in a time where cognitive science is growing in parallel to the need of a renewed awareness of the body’s agency in our manyfold interactions with the world.

    It is a reflection on the observation of bodily movements in artistic settings, and one that views human social interactions, multimodal communication, and cognitive processes through a different lens—that of the close collaboration between performing artists, designers, and scholars.

    This collection focuses simultaneously on methods and technologies for creating, documenting, or representing dance data. The editors highlight works focusing on the dancers’ embodied minds, including research using neural, cognitive, behavioural, and linguistic data in the context of dance composition processes. Each chapter deals with dance data from an interdisciplinary perspective, presenting theoretical and methodological discussions emerging from empirical studies, as well as more experimental ones.

    The book, which includes digital Support Material on the volume's Routledge website, will be of great interest to students and scholars in contemporary dance, neuro-cognitive science, intangible cultural heritage, performing arts, cognitive linguistics, embodiment, design, new media, and creativity studies.

    List of Illustrations

    List of Contributors

    Introduction

    CARLA FERNANDES

    Part I

    Performance-as-Research: Dance data from the artists’ perspectives

    1 Negotiating deliberate choice-making: Insights from an interdisciplinary and multimodal encounter during the making of a New Contemporary Dance

    SYLVIA RIJMER

    2 Dance | Data | Storytelling

    STEPHAN JÜRGENS

    3 Enabling multimodal interaction in mixed-abled dance: Insights into creating highly accessible teaching tools for inclusive cultural work

    SUSANNE QUINTEN AND MIA SOPHIA BILITZA

    Part II

    Dance documentation and dance scores

    4 Recording "Effect": A case study in technical, practical, and critical perspectives on dance data creation

    DAVID RITTERSHAUS, ANTON KOCH, SCOTT DELAHUNTA, AND FLORIAN JENETT

    5 Digital-born artworks and interactive experience: Documentation and archiving

    PAULA VARANDA

    6 Dance scoring and en-action as a creative tool for dance documentation

    BERTHA BERMÚDEZ-PASCUAL

    7 Terpsicore dance and performing arts archive

    DANIEL TÉRCIO, CATARINA CANELAS, AND ANA LUÍSA VALDEIRA

    Part III

    Computational dance data: Between the real and the virtual

    8 Augmented seeing and sensing

    ANGUS G. FORBES

    9 Motion capture and the digital dance aesthetic: Using inertial sensor motion tracking for devising and producing contemporary dance performance

    DANIEL STRUTT

    10 Capturing and visualizing 3D dance data: Challenges and lessons learnt

    CLÁUDIA RIBEIRO, RAFAEL KUFFNER, AND CARLA FERNANDES

    Part IV

    The brain’s experience of dance

    11 The embodied neuroaesthetics of watching dance

    EMILY S. CROSS AND REBECCA SMITH

    12 Dancing neurons: Common brain activity fMRI analysis of the cerebral phenomena behind dance perception

    SOFIA AMARAL MARTINS AND FRANK POLLICK

    13 "I see something, and I like it": Unveiling a choreographer’s decision-making process using quantitative and qualitative methods

    ANA RITA FONSECA, RODRIGO ABRIL-DE-ABREU, AND CARLA FERNANDES

    Part V

    Dance expertise and cognition

    14 Dance expertise, embodied cognition, and the body in the brain

    BETTINA BLÄSING

    15 What makes dancers extraordinary? Insights from a cognitive science perspective

    CARLA FERNANDES, VITO EVOLA, AND JOANNA SKUBISZ

    16 The role of dance experience, visual processing strategies, and quantitative movement features in recognition of emotion from whole-body movements

    REBECCA SMITH AND FRANK POLLICK

    Part VI

    Cognitive metaphor and gestures in dance and theatre

    17 Unpeeling meaning: An analogy and metaphor identification and analysis tool for modern and post-modern dance, and beyond

    VICKY J . FISHER

    18 Understanding non-verbal metaphor: A cognitive approach to metaphor in dance

    LACEY OKONSKI, JULIE MADDEN, AND KAITLIN TOTHPAL

    19 Study on hand movements accompanied during the description of dance appreciation

    ZI HYUN KIM AND HEDDA LAUSBERG

    20 Reduction of gesticulation and information patterning strategies in acted speech

    GIORGINA CANTALINI AND MASSIMO MONEGLIA

    21 Lines of experience: Towards a research method

    MICHAEL O’CONNOR

    Note about Funding

    Index

     

    Contributors

    Rodrigo Abril-de-Abreu is an Invited Researcher at BlackBox, Faculdade de
    Ciências Sociais e Humanas, Universidade Nova de Lisboa. He holds a PhD
    in Neuroscience, with a specialization in Social Behaviour. His main
    research interests lie in the topics of cooperation, empathy, and social
    learning in human and non-human animals.

    Bertha Bermúdez-Pascual is an Artist, Independent Researcher, and an
    External PhD student at the Faculty of Humanities, Capaciteitsgroep Media &
    Cultuur, Amsterdam University, Amsterdam, The Netherlands. Her research
    interests focus on dance transmission, documentation, archiving, and
    dissemination using digital media. She has coordinated transdisciplinary
    research projects (Capturing) Intention, Inside Movement Knowledge, and
    Pre-choreographic Elements between 2005 and 2014. Currently she is
    finishing her PhD Performing Archives at the Amsterdam University while
    working as independent advisor and coordinator for dance projects and artistic
    processes.

    Mia Sophia Bilitza is a Research Assistant at the faculty of Rehabilitation
    Science in Music and Movement in Rehabilitation and Pedagogy in
    Disability, TU Dortmund University, Germany. She is currently obtaining
    her PhD about respect behaviour in dance contexts and its possibilities for
    change in heterogenous societies. Her research interests are respect, dance,
    integration, and inclusion. She is also a choreographer and dance manager
    and works internationally in the field of community dance.

    Bettina Bläsing is a Lecturer in Rehabilitation Science at the Technical
    University Dortmund, Germany. After completing her PhD in Biology,
    she worked as a Postdoctoral Researcher at the Max-Planck-Institute for
    Evolutionary Anthropology at Leipzig and at the Center of Excellence
    Cognitive Interaction Technology at Bielefeld. In 2019, she received the
    venia legendi in Sport Science at Bielefeld University for completing her
    habilitation on memory, learning, and expertise in dance.

    Catarina Canelas is a Junior Researcher at Instituto de Etnomusicologia –
    Centro De Estudos Em Música e Dança at the Faculty of Human Kinetics of
    the University of Lisbon (FMH-UL). She has been working on Terpsicore
    (dance and performing arts archive) since 2017. This is her first article to be
    published. She has both degrees in Dance from FMH-UL and in Social
    Policy from Instituto Superior de Ciências Sociais e Políticas, Lisbon
    University.

    Giorgina Cantalini is a Didactic Manager at the Civica Scuola di Teatro Paolo
    Grassi and Professor of Italian and Communication at the Civica Scuola
    Interpreti e Traduttori Altiero Spinelli, both in Milan, Italy. She obtained her
    PhD in Linguistics at Roma Tre University in 2018 with a dissertation on
    Gesture/Prosody Synchronization in Acting and Spontaneous Speech. She is
    an actress and a drama professor as well, and she has developed a technique for
    exploiting body movements for expressive reading.

    Emily S. Cross is a Professor of Social Robotics within the Institute of
    Neuroscience and Psychology at the University of Glasgow in Scotland and
    Professor of Human Neuroscience within the Department of Cognitive
    Science at Macquarie University in Australia. She obtained her PhD in
    Cognitive Neuroscience from Dartmouth College in 2008 and has danced
    and toured with several contemporary dance companies in the USA, UK,
    and NZ. She currently directs the Social Brain in Action laboratory, which
    explores how different kinds of experience shape brain and behaviour.

    Scott deLahunta is a Professor of Dance, Centre for Dance Research,
    Coventry University (UK) and co-directing (with Florian Jenett) Motion
    Bank, Hochschule Mainz – University of Applied Sciences, Germany. He
    has worked as writer, researcher, and organiser on a range of international
    projects, bringing performing arts with a focus on choreography into
    conjunction with other disciplines and practices. http://www.sdela.dds.nl.

    Vito Evola is currently a Researcher in Cognitive Linguistics and Multimodal
    Communication at the Universidade Nova de Lisboa, having previously
    lectured and conducted research at universities in Palermo, Berkeley,
    Cleveland, Aachen, and Geneva. His research lies at the intersection of
    language, culture, and cognition, and analyses data from both common
    and more specialized contexts, such as patient–doctor interactions,
    psychotherapy and forensic interviews, religious discourse, and the
    performing arts.

    Carla Fernandes is Principal Investigator, Head of ICNOVA’s Research
    Group on Performance & Cognition, and Professor at FCSH – Universidade
    NOVA de Lisboa, where she directs the ERC-funded “BlackBox LAB
    Arts&Cognition.” Her current research focus is in the intersection of
    Performing Arts and Cognitive Science, Multimodal Communication,
    Intangible Cultural Heritage and New Media, fascinated by the complexity
    of the human mind and non-verbal behavior in creativity settings. She
    holds a PhD in Linguistics, supervises numerous PhD and MA theses, and is
    author in international indexed journals and books.

    Vicky J. Fisher is an Affiliated Researcher with the Multimodal Language
    and Cognition Group, Radboud University/Max Planck Institute for
    Psycholinguistics, Nijmegen, The Netherlands. Originally from England,
    Vicky is a participatory dance practitioner, founder member of CandoCo
    Dance Company, and taught dance theory and practice for over 20 years.
    Her research focus is on dance as a form of embodied analogy/metaphor,
    integrating approaches from dance practice, multimodal linguistics, and
    cognitive psychology.

    Ana Rita Fonseca is a Postdoctoral Researcher at BlackBox, Faculdade de
    Ciências Sociais e Humanas, Universidade Nova de Lisboa. She obtained
    her PhD in Neuroscience from Universidade Nova de Lisboa in 2016. Her
    research interests include Dance-making, Movement Creation, and
    Creativity.

    Angus G. Forbes is an Associate Professor of Computational Media at the
    University of California, Santa Cruz, where he directs the Creative Coding
    Lab, an interdisciplinary research group that investigates topics in
    visualization, graphics, human-computer interaction, and machine
    learning. More information about the lab can be found at https://
    creativecoding.soe.ucsc.edu/.

    Florian Jenett is a Professor of Computer Science, Department of Design,
    Hochschule Mainz – University of Applied Sciences, Germany. He has
    been running his own art studio since 2007, has shown works in various
    exhibitions, and is a long-time contributor to Processing, an international
    Creative Coding open-source project. From 2010 onwards, he has been
    researching for the Motion Bank that he co-directs alongside Scott
    deLahunta since 2014.

    Stephan Jürgens is a Postdoc Researcher at ITI-LARSyS, University of
    Madeira, Portugal. He holds a PhD in Contemporary Choreography and
    New Media Technologies. His research interests concentrate on the design
    of creative strategies for live performance involving interactive systems.
    Recently, Stephan has been involved in several 3D media productions
    exploring dance in VR.

    Zi Hyun Kim obtained her PhD in Sports Science from German Sport
    University Cologne in 2016. At present, she is a Research Scholar at the
    World Ethnic Dance Institute of Korea National University of Arts in
    Seoul, Korea, and interested in cross-cultural comparison studies on hand
    movement in connection with dance appreciation.

    Anton Koch is a Research Assistant with Motion Bank, Hochschule Mainz –
    University of Applied Sciences, Germany. He is the lead engineer and
    technological strategist, focusing on data modelling, experimental
    application infrastructures and deployment strategies for research projects
    in the humanities.

    Rafael Kuffner dos Anjos is a Postdoctoral Researcher at CMIC, Victoria
    University of Wellington, Wellington, New Zealand. He obtained his PhD
    in Information Systems and Computer Engineering at the University of
    Lisbon in 2018. His research interests include Computer Graphics, Imagebased
    rendering, and XR.

    Hedda Lausberg is a Full Professor in the Department of Neurology,
    Psychosomatic Medicine and Psychiatry at German Sport University
    Cologne, Germany. She is a neurologist, specialist in psychosomatic
    medicine and psychotherapy, and psychiatrist.

    Julie Madden is a Lecturer at the University of Tennessee at Chattanooga.
    She completed her undergraduate training at the University of California,
    Santa Cruz, and received her PhD in Cognitive Psychology from Florida
    State University. Dr. Madden is currently, in collaboration with Dr. Mike
    Kaschak, putting the final touches on a book chapter on best practices for
    replication science, specifically focusing on research in embodied cognition.

    Sofia Amaral Martins is a Dance Science Teacher at Neuroscience of
    Dance, Portugal. Upon attending the University of Glasgow in 2017, she
    decided to join her two passions by doing her research thesis on the cerebral
    phenomena beyond dance. She went on to develop the Substancia Nigra
    dance project, where she teaches neuroscience of dance workshops
    worldwide. The project aims to break the science art division between
    dancers and neuroscience and instigate the use of dance as medicine.

    Massimo Moneglia is a Professor of Linguistics at the Department of Letters
    and Philosophy of the University of Florence, Italy, where he currently
    directs the LABLITA research unit devoted to the collection and
    annotation of multilingual and multimodal language resources. Among
    his main achievements are the C-ORAL-ROM Multilingual Corpus of
    Spontaneous Speech and the IMAGACT Cross-linguistic Ontology of
    Action.

    Michael O’Connor is currently a PhD Candidate at the Vrije Universiteit in
    Amsterdam, NL. He is a choreographer and dancer, whose work is
    influenced by cognitive science, social, and philosophical theories.

    Lacey Okonski holds a PhD in Psychology. Previously a Fulbright Fellow
    and Visiting Professor in Brazil, she is currently at Umeå University,
    Sweden. Her research interests include psycholinguistics, metaphor, and
    dance. Her work has been featured in various publications, such as the
    Journal of Pragmatics, Metaphor and the Social World, and The American Journal
    of Psychology
    . Dr. Okonski has ballet training and was a performing member
    of Mambo Romero Latin Dance Company in San Francisco.

    Frank Pollick is a Professor of Psychology at the University of Glasgow, UK.
    He is interested in how we perceive human movement and how this
    movement is interpreted within brain mechanisms of emotion and higher
    order cognition. He has previously worked with the Jang Seon-Hee Ballet
    Company and the Rosie Kay Dance Company on dance neuroscience
    projects. He was also a member of the AHRC-funded Watching Dance
    Project.

    Susanne Quinten is a Substitute Professor at the faculty of Rehabilitation
    Sciences, TU Dortmund University, Germany. Her research interests
    include arts and disability, and arts education with particular regard to
    mixed-abled dance. She is the editor of several books and publishing
    journals.

    Cláudia Ribeiro is a Postdoctoral Researcher at FCUL, in the research group
    LASIGE. She obtained her PhD in Information Systems and Computer
    Engineering at the University of Lisbon. Her research interests include
    machine learning, deep learning (focusing on emotion recognition in
    multimedia content), and interactive systems.

    Sylvia Rijmer is a Professor of Dance at the Escola Superior de Dança (IPL)
    and holds a BFA in Dance, and a Master’s Degree in Dance Teaching. She
    is also an Independent Dance-Maker and Researcher based in Lisbon,
    Portugal. Formally trained (Nationale Balletacademie, Elmhurst, Juilliard),
    she has danced professionally in various repertoire and freelance dance
    companies (Ballet Gulbenkian, Stadttheater Bern Ballet, Drift among
    others). Her current choreographic research takes an experimental
    approach, which seeks to bridge the gap between Arts and Sciences to
    encounter alternative possibilities for making and experiencing dance.

    David Rittershaus is a Dance and Theatre Scholar with Motion Bank,
    Hochschule Mainz – University of Applied Sciences, Germany. He is
    working on his dissertation on the digital documentation and study of
    choreographic processes in cooperation with the University of Giessen
    (Institute for Applied Theatre Studies). Since 2020, he has been teaching
    regularly at the Department of Theatre Studies at Goethe University Frankfurt.

    Joanna Skubisz is a PhD Candidate in General Linguistics at the
    Universidade Nova de Lisboa, Portugal, and a former young Researcher
    in the BlackBox project (2014–2019). Research on head gestures marks the
    beginning of her early scientific work. Her research interest includes
    reporting quantitative methods in gesture studies. Currently, in her PhD
    dissertation, she investigates the impact of vocalizations and touch in dance
    practice.

    Rebecca Smith is a PhD Student within the Institute of Neuroscience and
    Psychology at the University of Glasgow in Scotland. As a classically trained
    ballet dancer, she brings considerable dance expertise to bear on her PhD
    work, which sits at the intersection of social psychology, dance, and
    artificial intelligence/robotics.

    Daniel Strutt is a Lecturer in the Department of Media, Communications
    and Cultural Studies at Goldsmiths, University of London, teaching film
    theory and analysis alongside social, cultural, and economic theory. His
    book The Digital Image and Reality: Affect, Metaphysics and Post-Cinema
    was published with Amsterdam University Press in 2019. With previous
    research work on CREATe and the AHRC Creative Economy
    Programme, he also does collaborative work with digital audiovisual
    artists. https://www.gold.ac.uk/media-communications/staff/strutt-dan/

    Daniel Tércio is an Associated Professor at the University of Lisbon, Faculty
    of Human Kinetics, and Researcher at INET-MD. He has published more
    than 20 articles in specialized journals, such as Performance Research (UK)
    and Repertório (Brazil). He has authored several book chapters, some of
    which appear in volumes published by prestigious publishing houses, such
    as Peter Lang and SAGE. As a critic, his dance reviews appear regularly in
    the Portuguese press since 2004.

    Kaitlin Tothpal received her Bachelor of Arts degree in psychology and
    education from the University of California, Santa Cruz. She began her
    dance training (ballet, tap, jazz, etc.) at the age of three in Los Angeles
    County. Kaitlin has experience of teaching dance as well as dancing
    professionally. She also has experience in the field of behavior therapy for
    autistic children to young adults.

    Ana Luísa Valdeira is a Researcher at Instituto de Etnomusicologia – Centro
    de Estudos em Música e Danca, and Editor at Imprensa da Universidade de
    Lisboa (University of Lisbon Press). She obtained her PhD in Arts from
    University of Lisbon in 2020.

    Paula Varanda obtained her PhD from Middlesex University in Media and
    Performing Arts in 2016, London, UK. She was a Dance Critic for
    PÚBLICO newspaper for 12 years in Portugal and published books and
    articles about dance, performance, technology, and community practices.
    Her research interests are on aesthetics, arts education and audience
    development, the body and new media in contemporary societies, and
    cultural policies that foster access and sustainability to the arts.

    Biography

    Carla Fernandes is Principal Investigator, Head of ICNOVA’s Research Group on Performance & Cognition, and Professor at FCSH – Universidade Nova de Lisboa, where she directs the ERC-funded "BlackBox LAB Arts&Cognition." Her current research focus is in the intersection of Performing Arts and Cognitive Science, Multimodal Communication, Intangible Cultural Heritage, and New Media. She is fascinated by the complexity of the human mind and non-verbal behavior in creativity settings. She holds a PhD in Linguistics, supervises numerous PhD and MA theses, and is author in international indexed peer-reviewed journals and books.

    Vito Evola is currently a Researcher in Cognitive Linguistics and Multimodal Communication at the Universidade Nova de Lisboa, having previously lectured and conducted research at universities in Palermo, Berkeley, Cleveland, Aachen, and Geneva. His research lies at the intersection of language, culture, and cognition, and analyses data from both common and more specialized contexts, such as patient–doctor interactions, psychotherapy and forensic interviews, religious discourse, and the performing arts.

    Cláudia Ribeiro is a Postdoctoral Researcher at FCUL, in the research group LASIGE. She obtained her PhD in Information Systems and Computer Engineering at the University of Lisbon. Her research interests include machine learning, deep learning (focusing on emotion recognition in multimedia content), and interactive systems.