João Fiadeiro
First Case Study: October 2014 to March 2016 More

Introduction: Composition in Real Time method

Performance Studies & Information Vizualisation

Multimodal Communication & Cognition

Computer Vision & Motion Tracking

Floor Study: "Silent" Turn-Taking

The “Multimodal Communication & Cognition” (MMCC) strand is one of the three thematic research lines co-existing in the BlackBox – Arts & Cognition.

The MMCC strand develops exploratory research on the interaction between linguistics and other research domains with a particular focus on the dynamic character of multimodal communication (the study of language in all its modalities in human interaction, including speech, gesture and other body communication) and the performing arts in the broader context of Cognitive Linguistics.

It promotes interaction with researchers from disciplines such as gesture studies, (social) neuroscience, performance studies, cognitive semiotics, computer science, anthropology, discourse, and communication sciences. With a strong interest in the creative and decision-making processes underlying the performing arts, it attempts to contribute to a wider understanding of language (both verbal and non-verbal) and, consequently, of the complexity of the human mind.

This thematic strand offers an opportunity to undergo interdisciplinary research and present research findings concerning human behaviour and agency in different types of communication and their cognitive, cultural, narrative, technological, social, textual, or discourse functions.

Performing arts practices in general are extremely rich in instances of creativity and expressivity, which therefore makes them rather challenging for linguistic research, as they provide a diversified corpus for the analysis of full body enactments and verbal vs. non-verbal interactions.

The MMCC strand currently includes Prof. Carla Fernandes, Dr. Vito Evola (post-doc), and Joanna Skubisz (PhD student).

 

Video examples

Turn management with the Experts was much more fluid compared to, for example, the Dancers group which was not as confident with the method. As expected, Non-Performers rely much more on those strategies common in verbal social communication, such as gaze exchange.

 

1. Nonperformers

In fact Non-performers, had more gazes to each other and to Fiadeiro, probably looking for confirmation before and after taking a turn. Their posture sitting on the chairs was stiffer than in the other 2 groups.

2. Dancers

The Dancers were the ones needing less time to take turns. They are used to performing, but not like this. They are quite determined and eager to move and perform! They often confront the choreographer, by asking questions and even suggesting other possibilities.

3. Experts

The Experts took considerably more time to take turns than the other groups. There’s somehow a similarity to meditation practices, where silence and control over body movements seem to rule. They are focused on the table, almost never look at their partners or the choreographer.