Sylvia Rijmer
Third Case Study: May 2018 to October 2019More

Multimodal Communication & Cognition

Computer Vision & Motion Tracking

Joanna Skubisz

PhD student and researcher

I'm a PhD grant holder in the ERC-funded "BlackBox: Arts & Cognition" project (Multimodal Communication / Linguistic section), where I study the verbal and non-verbal behavior of people from Performing Arts domain. In particular, I investigate the form and function of non-lexical vocalizations and tactile communication in choreographer-dancer collaborative work and study how these phenomena modulate social interaction. My research interests are Gesture Studies, Multimodal Communication, Cognitive Sciences, and quantitative methodologies.

I received my Master of Arts degree in German Philology and German as a Foreign Language at Bielefeld University (2014), writing a thesis about the communicative functions of listeners’ non-verbal behaviour, focusing head-gesture movements. In brief, I was questioning if and how communicative feedback functions can be assigned to head-gestures, independently from speech.

As a research assistant (2011-2014) in the "Rhythm and timing in dialogue project" (PI: Prof. Petra Wagner), I collected, annotated (using ELAN and PRAAT), analyzed, pre- and post-processed large speech and gesture data, as well as gained insights into the maintenance process and ethical issues of multimodal corpora.

I have six years of teaching experience to graduate and undergraduate students from Bielefeld University, where I taught "Introduction to German linguistics" (2008-2010) and "Introduction to language teaching" (2010-2013). Teaching was a natural outcome of my previous qualifications from PWSZ Konin (Poland), where I studied methodology and didactics of German language and received my Bachelor's degree (2009).

Within BlackBox project, I am investigating language (speech and gestures) and language like phenomena (non-lexical vocalizations and touch), and study how they shape the social context of performers being engaged in creative activities. In my PhD thesis, I am researching the iconic relationship between choreographer’s non-lexical vocalizations (i.e. their prosodic and acoustic features) and dancer’s body movements.


1. Evola, V., Skubisz, J. and Fernandes C. (2016). The Role of Eye Gaze and Body Movements in Turn-Taking during a Contemporary Dance Improvisation. In: Gilmartin, E., Cerrato, L., Campbell, N. (eds). Proceedings from the 3rd European Symposium on Multimodal Communication, Dublin, September 17-18, 2015. Linköping: LiU Press. pp.24-31. Full Paper

2. Malisz, Z., Włodarczak, M., Buschmeier, H., Skubisz, J., Kopp, S., & Wagner, P.(2016). The ALICO corpus: analysing the active listener. Language resources and evaluation, 50(2), 411-442. Full Paper