Freie Universität Berlin, February 4, 2016

During the workshop “Media Archives and Corpus Analysis”, Cinepoetics was honored to hold a special discussion with Mark Turner, Institute Professor and Professor of Cognitive Science at Case Western Reserve University. In the early 1990s, Turner, together with Gilles Fauconnier, founded the theory of conceptual blending. More recently, he co-directs the “Distributed Little Red Hen Lab”, a consortium for research into multimodal communication. To engage in a scholarly discussion with Turner, Jana Bressem (Academic Assistant at the chair German Linguistics, Technical University Chemnitz), Silva Ladewig (Assistant Professor at the chair for Language Use and Multimodal Communication, European University Viadrina), and Cornelia Müller (Professor for Language Use and Multimodal Communication, European University Viadrina) gave a talk on „Verbo-Kinesic Constructions.“

In a short introduction, Michael Wedel pointed out the research goals of Cinepoetics as well as the scholarly motivation behind this project. Afterwards, Cornelia Müller, Silva Ladewig, and Jana Bressem shared some findings of their detailed analysis of verbo-kinesic constructions in language and gesture. This led to a discussion of possible fields of cooperation with Turner’s work and a presentation of his “Distributed Little Red Hen Lab.”

In gesture research, as Müller, Ladewig, and Bressem pointed out, media specifics tend to be neglected. For their work on gesture, it is crucial to understand gesture as a medium that can thus be the subject of form-based, linguistic and semiotic analysis. Most importantly, one must recognize the simultaneous character of gestures as well as their linear combinations (including gestures used over time and by both hands). Based on these perspectives on gesture research, the group was able to look at a sort of “grammar” of gestures, which also includes multimodal phenomena as gestures are used mostly in combination with spoken language.

Müller, Ladewig, and Bressem are especially interested in recurrent gestures, since they are conventionalized, maintain a stable form-meaning relationship, and may depict concrete or abstract entities. This can be summarized as the referential function, whereas the performative function of recurrent gestures lies in their communicative actions. For Müller, Ladewig, and Bressem, a recurrent gesture “is used repeatedly in different contexts and its formational and semantic core remains stable among different contexts and speakers.”

Furthermore, the scholars discussed some recurrent gestures in detail to uncover the framework underlying this concept. Three of the most used recurrent gestures are the Palm-Up-Open-Hand (PUOH), the cyclic gesture, and the throwing away gesture. The PUOH shows openness for ideas or underlines the obviousness of something that has just been said. It is based on a giving/receiving-motion and can be traced as one of the earliest gestures in child development (after the pointing gesture). The other concepts mark continuity – as the cyclic gesture is based on a wheel – or, in the case of the throwing away gesture, negative assessment.

The presentation ended with an emphasis on possible collaborations between Mark Turner’s work and studies of Müller, Ladewig, and Bressem. In using the massive data of the Distributed Red Hen Lab, verbo-kinesic constructions could be further examined. In addition, cross-cultural observations for certain recurrent gestures might shed new light on this subject.

In the subsequent presentation, Mark Turner gave an in-depth look into the complex system behind the Distributed Little Red Hen Lab. Also, he took the time to share his view on the poetics of film in relation to the field of cognitive studies.

As Turner pointed out, film can move ancient human patterns into the field of perception, e.g. showing something we can only imagine (like the zooming motion of a camera). Cognitive science, according to him, is interested in these patterns made visible by film. Thus, film is “like a sandbox for the cognitive sciences.” In this way, cinepoetics could be paraphrased as “the poetics of mind,” Turner remarked. 

When he introduced the Distributed Little Red Hen Lab, Turner offered valuable insights into the history and form of the project. Its database consists of 300.00 hours of footage from news reports and non-fiction TV programs including closed captions. Since these captions are linked to the search engine, users are able to precisely look for certain phenomena. Over three billion words can be searched through UNIX-engines, while every day about 150 hours of new footage is made available.

With this vast amount of data, one might look for word constructions that evoke certain gestures. Here, Turner pointed out the use of comparative correlatives in the form of “the Xer the Yer.”, which allows a gesture scholar to analyze a pattern like “the closer the storm front gets, the stronger the cool moist onshore winds become” in a weather report. The retrieved data makes visible, the kinds of gestures used to underline the above mentioned comparative correlative.

Concluding his talk, Turner emphasized the importance of the collaborative aspect behind the Distributed Little Red Hen Lab. Scholars from different fields can benefit greatly from the many hours of collected audiovisual material. Vice versa, the lab needs professional input to perfect and differentiate the search engine. This added value to both sides is what made his project so interesting, Turner explained.