We understand film as the aesthetic modeling of space and time, which constantly reframes and refigures the dynamic conditions of our perception, our sensation, and our understanding. The goal of our method development is to make the aesthetical patterns intersubjectively describable that are effective within this temporal and spatial dynamic.

The detailed and descriptive analysis of spatiotemporal staging patterns within the means of digital tools is one significant basis of our analytical studies. We pursue a decidedly qualitative perspective with the development of such computer-based methods: We describe the spatiotemporal movement patterns of cinematic staging as dynamics – empirically given within the audio-visual images –, which can each be ascribed specific affective qualities. Thus, we can take into consideration the patterns of audio-visual compositions, which shape the temporally unfolding experience of the audience.

This perspective is theoretically grounded in a methodological concept focusing on the idea of expressive movement. The profiling of said idea is based on extensive cultural-historical research and interconnects various fields of study: early film theory (Münsterberg, Eisenstein, Balázs), sociology (Simmel), anthropology (Wundt, Bühler, Plessner), art theory (Fiedler, Dewey), neo-phenomenology (Sobchack), as well as film philosophy (Deleuze).

The idea of cinematic expressive movement indicates an expressive dimension of audio-visual images, which is not conveyed via the presented contents and representations, but which unfolds within the temporal composition of the cinematic means of staging. Within the orchestration of camera movement, editing rhythms, lighting, image composition, sound, and music, a new and dynamic modeling of space and time keeps constantly emerging. The patterns at the base of this modeling present themselves as a specific audio-visual Gestalt respectively, with specific expressive ‘colorations’, comparable with the melody of a piece of music (the affective qualities of which are only given within the temporal interaction of the various sounds through time, they are not, however, present in each individual note). The melody emerges through the process of hearing, just like the spatiotemporal Gestalt of the expressive movement does not unfold within the single image, but rather within the duration of the audience’s viewing and listening.

In order to make this dimension of audio-visual expressivity visible, we have developed a systematic, computer-based method of analysis to identify and describe the respective patterns and dynamics in their temporal dimension: the eMAEX-system („electronically based media analysis of expressive movement images”). It is not a software, but rather a methodologically based routine of segmentation and descriptive analysis of audio-visual images that uses various digital tools.

Further Reading:

• Kappelhoff, Hermann: Matrix der Gefühle. Das Kino, das Melodrama und das Theater der Empfindsamkeit. Berlin 2004.

• Kappelhoff, Hermann: Kognition und Reflexion: Zur Theorie filmischen Denkens. Berlin / Boston 2018.

• Kappelhoff, Hermann: Front Lines of Community. Hollywood Between War and Democracy. Berlin / Boston 2018.

• Müller, Cornelia / Kappelhoff, Hermann: Cinematic Metaphor. Experience – Affectivity – Temporality. Berlin / Boston 2018.

• Kappelhoff, Hermann / Bakels, Jan-Hendrik: Das Zuschauergefühl. Möglichkeiten qualitativer Medienanalyse. In: ZfM – Zeitschrift für Medienwissenschaft 5/2 (2011), pp. 78–95.

• Scherer, Thomas / Greifenstein, Sarah / Kappelhoff, Hermann: Expressive movements in audiovisual media: Modulating affective experience. In: Cornelia Müller, Alan Cienki, Ellen Fricke (Hg.): Body – Language – Communication. An International Handbook on Multimodality in Human Interaction. Berlin / New York 2014, pp. 2081–2092.

• Bakels, Jan-Hendrik: Audiovisuelle Rhythmen. Filmmusik, Bewegungskomposition und die dynamische Affizierung des Zuschauers. Berlin / Boston 2017.

• Greifenstein, Sarah: Tempi der Bewegung – Modi des Gefühls. Expressivität, heitere Affekte und die Screwball Comedy. Berlin / Boston 2020.

• Grotkopp, Matthias: Cinematic Poetics of Guilt. Audiovisual Accusations as a Mode of Commonality. Berlin / Boston 2021 (German 2017).

• Bakels, Jan Hendrik / Greifenstein, Sarah / Kappelhoff, Hermann: Die Poiesis des Filme-Sehens. Methoden der Analyse audiovisueller Bilder. Berlin / Boston (in prep.).