Multimodality and the Moving Image

The seminar on Multimodality and the Moving Image pushed forward thinking about this topic in several areas.


Theo van Leeuwen’s presentation revisited the key concepts of Reading Images, proposing new moves such as the addition of an aesthetic metafunction to the functions of representation, interaction and textuality derived from Halliday.


He demonstrated how conceptual and narrative functions could be analysed in tabular form using examples of animation from film title sequences and a Chinese public health video. He argued that animation was an increasingly important form in contemporary society, and that multimodal analysis could help clarify its nature not only for academics but for practitioners and animation software designers.


Andrew Burn presented an analysis of a machinima film by 11 year-olds, arguing that it could be analysed as an example of the modes assembled by the moving image (in his term the kineikonic mode), but also the specific modal configurations of animation such as the separation of virtual action and real voice. He argued that the manipulation of digital figures was a new feature of machinima and games, requiring new conceptions of virtual embodiment. He argued that multimodal film analysis could move film studies beyond a narrow focus on the “grammar” of filming and editing, and attend to the promiscuous mingling of cultural forms characteristic of the digital era.


Mark Reid presented examples of young people’s film made with the British Film Institute under the international Le Cinema: Cent Ans de Jeunesse programme. He demonstrated how these could be analysed in terms of contributory modes such as voice and dramatic action, using Burn’s framework. But he also showed how an independent film aesthetic attending to the formal properties of cinematic structures such as the long shot provided both a productive creative constraint as well as a cultural purpose for such work.


John Potter argued for the critical synthesis of multimodal analysis of the moving image with sociocultural theories of identity and culture. He described the film-making of primary school children in informal contexts. His main example demonstrated how a multimodal analysis could reveal uses of movement, gesture, objects and the built environment to construct narratives of selfhood and personal histories. He also referred to a recent project in which the making of films with I-pads produced new haptic manipulation of editing sequences through touch screens.


The discussions during the day developed many longstanding debates about film, multimodality, film studies and education. These included the following ideas.


Multimodality could extend the range of film semiotics – but how to make an impact on film studies and film education was a strategic question. Film educators – at least in the Anglophone world and the Nordic countries – had some history of engagement with multimodal approaches; but Film Studies (and Media Studies) in higher education and research were much less aware. The severe constraints of curriculum policy in England were referred to a number of times.


The alliance of film and drama raised productive questions – about character, narrative, embodiment – as well as possibilities for cross-curricular work in education.


Multimodality theory needed to do more work to develop its account of aesthetics in film, to address cultural distinctions between art film and popular cinema, the role of affect in film and film-making, and the questions of cultural value and processes of evaluation which inevitably arise in education. This work seemed urgent in the context of education, but also in clarifying the cultural role of film in different national contexts – the seminar included visitors from Turkey, Spain and South Korea, for example.