Emergent digital technologies, such as mobile, tangible, sensor and haptic based interfaces, offer a different perspective and approach to interaction. Their very nature offers opportunities for exploiting a wider range of perceptual experiences than traditional desktop computing. In particular, they offer opportunities to exploit bodily-based physical experiences in new ways, for example, through manipulation of physical objects linked to a variety of digitally augmentations; enhancing contextually based experience in real world environments; and promoting novel kinaesthetic experiences, thereby fostering new forms of interaction and new ways of thinking. Such digital environments are inherently multi-modal, requiring a methodological approach that takes multiple modalities into consideration. This project will investigate the role of emergent technologies in engendering ‘embodied’ learning experiences, specifically looking at ways in which they give rise to science learning experiences that are ‘embodied’; ways in which such ‘embodied learning experiences’ can foster scientific understanding and reasoning. In so doing the project aims to further develop notions of what ‘embodied learning’ means in the context of digital technologies, and to better understand the role of multiple modes of representation in communication and meaning making.

The project will run over 30 months (November 2011 – May 2014). Investigators on this project are Sara Price (Principal Investigator), Carey Jewitt and Mona Sakr.

Links and Resources

View the latest project news at the Embodiment Blog

To learn more about embodiment and related terms visit our Embodiment Glossary

For more information on embodied cognition and embodied interaction, see a review by Sara Price, George Roussos, Taciana Pontual Falcão, and Jennifer G. Sheridan: Technology and embodiment: relationships and implications for knowledge, creativity and communication.

Click here to watch a video presentation of the first study which looks at students’ experiences of time and place using mobile technologies to study historical settings.

Project Quarterly Updates:

December 2013

Over the last quarter, the project has focused on analysis of the second (of three) research study, examining students’ experiences of space and place when using mobile technologies and writing papers; planning for the third research study, which aims to investigate the role of whole-body interaction through sensor based technologies in fostering meaning making; running a training day on mobile technology research; and continued development of the online embodiment glossary, and embodiment project blog.

September 2013

Over the last quarter, the project has moved forward with work on three written papers, data collection for two research projects, the creation of an online embodiment glossary, and continued development on the project blog. A conference paper has been submitted to Computer-Human Interaction (CHI). The paper, which is entitled ‘Embodied Experiences of Historical Space: Conflation and Integration of the Physical and the Digital’, uses an analysis of students exploring the WWII history of a local area via mobile technologies to argue how digital augmentation changes the way students interact with the physical environment, and how they work to integrate physical and digital elements of their experience. Two working papers are currently being re-drafted and will be available online in the next quarter. One paper considers the methodological challenges of investigating mobile technologies and embodied experiences of place; it offers a multimodal framework for future research in this area. The other paper explores how digital environments cause us to rethink the ‘body’ in various ways: the role of the body in learning and communication; its role in the performance of identity; as a semiotic resource to be ‘read’ by others; and as the subject of representations.

Data collection has finished for the second main research study within this project. Data collection or a large study looking at students’ experiences of place when using mobile technologies was highly successful. The students involved enjoyed using a mobile environment on the iPad to explore on-site the WWII history of a local area. Through pre-exploration, in situ exploration and post-exploration reflections, we gathered data on how interaction with the mobile technologies changed the way they thought about and constructed place. The data collected through the study has since been archived and transcription and analysis of the videos of the exploration phase has begun. Preliminary analyses are the basis for the first paper mentioned above (‘Embodied Experiences of Historical Space’). Plans for further publications based on analysis are under way. In the coming quarter, we hope to share our findings with the school where the data was collected and discuss future participation in order to look at long-term implications of the experience.

The second research project with a visiting academic from Barcelona (Lucrezia Crescenzi), looking at the mode of touch when young children draw using iPads in a nursery setting, also completed data collection. The data is currently being analysed in order to understand more about very young children’s interaction with touch-screen interfaces. The findings will be discussed with practitioners in the children’s centre where the data was collected, and shared with the wider research community through a paper to be published in a special issue on Touch and Technology.

 

June 2013

Price, Jewitt and Sakr have made significant developments in the project over the last quarter including data collection for two new research studies, three publications, and the creation of a new blog on research in embodiment.

The first study is examining students’ experiences of body and place when using mobile technologies. Working closely with a London primary school, we have developed a mobile environment on the iPad to enable students to explore on-site the history of a local common linking with the history curriculum through which to understand how such experiences of mobile technologies can impact on how people think about and construct place. The project is using novel methods for data collection including a range of visual techniques, GPS trackers, and a range of video recorders – a body-cam, sense-cam, and head-mounted camera alongside researcher generated video. This design will enable MODE to explore and compare the potentials and limitations of different cameras and visual methods.

The second project, with Dr Lucrezia Crescenzi a MODE visiting academic from Spain, is exploring very young children’s use of the mode of touch when they draw using iPads in a nursery setting to understand more about children’s interaction with touch-screen interfaces. The pilot data has been collected. The study will be presented in a paper in a special issue of a journal on Touch and Technology.

A paper was presented at the International Conference of Human Computer Interaction (CHI); and a paper (Sakr, M., Jewitt, C. and Price, S. ‘The semiotic work of hands in scientific inquiry’) on the analysis of students exploring the properties of light via a tangible tabletop, that argues that the hands play an essential role in the process of inquiry has been accepted for publication in a special issue of Classroom Discourse on multimodality. Draft versions of two MODE working papers have also been produced. One paper considers the methodological challenges of investigating mobile technologies and embodied experiences of place; it offers a multimodal framework for future research in this area. The other explores how digital environments cause us to rethink the body in various ways: the role of the body in learning and communication; its role in the performance of identity; as a semiotic resource to be ‘read’ by others; and as the subject of representations.

April 2013

The project progressed significantly over the last quarter notably with the recruitment of a new researcher (Mona Sakr) and collaboration with a new visiting fellow (Lucrezia Crescenzi). In particular, the project has undertaken in depth analysis of empirical studies using a purpose built tangible tabletop learning environment. This has resulted in a number of publications and conference papers and a push on the dissemination of the methodological and substantive findings from the work as well as informing TCB events on embodiment.  We are currently writing a paper from the embodiment TCB seminar that compares different approaches to embodiment in digital environments. This includes problematizing and rethinking notions of scientific discourse in a multimodal tangible environment. We are currently designing the second study and have secured the participation of a London primary school for the study for the summer term. The study will focus on the use of mobiles and look at embodiment and location with a focus on spatial thinking and way finding.

In relation to the on going methodological development we have designed a small project with Lucrezia to explore the mode of touch when very young children draw and paint using iPads in a nursery setting. This work will employ a multimodal analysis to explore touch and result in a paper on touch as a mode for a special issue on Touch and Technology.

Oct 2012

The project progressed significantly over the last quarter. In particular, the project has undertaken empirical studies using a purpose built tangible tabletop learning environment in June/ July 2012. Twenty three students aged 10-11 years from a London primary school, accompanied by their teacher, and took part in the study. The study attended to both substantive issues and methodological issues. In relation to the first, we explored the role of action in meaning making in multimodal environments. To do this, students worked in pairs to explore concepts of the physics of light using a tangible tabletop interface. Sessions were video recorded to access data about how body positioning, orientation, gesture, and gaze are used when collaboratively engaging in meaning making of science concepts. During July – Sept we have undertaken analyses looking at the effect of body positioning on interaction and communication with the technology and one another specifically analysing details of the role of body position, gaze, manipulation and speech in shaping interaction. The analysis illustrates the interaction between these modes and ‘multimodal action flow’, particularly in terms of pace, rhythm and interaction structure, and the implications of this for interaction and the meaning making process.

In relation to the ongoing methodological development we explored multimodal methodological approaches in the context of tangible tabletop interaction. In particular we took a non-facilitated approach to student interaction with the interface, and examined different post interaction methodologies for reviewing interaction, using three alternative semi-structured interview approaches: straight verbal interview; verbal interview with video replay of students’ own session- video prompted recall; interview with the tangible table itself which could be used for demonstration and explanation. The comparative analysis of interview design shows how the nature of information elicited differs. Students have access to a different set of resources to recall, link to, and describe their physical embodied experience, and so the work that they are engaged in within the interviews becomes different.

June 2012

The project continues to develop a deeper understanding of notions of embodiment in the context of emergent digital technology environments; and to explore the application of multimodal methodologies for examining learning and meaning making. In particular,

• Empirical studies will be carried out in June 2012 to:
  1. Explore the role of action in meaning making in multimodal environments. To do this 12 pairs of students aged 10-11 years will explore concepts of the physics of light using a tangible tabletop interface. In particular, we aim to see how gesture, fine and gross motor interaction, posture, orientation and gaze are used when collaboratively engaging in meaning making of science concepts.
  2. Explore multimodal methodological approaches in the context of tangible tabletop interaction. In particular the study will enable the project to examine different post interaction methodologies for reviewing interaction; and to explore the value of both micro and macro analysis.

• Drawing on the literature review on embodiment the project is:
  1. Developing a glossary around key concepts of embodiment. This aims to draw together some key terms used in literature that concerns ideas around embodiment, and once developed will form an on-line searchable resource.
  2. Preparing a paper for submission to the British Journal of Educational Technology on Embodiment and Digital Technology in Education, outlining key theoretical approaches to embodiment, and their role in thinking about education/ learning with emergent digital technologies.

• Planning future empirical work around embodiment, space and place, working with different map-based representations and their relationship to physical experience of place and space, and the role of digital technologies in students’ perspective taking and understanding of location, direction and environment.

March 2012

To date the project has focused on developing a deeper understanding of theories of embodiment and how they might be realised in digital technology environments; developing our knowledge of multimodal methods and exploring the implications of using this approach for research; and developing empirical studies for implementation over the coming months. We have achieved this by:

(i) Doing an extensive literature review on embodiment, which traces its historical origins and the developing ideas around this notion to the present day; draws on the multi-disciplinary perspectives and approaches and where possible, making links across disciplines; specifically explores the relationship between these approaches and current thinking in Human Computer Interaction to identify key points of application to emergent digital technology environments (e.g. tangible, mobile, sensor). Currently we are specifying these key concepts of embodiment to inform empirical work. This will also be used to form the basis of a glossary.

(ii) Identifying key concepts from multimodality, and exploring their relationship with key concepts from embodiment in order to develop an effective methodological approach for these digital learning environments. We have explored the application of multimodal perspectives with video data from a tangible learning environment.

(iii) Developing empirical studies: study designs are based on concepts drawn from the literature and are designed to fit with school curriculum concepts being learned through the particular digital environments being investigated. It is anticipated that empirical work will begin in the next couple of months.