The Effects of Video Instructor’s Body Language on Students’ Distribution of Visual Attention: an Eye-tracking Study

Previous studies have shown that the instructor’s presence in video lectures has a positive effect on learners’ experience. However, it does increase the cost of video production and may increase learners’ cognitive load. An alternative to instructor’s presence is the use of embodied pedagogical agents that display limited but appropriate social signals. In this extended abstract, we report a small experimental study into the effects of video instructor’s behaviour on students’ learning experience, with the long term aim of better understanding which instructor’s social signals should be applied to pedagogical agents. We used eye-tracking technology and data visualisation techniques to collect and analyse students’ distribution of visual attention in relation to the instructor’s speech and body language. Participants also answered questions about their attitudes toward the instructor. The results suggest that the instructor’s gaze directed towards the lecture’s slides, or a pointing gesture towards the slides, is not enough to shift viewers’ attention. However, the combination of both is effective. An embodied pedagogical agent should be able to display a multimodal behaviour, combining gaze and gestures, to effectively direct the learners’ visual attention towards the relevant material. Furthermore, to make learners pay attention to the lecturer’s speech, the instructional agent should make use of pauses and emphasis.

Content

Author and article information

Contributors

Jiawen Zhang

Marie-Luce Bourguet

Gentiane Venture

Conference

Publication date: July 2018

Publication date (Print): July 2018

Pages: 1-5

Affiliations

[0001]Beijing University of Posts and Telecommunications

[0002]Queen Mary University of London

[0003]Tokyo University of Agriculture and Technology

Article

DOI: 10.14236/ewic/HCI2018.101

SO-VID: 8a1abcdf-5800-4022-9e19-6e2812dad46a

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 32

Conference location: Belfast, UK

Conference date: 4 - 6 July 2018

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Human Computer Interaction Conference

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.101

Self URI (journal page): https://ewic.bcs.org/

References

A. Bojko (2009). Informative or Misleading? Heatmaps Deconstructed. Human-Computer Interaction New Trends. Springer Berlin Heidelberg.
P. Chandler, & J. Sweller (1991). Cognitive load theory and the format of instruction Cognition & Instruction 8(4), 293–332.
C. M. Chen, & C. H. Wu (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance Computers & Education 80(5), 108–121.
R. C. Clark, & R. E. Mayer (2011). E-learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. Pfeiffer.
N. Garrett (2015). Eye-Tracking Analytics in Instructional Videos ISECON.
P. J. Guo J. Kim, & R. Rubin (2014). How video production affects student engagement: an empirical study of MOOC videos ACM Conference on Learning @ Scale Conference (Vol.43, pp. 41–50). ACM.
F.M. Hollands, & D. Tirthali (2014). MOOCs: Expectation sand reality. Full Report Center for Benefit Cost Studies of Education, Teachers College Columbia University, NY.
M. A. Just, & P. A. Carpenter (1980). A theory of reading: from eye fixations to comprehension Psychological Review 87(4), 329.
M. Kipp (2014). ANVIL: A Universal Video Research Tool. In J. Durand U. Gut G. Kristofferson (Eds.) Handbook of Corpus Phonology Oxford University Press 420–436.
R. F. Kizilcec K. Papadopoulos, & L. Sritanyaratana (2014). Showing face in video instruction: effects on information retention, visual attention, and affect 2095–2102.
M. L. Lai M. J. Tsai F. Y. Yang C. Y. Hsu T. C. Liu, & W. Y. Lee et al. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012 Educational Research Review 10(4), 90–115.
J. Li R. Kizilcec J. Bailenson, & W. Ju (2015). Social robots and virtual agents as lecturers for virdo instruction Computers in Human Behavior 55 1222–1230.
R. E. Mayer (2001). Multimedia Learning Cambridge University Press.
K. Ouwehand T. Van Gog, & F. Paas (2015). Designing effective video-based modeling examples using gaze and gesture cues Educational Technology & Society 18.
Z. Pi J. Hong, & J. Yang (2017). Effects of the instructor's pointing gestures on learning performance in video lectures British Journal of Educational Technology 48(4), 1020–1029.
K. Sharma P. Jermann, & P. Dillenbourg (2014). How Students Learn using MOOCs: An Eye-tracking Insight EMOOCs 2014, the Second MOOC European Stakeholders Summit.
K. Sharma H. S. Alavi P. Jermann, & P. Dillenbourg (2016). A gaze-based learning analytics model:in-video visual feedback to improve learner's attention in moocs 417–421.
SoftBank Robotics (2017) “Find out more about Pepper”. [Online] Available from: http://www.ald.softbankrobotics.com/en/robots/pepper [accessed 28 March 2018].
Y. Tian, & M. L. Bourguet (2016). Lecturers' Hand Gestures as Clues to Detect Pedagogical Significance in Video Lectures European Conference on Cognitive Ergonomics (pp.2). ACM.
J. Wang, & P. D. Antonenko (2017). Instructor presence in instructional video: effects on visual attention, recall, and perceived learning Computers in Human Behavior 71 79–89.
A. M. F. Yousef M. A. Chatti, & U. Schroeder (2014). Video-Based Learning: A Critical Analysis of The Research Published in 2003-2013 and Future Visions eLmL 2014 : The Sixth International Conference on Mobile, Hybrid, and On-line Learning (pp. 112–119).
J. R. Zhang (2012). Upper body gestures in lecture videos:indexing and correlating to pedagogical significance ACM International Conference on Multimedia (pp. 1389–1392). ACM.

Comments

Comment on this article

[1] A. Bojko (2009). Informative or Misleading? Heatmaps Deconstructed. Human-Computer Interaction New Trends. Springer Berlin Heidelberg.

[2] P. Chandler, & J. Sweller (1991). Cognitive load theory and the format of instruction Cognition & Instruction 8(4), 293–332.

[3] C. M. Chen, & C. H. Wu (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance Computers & Education 80(5), 108–121.

[4] R. C. Clark, & R. E. Mayer (2011). E-learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. Pfeiffer.

[5] N. Garrett (2015). Eye-Tracking Analytics in Instructional Videos ISECON.

[6] P. J. Guo J. Kim, & R. Rubin (2014). How video production affects student engagement: an empirical study of MOOC videos ACM Conference on Learning @ Scale Conference (Vol.43, pp. 41–50). ACM.

[7] F.M. Hollands, & D. Tirthali (2014). MOOCs: Expectation sand reality. Full Report Center for Benefit Cost Studies of Education, Teachers College Columbia University, NY.

[8] M. A. Just, & P. A. Carpenter (1980). A theory of reading: from eye fixations to comprehension Psychological Review 87(4), 329.

[9] M. Kipp (2014). ANVIL: A Universal Video Research Tool. In J. Durand U. Gut G. Kristofferson (Eds.) Handbook of Corpus Phonology Oxford University Press 420–436.

[10] R. F. Kizilcec K. Papadopoulos, & L. Sritanyaratana (2014). Showing face in video instruction: effects on information retention, visual attention, and affect 2095–2102.

[11] M. L. Lai M. J. Tsai F. Y. Yang C. Y. Hsu T. C. Liu, & W. Y. Lee et al. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012 Educational Research Review 10(4), 90–115.

[12] J. Li R. Kizilcec J. Bailenson, & W. Ju (2015). Social robots and virtual agents as lecturers for virdo instruction Computers in Human Behavior 55 1222–1230.

[13] R. E. Mayer (2001). Multimedia Learning Cambridge University Press.

[14] K. Ouwehand T. Van Gog, & F. Paas (2015). Designing effective video-based modeling examples using gaze and gesture cues Educational Technology & Society 18.

[15] Z. Pi J. Hong, & J. Yang (2017). Effects of the instructor's pointing gestures on learning performance in video lectures British Journal of Educational Technology 48(4), 1020–1029.

[16] K. Sharma P. Jermann, & P. Dillenbourg (2014). How Students Learn using MOOCs: An Eye-tracking Insight EMOOCs 2014, the Second MOOC European Stakeholders Summit.

[17] K. Sharma H. S. Alavi P. Jermann, & P. Dillenbourg (2016). A gaze-based learning analytics model:in-video visual feedback to improve learner's attention in moocs 417–421.

[18] SoftBank Robotics (2017) “Find out more about Pepper”. [Online] Available from: http://www.ald.softbankrobotics.com/en/robots/pepper [accessed 28 March 2018].

[19] Y. Tian, & M. L. Bourguet (2016). Lecturers' Hand Gestures as Clues to Detect Pedagogical Significance in Video Lectures European Conference on Cognitive Ergonomics (pp.2). ACM.

[20] J. Wang, & P. D. Antonenko (2017). Instructor presence in instructional video: effects on visual attention, recall, and perceived learning Computers in Human Behavior 71 79–89.

[21] A. M. F. Yousef M. A. Chatti, & U. Schroeder (2014). Video-Based Learning: A Critical Analysis of The Research Published in 2003-2013 and Future Visions eLmL 2014 : The Sixth International Conference on Mobile, Hybrid, and On-line Learning (pp. 112–119).

[22] J. R. Zhang (2012). Upper body gestures in lecture videos:indexing and correlating to pedagogical significance ACM International Conference on Multimedia (pp. 1389–1392). ACM.

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65