How teachers interpret displays of students’ gaze in reading comprehension assignments

Main Article Content

Carolien A. N. Knoop-van Campen
Ellen Kok
Roos van Doornik
Pam de Vries
Marleen Immink
Halszka Jarodzka
Tamara van Gog

Abstract

Reading comprehension is a central skill in secondary education. To be able to provide adaptive instruction, teachers need to be able to accurately estimate students’ reading comprehension. However, they tend to experience difficulties doing so. Eye tracking can uncover these reading processes by visualizing what a student looked at, in what order, and for how long, in a gaze display. The question is, however, whether teachers could interpret such displays. We, therefore, examined how teachers interpret gaze displays and perceived their potential use in education to foster tailored support for reading comprehension. Sixty teachers in secondary education were presented with three static gaze displays of students performing a reading comprehension task. Teachers were asked to report how they interpreted these gaze displays and what they considered to be the promises and pitfalls of gaze displays for education. Teachers interpreted in particular reading strategies in the gaze displays quite well, and also interpreted the displays as reflecting other concepts, such as motivation and concentration. Results showed that teachersinterpretations of the gaze displays were generally consistent across teachers and that teachers discriminated well between displays of different strategies. Teachers were generally positive about potential applications in educational practice. This study provides first insights into how teachers experience the utility of gaze displays as an innovative tool to support reading instruction, which is timely as rapid technological developments already enable eye tracking through webcams on regular laptops. Thus, using gaze displays in an educational setting seems to be an increasingly feasible scenario.

Article Details

How to Cite
Knoop-van Campen, C. A. N., Kok, E., van Doornik, R., de Vries, P., Immink, M., Jarodzka, H., & van Gog, T. (2021). How teachers interpret displays of students’ gaze in reading comprehension assignments. Frontline Learning Research, 9(4), 116 - 140. https://doi.org/10.14786/flr.v9i4.881
Section
Articles

References

Afflerbach, P., Pearson, P. D., & Paris, S. G. (2008). Clarifying differences between reading skills and reading strategies. The Reading Teacher, 61(5), 364-373. https://doi.org/https://doi.org/10.1598/RT.61.5.1

Almosallam, E. A., & Ouertani, H. C. (2013). Learning analytics: Definitions, applications and related fields. Proceedings of the first international conference on advanced data and information engineering (DaEng-2013),

Andreassen, R., Jensen, M. S., & Bråten, I. (2017). Investigating self-regulated study strategies among postsecondary students with and without dyslexia: A diary method study. Reading and Writing, 30(9), 1891-1916. https://doi.org/10.1007/s11145-017-9758-9

Authors. (under review). reference blinded for review.

Bahle, B., Mills, M., & Dodd, M. D. (2017). Human classifier: Observers can deduce task solely from eye movements. Attention Perception & Psychophysics, 79(5), 1415-1425. https://doi.org/10.3758/s13414-017-1324-7

Benfatto, M. N., Öqvist Seimyr, G., Ygge, J., Pansell, T., Rydberg, A., & Jacobson, C. (2016). Screening for dyslexia using eye tracking during reading. Plos One, 11(12), e0165508. https://doi.org/10.1371/journal.pone.0165508

Chisari, L. B., Mockevičiūtė, A., Ruitenburg, S. K., van Vemde, L., Kok, E. M., & van Gog, T. (2020). Effects of prior knowledge and joint attention on learning from eye movement modelling examples. Journal of Computer Assisted Learning, 36(4), 569-579. https://doi.org/10.1111/jcal.12428

de Koning, B. B., & Jarodzka, H. (2017). Attention guidance strategies for supporting learning from dynamic visualizations. In R. Lowe & R. Ploetzner (Eds.), Learning from Dynamic Visualization: Innovations in Research and Application (pp. 255-278). Springer International Publishing. https://doi.org/10.1007/978-3-319-56204-9_11

Donovan, T., Manning, D. J., & Crawford, T. (2008). Performance changes in lung nodule detection following perceptual feedback of eye movements Proceedings of SPIE - The International Society for Optical Engineering in San Diego, CA, http://dx.doi.org/10.1117/12.768503

Duffy, G. G. (2002). The case for direct explanation of strategies. In C. C. Block & M. Pressley (Eds.), Comprehension instruction: Research-based best practices (pp. 28-41). Guilford.

Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension. Journal of education, 189(1-2), 107-122. https://doi.org/10.1177/0022057409189001-208

Eder, T. F., Richter, J., Scheiter, K., Keutel, C., Castner, N., Kasneci, E., & Huettig, F. (2020). How to support dental students in reading radiographs: effects of a gaze‑based compare‑and‑contrast intervention. Advances in Health Sciences Education: Theory and Practice. https://doi.org/10.1007/s10459-020-09975-w

Emhardt, S. N., van Wermeskerken, M., Scheiter, K., & van Gog, T. (2020). Inferring task performance and confidence from displays of eye movements. Applied Cognitive Psychology, 34(6), 1430-1443. https://doi.org/https://doi.org/10.1002/acp.3721

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. MIT Press. http://books.google.nl/books?id=z4HQQgAACAAJ

Foulsham, T., & Lock, M. (2015). How the eyes tell lies: Social gaze during a preference task. Cognitive science, 39(7), 1704-1726. https://doi.org/10.1111/cogs.12211

Gambrell, L. B., Malloy, J. A., & Mazzoni, S. A. (2011). Evidence-based best practices for comprehensive literacy instruction. In L. M. Morrow & L. B. Gambrell (Eds.), Best practices in literacy instruction (fourth edition) (Vol. 4, pp. 11-56). The Guilford Press.

Guthrie, J. T., & Mosenthal, P. (1987). Literacy as multidimensional: Locating information and reading comprehension. Educational Psychologist, 22(3-4), 279-297. https://doi.org/10.1080/00461520.1987.9653053

Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication methods and measures, 1(1), 77-89. https://doi.org/10.1080/19312450709336664

Henneman, E. A., Cunningham, H., Fisher, D. L., Plotkin, K., Nathanson, B. H., Roche, J. P., Marquard, J. L., Reilly, C. A., & Henneman, P. L. (2014). Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices. Dimensions of Critical Care Nursing, 33(3), 129-135. https://doi.org/10.1097/DCC.0000000000000041

Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction, 25(0), 62-70. https://doi.org/10.1016/j.learninstruc.2012.11.004

Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: Pitfalls of learning analytics dashboards in the educational practice. European conference on technology enhanced learning, Tallinn, Estonia.

Katalayi, G. B., & Sivasubramaniam, S. (2013). Careful reading versus expeditious reading: Investigating the construct validity of a multiple-choice reading test. Theory and Practice in Language Studies, 3, 877-884. https://doi.org/10.4304/tpls.3.6.877-884

Knoop-van Campen, C., & Molenaar, I. (2020). How teachers integrate dashboards into their feedback practices. Frontline Learning Research, 8(4), 37-51. https://doi.org/10.14786/flr.v8i4.641

Knoop-van Campen, C. A. N., Wise, A., & Molenaar, I. (2021). The equalizing effect of teacher dashboards on feedback in K-12 classrooms. Interactive Learning Environments, 1-17. https://doi.org/10.1080/10494820.2021.1931346

Kok, E. M., Aizenman, A. M., Võ, M. L.-H., & Wolfe, J. M. (2017). Even if I showed you where you looked, remembering where you just looked is hard. Journal of vision, 17(12), 1-11. https://doi.org/10.1167/17.12.2

Kostons, D., van Gog, T., & Paas, F. (2009). How do I do? Investigating effects of expertise and performance-process records on self-assessment. Applied Cognitive Psychology, 23(9), 1256–1265. https://doi.org/10.1002/acp.1528

Krishnan, K. S. D. (2011). Careful versus expeditious reading: The case of the IELTS reading test. Academic Research International, 1(3), 25.

Liu, F. (2010). Reading abilities and strategies: A short introduction. International Education Studies, 3(3), 153-157. https://doi.org/10.5539/ies.v3n3p153

Mason, L., Pluchino, P., & Tornatora, M. C. (2015). Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning. Contemporary Educational Psychology, 41, 172-187. https://doi.org/10.1016/j.cedpsych.2015.01.004

Molenaar, I., & Knoop-van Campen, C. (2019). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies, 12(3), 347-355. https://doi.org/10.1109/TLT.2018.2851585

Murnane, R., Sawhill, I., & Snow, C. (2012). Literacy challenges for the twenty-first century: Introducing the issue. The Future of Children, 3-15. https://doi.org/10.1353/foc.2012.0013

Oudman, S., van de Pol, J., Bakker, A., Moerbeek, M., & van Gog, T. (2018). Effects of different cue types on the accuracy of primary school teachers' judgments of students' mathematical understanding. Teaching and Teacher Education, 76, 214-226. https://doi.org/https://doi.org/10.1016/j.tate.2018.02.007

Pikulski, J. J., & Chard, D. J. (2005). Fluency: Bridge between decoding and reading comprehension. The Reading Teacher, 58(6), 510-519. https://doi.org/10.1598/RT.58.6.2

Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A technology acceptance perspective of 95 teachers. International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3493

Rosengrant, D., Hearrington, D., & O’Brien, J. (2021). Investigating Student Sustained Attention in a Guided Inquiry Lecture Course Using an Eye Tracker. Educational psychology review, 33(1), 11-26. https://doi.org/10.1007/s10648-020-09540-2

Salmerón, L., Naumann, J., García, V., & Fajardo, I. (2017). Scanning and deep processing of information in hypertext: an eye tracking and cued retrospective think‐aloud study. Journal of Computer Assisted Learning, 33(3), 222-233. https://doi.org/10.1111/jcal.12152

Scheiter, K., Schubert, C., & Schüler, A. (2018). Self-regulated learning from illustrated text: Eye movement modelling to support use and regulation of cognitive processes during learning from multimedia. British Journal of Educational Psychology, 88(1), 80-94. https://doi.org/doi:10.1111/bjep.12175

Schutz, K. M., & Rainey, E. C. (2020). Making sense of modeling in elementary literacy instruction. The Reading Teacher, 73(4), 443-451. https://doi.org/10.1002/trtr.1863

Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. In P. J. Durlach & A. Lesgold (Eds.), Adaptive technologies for training and education (Vol. 7, pp. 1-35). Cambridge University Press. https://doi.org/10.1017/CBO9781139049580.004

Špakov, O., Siirtola, H., Istance, H., & Räihä, K. (2017). Visualizing the reading activity of people learning to read. Journal of Eye Movement Research, 10(5), 1-12. https://doi.org/10.16910/jemr.10.5.5

Tijmstra, J., & Boeije, H. (2011). Wetenschapsfilosofie in de context van de sociale wetenschappen. Boom Lemma.

Urquhart, S., & Weir, C. (1998). Reading in a Second Language: Process, Product and Practice. . Routledge.

van de Pol, J., de Bruin, A. B. H., van Loon, M. H., & van Gog, T. (2019). Students’ and teachers’ monitoring and regulation of students’ text comprehension: Effects of comprehension cue availability. Contemporary Educational Psychology, 56, 236-249. https://doi.org/10.1016/j.cedpsych.2019.02.001

Van de Pol, J., Van den Boom-Muilenburg, S. N., & Van Gog, T. (in press). Exploring the relations between teachers’ cue-utilization, monitoring and regulation of students’ text learning. Metacognition and Learning.

van Gog, T., & Jarodzka, H. (2013). Eye tracking as a tool to study and enhance cognitive and metacognitive processes in computer-based learning environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of metacognition and learning technologies (pp. 143-156). Springer Science+ Business media.

Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25(3), 785-791. https://doi.org/10.1016/j.chb.2009.02.007

Van Leeuwen, A., Knoop-van Campen, C., Molenaar, I., & Rummel, N. (in press). How teacher characteristics relate to how teachers use dashboards: results from two case studies. . Journal of Learning Analytics.

van Leeuwen, A., Rummel, N., & van Gog, T. (2019). What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations? International Journal of Computer-Supported Collaborative Learning, 14(3), 261-289. https://doi.org/10.1007/s11412-019-09299-x

van Wermeskerken, M., Litchfield, D., & van Gog, T. (2018). What am i looking at? Interpreting dynamic and static gaze displays. Cognitive science, 42(1), 220-252. https://doi.org/10.1111/cogs.12484

Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499-1514. https://doi.org/10.1007/s00779-013-0751-2

Võ, M. L. H., Aizenman, A. M., & Wolfe, J. M. (2016). You think you know where you looked? You better look again. Journal of experimental psychology: Human perception and performance, 42(10), 1477-1481. https://doi.org/10.1037/xhp0000264

Xhakaj, F., Aleven, V., & McLaren, B. M. (2017). Effects of a teacher dashboard for an intelligent tutoring system on teacher knowledge, lesson planning, lessons and student learning. European Conference on Technology Enhanced Learning,

Zelinsky, G. J., Peng, Y., & Samaras, D. (2013). Eye can read your mind: Decoding gaze fixations to reveal categorical search targets. Journal of vision, 13(14), 1-13. https://doi.org/10.1167/13.14.10