Assessing Students’ Interpretations of Histograms Before and After Interpreting Dotplots: A Gaze-Based Machine Learning Analysis

Main Article Content

Lonneke Boels
Alex Lyford
Arthur Bakker
Paul Drijvers


Many students persistently misinterpret histograms. Literature suggests that having students solve dotplot items may prepare for interpreting histograms, as interpreting dotplots can help students realize that the statistical variable is presented on the horizontal axis. In this study, we explore a special case of this suggestion, namely, how students’ histogram interpretations alter during an assessment. The research question is: In what way do secondary school students’ histogram interpretations change after solving dotplot items? Two histogram items were solved before solving dotplot items and two after. Students were asked to estimate or compare arithmetic means. Students’ gaze data, answers, and cued retrospective verbal reports were collected. We used students’ gaze data on four histogram items as inputs for a machine learning algorithm (MLA; random forest). Results show that the MLA can quite accurately classify whether students’ gaze data belonged to an item solved before or after the dotplot items. Moreover, the direction (e.g., almost vertical) and length of students’ saccades were different on the before and after items. These changes can indicate a change in strategies. A plausible explanation is that solving dotplot items creates readiness for learning and that reflecting on the solution strategy during recall then brings new insights. This study has implications for assessments and homework. Novel in the study is its use of spatial gaze data and its use of an MLA for finding differences in gazes that are relevant for changes in students’ task-specific strategies.

Article Details

How to Cite
Boels, L., Lyford, A., Bakker, A., & Drijvers, P. (2023). Assessing Students’ Interpretations of Histograms Before and After Interpreting Dotplots: A Gaze-Based Machine Learning Analysis. Frontline Learning Research, 11(2).
Author Biographies

Alex Lyford, Middlebury College, Middlebury, VT, USA

Department of Mathematics

Arthur Bakker, Utrecht University

Faculty of Science, Department of Mathematics, Freudenthal Institute

Paul Drijvers, Utrecht University

Faculty of Science, Department of Mathematics, Freudenthal Institute


Bakker, A. (2004a). Design research in statistics education: On symbolizing and computer tools. Doctoral thesis. Utrecht, the Netherlands: Utrecht University.

Bakker, A. (2004b). Reasoning about shape as a pattern in variability. Statistics Education Research Journal, 3(2), 64–83.

Bakker, A., Cai, J., English, L. Kaiser, G., Mesa, V., & Van Dooren, W. (2019). Beyond small, medium, or large: points of consideration when interpreting effect sizes. Educational Studies in Mathematics 102, 1–8.

Boels, L., Bakker, A., Van Dooren, W., & Drijvers, P. (2019). Conceptual difficulties when interpreting histograms: A review. Educational Research Review 28, art. 100291,

Bennett, R. E. (2011) Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice 18(1), 5–25,

Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth and Brooks/Cole Advanced Books and Software.

Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017). Classification and regression trees. Routledge.

Breiman, L. (2001). Random forests. Machine learning, 45(1), 5–32.

Capraro, M. M., Kulm, G., & Capraro, R. M. (2005). Middle grades: Misconceptions in statistical thinking. School Science & Mathematics 105(4), 165–174.

Catron, D. W. (1978). Immediate test-retest changes in WAIS scores among college males. Psychological Reports 43(1), 279–290.

Church, R.B., & Goldin-Meadow, S. (1986). The mismatch between gesture and speech as an index of transitional knowledge. Cognition 23(1), 43–71.

Clayden, A., & Croft, M. (1990). Statistical consultation—who's the expert? Annals of Mathematics and Artificial Intelligence 2, 65–75.

Cohen, S. (1996). Identifying impediments to learning probability and statistics from an assessment of instructional software. Journal of Educational and Behavioral Statistics 21(1), 35–54.

Cooper, L. L. (2018). Assessing students' understanding of variability in graphical representations that share the common attribute of bars. Journal of Statistics Education 26(2), 110–124.

Cooper, L. L., & Shore, F. S. (2008). Students' misconceptions in interpreting center and variability of data represented via histograms and stem-and-leaf plots. Journal of Statistics Education 16(2),

Dabos, M. (2014). A glimpse of two year college instructors' understanding of variation in histograms. In K. Makar, B. de Sousa, and R. Gould (Eds.). Sustainability in statistics education. Proceedings of the 9th international conference on teaching statistics (pp. 1–4). Flagstaff, Arizona, USA.

delMas, R., Garfield, J., & Ooms, A. (2005). Using assessment items to study students' difficulty reading and interpreting graphical representations of distributions. Proceedings of the fourth international research forum on statistical reasoning, literacy, and reasoning. Auckland, New Zealand: University of Auckland.

delMas, R., & Liu, Y. (2005). Exploring students' conceptions of the standard deviation. Statistics Education Research Journal 4(1), 55–82.

Dzeroski, S., & Zenko, B. (2004). Is combining classifiers with stacking better than selecting the best one? Machine Learning 54(3), 255–273.

Falleti, M. G., Maruff, P., Collie, A. & Darby, D. G. (2006). Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals, Journal of Clinical and Experimental Neuropsychology 28(7), 1095–1112.

Fisher, R. A. (1922). On the interpretation of χ2 from contingency tables, and the calculation of P. Journal of the Royal Statistical Society 85(1), 87-94.

Friedman, J., Hastie, T., & Tibshirani, R. (2001). The elements of statistical learning (Vol. 1, No. 10). Springer series in statistics.

Fuchs, A. F. (1967). Saccadic and smooth pursuit eye movements in the monkey. The Journal of Physiology 191(3), 609–631.

Gal, I. (1995). Statistical tools and statistical literacy: The case of the average. Teaching Statistics 17(3), 97–99.

Gal, I. (2002). Adults' statistical literacy: Meanings, components, responsibilities. International statistical review 70(1), 1–25.

Garcia Moreno-Esteva, E., Kervinen, A., Hannula, M. S., & Uitto, A. (2020). Scanning signatures: A graph theoretical model to represent visual scanning processes and a proof of concept study in biology Education. Education Sciences 10(5)

Garcia Moreno-Esteva, E., White, S. L. J., Wood, J. M., & Black, A. A. (2018). Application of mathematical and machine learning techniques to analyse eye tracking data enabling better understanding of children’s visual cognitive behaviours. Frontline Learning Research 6(3), 72–84.

Garfield, J. (2002). Histogram sorting. Statistics teaching and resource library (STAR).

Garfield, J. B., Ben-Zvi, D., Chance, B., Medina, E., Roseth, C., & Zieffler, A. (2008). Learning to reason about distribution. In J. Garfield & D. Ben-Zvi (Eds). Developing students’ statistical reasoning (pp. 165–186). Springer.

Godau, C. Haider, H., Hansen, S., Schubert, T., Frensch, P. A., Gaschler, R. (2014). Spontaneously spotting and applying shortcuts in arithmetic—a primary school perspective on expertise. Frontiers in Psychology – Cognition 5, art. e556, 1664–1078.

Goldberg, J. H., & Helfman, J. I. (2010). Comparing information graphics: A critical look at eye tracking. Proceedings of the 3rd BELIV’10 Workshop: Beyond time and errors: novel EvaLuation methods for Information Visualization, Atlanta, Georgia. 71–78.

Guan, Z., Lee, S., Cuddihy, E., & Ramey, J. (2006). The validity of the stimulated retrospective think-aloud method as measured by eye tracking. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1253–1262. Montréal, Québec, Canada.

Guerra-Carrillo, B. C., Bunge, S.A. (2018). Eye gaze patterns reveal how reasoning skills improve with experience. npj Science of Learning 3, art. 18.

Hansen, L. K., & Salamon, P. (1990). Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence 12, 993–1001.

Heilbronner, R. L., Sweet, J. J., Attix, D. K., Krull, K. R., Henry, G. K., & Hart, R. P. (2010). Official position of the american academy of clinical neuropsychology on serial neuropsychological assessments: the utility and challenges of repeat test administrations in clinical and forensic contexts, The Clinical Neuropsychologist, 24(8), 1267–1278.

Hessels, R. S., Niehorster, D. C., Nyström, M., Andersson, R., & Hooge, I. T. C. (2018). Is the eye-movement field confused about fixations and saccades? A survey among 124 researchers. Royal Society Open Science 5(8), 1–23.

Hinton-Bayre, A. D. (2010). Deriving Reliable Change Statistics from Test–Retest Normative Data: Comparison of Models and Mathematical Expressions. Archives of Clinical Neuropsychology 25(3), 244–256.

Hohn, R. W. (1992). An analysis of the components of curriculum-based assessment. Doctoral thesis. Denver, CO, USA: University of Denver.

Holmqvist, K., Örbom, S. L., Hooge, I. T. C., Niehorster, D. C., Alexander, R. G., Andersson, R., Benjamins, J. S., Blignaut, P., Brouwer, A-M., Chuang, L. L., Dalrymple, K. A., Drieghe, D., Dunn, M. J., Ettinger, U., Fiedler, S., Foulsham, T., Van der Geest, J. N., Witzner Hansen, D., Hutton, S., … Hessels, R. S. (2022). Eye tracking: empirical foundations for a minimal reporting guideline. Behavior Research Methods.

Hyönä, J. (2010). The use of eye movements in the study of multimedia learning. Learning and Instruction, 20(2), 172–176.

Jarodzka, H., Van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction 25, 62–70.

Kaakinen, J.K. (2021). What can eye movements tell us about visual perception processes in classroom contexts? Commentary on a special issue. Educational Psychology Review 33, 169–179.

Kaplan, J., Gabrosek, J., Curtiss, P., & Malone, C. (2014). Investigating student understanding of histograms. Journal of Statistics Education 22(2), 1–29.

Khalil, K. A. I. (2005). Expert-novice differences: Visual and verbal responses in a two-group comparison task (Master thesis.) Amherst, MA, USA: University of Massachusetts.

Konold, C., Higgins, T., Russell, S. J., & Khalil, K. (2015). Data seen through different lenses. Educational Studies in Mathematics 88(3), 305–325.

Konold, C., & Pollatsek, A. (2002). Data analysis as the search for signals in noisy processes. Journal for Research in Mathematics Education 33(4), 259–289.

Kragten, M., Admiraal, W., & Rijlaarsdam, G. (2015). Students’ learning activities while studying biological process diagrams. International Journal of Science Education 37(12), 1915–1937.

Lai, M., Tsai, M., Yang, F., Hsu, C., Liu, T., Lee, S. W., Lee, M., Chiou, G., Liang, J., & Tsai, C. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review 10, 90–115.

Lem, S., Onghena, P., Verschaffel, L., & Van Dooren, W. (2013). On the misinterpretation of histograms and box plots. Educational Psychology 33(2), 155–174.

Lievens, F., Reeve, C. L., & Heggestad, E. D. (2007). An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. Journal of Applied Psychology 92(6), 1672–1682.

Lumsden, J. (1976). Test theory. Annual Review of Psychology 27(1), 251 – 280.

Lyford, A. J. (2017). Investigating undergraduate student understanding of graphical displays of quantitative data through machine learning algorithms. Doctoral thesis. Athens, GA, USA: University of Georgia.

McGatha, M., Cobb, P., & McClain, K. (2002). An analysis of students' initial statistical understandings: Developing a conjectured learning trajectory. The Journal of Mathematical Behavior 21(3), 339–355.

McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157.

Meletiou-Mavrotheris, M. (2000). Students' understanding of variation: An untapped well in statistical reasoning. Doctoral thesis. Austin, TX, USA: University of Texas.

Mokros, J., & Russell, S. J. (1995). Children's concepts of average and representativeness. Journal for Research in Mathematics Education 26(1), 20–39.

Noguchi, K., Abel, R.S., Marmolejo-Ramos, F. & Konietschke, F. (2020). Nonparametric multiple comparisons. Behavioral Research, 52, 489–502.

Noguchi, K., Konietschke, F., Marmolejo-Ramos, F. Pauly, M. (2021). Permutation tests are robust and powerful at 0.5% and 5% significance levels. Behavioral Research, 53, 2712–2724.

Nuzzo, R. (2014). Scientific method: Statistical errors. Nature, 506, 150–152.

O’Dell, R.S. (2012). The Mean as Balance Point. Mathematics Teaching in the Middle School 18(3), 148–155.

Oohira, A., Okamoto, M., & Ozawa, T. (1981). 正常人の衝動性眼球運動最大速度について[Peak velocity of normal human saccadic eye movements (author's translation)]. 日限会誌 [Journal of the Japanese Society of Ophthalmology] 85(11), 2001–2007. or

Orquin, J. L., & Holmqvist, K. (2017). Threats to the validity of eye-movement research in psychology. Behavior Research Methods 50(4), 1645–1656.

Płomecka, M.B. Barańczuk-Turska, Z., Pfeiffer, C., & Langer, N. (2020). Aging effects and test–retest reliability of inhibitory control for saccadic eye movements. eNeuro 7(5)

Rokach, L., & Maimon, O. (2008). Data mining with decision trees: theory and applications. Toh Tuck Link, Singapore: World Scientific.

Russo, J. E. (2010). Eye fixations as a process trace. In M. Schulte-Mecklenbeck, A. Kühberger, and R. Ranyard (Eds.), Handbook of Process Tracing Methods for Decision Research (pp. 43–64). Psychology Press.

Scharfen, J., Jansen, K. & Holling, H. (2018). Retest effects in working memory capacity tests: A meta-analysis. Psychonomic Bulletin & Review 25, 2175–2199.

Schindler, M., & Lilienthal, A. J. (2019). Domain-specific interpretation of eye-tracking data: Towards a refined use of the eye-mind hypothesis for the field of geometry. Educational Studies in Mathematics 101, 123–139.

Setiawan, E. P., & Sukoco, H. (2021). Exploring first year university students’ statistical literacy: A case on describing and visualizing data. Journal on Mathematics Education 12(3), 427–448.

Strohmaier, A. R., MacKay, K. J., Obersteiner, A., & Reiss, K. M. (2020). Eye-tracking methodology in mathematics education research: A systematic literature review. Educational Studies in Mathematics 104, 147–200.

Tiefenbruck, B. F. (2007). Elementary teachers conceptions of graphical representations of categorical data. Doctoral dissertation. University of Minnesota, USA. .

Temkin, N. R., Heaton, R. K., Grant, I., & Dikmen, S. S. (1999). Detecting significant change in neuropsychological test performance: a comparison of four models. Journal of the International Neuropsychological Society 5(4), 357–369.

Tobii (n.d.). Tobii Studio. Users’ Manual. Version 3.4.5.

Van Gog, T., Paas, F., Van Merriënboer, J. J., & Witte, P. (2005). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied 11(4), 237.

Van Gog, T., & Jarodzka, H. (2013). Eye tracking as a tool to study and enhance cognitive and metacognitive processes in computer-based learning environments. In R. Azevedo, and V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 143–156). Springer.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.