Assessing Students’ Interpretations of Histograms Before and After Interpreting Dotplots: A Gaze-Based Machine Learning Analysis
Main Article Content
Abstract
Many students persistently misinterpret histograms. Literature suggests that having students solve dotplot items may prepare for interpreting histograms, as interpreting dotplots can help students realize that the statistical variable is presented on the horizontal axis. In this study, we explore a special case of this suggestion, namely, how students’ histogram interpretations alter during an assessment. The research question is: In what way do secondary school students’ histogram interpretations change after solving dotplot items? Two histogram items were solved before solving dotplot items and two after. Students were asked to estimate or compare arithmetic means. Students’ gaze data, answers, and cued retrospective verbal reports were collected. We used students’ gaze data on four histogram items as inputs for a machine learning algorithm (MLA; random forest). Results show that the MLA can quite accurately classify whether students’ gaze data belonged to an item solved before or after the dotplot items. Moreover, the direction (e.g., almost vertical) and length of students’ saccades were different on the before and after items. These changes can indicate a change in strategies. A plausible explanation is that solving dotplot items creates readiness for learning and that reflecting on the solution strategy during recall then brings new insights. This study has implications for assessments and homework. Novel in the study is its use of spatial gaze data and its use of an MLA for finding differences in gazes that are relevant for changes in students’ task-specific strategies.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Bakker, A. (2004a). Design research in statistics education: On symbolizing and computer tools. Doctoral thesis. Utrecht, the Netherlands: Utrecht University. https://dspace.library.uu.nl/handle/1874/893
Bakker, A. (2004b). Reasoning about shape as a pattern in variability. Statistics Education Research Journal, 3(2), 64–83. http://iase-web.org/documents/SERJ/SERJ3(2)_Bakker.pdf?1402525004
Bakker, A., Cai, J., English, L. Kaiser, G., Mesa, V., & Van Dooren, W. (2019). Beyond small, medium, or large: points of consideration when interpreting effect sizes. Educational Studies in Mathematics 102, 1–8. https://doi.org/10.1007/s10649-019-09908-4
Boels, L., Bakker, A., Van Dooren, W., & Drijvers, P. (2019). Conceptual difficulties when interpreting histograms: A review. Educational Research Review 28, art. 100291, https://doi.org/10.1016/j.edurev.2019.100291
Bennett, R. E. (2011) Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice 18(1), 5–25, https://doi.org/10.1080/0969594X.2010.513678
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth and Brooks/Cole Advanced Books and Software.
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017). Classification and regression trees. Routledge.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5–32.
Capraro, M. M., Kulm, G., & Capraro, R. M. (2005). Middle grades: Misconceptions in statistical thinking. School Science & Mathematics 105(4), 165–174. https://doi.org/10.1111/j.1949-8594.2005.tb18156.x
Catron, D. W. (1978). Immediate test-retest changes in WAIS scores among college males. Psychological Reports 43(1), 279–290. https://doi.org/10.2466/pr0.1978.43.1.279
Church, R.B., & Goldin-Meadow, S. (1986). The mismatch between gesture and speech as an index of transitional knowledge. Cognition 23(1), 43–71. https://doi.org/10.1016/0010-0277(86)90053-3
Clayden, A., & Croft, M. (1990). Statistical consultation—who's the expert? Annals of Mathematics and Artificial Intelligence 2, 65–75. https://doi.org/10.1007/BF01530997
Cohen, S. (1996). Identifying impediments to learning probability and statistics from an assessment of instructional software. Journal of Educational and Behavioral Statistics 21(1), 35–54. https://doi.org/10.2307/1165254
Cooper, L. L. (2018). Assessing students' understanding of variability in graphical representations that share the common attribute of bars. Journal of Statistics Education 26(2), 110–124. https://doi.org/10.1080/10691898.2018.1473060
Cooper, L. L., & Shore, F. S. (2008). Students' misconceptions in interpreting center and variability of data represented via histograms and stem-and-leaf plots. Journal of Statistics Education 16(2), https://doi.org/10.1080/10691898.2008.11889559
Dabos, M. (2014). A glimpse of two year college instructors' understanding of variation in histograms. In K. Makar, B. de Sousa, and R. Gould (Eds.). Sustainability in statistics education. Proceedings of the 9th international conference on teaching statistics (pp. 1–4). Flagstaff, Arizona, USA. https://icots.info/9/proceedings/pdfs/ICOTS9_C150_DABOS.pdf
delMas, R., Garfield, J., & Ooms, A. (2005). Using assessment items to study students' difficulty reading and interpreting graphical representations of distributions. Proceedings of the fourth international research forum on statistical reasoning, literacy, and reasoning. Auckland, New Zealand: University of Auckland. http://apps3.cehd.umn.edu/artist/articles/SRTL4_ARTIST.pdf
delMas, R., & Liu, Y. (2005). Exploring students' conceptions of the standard deviation. Statistics Education Research Journal 4(1), 55–82. http://iaseweb.org/documents/SERJ/SERJ4(1)_delMas_Liu.pdf
Dzeroski, S., & Zenko, B. (2004). Is combining classifiers with stacking better than selecting the best one? Machine Learning 54(3), 255–273. https://doi.org/10.1023/B:MACH.0000015881.36452.6e
Falleti, M. G., Maruff, P., Collie, A. & Darby, D. G. (2006). Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals, Journal of Clinical and Experimental Neuropsychology 28(7), 1095–1112. https://doi.org/10.1080/13803390500205718
Fisher, R. A. (1922). On the interpretation of χ2 from contingency tables, and the calculation of P. Journal of the Royal Statistical Society 85(1), 87-94. https://www.jstor.org/stable/2340521
Friedman, J., Hastie, T., & Tibshirani, R. (2001). The elements of statistical learning (Vol. 1, No. 10). Springer series in statistics. https://link.springer.com/book/10.1007/978-0-387-21606-5
Fuchs, A. F. (1967). Saccadic and smooth pursuit eye movements in the monkey. The Journal of Physiology 191(3), 609–631. https://doi.org/10.1113/jphysiol.1967.sp008271
Gal, I. (1995). Statistical tools and statistical literacy: The case of the average. Teaching Statistics 17(3), 97–99. https://doi.org/10.1111/j.1467-9639.1995.tb00720.x
Gal, I. (2002). Adults' statistical literacy: Meanings, components, responsibilities. International statistical review 70(1), 1–25. https://iase-web.org/documents/intstatreview/02.Gal.pdf
Garcia Moreno-Esteva, E., Kervinen, A., Hannula, M. S., & Uitto, A. (2020). Scanning signatures: A graph theoretical model to represent visual scanning processes and a proof of concept study in biology Education. Education Sciences 10(5) https://doi.org/10.3390/educsci10050141
Garcia Moreno-Esteva, E., White, S. L. J., Wood, J. M., & Black, A. A. (2018). Application of mathematical and machine learning techniques to analyse eye tracking data enabling better understanding of children’s visual cognitive behaviours. Frontline Learning Research 6(3), 72–84. https://doi.org/rg/10.14786/flr.v6i3.365
Garfield, J. (2002). Histogram sorting. Statistics teaching and resource library (STAR). https://amser.org/index.php?P=AMSER--ResourceFrame&resourceId=8554
Garfield, J. B., Ben-Zvi, D., Chance, B., Medina, E., Roseth, C., & Zieffler, A. (2008). Learning to reason about distribution. In J. Garfield & D. Ben-Zvi (Eds). Developing students’ statistical reasoning (pp. 165–186). Springer. https://link.springer.com/content/pdf/10.1007/978-1-4020-8383-9_8.pdf
Godau, C. Haider, H., Hansen, S., Schubert, T., Frensch, P. A., Gaschler, R. (2014). Spontaneously spotting and applying shortcuts in arithmetic—a primary school perspective on expertise. Frontiers in Psychology – Cognition 5, art. e556, 1664–1078. https://doi.org/10.3389/fpsyg.2014.00556
Goldberg, J. H., & Helfman, J. I. (2010). Comparing information graphics: A critical look at eye tracking. Proceedings of the 3rd BELIV’10 Workshop: Beyond time and errors: novel EvaLuation methods for Information Visualization, Atlanta, Georgia. 71–78. https://doi.org/10.1145/2110192.2110203
Guan, Z., Lee, S., Cuddihy, E., & Ramey, J. (2006). The validity of the stimulated retrospective think-aloud method as measured by eye tracking. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1253–1262. Montréal, Québec, Canada. https://doi.org/10.1145/1124772.1124961
Guerra-Carrillo, B. C., Bunge, S.A. (2018). Eye gaze patterns reveal how reasoning skills improve with experience. npj Science of Learning 3, art. 18. https://doi.org/10.1038/s41539-018-0035-8
Hansen, L. K., & Salamon, P. (1990). Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence 12, 993–1001. https://doi.org/10.1109/34.58871
Heilbronner, R. L., Sweet, J. J., Attix, D. K., Krull, K. R., Henry, G. K., & Hart, R. P. (2010). Official position of the american academy of clinical neuropsychology on serial neuropsychological assessments: the utility and challenges of repeat test administrations in clinical and forensic contexts, The Clinical Neuropsychologist, 24(8), 1267–1278. https://doi.org/10.1080/13854046.2010.526785
Hessels, R. S., Niehorster, D. C., Nyström, M., Andersson, R., & Hooge, I. T. C. (2018). Is the eye-movement field confused about fixations and saccades? A survey among 124 researchers. Royal Society Open Science 5(8), 1–23. https://doi.org/10.1098/rsos.180502
Hinton-Bayre, A. D. (2010). Deriving Reliable Change Statistics from Test–Retest Normative Data: Comparison of Models and Mathematical Expressions. Archives of Clinical Neuropsychology 25(3), 244–256. https://doi.org/10.1093/arclin/acq008
Hohn, R. W. (1992). An analysis of the components of curriculum-based assessment. Doctoral thesis. Denver, CO, USA: University of Denver. https://www.proquest.com/openview/abb4ea5179410900d6e2af9a7473f0e9/1?pq-origsite=gscholar&cbl=18750&diss=y
Holmqvist, K., Örbom, S. L., Hooge, I. T. C., Niehorster, D. C., Alexander, R. G., Andersson, R., Benjamins, J. S., Blignaut, P., Brouwer, A-M., Chuang, L. L., Dalrymple, K. A., Drieghe, D., Dunn, M. J., Ettinger, U., Fiedler, S., Foulsham, T., Van der Geest, J. N., Witzner Hansen, D., Hutton, S., … Hessels, R. S. (2022). Eye tracking: empirical foundations for a minimal reporting guideline. Behavior Research Methods. https://doi.org/10.3758/s13428-021-01762-8
Hyönä, J. (2010). The use of eye movements in the study of multimedia learning. Learning and Instruction, 20(2), 172–176. https://doi.org/10.1016/j.learninstruc.2009.02.013
Jarodzka, H., Van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction 25, 62–70. https://doi.org/10.1016/j.learninstruc.2012.11.004
Kaakinen, J.K. (2021). What can eye movements tell us about visual perception processes in classroom contexts? Commentary on a special issue. Educational Psychology Review 33, 169–179. https://doi.org/10.1007/s10648-020-09573-7
Kaplan, J., Gabrosek, J., Curtiss, P., & Malone, C. (2014). Investigating student understanding of histograms. Journal of Statistics Education 22(2), 1–29. http://jse.amstat.org/v22n2/kaplan.pdf
Khalil, K. A. I. (2005). Expert-novice differences: Visual and verbal responses in a two-group comparison task (Master thesis.) Amherst, MA, USA: University of Massachusetts. https://scholarworks.umass.edu/theses/2428
Konold, C., Higgins, T., Russell, S. J., & Khalil, K. (2015). Data seen through different lenses. Educational Studies in Mathematics 88(3), 305–325. https://doi.org/10.1007/s10649-013-9529-8
Konold, C., & Pollatsek, A. (2002). Data analysis as the search for signals in noisy processes. Journal for Research in Mathematics Education 33(4), 259–289. https://www.jstor.org/stable/749741
Kragten, M., Admiraal, W., & Rijlaarsdam, G. (2015). Students’ learning activities while studying biological process diagrams. International Journal of Science Education 37(12), 1915–1937. https://doi.org/10.1080/09500693.2015.1057775
Lai, M., Tsai, M., Yang, F., Hsu, C., Liu, T., Lee, S. W., Lee, M., Chiou, G., Liang, J., & Tsai, C. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review 10, 90–115. https://doi.org/10.1016/j.edurev.2013.10.001
Lem, S., Onghena, P., Verschaffel, L., & Van Dooren, W. (2013). On the misinterpretation of histograms and box plots. Educational Psychology 33(2), 155–174. https://doi.org/10.1080/01443410.2012.674006
Lievens, F., Reeve, C. L., & Heggestad, E. D. (2007). An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. Journal of Applied Psychology 92(6), 1672–1682. https://ink.library.smu.edu.sg/lkcsb_research/5693
Lumsden, J. (1976). Test theory. Annual Review of Psychology 27(1), 251 – 280. https://doi.org/10.1146/annurev.ps.27.020176.001343
Lyford, A. J. (2017). Investigating undergraduate student understanding of graphical displays of quantitative data through machine learning algorithms. Doctoral thesis. Athens, GA, USA: University of Georgia. https://iase-web.org/documents/dissertations/17.AlexanderLyford.Dissertation.pdf
McGatha, M., Cobb, P., & McClain, K. (2002). An analysis of students' initial statistical understandings: Developing a conjectured learning trajectory. The Journal of Mathematical Behavior 21(3), 339–355. https://doi.org/10.1016/S0732-3123(02)00133-5
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157. https://doi.org/10.1007/BF02295996
Meletiou-Mavrotheris, M. (2000). Students' understanding of variation: An untapped well in statistical reasoning. Doctoral thesis. Austin, TX, USA: University of Texas. http://iase-web.org/documents/dissertations/00.Meletiou.Dissertation.pdf
Mokros, J., & Russell, S. J. (1995). Children's concepts of average and representativeness. Journal for Research in Mathematics Education 26(1), 20–39. https://doi.org/10.2307/749226
Noguchi, K., Abel, R.S., Marmolejo-Ramos, F. & Konietschke, F. (2020). Nonparametric multiple comparisons. Behavioral Research, 52, 489–502. https://doi.org/10.3758/s13428-019-01247-9
Noguchi, K., Konietschke, F., Marmolejo-Ramos, F. Pauly, M. (2021). Permutation tests are robust and powerful at 0.5% and 5% significance levels. Behavioral Research, 53, 2712–2724. https://doi.org/10.3758/s13428-021-01595-5
Nuzzo, R. (2014). Scientific method: Statistical errors. Nature, 506, 150–152. http://www.nature.com/news/scientific-methodstatistical-errors-1.14700
O’Dell, R.S. (2012). The Mean as Balance Point. Mathematics Teaching in the Middle School 18(3), 148–155.
Oohira, A., Okamoto, M., & Ozawa, T. (1981). 正常人の衝動性眼球運動最大速度について[Peak velocity of normal human saccadic eye movements (author's translation)]. 日限会誌 [Journal of the Japanese Society of Ophthalmology] 85(11), 2001–2007. https://pubmed.ncbi.nlm.nih.gov/7337121/ or https://www.researchgate.net/publication/15862222_Peak_velocity_of_normal_human_saccadic_eye_movements_author%27s_transl
Orquin, J. L., & Holmqvist, K. (2017). Threats to the validity of eye-movement research in psychology. Behavior Research Methods 50(4), 1645–1656. https://doi.org/10.3758/s13428-017-0998-z
Płomecka, M.B. Barańczuk-Turska, Z., Pfeiffer, C., & Langer, N. (2020). Aging effects and test–retest reliability of inhibitory control for saccadic eye movements. eNeuro 7(5) https://doi.org/10.1523/ENEURO.0459-19.2020
Rokach, L., & Maimon, O. (2008). Data mining with decision trees: theory and applications. Toh Tuck Link, Singapore: World Scientific. https://doi.org/10.1142/9097
Russo, J. E. (2010). Eye fixations as a process trace. In M. Schulte-Mecklenbeck, A. Kühberger, and R. Ranyard (Eds.), Handbook of Process Tracing Methods for Decision Research (pp. 43–64). Psychology Press.
Scharfen, J., Jansen, K. & Holling, H. (2018). Retest effects in working memory capacity tests: A meta-analysis. Psychonomic Bulletin & Review 25, 2175–2199. https://doi.org/10.3758/s13423-018-1461-6
Schindler, M., & Lilienthal, A. J. (2019). Domain-specific interpretation of eye-tracking data: Towards a refined use of the eye-mind hypothesis for the field of geometry. Educational Studies in Mathematics 101, 123–139. https://doi.org/10.1007/s10649-019-9878-z
Setiawan, E. P., & Sukoco, H. (2021). Exploring first year university students’ statistical literacy: A case on describing and visualizing data. Journal on Mathematics Education 12(3), 427–448. https://ejournal.unsri.ac.id/index.php/jme/article/view/13202
Strohmaier, A. R., MacKay, K. J., Obersteiner, A., & Reiss, K. M. (2020). Eye-tracking methodology in mathematics education research: A systematic literature review. Educational Studies in Mathematics 104, 147–200. https://doi.org/10.1007/s10649-020-09948-1
Tiefenbruck, B. F. (2007). Elementary teachers conceptions of graphical representations of categorical data. Doctoral dissertation. University of Minnesota, USA. . https://conservancy.umn.edu/handle/11299/91699
Temkin, N. R., Heaton, R. K., Grant, I., & Dikmen, S. S. (1999). Detecting significant change in neuropsychological test performance: a comparison of four models. Journal of the International Neuropsychological Society 5(4), 357–369. https://doi.org/10.1017/s1355617799544068
Tobii (n.d.). Tobii Studio. Users’ Manual. Version 3.4.5. https://www.tobiipro.com/siteassets/tobii-pro/user-manuals/tobii-pro-studio-user-manual.pdf/?v=3.4.5
Van Gog, T., Paas, F., Van Merriënboer, J. J., & Witte, P. (2005). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied 11(4), 237. https://doi.org/10.1037/1076-898X.11.4.237
Van Gog, T., & Jarodzka, H. (2013). Eye tracking as a tool to study and enhance cognitive and metacognitive processes in computer-based learning environments. In R. Azevedo, and V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 143–156). Springer. https://doi.org/10.1007/978-1-4419-5546-3_10
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. https://www.hup.harvard.edu/catalog.php?isbn=9780674576292