Tracking Patterns in Self-Regulated Learning Using Students’ Self-Reports and Online Trace Data

Main Article Content

Nicolette van Halem
Chris van Klaveren
Hendrik Drachsler
Marcel Schmitz
Ilja Cornelisz


For decades, self-report instruments – which rely heavily on students’ perceptions and beliefs – have been the dominant way of measuring motivation and strategy use. An event-based measure based on online trace data arguably has the potential to remove analytical restrictions of self-report measures. The purpose of this study is therefore to triangulate constructs suggested in theory and measured using self-reported data with revealed online traces of learning behaviour. The results show that online trace data of learning behaviour are complementary to self-reports, as they explained a unique proportion of variance in student academic performance and reveal that self-reports explain more variance in online learning behaviour of prior weeks than variance in learning behaviour in succeeding weeks. Student motivation is, however, to a lesser extent captured with online trace data, likely because of its covert nature. In that respect, it is of importance to recognize the crucial role of self-reports in capturing student learning holistically. This manuscript is ‘frontline’ in the sense that event-based measurement methodologies using online trace data are relatively unexplored. The comparison with self-report data made in this manuscript sheds new light on the added value of innovative and traditional methods of measuring motivation and strategy use.

Article Details

How to Cite
van Halem, N., Klaveren, C. van, Drachsler, H., Schmitz, M., & Cornelisz, I. (2020). Tracking Patterns in Self-Regulated Learning Using Students’ Self-Reports and Online Trace Data. Frontline Learning Research, 8(3), 140 - 163.


Bera, S., & Liu, M. (2006). Cognitive tools, individual differences, and group processing as mediating factors in a hypermedia environment. Computers in Human Behavior, 22, 295-319.

Boekaerts, M., & Corno, L. (2005). Self‐regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54, 199-231.

Boekaerts, M., Pintrich, P. R., & Zeidner, M. (2000). Self-regulation: An introductory overview. In Handbook of self-regulation (pp. 1-9). Academic Press.

Charleer, S., Moere, A. V., Klerkx, J., Verbert, K., & De Laet, T. (2017). Learning analytics dashboards to support adviser-student dialogue. IEEE Transactions on Learning Technologies, 11, 389-399.

Cho, M. H., & Heron, M. L. (2015). Self-regulated learning: the role of motivation, emotion, and use of learning strategies in students’ learning experiences in a self-paced online mathematics course. Distance Education, 36, 80-99.

Cho, M. H., & Yoo, J. S. (2017). Exploring online students’ self-regulated learning with self-reported surveys and log files: a data mining approach. Interactive Learning Environments, 25, 970-982.

Coogan, J., Dancey, C. P., & Attree, E. A. (2005). WebCT: a useful support tool for psychology undergraduates–a Q methodological study. Psychology Learning and Teaching, 5, 61-66.

Cornelisz, I., & Van Klaveren, C. (2018). Student engagement with computerized practising: Ability, task value, and difficulty perceptions. Journal of Computer Assisted Learning, 34(6), 828-842.

DeNeui, D. L., & Dodge, T. L. (2006). Asynchronous learning networks and student outcomes: The utility of online learning components in hybrid courses. Journal of Instructional Psychology, 33, 256-260.

Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20, 391-409.

Greller, W. & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning Analytics. Journal of Educational Technology & Society, 15, 42-57. Retrieved from

Chauliac, M., Catrysse, L., Gijbels, D., & Donche, V. (2020). It is all in the surv-eye: Can eye tracking data shed light on the internal consistency in self-report questionnaires on cognitive processing strategies? Frontline Learning Research, 8(3), 26–39.

Duncan, T. G., & McKeachie, W. J. (2005). The making of the motivated strategies for learning questionnaire. Educational psychologist, 40, 117-128.

Efklides, A. (2011). Interactions of metacognition with motivation and affect in self-regulated learning: The MASRL model. Educational Psychologist, 46, 6-25.

Ellis, R. A., Marcus, G., & Taylor, R. (2005). Learning through inquiry: student difficulties with online course‐based Material. Journal of Computer Assisted Learning, 21, 239-252.

Entwistle, N., & McCune, V. (2004). The conceptual bases of study strategy inventories. Educational Psychology Review, 16, 325-345.

Fryer, L. K. (2017). Building bridges: Seeking structure and direction for higher education motivated learning strategy models. Educational Psychology Review, 29, 325-344.

Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68-84.

Gašević, D., Kovanovic, V., Joksimovic, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC Research Initiative. The International Review of Research in Open and Distributed Learning, 15, 134-176.

Guerra, J., Hosseini, R., Somyurek, S., & Brusilovsky, P. (2016). An intelligent interface for learning content: Combining an open learner model and social comparison to support self-regulated learning and engagement. In Proceedings of the 21st international conference on intelligent user interfaces (pp. 152-163). ACM.

Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2-3), 107-124.

Han, J., Pei, J., & Kamber, M. (2011). Data mining: concepts and techniques. Elsevier.

Heffner, M., & Cohen, S. H. (2005). Evaluating student use of web-based course material. Journal of Instructional Psychology, 32, 74-82.

Hilpert, J. C., Stempien, J., van der Hoeven Kraft, K. J., & Husman, J. (2013). Evidence for the latent factor structure of the MSLQ: A new conceptualization of an established questionnaire. SAGE Open, 3, 2158244013510305.

Hoskins, S. L., & Van Hooff, J. C. (2005). Motivation and ability: which students use online learning and what influence does it have on their achievement? British Journal of Educational Technology, 36, 177-192.

Howard-Rose, D., & Winne, P. H. (1993). Measuring component and sets of cognitive processes in self-regulated learning. Journal of Educational Psychology, 85, 591.

Huon, G., Spehar, B., Adam, P., & Rifkin, W. (2007). Resource use and academic performance among first year psychology students. Higher Education, 53, 1-27.

Iaconelli, R., & Wolters, C. A. (2020). Insufficient effort responding in surveys assessing self-regulated learning: Nuisance or fatal flaw? Frontline Learning Research, 8(3), 105–127.

Itmazi, J. A., & Megías, M. G. (2005). Survey: Comparison and evaluation studies of learning content management systems. Unpublished manuscript.

Jamieson-Noel, D., & Winne, P. H. (2003). Comparing Self-Reports to Traces of Studying Behavior as Representations of Students' Studying and Achievement. Zeitschrift für Pädagogische Psychologie/German Journal of Educational Psychology.

Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practice. In É. L., H. D., K. V., J. B., & M. P-S. (Eds.), Data Driven Approaches in Digital Education: 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia, September 12–15, 2017, Proceedings (Lecture Notes in Computer Science (LNCS); Vol. 10474). Cham: Springer International Publishing AG.

Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: preparing learning analytics dashboards for educational practice. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 31-40). ACM.

Jiang, L., Elen, J., & Clarebout, G. (2009). The relationships between learner variables, tool-usage behaviour and performance. Computers in Human Behavior, 25, 501-509.

Jupp, V. (2006). The Sage dictionary of social research methods. Sage.

Kim, D., Yoon, M., Jo, I. H., & Branch, R. M. (2018). Learning analytics to support self-regulated learning in asynchronous online courses: A case study at a women's university in South Korea. Computers & Education, 127, 233-251.

Knight, J. (2010). Distinguishing the learning approaches adopted by undergraduates in their use of online resources. Active Learning in Higher Education, 11, 67–76.

Lust, G., Collazo, N. A. J., Elen, J., & Clarebout, G. (2012). Content management systems: enriched learning opportunities for all? Computers in Human Behavior, 28, 795-808.

Lust, G., Elen, J., & Clarebout, G. (2013). Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education, 60, 385-395.

McKeachie, W. J., Pintrich, P. R., & Lin, Y. G. (1985). Teaching learning strategies. Educational Psychologist, 20, 153-160.

Molenda, M. (2008). Historical foundations. In M. J. Spector, M. D. Merrill, J. van Merrienboer, & M. P. Driscoll (Eds.). Handbook of research for educational communications and technology (pp. 5–20). Routledge.

Moeller, J., Dietrich, J., Viljaranta, J., & Kracke, B. (2020). Disentangling objective characteristics of learning situations from subjective perceptions thereof, using an experience sampling method design. Frontline Learning Research, 8(3), 63–85.

Muis, K. R., Winne, P. H., & Jamieson-Noel, D. (2007). Using a multitrait-multimethod analysis to examine conceptual similarities of three self-regulated learning inventories. British Journal of Educational Psychology, 77, 177–195.

Munoz, KD, & Van Duzer, J. (2005). BlackBoard vs. Moodle: A Comparison of Satisfaction with Online Teaching and Learning Tools. Humboldt State University.

Panadero, E. (2017). A review of self-regulated learning: six models and four directions for research. Frontiers in Psychology, 8, 422.

Panadero, E., Klug, J., & Järvelä, S. (2016). Third wave of measurement in the self-regulated learning field: when measurement and intervention come hand in hand. Scandinavian Journal of Educational Research, 60, 723-735.

Pintrich, P. R. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ).

Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92, 544.

Pintrich, P. R., McKeachie, W. J., & Lin, Y. G. (1987). Teaching a course in learning to learn. Teaching of Psychology, 14, 81-86.

Rogiers, A.; Merchie, E., & van Keer, H. (2020). Opening the black box of students’ text-learning processes: A process mining perspective. Frontline Learning Research, 8(3), 40–62.

Schmitz, M., Scheffel, M., van Limbeek, E., van Halem, N., Cornelisz, I., van Klaveren, C., ... & Drachsler, H. (2018). Investigating the Relationships Between Online Activity, Learning Strategies and Grades to Create Learning Analytics-Supported Learning Designs. In European Conference on Technology Enhanced Learning (pp. 311-325). Springer, Cham.

Taylor, S. E. & Brown, J. D. (1988). Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin, 103, 193–210.

Tock, J. L., & Moxley, J. H. (2017). A comprehensive reanalysis of the metacognitive self-regulation scale from the MSLQ. Metacognition and Learning, 12, 79-111.

Veenman, M. (2011). Learning to self-monitor and self-regulate. In R. Mayer & P. Alexander (Eds.), Handbook of research on learning and instruction (pp. 197–218). New York: Routledge.

Vriesema, C. C., & McCaslin, M. (2020) Experience and meaning in small-group contexts: Fusing observational and self-report data to capture self and other dynamics. Frontline Learning Research, 8(3), 128–141.

Wang, A. Y., & Newlin, M. H. (2000). Characteristics of students who enroll and succeed in psychology web-based classes. Journal of Educational Psychology, 92, 137.

Winne, P. H., & Baker, R. S. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5, 1-8. Retrieved from

Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation. Academic Press.

Zhou, M., & Winne, P. H. (2012). Modelling academic performance by self-reported versus traced goal orientation. Learning and Instruction, 22, 413-419.

Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45, 166–183.