Opening the black box of students’ text-learning processes: A process mining perspective

Main Article Content

Amélie Rogiers
Emmelien Merchie
Hilde Van Keer

Abstract

The current study uncovers secondary school students’ actual use of text-learning strategies during an individual learning task by means of a concurrent self-reported thinking aloud procedure. Think-aloud data of 51 participants with different learner profiles, distinguished based on a retrospective self-report questionnaire (i.e., 15 integrated strategy users, 15 information organizers, 10 mental learners, and 11 limited strategy users), were analysed by means of educational process mining. Both the frequency of students’ strategy use, as well as the temporal patterns between these strategies were studied. The process mining results clearly demonstrated differences between students with different learner profiles with respect to the frequency of their applied strategies, as well as concerning the sequences wherein strategies were applied throughout the course of students’ text-learning process. The added value of combining both retrospective and concurrent self-report measures of students’ strategies as well as conducting process mining analysis is discussed.

Article Details

How to Cite
Rogiers, A., Merchie, E., & Van Keer, H. (2020). Opening the black box of students’ text-learning processes: A process mining perspective. Frontline Learning Research, 8(3), 40-62. https://doi.org/10.14786/flr.v8i3.527
Section
Articles

References

Alexander, P. A. (1998). The nature of disciplinary and domain learning: The knowledge, interest, and strategic dimensions of learning from subject matter text. In C. R. Hynd (Ed.), Learning from texts across conceptual domains (pp. 263-287). New York, NY: Routledge.

Alexander, P. A., & Jetton, T. L. (2000). Learning from text: A multidimensional and developmental perspective. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. III, pp. 285-310). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Alexander, P. A., Grossnickle, E. M., Dumas, D., & Hattan, C. (2018). A retrospective and prospective examination of cognitive strategies and academic development: Where have we come in twenty-five years? In A. O’Donnell (Ed.), Oxford Handbook of Educational Psychology. Oxford, UK: Oxford University Press.

Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning, 9(2), 161–185. https://doi.org/10.1007/s11409-013-9107-6

Bergman, L. (2001). A person approach to adolescence: Some methodological challenges. Journal of Adolescent Research, 16(1), 28–53. https://doi.org/10.1177/0743558401161004

Boekaerts, M., & Corno, L. (2005). Self regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54(2), 199–231. https://doi.org/10.1111/j.1464-0597.2005.00205.x

Boekaerts, M., & Niemivirta, M. (2000). Self-regulated learning: Finding a balance between learning goals and ego-protective goals. In M. Boekaerts, P. R. Pintrich & M. Zeidner (Eds.), Handbook of self-regulation. San Diego, CA: Academic Press.

Bråten, I., & Samuelstuen, M. S. (2004). Does the influence of reading purpose on reports of strategic text processing depend on students’ topic knowledge? Journal of Educational Psychology, 96(2), 324-336. https://doi.org/10.1037/0022-0663.96.2.324

Båten, I., & Samuelstuen, M. S. (2007). Measuring strategic processing: comparing task-specific self-reports to traces. Metacognition and Learning, 2(1), 1-20. https://doi.org/10.1007/s11409-007-9004-y

Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.

Cromley, J. G., & Wills, T. W. (2016). Flexible strategy use by students who learn much versus little from text: transitions within think-aloud protocols. Journal of Research in Reading, 39(1), 50-71. https://doi.org/10.1111/1467-9817.12026

Deed, C., Lesko, T. M., & Lovejoy, V. (2014). Teacher adaptation to personalized learning spaces. Teacher Development, 18(3), 369–383. doi:10.1080/13664530.2014.919345

Dinsmore, D. L. (2018). Strategic Processing in Education. New York, NY: Routledge.

Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research, 69(2), 145-186. https://doi.org/10.3102/00346543069002145

Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215- 251. https://doi.org/10.1037//0033-295x.87.3.215

Fluxicon. (2019). Disco user’s guide. Retrieved from https://fluxicon.com/disco/files/Disco-UserGuide.pdf

Fox, E. (2009). The role of reader characteristics in processing and learning from informational text. Review of Educational Research, 79(1), 197-261. https://doi.org/10.3102/0034654308324654

Fryer, L. K., & Vermunt, J. D. (2017). Regulating approaches to learning: Testing learning strategy convergences across a year at university. British Journal of Educational Psychology, 88(1), 21-41. https://doi.org/10.1111/bjep.12169

Günther, C., & Van der Aalst, W. (2007). Fuzzy mining: Adaptive process simplification based on multi-perspective metrics. In G. Alonso, P. Dadam, & M. Rosemann (Eds.), International conference on business process management (BPM 2007) (pp. 328–343). Berlin, Germany: Springer.

Greene, J. A., Robertson, J., & Croker Costa, L.-J. (2011). Assessing self-regulated learning using think-aloud methods. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 313–328). New York, NY: Routledge.

McNamara, D. S., Ozuru, Y., Best, R., & O'Reilly, T. (2007). The 4-pronged comprehension strategy framework. In D. S. McNamara (Ed.), Reading Comprehension Strategies: Theories, Interventions, and Technologies (pp. 465-496): New York, NY: Erlbaum

Merchie, E., & Van Keer, H. (2014). Learning from Text in Late Elementary Education. Comparing Think-aloud Protocols with Self-reports. Procedia - Social and Behavioral Sciences, 112(2013), 489–496. https://doi.org/10.1016/j.sbspro.2014.01.1193

Merchie, E., Van Keer, H., & Vandevelde, S. (2014). Development of the Text-Learning Strategies Inventory: Assessing and profiling learning from texts in fifth and sixth grade. Journal of Psychoeducational Assessment, 32(6), 1-15. https://doi.org/10.1177/0734282914525155

Organisation for Economic Cooperation, Development [OECD]. (2006). Personalizing Education. Paris: OECD Publishing.

Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology, 8(422), 1–28. https://doi.org/10.3389/fpsyg.2017.00422

Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385–407. https://doi.org/10.1007/s10648-004-0006-x

Pressley, M., Borkowski, J. G., & Schneider, W. (1987). Cognitive strategies: Good strategy users coordinate metacognition and knowledge. Annals of Child Development, 4, 89-129.

Reimann, P. (2007). Time is precious: Why process analysis is essential for CSCL (and can also help to bridge between experimental and descriptive methods). In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Mice, minds, and society. Proceedings of the computer-supported collaborative learning conference (CSCL 2007) (pp. 598–607). New Brunswick, NJ: International Society of the Learning Sciences.

Rogiers, A., Merchie, E., & Van Keer, H. (2018). Fostering text-learning strategies in secondary education through explicit strategy-instruction. Paper presented at the International Conference of the EARLI SIG 2, Freiburg, Germany, 27-29 august, 2018.

Rogiers, A., Merchie, E., & Van Keer, H. (2019a). Learner profiles in secondary education: Occurrence and relationship with performance and student characteristics. The Journal of Educational Research, 112(3), 385-396. https://doi.org/10.1080/00220671.2018.1538093

Rogiers, A., Merchie, E., & Van Keer, H. (2019b). What they say is what they do? Comparing task-specific self-reports, think-aloud protocols, and study traces for measuring secondary school students’ text-learning strategies. European Journal of Psychology of Education, 1-18. https://doi.org/10.1007/s10212-019-00429-5

Samuelstuen, M. S., & Bråten, I. (2007). Examining the validity of self-reports on scales measuring students' strategic processing. British Journal of Educational Psychology, 77(2), 351-378. https://doi.org/10.1348/000709906X106147

Schellings, G. L. M., & Broekkamp, H. (2011). Signaling task awareness in think-aloud protocols from students selecting relevant information from text. Metacognition and Learning, 6, 65–82. https://doi.org/10.1007/s11409-010-9067-z

Slotte, V., Lonka, K., & Lindblom-Ylanne, S. (2001). Study-strategy use in learning from text. Does gender make any difference? Instructional Science, 29(3), 255-272. https://doi.org/10.1023/a:1017574300304

Schoor, C., & Bannert, M. (2012). Exploring regulatory processes during a computer-supported collaborative learning task using process mining. Computers in Human Behavior, 28(4), 1321–1331. https://doi.org/10.1016/j.chb.2012.02.016

Scott, D. B. (2008). Assessing text processing: A comparison of four methods. Journal of Literacy Research, 40, 290-316. doi:10.1080/10862960802502162

Simpson, M. L., & Nist, S. L. (2000). An update on strategic learning: It's more than textbook reading strategies. Journal of Adolescent & Adult Literacy, 43(6), 528-541.

Stoeger, H., Fleischmann, S., Obergriesser, S. (2015). Self-regulated learning (SRL) and the gifted learner in primary school: The theoretical basis and empirical findings on a research program dedicated to ensuring that all students learn to regulate their own learning. Asia Pacific Education Review, 16, 257-267. https://doi.org/ 10.1007/s12564-015-9376-7

Veenman, M. V. J. (2005). The assessment of metacognitive skills: What can be learned from multi-method designs? In C. Artett & B. Moschner (Eds.), Ledrnstrategien und metakognition. Implikationen für forschung und praxis (pp. 77-99). Münster: Waxmann.

Veenman, M. V. J. (2011). Alternative assessment of strategy use with self-report instruments: A discussion. Metacognition and Learning, 6(2), 205–211. https://doi.org/10.1007/s11409-011-9080-x

Veenman, M. V., Elshout, J. J., & Meijer, J. (1997). The generality vs. domain-specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7(2), 187–209. https://doi.org/10.1016/S0959-4752(96)00025-4

Wade, S. E., Trathen, W., & Schraw, G. (1990). An analysis of spontaneous study strategies. Reading Research Quarterly, 25(2), 147-166. https://doi.org/10.2307/747599

Weinstein, C. E., Jung, J., & Acee, T.W. (2011). Learning strategies In V. G. Aukrust (Ed.), Learning and Cognition in Education (pp. 137-143). Oxford, UK: Elsevier Limited.

Van der Aalst. (2011). Process mining: Discovery, conformance and enhancement of business processes. New York, NY: Springer.

van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method. A practical guide to modelling cognitive processes. London, UK: Academic Press.

Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64–71.

Young, K. A. (2005). Direct from the source: the value of ‘think-aloud’ data in understanding learning. Journal of Educational Enquiry, 6(1), 19-33.