Opening the black box of students’ text-learning processes: A process mining perspective
Main Article Content
Abstract
The current study uncovers secondary school students’ actual use of text-learning strategies during an individual learning task by means of a concurrent self-reported thinking aloud procedure. Think-aloud data of 51 participants with different learner profiles, distinguished based on a retrospective self-report questionnaire (i.e., 15 integrated strategy users, 15 information organizers, 10 mental learners, and 11 limited strategy users), were analysed by means of educational process mining. Both the frequency of students’ strategy use, as well as the temporal patterns between these strategies were studied. The process mining results clearly demonstrated differences between students with different learner profiles with respect to the frequency of their applied strategies, as well as concerning the sequences wherein strategies were applied throughout the course of students’ text-learning process. The added value of combining both retrospective and concurrent self-report measures of students’ strategies as well as conducting process mining analysis is discussed.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Alexander, P. A. (1998). The nature of disciplinary and domain learning: The knowledge, interest, and strategic dimensions of learning from subject matter text. In C. R. Hynd (Ed.), Learning from texts across conceptual domains (pp. 263-287). New York, NY: Routledge.
Alexander, P. A., & Jetton, T. L. (2000). Learning from text: A multidimensional and developmental perspective. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. III, pp. 285-310). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Alexander, P. A., Grossnickle, E. M., Dumas, D., & Hattan, C. (2018). A retrospective and prospective examination of cognitive strategies and academic development: Where have we come in twenty-five years? In A. O’Donnell (Ed.), Oxford Handbook of Educational Psychology. Oxford, UK: Oxford University Press.
Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning, 9(2), 161–185. https://doi.org/10.1007/s11409-013-9107-6
Bergman, L. (2001). A person approach to adolescence: Some methodological challenges. Journal of Adolescent Research, 16(1), 28–53. https://doi.org/10.1177/0743558401161004
Boekaerts, M., & Corno, L. (2005). Self regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54(2), 199–231. https://doi.org/10.1111/j.1464-0597.2005.00205.x
Boekaerts, M., & Niemivirta, M. (2000). Self-regulated learning: Finding a balance between learning goals and ego-protective goals. In M. Boekaerts, P. R. Pintrich & M. Zeidner (Eds.), Handbook of self-regulation. San Diego, CA: Academic Press.
Bråten, I., & Samuelstuen, M. S. (2004). Does the influence of reading purpose on reports of strategic text processing depend on students’ topic knowledge? Journal of Educational Psychology, 96(2), 324-336. https://doi.org/10.1037/0022-0663.96.2.324
Båten, I., & Samuelstuen, M. S. (2007). Measuring strategic processing: comparing task-specific self-reports to traces. Metacognition and Learning, 2(1), 1-20. https://doi.org/10.1007/s11409-007-9004-y
Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.
Cromley, J. G., & Wills, T. W. (2016). Flexible strategy use by students who learn much versus little from text: transitions within think-aloud protocols. Journal of Research in Reading, 39(1), 50-71. https://doi.org/10.1111/1467-9817.12026
Deed, C., Lesko, T. M., & Lovejoy, V. (2014). Teacher adaptation to personalized learning spaces. Teacher Development, 18(3), 369–383. doi:10.1080/13664530.2014.919345
Dinsmore, D. L. (2018). Strategic Processing in Education. New York, NY: Routledge.
Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research, 69(2), 145-186. https://doi.org/10.3102/00346543069002145
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215- 251. https://doi.org/10.1037//0033-295x.87.3.215
Fluxicon. (2019). Disco user’s guide. Retrieved from https://fluxicon.com/disco/files/Disco-UserGuide.pdf
Fox, E. (2009). The role of reader characteristics in processing and learning from informational text. Review of Educational Research, 79(1), 197-261. https://doi.org/10.3102/0034654308324654
Fryer, L. K., & Vermunt, J. D. (2017). Regulating approaches to learning: Testing learning strategy convergences across a year at university. British Journal of Educational Psychology, 88(1), 21-41. https://doi.org/10.1111/bjep.12169
Günther, C., & Van der Aalst, W. (2007). Fuzzy mining: Adaptive process simplification based on multi-perspective metrics. In G. Alonso, P. Dadam, & M. Rosemann (Eds.), International conference on business process management (BPM 2007) (pp. 328–343). Berlin, Germany: Springer.
Greene, J. A., Robertson, J., & Croker Costa, L.-J. (2011). Assessing self-regulated learning using think-aloud methods. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 313–328). New York, NY: Routledge.
McNamara, D. S., Ozuru, Y., Best, R., & O'Reilly, T. (2007). The 4-pronged comprehension strategy framework. In D. S. McNamara (Ed.), Reading Comprehension Strategies: Theories, Interventions, and Technologies (pp. 465-496): New York, NY: Erlbaum
Merchie, E., & Van Keer, H. (2014). Learning from Text in Late Elementary Education. Comparing Think-aloud Protocols with Self-reports. Procedia - Social and Behavioral Sciences, 112(2013), 489–496. https://doi.org/10.1016/j.sbspro.2014.01.1193
Merchie, E., Van Keer, H., & Vandevelde, S. (2014). Development of the Text-Learning Strategies Inventory: Assessing and profiling learning from texts in fifth and sixth grade. Journal of Psychoeducational Assessment, 32(6), 1-15. https://doi.org/10.1177/0734282914525155
Organisation for Economic Cooperation, Development [OECD]. (2006). Personalizing Education. Paris: OECD Publishing.
Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology, 8(422), 1–28. https://doi.org/10.3389/fpsyg.2017.00422
Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385–407. https://doi.org/10.1007/s10648-004-0006-x
Pressley, M., Borkowski, J. G., & Schneider, W. (1987). Cognitive strategies: Good strategy users coordinate metacognition and knowledge. Annals of Child Development, 4, 89-129.
Reimann, P. (2007). Time is precious: Why process analysis is essential for CSCL (and can also help to bridge between experimental and descriptive methods). In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Mice, minds, and society. Proceedings of the computer-supported collaborative learning conference (CSCL 2007) (pp. 598–607). New Brunswick, NJ: International Society of the Learning Sciences.
Rogiers, A., Merchie, E., & Van Keer, H. (2018). Fostering text-learning strategies in secondary education through explicit strategy-instruction. Paper presented at the International Conference of the EARLI SIG 2, Freiburg, Germany, 27-29 august, 2018.
Rogiers, A., Merchie, E., & Van Keer, H. (2019a). Learner profiles in secondary education: Occurrence and relationship with performance and student characteristics. The Journal of Educational Research, 112(3), 385-396. https://doi.org/10.1080/00220671.2018.1538093
Rogiers, A., Merchie, E., & Van Keer, H. (2019b). What they say is what they do? Comparing task-specific self-reports, think-aloud protocols, and study traces for measuring secondary school students’ text-learning strategies. European Journal of Psychology of Education, 1-18. https://doi.org/10.1007/s10212-019-00429-5
Samuelstuen, M. S., & Bråten, I. (2007). Examining the validity of self-reports on scales measuring students' strategic processing. British Journal of Educational Psychology, 77(2), 351-378. https://doi.org/10.1348/000709906X106147
Schellings, G. L. M., & Broekkamp, H. (2011). Signaling task awareness in think-aloud protocols from students selecting relevant information from text. Metacognition and Learning, 6, 65–82. https://doi.org/10.1007/s11409-010-9067-z
Slotte, V., Lonka, K., & Lindblom-Ylanne, S. (2001). Study-strategy use in learning from text. Does gender make any difference? Instructional Science, 29(3), 255-272. https://doi.org/10.1023/a:1017574300304
Schoor, C., & Bannert, M. (2012). Exploring regulatory processes during a computer-supported collaborative learning task using process mining. Computers in Human Behavior, 28(4), 1321–1331. https://doi.org/10.1016/j.chb.2012.02.016
Scott, D. B. (2008). Assessing text processing: A comparison of four methods. Journal of Literacy Research, 40, 290-316. doi:10.1080/10862960802502162
Simpson, M. L., & Nist, S. L. (2000). An update on strategic learning: It's more than textbook reading strategies. Journal of Adolescent & Adult Literacy, 43(6), 528-541.
Stoeger, H., Fleischmann, S., Obergriesser, S. (2015). Self-regulated learning (SRL) and the gifted learner in primary school: The theoretical basis and empirical findings on a research program dedicated to ensuring that all students learn to regulate their own learning. Asia Pacific Education Review, 16, 257-267. https://doi.org/ 10.1007/s12564-015-9376-7
Veenman, M. V. J. (2005). The assessment of metacognitive skills: What can be learned from multi-method designs? In C. Artett & B. Moschner (Eds.), Ledrnstrategien und metakognition. Implikationen für forschung und praxis (pp. 77-99). Münster: Waxmann.
Veenman, M. V. J. (2011). Alternative assessment of strategy use with self-report instruments: A discussion. Metacognition and Learning, 6(2), 205–211. https://doi.org/10.1007/s11409-011-9080-x
Veenman, M. V., Elshout, J. J., & Meijer, J. (1997). The generality vs. domain-specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7(2), 187–209. https://doi.org/10.1016/S0959-4752(96)00025-4
Wade, S. E., Trathen, W., & Schraw, G. (1990). An analysis of spontaneous study strategies. Reading Research Quarterly, 25(2), 147-166. https://doi.org/10.2307/747599
Weinstein, C. E., Jung, J., & Acee, T.W. (2011). Learning strategies In V. G. Aukrust (Ed.), Learning and Cognition in Education (pp. 137-143). Oxford, UK: Elsevier Limited.
Van der Aalst. (2011). Process mining: Discovery, conformance and enhancement of business processes. New York, NY: Springer.
van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method. A practical guide to modelling cognitive processes. London, UK: Academic Press.
Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64–71.
Young, K. A. (2005). Direct from the source: the value of ‘think-aloud’ data in understanding learning. Journal of Educational Enquiry, 6(1), 19-33.