Learning analytics for academic paths: student evaluations of two dashboards for study planning and monitoring
Main Article Content
Abstract
An in-depth understanding of student experiences and evaluations of learning analytics dashboards (LADs) is needed to develop supportive learning analytics tools. This study investigates how students (N = 140) evaluated two student-facing LADs as a support for academic path-level self-regulated learning (SRL) through the concrete processes of planning and monitoring studies. Aim of the study was to gain new understanding about student perspectives for LAD use on academic path-level context. The study specifically focused on the student evaluations of the dashboard support and challenges, and the differences of student evaluations based on their self-efficacy beliefs and resource management strategies. The findings revealed that students evaluated dashboard use helpful for their study planning and monitoring, while the challenge aspects mostly included further information needs and development ideas. Students with higher self-efficacy evaluated the dashboards as more helpful for study planning than those with lower self-efficacy, and students with lower help seeking skills evaluated the dashboards as more helpful for study monitoring than those with higher help seeking skills. The results indicate that the design of LAD can help students to focus on different aspects of study planning and monitoring and that students with different beliefs and capabilities might benefit from different LAD designs and use practices. The study provides theory-informed approach for investigating LAD use in academic path-level context and extends current understanding of students as users of LADs.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Author et al. (2021)
Author et al. (2021)
Authors et al. (2021)
Aguilar, S. J., & Baek, C. (2019). Motivated information seeking and graph comprehension among college students. Proceedings of the 9th International Conference on Learning Analytics & Knowledge (pp. 280–289).
Aguilar, S. J., Karabenick, S. A., Teasley, S. D., & Baek, C. (2021). Associations between learning analytics dashboard exposure and motivation and self-regulated learning. Computers and Education, 162. https://doi.org/10.1016/j.compedu.2020.104085
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological Review, 84(2), 191.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315. https://doi.org/10.1207/s15327809jls0603_1
Duncan, T. G., & McKeachie, W. J. (2005). The making of the motivated strategies for learning questionnaire. Educational Psychologist, 40(2), 117–128.
Dresel, M., Schmitz, B., Schober, B., Spiel, C., Ziegler, A., Engelschalk, T., ... & Steuer, G. (2015). Competencies for successful self-regulated learning in higher education: structural model and indications drawn from expert interviews. Studies in Higher Education, 40(3), 454-470.
Ellis, R. A., & Bliuc, A. M. (2019). Exploring new elements of the student approaches to learning framework: The role of online learning technologies in student learning. Active Learning in Higher Education, 20(1), 11-24.
Greene, J. A., & Azevedo, R. (2007). A theoretical review of Winne and Hadwin's model of self-regulated learning: New perspectives and directions. Review of Educational Research, 77(3), 334–372. https://doi.org/10.3102/003465430303953
Gutiérrez, F., Seipp, K., Ochoa, X., Chiluiza, K., De Laet, T., & Verbert, K. (2020). LADA: A learning analytics dashboard for academic advising. Computers in Human Behavior, 107, 105826.
Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., & Zdrahal, Z. (2020). The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. The Internet and Higher Education, 45, 100725.
Ifenthaler, D., & Yau, J. Y. (2020). Utilising learning analytics to support study success in higher education: A systematic review. Educational Technology Research and Development, 68(4), 1961–1990. https://doi.org/10.1007/s11423-020-09788-z
Jansen, R., Leeuwen, A. van, Janssen, J., & Kester, L. (2020). A mixed method approach to studying self-regulated learning in MOOCs: combining trace data with interviews. Frontline Learning Research, 8(2), 35–64. https://doi.org/10.14786/flr.v8i2.539
Jivet, I., Scheffel, M., Schmitz, M., Robbers, S., Specht, M., & Drachsler, H. (2020). From students with love: An empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education. The Internet and Higher Education 47, 100758.
Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: Preparing learning analytics dashboards for educational practice. ACM International Conference Proceeding Series (2018), 31–40. https://doi.org/10.1145/3170358.3170421
Järvelä, S., Hurme, T. R., & Järvenoja, H. (2010). Self-regulation and motivation in computer-supported collaborative learning environments. In Learning across sites (pp. 342-357). Routledge.
Kyndt, E., Donche, V., Trigwell, K., & Lindblom-Ylänne, S. (Eds.). (2017). Higher education transitions: theory and research. Taylor & Francis.
Khiat, H. (2022). Using automated time management enablers to improve self-regulated learning. Active Learning in Higher Education, 23(1), 3-15.
Lim, L., Dawson, S., Joksimovic, S., & Gašević, D. (2019). Exploring students' sensemaking of learning analytics dashboards: Does frame of reference make a difference? 9th International Learning Analytics and Knowledge Conference (LAK19), March 2019, Tempe, AZ, USA. ACM, New York, NY, USA. https://doi.org/10.1145/3303772.3303804
Li, Q., Xu, D., Baker, R., Holton, A., & Warschauer, M. (2022). Can student-facing analytics improve online students’ effort and success by affecting how they explain the cause of past performance?. Computers & Education, 185, 104517
Ludvigsen, S., Lund, A., & Rasmussen, I. (2011a). Introduction: Learning across sites; new tools, infrastructures and practices. In Learning across sites (pp. 13-26). Routledge.
Ludvigsen, S., Rasmussen, I., Krange, I., Moen, A., & Middleton, D. (2011b). Intersecting trajectories of participation. Learning across sites: New tools, infrastructures and practices, 105-121.
Matcha, W., Uzir, N. A., Gasevic, D., & Pardo, A. (2020). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 13(2), 226–245. https://doi.org/10.1109/TLT.2019.2916802
Mendez, G., Galárraga, L., & Chiluiza, K. (2021, May). Showing academic performance predictions during term planning: Effects on students’ decisions, behaviors, and preferences. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-17).
Nguyen, L. T., & Ikeda, M. (2015). The effects of ePortfolio-based learning model on student self-regulated learning. Active Learning in Higher Education, 16(3), 197-209.
Ochoa, X., & Wise, A. F. (2021). Supporting the shift to digital with student-centered learning analytics. Educational Technology Research and Development, 69(1), 357–361.
Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology, 8, 422. https://dx.doi.org/10.3389/fpsyg.2017.00422.
Pintrich, P. R., Smith, P.R., Garcia, T. & McKeahie, W. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). https://eric.ed.gov/?id=ED338122
Rets, I., Herodotou, C., Bayer, V., Hlosta, M., & Rienties, B. (2021). Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students. International Journal of Educational Technology in Higher Education, 18(1), 1–23. https://doi.org/10.1186/s41239-021-00284-9
Salkind, N. J. (Ed.). (2010). Encyclopedia of research design (Vol. 1). Sage.
Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior, 78, 397–407.
Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26(3-4), 207–231. https://doi.org/10.1080/00461520.1991.9653133
Schunk, D. H., & Greene, J. A. (2018). Handbook of self-regulation of learning and performance: Second edition. Routledge.
Stenalt, M. H. (2021). Digital student agency: Approaching agency in digital contexts from a critical perspective. Frontline Learning Research, 9(3), 52–68. https://doi.org/10.14786/flr.v9i3.697
Teasley, S. D. (2017). Student facing dashboards: One size fits all? Technology, Knowledge and Learning, 22(3), 377–384. https://doi.org/10.1007/s10758-017-9314-3
Tempelaar, D. (2020) Supporting the less-adaptive student: the role of learning analytics, formative assessment and blended learning, Assessment & Evaluation in HigherEducation, 45:4, 579-593, DOI: 10.1080/02602938.2019.1677855
Vanthournout, G., Catrysse, L., Coertjens, L., Gijbels, D., & Donche, V. (2017). The development of learning strategies in higher education: Impact of gender and prior education. In Higher education transitions (pp. 135-154). Routledge.
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in human behavior, 89, 98-110.
Viberg, O., Khalil, M., & Baars, M. (2020). Self-regulated learning and learning analytics in online learning environments: A review of empirical research. In Proceedings of the tenth international conference on learning analytics & knowledge (pp. 524-533).
White, E. R. (2015). Academic advising in higher education: A place at the core. The Journal of General Education (University Park, Pa.), 64(4), 263–277. https://doi.org/10.5325/jgeneeduc.64.4.0263
Wiedbusch, M. D., Kite, V., Yang, X., Park, S., Chi, M., Taub, M., & Azevedo, R. (2021). A theoretical and evidence-based conceptual design of metadash: An intelligent teacher dashboard to support teachers' decision making and students’ self-regulated learning. In Frontiers in education (Vol. 6, p. 570229). Frontiers Media SA.
Winne, P. (2011). A cognitive and metacognitive analysis of self-regulated learning. In B. J. Zimmerman, & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance. Routledge. https://doi.org/10.4324/9780203839010-6
Winne, P., & Hadwin, A. (1998). Studying as self-regulated learning. In Metacognition in educational theory and practice.
Yilmaz, F. G. (2022). Utilizing learning analytics to support students' academic self-efficacy and problem-solving skills. The Asia-Pacific Education Researcher, 31(2), 175–191.
Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Sage Publications.