Measuring academic learning and exam self-efficacy at admission to university and its relation to first-year attrition an IRT-based multi-program validity study
Main Article Content
Abstract
Self-efficacy is associated with both academic performance and attrition in higher education. Whether it is possible to measure students’ academic self-efficacy after admission and prior to commencing higher education (i.e. pre-academic self-efficacy) in a valid and reliable way has hardly been studied. Aims: 1) to evaluate the construct validity and psychometric properties of two short scales to measure Pre-Academic Learning Self-Efficacy (PAL-SE) and Pre-Academic Exam Self-Efficacy (PAE-SE) using Rasch measurement models, 2) to investigate whether pre-academic self-efficacy was associated with half-year attrition across degree programs and institutions. Data consisted of 2686 Danish students admitted to nine different university degree programs across two institutions. Item analyses showed both scales to be essentially objective and construct valid, however, all items from the PAE-SE and two from the PAL-SE were locally dependent. Differential item functioning was found for the PAL-SE relative to degree programs. Reliability of the PAE-SE was .77, and varied for the PAL-SE from .79 to .86 across degree programs. Targeting was good only for the PAL-SE, thus we proceeded with the PAL-SE. PAL-SE was found to be associated with half-year attrition: A difference in PAL-SE from minimum to maximum was associated with a difference in half-year attrition of approximately 7%. This association was found both in the bivariate model and in the multivariate models with control of degree program, and with control of degree program and individual covariates such as earlier educational achievement and social background variables. Results thus also indicate that PAL-SE has a causal effect on half-year attrition.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Andersen, E. B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38(1), 123–140. DOI: 10.1007/BF02291180
Aryee, M. (2017). College students’ persistence and degree completion in science, technology, engineering, and mathematics (STEM): The role of non-cognitive attributes of self-efficacy, outcome expectations, and interest, (Doctoral dissertation). Available from: ProQuest Dissertations and Theses databases. (UMI No. 10264673)
Bandura, A. (1997). Self-efficacy. The exercise of control. New York, NY: Freeman. DOI: 10.5860/CHOICE.35-1826
Bartimote-Aufflick, K., Bridgeman, A., Walker, R., Sharma, M., & Smith, L. (2015). The study, evaluation, and improvement of university student self-efficacy. Studies in Higher Education, (ahead-of-print), 1-25. DOI: 10.1080/03075079.2014.999319
Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300. DOI: 10.1111/j.2517-6161.1995.tb02031.x
Breen, R.; Karlson, K. B. & Holm, A. (2018). Interpreting and Understanding Logits, Probits, and Other Nonlinear Probability Models. Annual Review of Sociology, 44, 4.1–4.16. DOI: 10.1146/annurev-soc-073117-041429
Burger, K. & Samuel, R. (2017). The Role of Perceived Stress and Self-Efficacy in Young People’s Life Satisfaction: A Longitudinal Study. Journal of Youth and Adolescence, 46, 78–90. DOI: 10.1007/s10964-016-0608-x
Cox, D. R.; Spjøtvoll, E.; Johansen, S.; van Zwet, W. R.; Bithell, J. F. & Barndorff-Nielsen, O. (1977). The Role of Significance Tests [with Discussion and Reply]. Scandinavian Journal of Statistics, 4, 49–70.
Credé, M., & Phillips, L. A. (2011). A meta-analytic review of the motivated strategies for learning questionnaire. Learning and Individual Differences, 21(4), 337–346. DOI: 10.1016/j.lindif.2011.03.002
De Clercq, M.; Galand, B. & Frenay, M. (2017). Transition from high school to university: a person-centered approach to academic achievement. European journal of psychology of education, 32(1), 39-59. DOI: 10.1007/s10212-016-0298-5
Devonport, T. J. & Lane, A. M. (2006). Relationships between self-efficacy, coping and student retention. Social Behavior and Personality: an international journal, 34(2), 127-138. DOI: 10.2224/sbp.2006.34.2.127
Duncan, T. G. & McKeachie, W. J. (2005). The making of the motivated strategies for learning questionnaire. Educational Psychologist, 40(2), 117–128. DOI: 10.1207/s15326985ep4002_6.
Elliott, D. C. (2016). The impact of self beliefs on post-secondary transitions: The moderating effects of institutional selectivity. Higher Education, 71(3), 415-431. DOI: 10.1007/s10734-015-9913-7
Ferla, J.; Valcke, M. & Cai, Y. (2009). Academic self-efficacy and academic self-concept: Reconsidering structural relationships. Learning and Individual Differences, 19(4), 499-505. DOI: 10.1016/j.lindif.2009.05.004
Ginsborg, J.; Kreutz, G.; Thomas, M. & Williamon, A. (2009). Healthy behaviours in music and non-music performance students. Health Education, 109(3), 242-258. DOI: 10.1108/09654280910955575
Greene, W. H. (2011). Econometric Analysis. Seventh Edition. Upper Saddle River, Prentice Hall.
Hamon, A. & Mesbah, M. (2002). Questionnaire reliability under the Rasch model. In: Mesbah, M.; Cole, B. F. & Lee, M. T. (Eds.). Statistical Methods for Quality of Life Studies. Dordrecht: Kluwer Academic Publishers, pp. 155-68. DOI: 10.1007/978-1-4757-3625-0_13
Heckman, J.J.; Humphries, J.E. and Veramendi, G. (2016). Returns to Education: The Causal Effects of Education on Earnings, Health and Smoking. NBER Working Paper, May 2016 (22291).
Huang, C. (2013). Gender differences in academic self-efficacy: a meta-analysis. European Journal of Psychology of Education, 28(1), 1–35. DOI: 10.1007/s10212-011-0097-y
Kelderman, H. (1984). Loglinear Rasch model tests. Psychometrika, 49, 223-245. DOI: 10.1007/bf02294174
Kreiner, S. (2003). Introduction to DIGRAM. Copenhagen: Department of Biostatistics, University of Copenhagen.
Kreiner, S. (2007). Validity and objectivity. Reflections on the role and nature of Rasch Models. Nordic Psychology, 59, 268-298. DOI: 10.1027/1901-2276.59.3.268
Kreiner, S. (2013). The Rasch model for dichotomous items. In Christensen, K. B.; Kreiner, S. & Mesbah, M. (Eds.) Rasch models in health. London: ISTE Ltd, Wiley, pp. 5–26. DOI: 10.1002/9781118574454.ch1
Kreiner, S. & Christensen, K.B. (2002). Graphical Rasch models. In: Mesbah, M.; Cole, B. F. & Lee, M. T. (Eds.) Statistical methods for quality of life studies. Dordrecht: Kluwer Academic Publishers, pp. 187–203. DOI: 10.1007/978-1-4757-3625-0_15
Kreiner, S. & Christensen, K. B. (2004). Analysis of local dependence and multidimensionality in graphical loglinear Rasch models. Communication in Statistics – Theory and Methods, 33(6), 1239–1276, DOI: 10.1081/STA-120030148
Kreiner, S. & Christensen, K. B. (2007). Validity and Objectivity in health-related Scales: Analysis by Graphical Loglinear Rasch models. In von Davier, M. & Carstensen, C. H. (Eds.) Multivariate and Mixture Distribution Rasch Models, New York, Springer, pp. 329-346. DOI: 10.1007/978-0-387-49839-3_21
Kreiner, S. & Christensen, K. B. (2013). Person Parameter Estimation and Measurement in Rasch Models. In Christensen, K. B.; Kreiner, S. & Mesbah, M. (Eds.) Rasch models in health. London: ISTE Ltd, Wiley, pp. 63–78. DOI: 10.1002/9781118574454.ch4
Kreiner, S. & Nielsen, T. (2013). Item analysis in DIGRAM 3.04. Part I: Guided tours. Research report 2013/06. University of Copenhagen, Department of Public Health.
Lent, R. W.; Miller, M. J.; Smith, P. E.; Watford, B. A.; Lim, R. H. & Hui, K. (2016). Social cognitive predictors of academic persistence and performance in engineering: Applicability across gender and race/ethnicity. Journal of Vocational Behavior, 94, 79-88. DOI: 10.1016/j.jvb.2016.02.012
Lindstrøm, C. & Sharma, M. D. (2011). Self-efficacy of first year university physics students: Do gender and prior formal instruction in physics matter? International Journal of Innovation in Science and Mathematics Education (formerly CAL-laborate International), 19(2), 1–19.
Luszczynska, A.; Gutiérrez-Doña, B. & Schwarzer, R. (2005). General self-efficacy in various domains of human functioning: Evidence from five countries. International Journal of Psychology, 40(2), 80-89. DOI: 10.1080/00207590444000041
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174, DOI: 10.1007/BF02296272
Mellenbergh, G. J. (1989). Item Bias and Item Response Theory. International Journal of Educational Research, 13, 127-143. DOI: 10.1016/0883-0355(89)90002-5
Meredith, W. (1993). Psychometrika, 58(4), 525-543. DOI:10.1007/BF02294825
Mesbah, M. & Kreiner, S. (2013). The Rasch model for ordered polytomous items. In Christensen, K. B.; Kreiner, S. & Mesbah, M. (Eds.) Rasch models in health. London: ISTE Ltd, Wiley, pp. 27-42. DOI: 10.1002/9781118574454.ch2
Multon, K. D.; Brown, S. D. & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. Journal of counseling psychology, 38(1), 30-38.DOI: 10.1037/0022-0167.38.1.30
Muris, P. (2002). Relationships between self-efficacy and symptoms of anxiety disorders and depression in a normal adolescent sample. Personality and Individual Differences, 32(2), 337–348. DOI: 10.1016/s01918869(01)00027-7
Nielsen, T.; Dammeyer, J.; Vang, M. L. & Makransky, G. (2018). Gender fairness in self-efficacy? A Rasch-based validity study of the General Academic Self-efficacy scale (GASE). Scandinavian Journal of Educational Research, 62(5), 664-681. DOI: 10.1080/00313831.2017.1306796
Nielsen, T. & Kreiner, S. (2013). Improving items that do not fit the Rasch model: exemplified with the physical functioning scale of the SF-36. Annales de L’I.S.U.P. Publications de L’Institut
de Statistique de L’Université de Paris, Numero Special, 57(1-2), 91-108.
Nielsen, T.; Makransky, G.; Vang, M. L. & Dammeyer, J. (2017). How specific is specific self-efficacy? A construct validity study using Rasch measurement models. Studies in Educational Evaluation, 57, 87-97. DOI: 10.1016/j.stueduc.2017.04.003
Pajares, F. (1997). Current directions in self-efficacy research. In M. Maehr & P. R. Pintrich (Eds.), Advances in motivation and achievement (Vol. 10). Greenwich, CT: JAI Press, pp. 1–49.
Pintrich, P.R.; Smith, D.A.F.; Garcia, T. & McKeachie, W. J. (1991). A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). Technical Report No. 91-8-004. The Regents of The University of Michigan.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Danish Institute for Educational Research.
Richardson, M.; Abraham, C. & Bond, R. (2012). Psychological correlates of university students' academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353-387. DOI: 10.1037/a0026838.
Saleh D.; Camart N. & Romo L. (2017). Predictors of Stress in College Students. Frontiers in Psychology, 8(19). DOI: 10.3389/fpsyg.2017.00019
Scherbaum, C. A.; Cohen-Charash, Y. & Kern, M. J. (2006). Measuring general self-efficacy: A comparison of three measures using item response theory. Educational and Psychological Measurement, 66(6), 1047–1063. DOI:10. 1177/0013164406288171
Scherer, R. & Siddiq, F. (2015). Revisiting teachers’ computer self-efficacy: A differentiated view on gender differences. Computers in Human Behavior, 53, 48–57. DOI:10.1016/j.chb.2015.06.038
Schunk, D. & Pajares, F. (2002). The development of academic self-efficacy. In A. Wigfield, & J. Eccles (Eds.), Development of achievement motivation. San Diego: Academic Press, pp. 15-31. DOI: 10.1016/b978-012750053-9/50003-6
Schwarzer, R. & Jerusalem, M. (1995). Generalized self-efficacy scale. Measures in health psychology: A user’s portfolio. Causal and Control Beliefs, 1, 35–37.
Tahmassian, K. & Jalali Moghadam, N. (2011). Relationship between self-efficacy and symptoms of anxiety, Depression, worry and social avoidance in a normal sample of students. Iranian Journal of Psychiatry and Behavioral Sciences, 5(2), 91–98.
Thomas, D. (2014). Factors that influence college completion intention of undergraduate students. The Asia-Pacific Education Researcher, 23(2), 225-235. DOI: 10.1007/s40299-013-0099-4
van der Linden, W. J. & Hambleton, R. K. (1997). Handbook of modern item response theory. Springer-Verlag, New York. DOI: 10.1007/978-1-4757-2691-6
van Herpen, S. G.; Meeuwisse, M.; Hofman, W. A.; Severiens, S. E. & Arends, L. R. (2017). Early predictors of first-year academic success at university: pre-university effort, pre-university self-efficacy, and pre-university reasons for attending university. Educational Research and Evaluation, 23(1-2), 52-72. DOI: 10.1080/13803611.2017.1301261
Vuong, M.; Brown-Welty, S. & Tracz, S. (2010). The effects of self-efficacy on academic success of first-generation college sophomore students. Journal of college student development, 51(1), 50-64. DOI: 10.1353/csd.0.0109
Watt, H. M. G.; Ehrich, J.; Stewart, S.E.; Snell, T.; Bucich, M.; Jacobs, N.; Furlonger, B. & English, D. (2019). Development of the Psychologist and Counsellor Self-Efficacy Scale. Higher Education, Skills and Work-Based Learning, (Early cite). DOI: 10.1108/HESWBL-07-2018-0069
Williams, B. W.; Kessler, H. A. & Williams, M. V. (2014). Relationship among practice change, motivation, and self‐efficacy. Journal of continuing education in the health professions, 34(S1), 5-10. DOI: 10.1002/chp.21235.
Yong, F. L. (2010). A study on the self-efficacy and expectancy for success of pre-university students. European Journal of Social Sciences, 13(4), 514-524.
Young, A. M., Wendel, P. J., Esson, J. M., & Plank, K. M. (2018). Motivational decline and recovery in higher education STEM courses. International Journal of Science Education, 40(9), 1016-1033. DOI: 10.1080/09500693.2018.1460773
Zimmerman, B. J.; Bandura, A. & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American educational research journal, 29(3), 663-676. DOI: 10.2307/1163261