Measuring academic learning and exam self-efficacy at admission to university and its relation to first-year attrition an IRT-based multi-program validity study

Main Article Content

Tine Nielsen
Ida Sophie Friderichsen
Bjarke Tarpgaard Hartkopf

Abstract

Self-efficacy is associated with both academic performance and attrition in higher education. Whether it is possible to measure students’ academic self-efficacy after admission and prior to commencing higher education (i.e. pre-academic self-efficacy) in a valid and reliable way has hardly been studied. Aims: 1) to evaluate the construct validity and psychometric properties of two short scales to measure Pre-Academic Learning Self-Efficacy (PAL-SE) and Pre-Academic Exam Self-Efficacy (PAE-SE) using Rasch measurement models, 2) to investigate whether pre-academic self-efficacy was associated with half-year attrition across degree programs and institutions. Data consisted of 2686 Danish students admitted to nine different university degree programs across two institutions. Item analyses showed both scales to be essentially objective and construct valid, however, all items from the PAE-SE and two from the PAL-SE were locally dependent. Differential item functioning was found for the PAL-SE  relative to degree programs. Reliability of the PAE-SE was .77, and varied for the PAL-SE  from .79 to .86 across degree programs. Targeting was good only for the PAL-SE, thus we proceeded with the PAL-SE. PAL-SE was found to be associated with half-year attrition: A difference in PAL-SE from minimum to maximum was associated with a difference in half-year attrition of approximately 7%. This association was found both in the bivariate model and in the multivariate models with control of degree program, and with control of degree program and individual covariates such as earlier educational achievement and social background variables. Results thus also indicate that PAL-SE has a causal effect on half-year attrition.

Article Details

How to Cite
Nielsen, T., Friderichsen, I. S., & Tarpgaard Hartkopf, B. (2019). Measuring academic learning and exam self-efficacy at admission to university and its relation to first-year attrition: an IRT-based multi-program validity study. Frontline Learning Research, 7(3), 91-118. https://doi.org/10.14786/flr.v7i3.503
Section
Articles
Author Biographies

Ida Sophie Friderichsen, University of Copenhagen

Department of Psychology, Research Assistant

Bjarke Tarpgaard Hartkopf, The Danish Evaluation Institute

Higher Education section, Chief consultant

References

Andersen, E. B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38(1), 123–140. DOI: 10.1007/BF02291180

Aryee, M. (2017). College students’ persistence and degree completion in science, technology, engineering, and mathematics (STEM): The role of non-cognitive attributes of self-efficacy, outcome expectations, and interest, (Doctoral dissertation). Available from: ProQuest Dissertations and Theses databases. (UMI No. 10264673)

Bandura, A. (1997). Self-efficacy. The exercise of control. New York, NY: Freeman. DOI: 10.5860/CHOICE.35-1826

Bartimote-Aufflick, K., Bridgeman, A., Walker, R., Sharma, M., & Smith, L. (2015). The study, evaluation, and improvement of university student self-efficacy. Studies in Higher Education, (ahead-of-print), 1-25. DOI: 10.1080/03075079.2014.999319

Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300. DOI: 10.1111/j.2517-6161.1995.tb02031.x

Breen, R.; Karlson, K. B. & Holm, A. (2018). Interpreting and Understanding Logits, Probits, and Other Nonlinear Probability Models. Annual Review of Sociology, 44, 4.1–4.16. DOI: 10.1146/annurev-soc-073117-041429

Burger, K. & Samuel, R. (2017). The Role of Perceived Stress and Self-Efficacy in Young People’s Life Satisfaction: A Longitudinal Study. Journal of Youth and Adolescence, 46, 78–90. DOI: 10.1007/s10964-016-0608-x

Cox, D. R.; Spjøtvoll, E.; Johansen, S.; van Zwet, W. R.; Bithell, J. F. & Barndorff-Nielsen, O. (1977). The Role of Significance Tests [with Discussion and Reply]. Scandinavian Journal of Statistics, 4, 49–70.

Credé, M., & Phillips, L. A. (2011). A meta-analytic review of the motivated strategies for learning questionnaire. Learning and Individual Differences, 21(4), 337–346. DOI: 10.1016/j.lindif.2011.03.002

De Clercq, M.; Galand, B. & Frenay, M. (2017). Transition from high school to university: a person-centered approach to academic achievement. European journal of psychology of education, 32(1), 39-59. DOI: 10.1007/s10212-016-0298-5

Devonport, T. J. & Lane, A. M. (2006). Relationships between self-efficacy, coping and student retention. Social Behavior and Personality: an international journal, 34(2), 127-138. DOI: 10.2224/sbp.2006.34.2.127

Duncan, T. G. & McKeachie, W. J. (2005). The making of the motivated strategies for learning questionnaire. Educational Psychologist, 40(2), 117–128. DOI: 10.1207/s15326985ep4002_6.

Elliott, D. C. (2016). The impact of self beliefs on post-secondary transitions: The moderating effects of institutional selectivity. Higher Education, 71(3), 415-431. DOI: 10.1007/s10734-015-9913-7

Ferla, J.; Valcke, M. & Cai, Y. (2009). Academic self-efficacy and academic self-concept: Reconsidering structural relationships. Learning and Individual Differences, 19(4), 499-505. DOI: 10.1016/j.lindif.2009.05.004

Ginsborg, J.; Kreutz, G.; Thomas, M. & Williamon, A. (2009). Healthy behaviours in music and non-music performance students. Health Education, 109(3), 242-258. DOI: 10.1108/09654280910955575

Greene, W. H. (2011). Econometric Analysis. Seventh Edition. Upper Saddle River, Prentice Hall.

Hamon, A. & Mesbah, M. (2002). Questionnaire reliability under the Rasch model. In: Mesbah, M.; Cole, B. F. & Lee, M. T. (Eds.). Statistical Methods for Quality of Life Studies. Dordrecht: Kluwer Academic Publishers, pp. 155-68. DOI: 10.1007/978-1-4757-3625-0_13

Heckman, J.J.; Humphries, J.E. and Veramendi, G. (2016). Returns to Education: The Causal Effects of Education on Earnings, Health and Smoking. NBER Working Paper, May 2016 (22291).

Huang, C. (2013). Gender differences in academic self-efficacy: a meta-analysis. European Journal of Psychology of Education, 28(1), 1–35. DOI: 10.1007/s10212-011-0097-y

Kelderman, H. (1984). Loglinear Rasch model tests. Psychometrika, 49, 223-245. DOI: 10.1007/bf02294174

Kreiner, S. (2003). Introduction to DIGRAM. Copenhagen: Department of Biostatistics, University of Copenhagen.

Kreiner, S. (2007). Validity and objectivity. Reflections on the role and nature of Rasch Models. Nordic Psychology, 59, 268-298. DOI: 10.1027/1901-2276.59.3.268

Kreiner, S. (2013). The Rasch model for dichotomous items. In Christensen, K. B.; Kreiner, S. & Mesbah, M. (Eds.) Rasch models in health. London: ISTE Ltd, Wiley, pp. 5–26. DOI: 10.1002/9781118574454.ch1

Kreiner, S. & Christensen, K.B. (2002). Graphical Rasch models. In: Mesbah, M.; Cole, B. F. & Lee, M. T. (Eds.) Statistical methods for quality of life studies. Dordrecht: Kluwer Academic Publishers, pp. 187–203. DOI: 10.1007/978-1-4757-3625-0_15

Kreiner, S. & Christensen, K. B. (2004). Analysis of local dependence and multidimensionality in graphical loglinear Rasch models. Communication in Statistics – Theory and Methods, 33(6), 1239–1276, DOI: 10.1081/STA-120030148

Kreiner, S. & Christensen, K. B. (2007). Validity and Objectivity in health-related Scales: Analysis by Graphical Loglinear Rasch models. In von Davier, M. & Carstensen, C. H. (Eds.) Multivariate and Mixture Distribution Rasch Models, New York, Springer, pp. 329-346. DOI: 10.1007/978-0-387-49839-3_21

Kreiner, S. & Christensen, K. B. (2013). Person Parameter Estimation and Measurement in Rasch Models. In Christensen, K. B.; Kreiner, S. & Mesbah, M. (Eds.) Rasch models in health. London: ISTE Ltd, Wiley, pp. 63–78. DOI: 10.1002/9781118574454.ch4

Kreiner, S. & Nielsen, T. (2013). Item analysis in DIGRAM 3.04. Part I: Guided tours. Research report 2013/06. University of Copenhagen, Department of Public Health.

Lent, R. W.; Miller, M. J.; Smith, P. E.; Watford, B. A.; Lim, R. H. & Hui, K. (2016). Social cognitive predictors of academic persistence and performance in engineering: Applicability across gender and race/ethnicity. Journal of Vocational Behavior, 94, 79-88. DOI: 10.1016/j.jvb.2016.02.012

Lindstrøm, C. & Sharma, M. D. (2011). Self-efficacy of first year university physics students: Do gender and prior formal instruction in physics matter? International Journal of Innovation in Science and Mathematics Education (formerly CAL-laborate International), 19(2), 1–19.

Luszczynska, A.; Gutiérrez-Doña, B. & Schwarzer, R. (2005). General self-efficacy in various domains of human functioning: Evidence from five countries. International Journal of Psychology, 40(2), 80-89. DOI: 10.1080/00207590444000041

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174, DOI: 10.1007/BF02296272

Mellenbergh, G. J. (1989). Item Bias and Item Response Theory. International Journal of Educational Research, 13, 127-143. DOI: 10.1016/0883-0355(89)90002-5

Meredith, W. (1993). Psychometrika, 58(4), 525-543. DOI:10.1007/BF02294825

Mesbah, M. & Kreiner, S. (2013). The Rasch model for ordered polytomous items. In Christensen, K. B.; Kreiner, S. & Mesbah, M. (Eds.) Rasch models in health. London: ISTE Ltd, Wiley, pp. 27-42. DOI: 10.1002/9781118574454.ch2

Multon, K. D.; Brown, S. D. & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. Journal of counseling psychology, 38(1), 30-38.DOI: 10.1037/0022-0167.38.1.30

Muris, P. (2002). Relationships between self-efficacy and symptoms of anxiety disorders and depression in a normal adolescent sample. Personality and Individual Differences, 32(2), 337–348. DOI: 10.1016/s01918869(01)00027-7

Nielsen, T.; Dammeyer, J.; Vang, M. L. & Makransky, G. (2018). Gender fairness in self-efficacy? A Rasch-based validity study of the General Academic Self-efficacy scale (GASE). Scandinavian Journal of Educational Research, 62(5), 664-681. DOI: 10.1080/00313831.2017.1306796

Nielsen, T. & Kreiner, S. (2013). Improving items that do not fit the Rasch model: exemplified with the physical functioning scale of the SF-36. Annales de L’I.S.U.P. Publications de L’Institut

de Statistique de L’Université de Paris, Numero Special, 57(1-2), 91-108.

Nielsen, T.; Makransky, G.; Vang, M. L. & Dammeyer, J. (2017). How specific is specific self-efficacy? A construct validity study using Rasch measurement models. Studies in Educational Evaluation, 57, 87-97. DOI: 10.1016/j.stueduc.2017.04.003

Pajares, F. (1997). Current directions in self-efficacy research. In M. Maehr & P. R. Pintrich (Eds.), Advances in motivation and achievement (Vol. 10). Greenwich, CT: JAI Press, pp. 1–49.

Pintrich, P.R.; Smith, D.A.F.; Garcia, T. & McKeachie, W. J. (1991). A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). Technical Report No. 91-8-004. The Regents of The University of Michigan.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Danish Institute for Educational Research.

Richardson, M.; Abraham, C. & Bond, R. (2012). Psychological correlates of university students' academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353-387. DOI: 10.1037/a0026838.

Saleh D.; Camart N. & Romo L. (2017). Predictors of Stress in College Students. Frontiers in Psychology, 8(19). DOI: 10.3389/fpsyg.2017.00019

Scherbaum, C. A.; Cohen-Charash, Y. & Kern, M. J. (2006). Measuring general self-efficacy: A comparison of three measures using item response theory. Educational and Psychological Measurement, 66(6), 1047–1063. DOI:10. 1177/0013164406288171

Scherer, R. & Siddiq, F. (2015). Revisiting teachers’ computer self-efficacy: A differentiated view on gender differences. Computers in Human Behavior, 53, 48–57. DOI:10.1016/j.chb.2015.06.038

Schunk, D. & Pajares, F. (2002). The development of academic self-efficacy. In A. Wigfield, & J. Eccles (Eds.), Development of achievement motivation. San Diego: Academic Press, pp. 15-31. DOI: 10.1016/b978-012750053-9/50003-6

Schwarzer, R. & Jerusalem, M. (1995). Generalized self-efficacy scale. Measures in health psychology: A user’s portfolio. Causal and Control Beliefs, 1, 35–37.

Tahmassian, K. & Jalali Moghadam, N. (2011). Relationship between self-efficacy and symptoms of anxiety, Depression, worry and social avoidance in a normal sample of students. Iranian Journal of Psychiatry and Behavioral Sciences, 5(2), 91–98.

Thomas, D. (2014). Factors that influence college completion intention of undergraduate students. The Asia-Pacific Education Researcher, 23(2), 225-235. DOI: 10.1007/s40299-013-0099-4

van der Linden, W. J. & Hambleton, R. K. (1997). Handbook of modern item response theory. Springer-Verlag, New York. DOI: 10.1007/978-1-4757-2691-6

van Herpen, S. G.; Meeuwisse, M.; Hofman, W. A.; Severiens, S. E. & Arends, L. R. (2017). Early predictors of first-year academic success at university: pre-university effort, pre-university self-efficacy, and pre-university reasons for attending university. Educational Research and Evaluation, 23(1-2), 52-72. DOI: 10.1080/13803611.2017.1301261

Vuong, M.; Brown-Welty, S. & Tracz, S. (2010). The effects of self-efficacy on academic success of first-generation college sophomore students. Journal of college student development, 51(1), 50-64. DOI: 10.1353/csd.0.0109

Watt, H. M. G.; Ehrich, J.; Stewart, S.E.; Snell, T.; Bucich, M.; Jacobs, N.; Furlonger, B. & English, D. (2019). Development of the Psychologist and Counsellor Self-Efficacy Scale. Higher Education, Skills and Work-Based Learning, (Early cite). DOI: 10.1108/HESWBL-07-2018-0069

Williams, B. W.; Kessler, H. A. & Williams, M. V. (2014). Relationship among practice change, motivation, and self‐efficacy. Journal of continuing education in the health professions, 34(S1), 5-10. DOI: 10.1002/chp.21235.

Yong, F. L. (2010). A study on the self-efficacy and expectancy for success of pre-university students. European Journal of Social Sciences, 13(4), 514-524.

Young, A. M., Wendel, P. J., Esson, J. M., & Plank, K. M. (2018). Motivational decline and recovery in higher education STEM courses. International Journal of Science Education, 40(9), 1016-1033. DOI: 10.1080/09500693.2018.1460773

Zimmerman, B. J.; Bandura, A. & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American educational research journal, 29(3), 663-676. DOI: 10.2307/1163261