Modeling and Measuring Domain-Specific Quantitative Reasoning in Higher Education Business and Economics

Main Article Content

Susanne Schmidt
Olga Zlatkin-Troitschanskaia
Richard J. Shavelson


Quantitative reasoning is considered a crucial prerequisite for acquiring domain-specific expertise in higher education. To ascertain whether students are developing quantitative reasoning, validly assessing its development over the course of their studies is required. However, when measuring quantitative reasoning in an academic study program, it is often confounded with other skills. Following a situated approach, we focus on quantitative reasoning in the domain of business and economics and define domain-specific quantitative reasoning primarily as a skill and capacity that allows for reasoned thinking regarding numbers, arithmetic operations, graph analyses, and patterns in real-world business and economics tasks, leading to problem solving. As many studies demonstrate, well-established instruments for assessing business and economics knowledge like the Test of Understanding College Economics (TUCE) and the Examen General para el Egreso de la Licenciatura (EGEL) contain items that require domain-specific quantitative reasoning skills. In this study, we follow a new approach and assume that assessing business and economics knowledge offers the opportunity to extract domain-specific quantitative reasoning as the skill for handling quantitative data in domain-specific tasks. We present an approach where quantitative reasoning – embedded in existing measurements from TUCE and EGEL tasks – will be empirically extracted. Hereby, we reveal that items tapping domain-specific quantitative reasoning constitute an empirically separable factor within a Confirmatory Factor Analysis and that this factor (domain-specific quantitative reasoning) can be validly and reliably measured using existing knowledge assessments. This novel methodological approach, which is based on obtaining information on students’ quantitative reasoning skills using existing domain-specific tests, offers a practical alternative to broad test batteries for assessing students’ learning outcomes in higher education.

Article Details

How to Cite
Schmidt, S., Zlatkin-Troitschanskaia, O., & Shavelson, R. J. (2023). Modeling and Measuring Domain-Specific Quantitative Reasoning in Higher Education Business and Economics. Frontline Learning Research, 11(1), 40–56.


AERA (American Education Research Association), APA (American Psychological Association), & NCME (National Council on Measurement in Education). (2014). Standards for educational and psychological testing. AERA.

Alexander, P. A., & Jetton, T. L. (2003). Learning from traditional and alternative texts: New conceptualization for an information age. In A. C. Graesser, M. A. Gernsbacher & S. R. Goldman (Eds.), Handbook of Discourse Processes (pp. 199–241). Lawrence Erlbaum Associates.

Anderson, J. R. (2005). Cognitive Psychology and its Implications (6th ed.). Worth.

Association of American Colleges and Universities (AAC&U). (2008). College learning for the new global century.

Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94.

Ball, D. L. (2003). Mathematical proficiency for all students: Toward a strategic research and development program in mathematics education. RAND Mathematics Study Panel.

Ballard, C. L., & Johnson, M. F. (2004). Basic math skills and performance in an introductory economics class. The Journal of Economic Education, 35(1), 3–23.

Bleske-Rechek, A., & Browne, K. (2014). Trends in GRE scores and graduate enrollments by gender and ethnicity. Intelligence, 46, 25–34.

Brückner, S., Förster, M., Zlatkin-Troitschanskaia, O., Happ, R., Walstad, W.B., Yamaoka, M., & Asano, T. (2015a). Gender effects in assessment of economic knowledge and understanding: Differences among undergraduate business and economics students in Germany, Japan, and the United States. Peabody Journal of Education, 90(4), 503–518.

Brückner, S., Förster, M., Zlatkin-Troitschanskaia, O., & Walstad, W. B. (2015b). Effects of prior economic education, native language, and gender on economic knowledge of first-year students in higher education. A comparative study between Germany and the USA. Studies in Higher Education, 40(3), 437–453.

Brückner, S., & Pellegrino, J. W. (2016). Integrating the analysis of mental operations into multilevel models to validate an assessment of higher education students’ competency in business and economics. Journal of Educational Measurement, 53(3), 293–312.

Byrne, M., & Flood, B. (2008). Examining the relationship among background variables and academic performance of first year accounting students at an Irish University. Journal of Accounting Education, 26(4), 202–212.

Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge University Press.

CENEVAL (Centro Nacional de Evaluación para la Educación Superior). (2011). Examen General para el Egreso de la Licenciatura en Administración. EGEL-ADMON. CENEVAL.

Davidson, M., & McKinney, G. G. R. (2001). Quantitative reasoning: An overview. Dialogue, 8, 1–5.

ETS (Educational Testing Service). (2016). GRE: Guide to the use of scores 2016-17.

Elrod, S. (2014). Quantitative Reasoning: The Next "Across the Curriculum" Movement. Peer Review, 16(3), 4–8.

Follette, K., Buxner, S., Dokter, E., McCarthy, D., Vezino B., Brock, L., & Prather, E. (2017). The Quantitative Reasoning for College Science (QuaRCS) Assessment 2: Demographic, Academic and Attitudinal Variables as Predictors of Quantitative Ability. Numeracy, 10(1), 1–33.

Förster, M., Brückner, S., & Zlatkin-Troitschanskaia, O. (2015). Assessing the financial knowledge of university students in Germany. Empirical Research in Vocational Education and Training, 7(6), 1–20.

Frey, A., Hartig, J., & Rupp, A. A. (2009). An NCME Instructional Module on Booklet designs in large-scale assessments of student achievement: Theory and practice. Educational Measurement: Issues and Practice, 28(3), 39–53.

Fritsch, S., Berger, S, Seifried, J., Bouley, F., Wuttke, E., Schnick-Vollmer, K., & Schmitz, B. (2015). The impact of university teacher training on prospective teachers’ CK and PCK – a comparison between Austria and Germany. Empirical Research in Vocational Education and Training, 7(4), 1–20. https://doi.rog/10.1186/s40461-015-0014-8

Gaze, E. C., Montgomery, A., Kilic-Bahi, S., Leoni, D., Misener, L., & Taylor, C. (2014). Towards developing a quantitative literacy/reasoning assessment instrument. Numeracy, 7(2), 4.

Happ, R., Zlatkin-Troitschanskaia, O., & Förster, M. (2018). How prior economic education influences beginning university students’ knowledge of economics. Empirical Research in Vocational Education and Training, 10(5), 1–20.

Hart Research Associates. (2009). Learning and assessment: Trends in undergraduate education. A survey among members of the Association of American Colleges and Universities.

ITC (International Test Commission). (2005). ITC Guidelines for Translating and Adapting Tests.

Kuncel, N. R., Wee, S., Serafin, L., & Hezlett, S. A. (2010). The validity of the Graduate Record Examination for master’s and doctoral programs: A meta-analytic investigation. Educational and Psychological Measurement, 70(2), 340–352.

Lusardi, A., & Wallace, D. (2013). Financial literacy and quantitative reasoning in the high school and college classroom. Numeracy, 6(2), Article 1.

Madison, B. L. (2009). All the more reason for QR across the curriculum. Numeracy, 2(1), Article 1.

Mehler, A., Zlatkin-Troitschanskaia, O., Hemati, W., Molerov, D., Lücking, A., & Schmidt, S. (2018). Integrating computational linguistic analysis of multilingual learning data and educational measurement approaches to explore student learning in higher education. In O. Zlatkin-Troitschanskaia, G. Wittum & A. Dengel (Eds.), Positive learning in the age of information (pp. 145–193). Springer VS.

Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice, 25(4), 6–20.

Muthén, L. K., & Muthén, B. O. (2012). Mplus user’s guide (7th ed.). Muthén & Muthén.

NRC (National Research Council). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century.

OECD (Organisation for Economic Co-operation and Development). (2013). The Survey of Adult Skills: Reader’s Companion.

O’Neill, P. B., & Flynn, D. T. (2013). Another curriculum requirement? Quantitative reasoning in economics: Some first steps. American Journal of Business Education, 6(3), 339–346.

Owen, A. L. (2012). Student characteristics, behavior, and performance in economics classes. In G. M. Hoyt & K. M. McGoldrick (Eds.), International Handbook on Teaching and Learning Economics (pp. 341–350). Edward Elgar.

Rocconi, L. M., Lambert, A. D., McCormick, A. C., & Sarraf, S. A. (2013). Making college count: an examination of quantitative reasoning activities in higher education. Numeracy, 6(2), 1–20.

Roohr, K. C., Graf, E. A., & Liu, O. L. (2014). Assessing quantitative literacy in higher education: An overview of existing research and assessments with recommendations for next-generation assessment. ETS Research Report Series, 2014(2), 1–26.

Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75(2), 24–248.

Schreiber, J. B., Nora, A., Stage, F. C., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results. A review. The Journal of Educational Research, 99(6), 323–338.

Schuler, H., Funke, U., & Baron-Boldt, J. (1990). Predictive validity of high-school grades: A meta-analysis. Applied Psychology: An International Review, 39(1), 89–103.

Shavelson, R. J. (2008). Reflections on quantitative reasoning: An assessment perspective. In B. L. Madison & L. A. Steen (Eds.), Calculation vs. context. Quantitative literacy and its implications for teacher education (pp. 27–44). Mathematical Association of America.

Shavelson, R. J., & Huang, L. (2003). Responding responsibly to the frenzy to assess learning in higher education. Change. The Magazin of Higher Learning, 35(1), 10–19.

Shavelson, R. J., Marino, J. P., Zlatkin-Troitschanskaia, O., & Schmidt, S. (2019). Reflections on the assessment of quantitative reasoning. In B. L. Madison & L. A. Steen (Eds.), Calculation vs. context: Quantitative literacy and its implications for teacher education. Mathematical Association of America.

Tiffin, P. A., McLachlan, J. C., Webster, L., & Nicholson, S. (2014). Comparison of the sensitivity of the UKCAT and A levels to sociodemographic characteristics: A national study. BMC Medical Education, 14(7), 1–12.

Walstad, W. B., Watts, M. W., & Rebeck, K. (2007). Test of understanding in college economics: Examiner’s manual (4th ed.). National Council on Economic Education.

Williams, M. L., Waldauer, C., & Duggal, V.G. (1992). Gender differences in economic knowledge: An extension of the analysis. The Journal of Economic Education, 23(3), 219–231.

Yamaoka, M., Walstad, W. B., Watts, M. W., Asana, T., & Abe, S. (Eds.). (2010). Comparative studies on economic education in Asia-Pacific region. Shumpusha.

Zahner, D. (2013). Reliability and validity of CLA +. Council for Aid to Education.

Zlatkin-Troitschanskaia, O., Förster, M., Brückner, S., & Happ, R. (2014). Insights from a German assessment of business and economics competence. In H. Coates (Ed.), Higher education learning outcomes assessment (pp. 175–200). Peter Lang.

Zlatkin-Troitschanskaia, O., Förster, M., Schmidt, S., Brückner, S., & Beck, K. (2015). Erwerb wirtschaftswissenschaftlicher Fachkompetenz im Studium. Eine mehrebenenanalytische Betrachtung von hochschulischen und individuellen Einflussfaktoren [Acquisition of economic competence over the course of studies. A multilevel consideration of academic and individual determinants]. In S. Blömeke & O. Zlatkin-Troitschanskaia (Eds.), Kompetenzen von Studierenden (pp. 116–134). Beltz Juventa.

Zlatkin-Troitschanskaia, O., Shavelson, R. J., & Pant, H. A. (2017). Assessment of learning outcomes in higher education – international comparisons and perspectives. In C. Secolsky & B. Denison (Eds.), Handbook on measurement, assessment and evaluation in higher education (2nd ed.). Routledge.