Psychometric properties of the Epistemological Development in Teaching Learning Questionnaire (EDTLQ): An inventory to measure higher order epistemological development

Main Article Content

Sofia Kjellström
Hudson Golino
Rebecca Hamer
Erik Jan van Rossum
Ellen Almers

Abstract

Qualitative research supports a developmental dimension in views on teaching and learning, but there are currently no quantitative tools to measure the full range of this development. To address this we developed the Epistemological Development in Teaching and Learning Questionnaire (EDTLQ). In the current study the psychometric properties of the EDTLQ were examined using a sample (N= 232) of teachers from a Swedish University. A confirmatory factor and Rasch analysis showed that the items of the EDTLQ form a unidimensional scale, implying a single latent variable (eg epistemological development). Item and person separation reliability, showed satisfactory levels of fit indicating that the response alternatives differentiate appropriately. Endorsement of the statements reflected the preferred constructivist learning-teaching environment of the response group. The EDTLQ is innovative since is the first quantitative survey to measure epistemological development and it has a potential to be used as an apt tool for teachers to monitor the development of students as well as to offer professional development opportunities to the teachers.

Article Details

How to Cite
Kjellström, S., Golino, H., Hamer, R., van Rossum, E. J., & Almers, E. (2016). Psychometric properties of the Epistemological Development in Teaching Learning Questionnaire (EDTLQ): An inventory to measure higher order epistemological development. Frontline Learning Research, 4(5), 1–33. https://doi.org/10.14786/flr.v4i5.239
Section
Articles
Author Biographies

Sofia Kjellström, Jönköping University, School of Health and Welfare, The Jönköping Academy for Improvments of Health and Welfare, Jönköping

The Jönköping Academy for Improvments in Health and Welfare

PhD, associate professor

Hudson Golino, Universidade Salgado de Oliveira, Rio de Janeiro, Brazil

PhD

Rebecca Hamer, International Baccalaureate The Hague

PhD

Erik Jan van Rossum, University of Twente

PhD

References

References
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–574.
Andrich, D. (1988). Rasch Models for Measurement. Newbury Park, CA: Sage.
Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1), 7 – 16.
Andrich, D. (2011). Rating scales and Rasch measurement. Expert Review Pharmacoeconomics Outcomes Reserach, 11(5), 571–585.
Arum, R., & J. Roksa, (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.
Baxter Magolda, M.B. (1992). Knowing and reasoning in college. San Francisco, CA: Jossey-Bass.
Baxter Magolda, M.B. (2001). Making their own way. Sterling, VA: Stylus Publishing.
Belenky, M.F., Clinchy, B.M., Goldberger, N.R., & Tarule, J.M. (1986 ed. 1997). Women’s ways of knowing: The development of self, voice and mind. New York, NY: Basic Books.
Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246.
Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606.
Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ US: Lawrence Erlbaum Associates Publishers.
Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In: Bollen, K. A. & J. S. Long (Eds.), Testing structural equation models (pp.136 -162). Newbury Park, CA: Sage.
Commons, M.L., & Ross, S. N. (2008a). Special Issue: Postformal Thought and Hierarchical complexity. World Future: Journal of General Evolution, 64(5-7), 297-562.
Commons, M.L., & Ross, S. N. (2008b). What postformal thought is, and why it matters, World Future: Journal of General Evolution, 64(5-7), 321-329.
Commons, M.L. (1989). Adult development. Vol. 1, Comparisons and applications of developmental models. New York, NY: Praeger.
Commons, M.L. (1990). Adult development. Vol. 2, Models and methods in the study of adolescent and adult thought. New York, NY: Praeger.
Dawson, T.L. (2006). The meaning and measurement of conceptual development in adulthood. In C.H. Hoare (Ed.), Handbook of adult development and learning (pp 433.454). Oxford, UK: Oxford University Press.
Dawson, T., Goodheart, E., Draney, K., Wilson, M., & Commons, M. (2010). Concrete, abstract, formal, and systematic operations as observed in a 'Piagetian' balance-beam task series. Journal of Applied Measurement, 11(1), 11-23.
Dawson, T. L., Xie, Y., & Wilson, M. (2003). Domain-general and domain-specific developmental assessments: Do they measure the same thing? Cognitive Development, 18(1), 61-78.
Demick, J., & Andreoletti, C. (2003). Handbook of Adult Development. The Springer Series in Adult Development and Aging. New York, NY: Springer.
Demetriou, A., & Kyriakides, L. (2006). The functional and developmental organization of cognitive developmental sequences. British Journal of Educational Psychology, 76(2), 209-242.
Embretson, S.E., & Reise, S. P. (2000). Item response theory for psychologists. London, UK: Erlbaum.
Entwistle, N.J., & Ramsden, P. (1983). Understanding Student Learning. New York, NY: Nichols.
Epskamp, S. (2014). semPlot: Path diagrams and visual analysis of various SEM packages' output. R package version 1.0.1. Retrieved from http://CRAN.R-project.org/package=semPlot
Fischer, K.W., & Pruyne, E, (2003) Reflective thinking in adulthood: Emergence, development, and variation. In J. Demick & C. Andreoletti (Eds.), Handbook of adult development (pp 169-198). New York, NY: Kluwer Academic/Plenum.
Glas, C.A. (2007). Multivariate and Mixture Distribution Rasch Models. New York, NY: Springer-Verlag.
Gow, L., & D. Kember, D. (1990). Does higher education promote independent learning? Higher Education, 19(3), 307-322.
Golino, H.F., Gomes, C.M.A., Commons, M.L.C., & Miller, P. M. (2014). The Construction and Validation of a Developmental Test for Stage Identification: Two Exploratory Studies. Behavioral Development Bulletin, 19(3), 37-54.
Hoare, C.H., (2006). Handbook of adult development and learning. Oxford, UK: Oxford University Press.
Hambleton, R. K. (2000). Response to Hays et al and McHorney and Cohen: Emergence of Item Response Modeling in Instrument Development and Data Analysis. Medical Care, 38(9), II-60.
Hambleton, R. K., & Jones, R. W. (1993). An NCME Instructional Module on. Educational measurement: issues and practice, 12(3), 38-47.
Hamer, R., & Van Rossum, E.J. (2010, November). De zes talen in het onderwijs: communicatie en miscommunicatie [The six languages in education: communication and miscommunication]. Onderwijsvernieuwing, 17.
Hamer, R., & Van Rossum, E.J. (in press). Six Languages in Education: Looking for Postformal Thinking. Behavioral Developmental Bulletin, (x), pp. yyy-zzz. DOI: xxxxxx
Hibbard, J., Collins, P., Mahoney, E., & Baker, L. (2010). The development and testing of a measure assessing clinician beliefs about patient self-management. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy, 13(1), 65-72.
Hornik, K. (2014). The R FAQ. Retrieved from http://CRAN.R-project.org/doc/FAQ/R-FAQ.html
Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
Kegan, R. (1994). In over our Heads: The mental demands of modern life. Cambridge, MA: Harvard University Press.
King, P.M., & Kitchener, K.S. (1994). Developing Reflective Judgment. San Francisco, CA: Jossey-Bass Publishers
King, P.M., & Kitchener, K.S. (2004). Reflective judgment: Theory and research on the development of epistemic assumptions through adulthood. Educational Psychologist, 39(1), p. 5-18.
Kjellström, S., & Ross, S.N. (2011). Older persons reasoning about responsibility for health: variations and predictions. International Journal of Aging & Human Development, 72(2), 99-124.
Kjellström, S., & Sjölander, P. (2013). The level of development of nursing staff’s value system predicts their views on paternalistic care and personal autonomy. International Journal of Aging and Later Life. 9(1), p. 35-68.
Kuhn, D. (1991). The Skills of Argument. Cambridge, UK: Cambridge University Press:
Linacre J. M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16 (2), 878.
Lindblom-Ylänne, S., & Lonka, K. (1999). Individual ways of interacting with the learning environment – are they related to study success? Learning and Instruction, 9(1), 1-18.
Linder, C.J. (1992). Is teacher-reflected epistemology a source of conceptual difficulty in physics? International Journal of Science Education, 14(1), 111-121.
Mair P., & Hatzinger, R. (2007a). Extended Rasch Modeling: The eRm package for the application of IRT models in R. Journal of Statistical Software, 20(9), 1–20.
Mair P., & Hatzinger, R. (2007b). CML based estimation of extended Rasch models with the eRm package in R. Psychology Science, 49, 26–43.
Mair, P., Hatzinger, R., & Maier, M.J. (2014). eRm: Extended Rasch Modeling. R package version 0.15-4. Retrieved from http://erm.r-forge.r-project.org/
Mishra, P., & Kereluik, K. (2011). What 21st century learning? A review and a synthesis. In C. D. Maddux, M. J.Koehler, P. Mishra, & C. Owens (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2011 (pp. 3301–3312). Chesapeake, VA: AACE.
Newble, D.I., & Clarke, R.M. (1986). The approaches to learning of students in a traditional and an innovative problem-based medical school, Medical Education, 20(4), 267-273.
Panayides, P., Robinson, C., & Tymms, P. (2010). The assessment revolution that has passed England by: Rasch measurement. British Educational Research Journal, 36(4), 611-626.
Perry, W.G. (1970). Forms of intellectual and ethical development in the college years: A scheme. New York, NY: Holt, Rinehart & Winston.
Pornprasertmanit, S., Miller, P., Schoemann, A., & Rosseel, Y. (2014). semTools: Useful tools for structural equation modeling.. R package version 0.4-6. Retrieved from http://CRAN.R-project.org/package=semTools
R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.R-project.org/.
Rasch, G. (1960). Probabilistic Models for some Intelligence and Attainment Tests. Danish Institute for Educational Research, Copenhagen.
Raykov, T. (2012). Scale construction and development using structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 472-494). New York, NY: Guilford.
Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1-36. Retrieved from http://www.jstatsoft.org/v48/i02/.
Säljö, R. (1994). Minding action; Conceiving of the world versus participating in cultural practices. Nordisk Pedagogik, 14(2), 71-80.
Säljö, R. (1997). Talk as Data and Practice – a critical look at phenomenographic inquiry and the appeal to experience. Higher Education Research & Development, 16(2), 173-190.
Schommer, M. (1998). The influence of age and education on epistemological beliefs. British Journal of Educational Psychology, 68(4), 551-561.
Sjölander, P., Lindström, N., Ericsson, A., & Kjellström, S. (2013). A pattern recognition method for disclosing different levels of value system from questionnaire data, Behavioral Developmental Bulletin, 19(3), 114-127.
Van Rossum, E.J., & Hamer, R.N. (2010). The Meaning of Learning and Knowing. Rotterdam, Netherlands: Sense Publishers.
Van Rossum, E.J., & Hamer R. (2011). Analysing deep learning and understanding: getting the meanings clear first. Paper presented at the EARLI conference, Exeter, U.K.
Van Rossum, E.J., & Hamer R. (2012). Analysing Higher Education Teachers’ Learning-Teaching Conceptions with a Model of Student Thinking. Paper presented at the AERA conference, Vancouver, Canada.
Van Rossum, E.J., & Hamer, R. (2013). The relationship between students’ conception of good teaching and their views on a good textbook. Paper presented at the EARLI conference, Munich, Germany.
Van Rossum, E.J., & Schenk, S.M. (1984). The relationship between learning conception, study strategy and learning outcome. British Journal of Educational Psychology, 54(1), 73-83.
Vermunt, J.D. (1996). Metacognitive, cognitive and affective aspects of learning styles and strategies: A phenomenographic analysis. Higher Education, 31(1), 25-50.
Voogt, J, Erstad, O., Dede, C., & Mishra P. (2013) Challenges to learning and schooling in the digital networked world of the 21st century. Journal of Computer Assisted Learning, 29, 403–413, doi: 10.1111/jcal.12029
West, E.J., (2004) Perry's legacy: models of epistemological development. Journal of adult development, 112, 61-70. Wright, B. D. & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
Yerrick, R.K., Pedersen, J.E., & Arnason, J. (1998). “We're just spectators”: A case study of science teaching, epistemology, and classroom management, Science Education, 82(6), 619-648.