Challenges and learning outcomes of educational design research for PhD students
Main Article Content
Abstract
Educational design research (EDR) is described as a complex research approach. The challenges resulting from this complexity are typically described as procedural, whereas EDR might also be challenging for different reasons, specifically for early career researchers. Yet challenging experiences may be noteworthy in the process of learning to do research and becoming a researcher. To explore this issue further, we engaged in a collaborative self-study, and conducted a narrative cross-case analysis of two PhD candidates’ experiences of engaging in EDR, focusing on challenges and learning outcomes. We find indications that the challenges of EDR might be more related to EDR’s relatively new and minority position in educational sciences and the role a (early career) researcher needs to assume in EDR. Retrospectively, the challenges appear closely related to learning outcomes, which are described in terms of a more profound understanding of research (quality) and of oneself as a researcher. As such, insights gained by self-study of research practices provide a complementary perspective to existing literature on EDR and becoming a researcher.
Article Details
How to Cite
Bronkhorst, L. H., & de Kleijn, R. A. (2016). Challenges and learning outcomes of educational design research for PhD students. Frontline Learning Research, 4(3), 75–91. https://doi.org/10.14786/flr.v4i3.198
Section
Articles
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Akkerman, S. F., Bronkhorst, L. H., & Zitter, I. (2013). The complexity of educational design research. Quality & Quantity, 47(1), 421-439. doi: 10.1007/s11135-011-9527-9
Anderson, T., & Shattuck, J. (2012). Design-based research: a decade of progress in education research? Educational researcher, 41(1), 16-25. doi: 10.3102/0013189X11428813
Andres, L., Bengtsen, S. S., Castaño, L. G., Crossouard, B., Keefer, J. M., & Pyhältö, K. (2015). Drivers and interpretations of doctoral education today: National comparisons. Frontline Learning Research, 3(3), 1-18. doi: 10.14786/flr.v3i3.177
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1-14. doi: 10.1207/s15327809jls1301_1
Barnacle, R. (2005). Research education ontologies: Exploring doctoral becoming. Higher Education Research & Development, 24(2), 179-188. doi: 10.1080/07294360500062995
Baptista, A., Frick, L., Holley, K., Remmik, M., Tesch, J., & Åkerlind, G. (2015). The doctorate as an original contribution to knowledge: Considering relationships between originality, creativity, and innovation. Frontline Learning Research, 3(3), 51-63. doi: 10.14786/flr.v3i3.147
Biesta, G. J. (2010). Why ‘what works’ still won’t work: From evidence-based education to value-based education. Studies in Philosophy and Education, 29(5), 491-503. doi: 10.1007/s11217-010-9191-x
Bronkhorst, L. H., Meijer, P. C., Koster, B., & Vermunt, J. D. (2011). Fostering meaning oriented learning and deliberate practice in teacher education. Teaching and Teacher Education, 27, 1120-1130. doi: 10.1016/j.tate.2011.05.008
Bronkhorst, L. H., Meijer, P. C., Koster, B., Akkerman, S. F., & Vermunt, J. D. (2013). Consequential research designs in research on teacher education. Teaching and Teacher Education, 33, 90-99. doi: 10.1016/j.tate.2013.02.007
Bronkhorst, L. H., van Rijswijk, M. M., Meijer, P. C., Koster, B., & Vermunt, J. D. (2013). University teachers’ collateral transitions: continuity and discontinuity between research and teaching. Infancía y Aprendizaje, 36, 293-308. doi: 10.1174/021037013807532972
Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141-178. doi: 10.1207/s15327809jls0202_2
Bullough, R. V., & Pinnegar, S. (2001). Guidelines for quality in autobiographical forms of self-study research. Educational researcher, 30(3), 13-21. doi: 10.3102/0013189X030003013
Castelló, M., Kobayaski, S., McGinn, M., Pechar, H., Vekkaila, J., & Wisker, G. (2015). Researcher identity in transition: Signals to identify and managesSpheres of activity in a risk-career. Frontline Learning Research, 3(3), 35-50. doi: 10.14786/flr.v3i3.149
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15-42. doi: 10.1207/s15327809jls1301_2
Dede, C. (2004). If design-based research is the answer, what is the question? A commentary on Collins, Joseph, and Bielaczyc; diSessa and Cobb; and Fishman, Marx, Blumenthal, Krajcik, and Soloway in the JLS special issue on design-based research. Journal of the Learning Sciences, 13(1), 105-114. doi: 10.1207/s15327809jls1301_5
Edwards, A., Sebba, J., & Rickinson, M. (2007). Working with users: Some implications for educational research. British Educational Research Journal, 33(5), 647-661. doi: 10.1080/01411920701582199
Engeström, Y. (2011a). From design experiments to formative interventions. Theory & Psychology, 21, 598-628. doi:10.1177/0959354311419252
Engeström, Y. (2011b). Intervening to shape the future. Keynote given at the Annual AERA Conference, New Orleans, LO.
Evans, L. (2010). Developing the European researcher: ‘extended’ professionality within the Bologna process. Professional Development in Education, 36, 663-677. doi:10.1080/19415251003633573
González-Ocampo, G., Kiley, M., Lopes, A., Malcolm, J., Menezes, I., Morais, R., & Virtanen, V. (2015). The curriculum question in doctoral education. Frontline Learning Research, 3(3), 19-34. doi: 10.14786/flr.v3i3.191
Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries. Educational Technology Research and Development, 29(2), 75-91. doi:10.1007/BF02766777
Haigh, N. (2012). Historical research and research in higher education: reflections and recommendations from a self-study. Higher Education Research & Development, 31(5), 689-702. doi: 10.1080/07294360.2012.689955
Hall, L., & Burns, L. (2009). Identity development and mentoring in doctoral education. Harvard Educational Review, 79(1), 49-70. doi: 10.17763/haer.79.1.wr25486891279345
Hamilton, M. L., Smith, L., & Worthington, K. (2008). Fitting the methodology with the research: An exploration of narrative, self-study and auto-ethnography. Studying Teacher Education, 4(1), 17-28. doi: 10.1080/17425960801976321
Herrington, J., McKenney, S., Reeves, T., & Oliver, R. (2007). Design-based research and doctoral students: Guidelines for preparing a dissertation proposal. In C. Montgomerie, & J. Seale (Eds.), Proceedings of EdMEdia 2007: Wolrd conference on education multimedia, hypermedia & telecommunications (pp. 4089-4097). Chesapeake, VA: AACE.
Holt, N. L. (2003). Representation, legitimation, and autoethnography: An autoethnographic writing story. International Journal of Qualitative Methods, 2(1), 18-28. doi: 10.1177/160940690300200102
Hopwood, N. (2010). Doctoral students as journal editors: non‐formal learning through academic work. Higher Education Research & Development, 29(3), 319-331. doi: 10.1080/07294360903532032
Kelly, A. E. (2003). Theme issue: The role of design in educational research. Educational Researcher, 32(1), 3-4. doi: 10.3102/0013189X032001003
Kelly, A. E. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 115-128. doi: 10.1207/s15327809jls1301_6
Kleijn, R. A. M. de, Bronkhorst, L. H., Meijer, P. C., Pilot, A., & Brekelmans, M. (2014). Understanding the up, back, and forward-component in master's thesis supervision with adaptivity. Studies in Higher Education, 1-17. doi: 10.1080/03075079.2014.980399
Kleijn, R. A. M. de, Meijer, P. C., Brekelmans, M., & Pilot, A. (2015). Adaptive research supervision: exploring expert thesis supervisors' practical knowledge. Higher Education Research & Development, 34(1), 117-130. doi: 10.1080/07294360.2014.934331
Lee, S., & Roth, W. (2003). Becoming and belonging: Learning qualitative research through legitimate peripheral participation. Forum: Qualitative Social Research, 4(2), Available online at: http://www.qualitative-research.net/index.php/fqs/article/view/708 (accessed October 26, 2012).
Lichtman, M. (2006). Qualitative research in education. A user's guide. Thousand Oaks, London, New Delhi: Sage Publications, Inc.
Loughran, J. (2007). Researching teacher education practices responding to the challenges, demands, and expectations of self-study. Journal of Teacher Education, 58(1), 12-20. doi: 10.1177/0022487106296217
Maxwell, J. A. (2004a). Causal explanation, qualitative research, and scientific inquiry in education. Educational researcher, 33(2), 3-11. doi: 10.3102/0013189X033002003
Maxwell, J. A. (2004b). Using qualitative methods for causal explanation. Field Methods, 16(3), 243-264. doi:10.1177/1525822X04266831
Meijer, P. C., de Graaf, G., & Meirink, J. (2011). Key experiences in student teachers’ development. Teachers and Teaching: theory and practice, 17(1), 115-129. doi: 10.1080/13540602.2011.538502
Newbury, D. (2002). Doctoral education in design, the process of research degree study, and the ‘trained researcher’. Art, Design and Communication in Higher Education, 1(3): 149-159. doi: 10.1386/adch.1.3.149
Pallas, A. M. (2001). Preparing education doctoral students for epistemological diversity. Educational researcher, 6-11. doi: jstor.org/stable/3594455
Penuel, W. R. (2014). Emerging forms of formative intervention research in education. Mind, Culture, and Activity, 21(2), 97-117. doi: 10.1080/10749039.2014.884137
Petrarca, D., & Bullock, S. M. (2014). Tensions between theory and practice: Interrogating our pedagogy through collaborative self-study. Professional Development in Education, 5, 265-281. doi:10.1080/19415257.2013.801876
Reeves, T. C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 96-115. doi:10.1007/BF02961476
Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25-28. doi: 10.3102/0013189X032001025
Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39(4), 199-201. doi:10.1207/s15326985ep3904_1
Svihla, V. (2014). Advances in design-based research. Frontline Learning Research, 2(4), 35-45. doi: 10.14786/flr.v2i4.114
Stubb, J., Pyhältö, K., & Lonka, K. (2011). Balancing between inspiration and exhaustion: PhD students' experienced socio-psychological well-being. Studies in Continuing Education, 33(1), 33-50. doi: 10.1080/0158037X.2010.515572
van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, N. Nieveen, R. M. Branch, K. L. Gustafson & T. Plomp (Eds.), Design methodology and developmental research in education and training (pp. 1-14). The Netherlands: Kluwer Academic Publishers.
Wilhelm, R. W., Craig, M. T., Glover, R. J., Allen, D. D., & Huffman, J. B. (2000). Becoming qualitative researchers: A collaborative approach to faculty development. Innovative Higher Education, 24(4), 265-278. doi: 10.1023/B:IHIE.0000047414.56668.5b
Williams, J., & Ritter, J. K. (2010). Constructing new professional identities through self‐study: From teacher to teacher educator. Professional Development in Education, 36, 77-92. doi:10.1080/19415250903454833
Wisker, G., Robinson, G., Trafford, V., Creighton, E., & Warnes, M. (2003). Recognising and overcoming dissonance in postgraduate student research. Studies in Higher Education, 28(1), 91-105. doi: 10.1080/03075070309304
Zwart, R. C., Smit, B., & Admiraal, W. F. (2015). A closer look at teacher research: a review study into the nature and value of research conducted by teachers. Pedagogische Studien, 92(2), 131-149.
Anderson, T., & Shattuck, J. (2012). Design-based research: a decade of progress in education research? Educational researcher, 41(1), 16-25. doi: 10.3102/0013189X11428813
Andres, L., Bengtsen, S. S., Castaño, L. G., Crossouard, B., Keefer, J. M., & Pyhältö, K. (2015). Drivers and interpretations of doctoral education today: National comparisons. Frontline Learning Research, 3(3), 1-18. doi: 10.14786/flr.v3i3.177
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1-14. doi: 10.1207/s15327809jls1301_1
Barnacle, R. (2005). Research education ontologies: Exploring doctoral becoming. Higher Education Research & Development, 24(2), 179-188. doi: 10.1080/07294360500062995
Baptista, A., Frick, L., Holley, K., Remmik, M., Tesch, J., & Åkerlind, G. (2015). The doctorate as an original contribution to knowledge: Considering relationships between originality, creativity, and innovation. Frontline Learning Research, 3(3), 51-63. doi: 10.14786/flr.v3i3.147
Biesta, G. J. (2010). Why ‘what works’ still won’t work: From evidence-based education to value-based education. Studies in Philosophy and Education, 29(5), 491-503. doi: 10.1007/s11217-010-9191-x
Bronkhorst, L. H., Meijer, P. C., Koster, B., & Vermunt, J. D. (2011). Fostering meaning oriented learning and deliberate practice in teacher education. Teaching and Teacher Education, 27, 1120-1130. doi: 10.1016/j.tate.2011.05.008
Bronkhorst, L. H., Meijer, P. C., Koster, B., Akkerman, S. F., & Vermunt, J. D. (2013). Consequential research designs in research on teacher education. Teaching and Teacher Education, 33, 90-99. doi: 10.1016/j.tate.2013.02.007
Bronkhorst, L. H., van Rijswijk, M. M., Meijer, P. C., Koster, B., & Vermunt, J. D. (2013). University teachers’ collateral transitions: continuity and discontinuity between research and teaching. Infancía y Aprendizaje, 36, 293-308. doi: 10.1174/021037013807532972
Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141-178. doi: 10.1207/s15327809jls0202_2
Bullough, R. V., & Pinnegar, S. (2001). Guidelines for quality in autobiographical forms of self-study research. Educational researcher, 30(3), 13-21. doi: 10.3102/0013189X030003013
Castelló, M., Kobayaski, S., McGinn, M., Pechar, H., Vekkaila, J., & Wisker, G. (2015). Researcher identity in transition: Signals to identify and managesSpheres of activity in a risk-career. Frontline Learning Research, 3(3), 35-50. doi: 10.14786/flr.v3i3.149
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15-42. doi: 10.1207/s15327809jls1301_2
Dede, C. (2004). If design-based research is the answer, what is the question? A commentary on Collins, Joseph, and Bielaczyc; diSessa and Cobb; and Fishman, Marx, Blumenthal, Krajcik, and Soloway in the JLS special issue on design-based research. Journal of the Learning Sciences, 13(1), 105-114. doi: 10.1207/s15327809jls1301_5
Edwards, A., Sebba, J., & Rickinson, M. (2007). Working with users: Some implications for educational research. British Educational Research Journal, 33(5), 647-661. doi: 10.1080/01411920701582199
Engeström, Y. (2011a). From design experiments to formative interventions. Theory & Psychology, 21, 598-628. doi:10.1177/0959354311419252
Engeström, Y. (2011b). Intervening to shape the future. Keynote given at the Annual AERA Conference, New Orleans, LO.
Evans, L. (2010). Developing the European researcher: ‘extended’ professionality within the Bologna process. Professional Development in Education, 36, 663-677. doi:10.1080/19415251003633573
González-Ocampo, G., Kiley, M., Lopes, A., Malcolm, J., Menezes, I., Morais, R., & Virtanen, V. (2015). The curriculum question in doctoral education. Frontline Learning Research, 3(3), 19-34. doi: 10.14786/flr.v3i3.191
Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries. Educational Technology Research and Development, 29(2), 75-91. doi:10.1007/BF02766777
Haigh, N. (2012). Historical research and research in higher education: reflections and recommendations from a self-study. Higher Education Research & Development, 31(5), 689-702. doi: 10.1080/07294360.2012.689955
Hall, L., & Burns, L. (2009). Identity development and mentoring in doctoral education. Harvard Educational Review, 79(1), 49-70. doi: 10.17763/haer.79.1.wr25486891279345
Hamilton, M. L., Smith, L., & Worthington, K. (2008). Fitting the methodology with the research: An exploration of narrative, self-study and auto-ethnography. Studying Teacher Education, 4(1), 17-28. doi: 10.1080/17425960801976321
Herrington, J., McKenney, S., Reeves, T., & Oliver, R. (2007). Design-based research and doctoral students: Guidelines for preparing a dissertation proposal. In C. Montgomerie, & J. Seale (Eds.), Proceedings of EdMEdia 2007: Wolrd conference on education multimedia, hypermedia & telecommunications (pp. 4089-4097). Chesapeake, VA: AACE.
Holt, N. L. (2003). Representation, legitimation, and autoethnography: An autoethnographic writing story. International Journal of Qualitative Methods, 2(1), 18-28. doi: 10.1177/160940690300200102
Hopwood, N. (2010). Doctoral students as journal editors: non‐formal learning through academic work. Higher Education Research & Development, 29(3), 319-331. doi: 10.1080/07294360903532032
Kelly, A. E. (2003). Theme issue: The role of design in educational research. Educational Researcher, 32(1), 3-4. doi: 10.3102/0013189X032001003
Kelly, A. E. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 115-128. doi: 10.1207/s15327809jls1301_6
Kleijn, R. A. M. de, Bronkhorst, L. H., Meijer, P. C., Pilot, A., & Brekelmans, M. (2014). Understanding the up, back, and forward-component in master's thesis supervision with adaptivity. Studies in Higher Education, 1-17. doi: 10.1080/03075079.2014.980399
Kleijn, R. A. M. de, Meijer, P. C., Brekelmans, M., & Pilot, A. (2015). Adaptive research supervision: exploring expert thesis supervisors' practical knowledge. Higher Education Research & Development, 34(1), 117-130. doi: 10.1080/07294360.2014.934331
Lee, S., & Roth, W. (2003). Becoming and belonging: Learning qualitative research through legitimate peripheral participation. Forum: Qualitative Social Research, 4(2), Available online at: http://www.qualitative-research.net/index.php/fqs/article/view/708 (accessed October 26, 2012).
Lichtman, M. (2006). Qualitative research in education. A user's guide. Thousand Oaks, London, New Delhi: Sage Publications, Inc.
Loughran, J. (2007). Researching teacher education practices responding to the challenges, demands, and expectations of self-study. Journal of Teacher Education, 58(1), 12-20. doi: 10.1177/0022487106296217
Maxwell, J. A. (2004a). Causal explanation, qualitative research, and scientific inquiry in education. Educational researcher, 33(2), 3-11. doi: 10.3102/0013189X033002003
Maxwell, J. A. (2004b). Using qualitative methods for causal explanation. Field Methods, 16(3), 243-264. doi:10.1177/1525822X04266831
Meijer, P. C., de Graaf, G., & Meirink, J. (2011). Key experiences in student teachers’ development. Teachers and Teaching: theory and practice, 17(1), 115-129. doi: 10.1080/13540602.2011.538502
Newbury, D. (2002). Doctoral education in design, the process of research degree study, and the ‘trained researcher’. Art, Design and Communication in Higher Education, 1(3): 149-159. doi: 10.1386/adch.1.3.149
Pallas, A. M. (2001). Preparing education doctoral students for epistemological diversity. Educational researcher, 6-11. doi: jstor.org/stable/3594455
Penuel, W. R. (2014). Emerging forms of formative intervention research in education. Mind, Culture, and Activity, 21(2), 97-117. doi: 10.1080/10749039.2014.884137
Petrarca, D., & Bullock, S. M. (2014). Tensions between theory and practice: Interrogating our pedagogy through collaborative self-study. Professional Development in Education, 5, 265-281. doi:10.1080/19415257.2013.801876
Reeves, T. C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 96-115. doi:10.1007/BF02961476
Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25-28. doi: 10.3102/0013189X032001025
Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39(4), 199-201. doi:10.1207/s15326985ep3904_1
Svihla, V. (2014). Advances in design-based research. Frontline Learning Research, 2(4), 35-45. doi: 10.14786/flr.v2i4.114
Stubb, J., Pyhältö, K., & Lonka, K. (2011). Balancing between inspiration and exhaustion: PhD students' experienced socio-psychological well-being. Studies in Continuing Education, 33(1), 33-50. doi: 10.1080/0158037X.2010.515572
van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, N. Nieveen, R. M. Branch, K. L. Gustafson & T. Plomp (Eds.), Design methodology and developmental research in education and training (pp. 1-14). The Netherlands: Kluwer Academic Publishers.
Wilhelm, R. W., Craig, M. T., Glover, R. J., Allen, D. D., & Huffman, J. B. (2000). Becoming qualitative researchers: A collaborative approach to faculty development. Innovative Higher Education, 24(4), 265-278. doi: 10.1023/B:IHIE.0000047414.56668.5b
Williams, J., & Ritter, J. K. (2010). Constructing new professional identities through self‐study: From teacher to teacher educator. Professional Development in Education, 36, 77-92. doi:10.1080/19415250903454833
Wisker, G., Robinson, G., Trafford, V., Creighton, E., & Warnes, M. (2003). Recognising and overcoming dissonance in postgraduate student research. Studies in Higher Education, 28(1), 91-105. doi: 10.1080/03075070309304
Zwart, R. C., Smit, B., & Admiraal, W. F. (2015). A closer look at teacher research: a review study into the nature and value of research conducted by teachers. Pedagogische Studien, 92(2), 131-149.