Constructing nomological nets on the basis of process analyses to strengthen CSCL research

Main Article Content

Karsten Stegmann

Abstract

Due to the nature of collaborative learning, realising perfectly controlled experiments often requires an unreasonable amount of resources and sometimes it is not possible at all. Against this background, I propose to augment as good as feasible experimental design with a nomological net of relations between instructional support (intervention), learning processes and learning outcomes. Nomological networks are known from construct validity. In construct validity, the relations between variables (e.g. group differences, correlation matrices) are used to provide evidence for the validity of a measure. By adding multiple process and outcome variables together with the corresponding relations between intervention, process and outcome, the validity of causal relations found can be strengthened. I suggest adopting quality criteria from good research designs to evaluate the nomological nets. The resulting net needs to be (1) theory grounded, (2) situational, (3) feasible, (4) redundant, and (5) efficient. By making these nomological nets explicit and by designing them according to the presented criteria, CSCL research becomes more potent: the risk of inconclusive results is reduced while results that form a consistent nomological net can be interpreted with a stronger confidence, even if the experimental design has some flaws. If this becomes standard in CSCL research, it can be expected to contribute significantly better to knowledge accumulation in this area of research.

Article Details

How to Cite
Stegmann, K. (2014). Constructing nomological nets on the basis of process analyses to strengthen CSCL research. Frontline Learning Research, 2(4), 25–34. https://doi.org/10.14786/flr.v2i4.112
Section
Articles

References

Aronson, E. (1978). The jigsaw classroom. London: Sage.
Clark, D. B., D'Angelo, C. M. & Menekse, M. (2009). Initial structuring of online discussions to improve learning and argumentation: Incorporating students' own explanations as seed comments versus an augmented-preset approach to seeding discussions. Journal of Science Education and Technology, 18,
321-333. doi:10.1007/s10956-009-9159-1
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis for field setting. MA: Houghton Mifflin.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological bulletin, 52(4), 281. doi:10.1037/h0040957
De Jong & van Joolingen (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68, 179-201. doi:10.3102/00346543068002179
Eberle, J., Stegmann, K., Lund, K., Barrat, A., Sailer, M., & Fischer, F. (2013). Fostering learning and collaboration in a scientific community – evidence from an experiment using RFID devices to measure collaborative processes. In N. Rummel, M. Kapur, M. Nathan, & S. Puntambekar, S. (Eds.), To See the World and a Grain of Sand: Learning across Levels of Space, Time, and Scale: CSCL 2013 Conference Proceedings Volume 1 — Full Papers & Symposia (pp. 169-175). International Society of the Learning Sciences.
Jeong, A., Clark, D. B., Sampson, V. D., & Menekse, M. (2010). Sequential analysis of scientific argumentation in asynchronous online discussion environments. In S. Puntambekar, G. Erkens & C. Hmelo-Silver (Eds.), Analyzing Interactions in CSCL: Methodologies, Approaches and Issues. Berlin: Springer. doi:10.1007/978-1-4419-7710-6_10
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56-66. doi:10.1080/00461520.2012.748005
Fischer, S., Lowe, R. K., & Schwan, S. (2008). Effects of presentation speed of a dynamic visualization on the understanding of a mechanical system. Applied Cognitive Psychology, 22(8), 1126-1141. doi:10.1002/acp.1426
Kobarg, M., Prenzel, M., & Seidel, T. (2011). An international comparison of science teaching and learning. Further results from PISA 2006. Münster: Waxmann Verlag.
Koschmann, T. D. (1994). Toward a theory of computer support for collaborative learning. The Journal of the Learning Sciences, 3(3), 219-225. doi:10.1207/s15327809jls0303_1
Maas, C. J. M., & Hox, J. (2005). Sufficient samples sizes for multilevel modeling. Methodology, 1,
86–92. doi:10.1027/1614-1881.1.3.86
Mayfield, E., & Rosé, C. P. (2012). LightSIDE: Open Source Machine Learning for Text Accessible to Non-Experts. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Grading (pp. 124-135). New York: Routledge.
Mäkitalo-Siegl, K., Stegmann, K., Frete, A., & Streng, S. (2012). Orchestrating computer-supported collaborative learning: Effects of knowledge sharing and shared knowledge. In S. Abramovich (Ed.), Computers in education (pp. 75-91). Commack, NY: Nova Science Publishers.
Molinari, G., Chanel, G., Betrancourt, M., Pun, T, & Bozelle, C. (2013). Emotion Feedback during Computer-Mediated Collaboration: Effects on Self-Reported Emotions and Perceived Interaction. In N. Rummel, M. Kapur, M. Nathan, & S. Puntambekar, S. (Eds.), To See the World and a Grain of Sand: Learning across Levels of Space, Time, and Scale: CSCL 2013 Conference Proceedings Volume 1 — Full Papers & Symposia (pp. 336-343). International Society of the Learning Sciences.
Mu, J., Stegmann, K., Mayfield, E., Rosé, C. & Fischer, F. (2012). The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions. International Journal of Computer-Supported Collaborative Learning, 7(2), 285–305. doi:10.1007/s11412-012-9147-y
Osborne, J., & Henessy, S. (2003). Literature review in science education and the role of ICT: promise, problems and future directions. Bristol: NESTA Futurelab. Retrieved from http://www.futurelab.org.uk/resources/publications_reports_articles/literature_reviews /Literature_Review380
Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition & Instruction, 1(2), 117-175. doi:10.1207/s1532690xci0102_1
Rosé, C. P., Wang, Y. C., Arguello, J., Stegmann, K., Weinberger, A., & Fischer, F. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3(3), 237-271. doi:10.1007/s11412-007-9034-0
Stahl, G. (2006). Group cognition. Cambridge, MA: MIT Press.
Stegmann, K. & Fischer, F. (2011). Quantifying qualities in collaborative knowledge construction: the analysis of online discussions. In S. Puntambekar, G. Erkens & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: methods, approaches and issues (pp. 247-268). New York: Springer. doi:10.1007/978-1-4419-7710-6_12
Stegmann, K., Weinberger, A., & Fischer, F. (2011). Aktives Lernen durch Argumentieren: Evidenz für das Modell der argumentativen Wissenskonstruktion in Online-Diskussionen [Active learning by argumentation: Evidence for the model of argumentative knowledge construction in online discussions.]. Unterrichtswissenschaft, 39(3), 231–244.
Stegmann, K., Wecker, C., Weinberger, A. & Fischer, F. (2012). Collaborative argumentation and cognitive elaboration in a computer-supported collaborative learning environment. Instructional Science, 40(2), 297-323. doi:10.1007/s11251-011-9174-5
Stegmann, K., Weinberger, A., & Fischer, F. (2007). Facilitating argumentative knowledge construction with computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(4), 421-447. doi:10.1007/978-0-387-36949-5_12
Strijbos, J.-W., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers & Education, 46(1), 29-48. doi:10.1016/j.compedu.2005.04.002
Trochim, W., & Land, D. (1982). Designing designs for research. The Researcher, 1(1), 1-6.
van Boxtel, C., van der Linden, J., Roelofs, E., & Erkens, G. (2002). Collaborative Concept Mapping: Provoking and Supporting Meaningful Discourse. Theory Into Practice, 41(1), 40-46. doi:10.1207/s15430421tip4101_7
Wecker, C. (2013). How to support prescriptive statements by empirical research: Some missing parts. Educational Psychology Review, 25(1), 1-18. doi:10.1007/s10648-012-9208-9
Wecker, C., Stegmann, K., & Fischer, F. (2012). Lern- und Kooperationsprozesse: Warum sind sie interessant und wie können sie analysiert werden? [Learning and Cooperation Processes in Case-Based Learning. Interesting Issues and Analysis Approaches] REPORT: Zeitschrift für Weiterbildungsforschung, 35(3), 30-41. doi:10.3278/REP1203W
Wegerif, R. (2002). Thinking skills, technology and learning: a review of the literature for NESTA FutureLab. Bristol: NESTA Futurelab. Retrieved from: http://www.futurelab.org.uk/resources/publications_reports_articles/literature_reviews /Literature_Review394
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46(1), 71-95. doi:10.1016/j.compedu.2005.04.003
Weinberger, A., Marttunen, M., Laurinen, L., & Stegmann, K. (2013). Inducing socio-cognitive conflict in Finnish and German groups of online learners by CSCL script. International Journal of Computer-Supported Collaborative Learning, 8(3), 333-349. doi:10.1007/s11412-013-9173-4
Weinberger, A., Stegmann, K., & Fischer, F. (2005). Computer-supported collaborative learning in higher education: Scripts for argumentative knowledge construction in distributed groups. In T. Koschmann, D. D. Suthers & T.-W. Chan (Eds.), Computer Supported Collaborative Learning 2005: The Next 10 Years! Proceedings of the International Conference on Computer Supported Collaborative Learning 2005 (pp.
717-726). Mahwah, NJ: Lawrence Erlbaum. doi:10.3115/1149293.1149387
Weinberger, A., Stegmann, K., & Fischer, F. (2007). Knowledge convergence in collaborative learning: Concepts and assessment. Learning and Instruction, 17(4), 416-426. doi:10.1016/j.learninstruc.2007.03.007
Weinberger, A., Stegmann, K., & Fischer, F. (2010). Learning to argue online: Scripted groups surpass individuals (unscripted groups do not). Computers in Human Behavior, 26(4), 506-515. doi:10.1016/j.chb.2009.08.007
Wise, A. F., & Chiu, M. M. (2011). Analyzing temporal patterns of knowledge construction in a role-based online discussion. International Journal of Computer-Supported Collaborative Learning, 6(3),
445-470. doi:10.1007/s11412-011-9120-1
Zottmann, J., Stegmann, K., Strijbos, J. W., Vogel, F., Wecker, C., & Fischer, F. (2013). Computer-supported collaborative learning with digital video cases in teacher education: The impact of teaching experience on knowledge convergence. Computers in Human Behavior, 29(5), 2100-2108. doi: 10.1016/j.chb.2013.04.014