Using New Models to Analyze Complex Regularities of the World
Main Article Content
Abstract
This commentary to the recent article by Musso et al. (2013) discusses issues related to model fitting, comparison of classification accuracy of generative and discriminative models, and two (or more) cultures of data modeling. We start by questioning the extremely high classification accuracy with an empirical data from a complex domain. There is a risk that we model perfect nonsense perfectly. Our second concern is related to the relevance of comparing multilayer perceptron neural networks and linear discriminant analysis classification accuracy indices. We find this problematic, as it is like comparing apples and oranges. It would have been easier to interpret the model and the variable (group) importance’s if the authors would have compared MLP to some discriminative classifier, such as group lasso logistic regression. Finally, we conclude our commentary with a discussion about the predictive properties of the adopted data modeling approach.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Breiman, L. (2001b). Statistical Modeling: The Two Cultures. Statistical Science, 16(3), 199–231. doi:10.1214/ss/1009213726
Chipman, H. A., George, E. I., & McCulloch, R. E. (2010). BART: Bayesian Additive Regression Trees. The Annals of Applied Statistics, 4(1), 266–298. doi:10.1214/09-aoas285
Demšar, J. (2006). Statistical comparison of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.
Correa, M., Bielza, C., & Pamies-Teixeira, J. (2009). Comparison of Bayesian networks and artificial neural networks for quality detection in a machining process. Expert Systems with Applications, 36, 7270–7279. doi:10.1016/j.eswa.2008.09.024
Lek, S., & Guegan, J. F. (1999). Artificial neural networks as a tool in ecological modelling, an introduction. Ecological Modelling, 120, 65–73. doi:10.1016/s0304-3800(99)00092-7
Meier, L., van de Geer, S., & Bühlmann, P. (2008). The group lasso for logistic regression. Journal of the Royal Statistical Society: Series B, 70(Part 1), 53-71. doi:10.1111/j.1467-9868.2007.00627.x
Musso, M. F., Kyndt, E., Cascallar, E. C., & Dochy, F. (2013). Predicting general academic performance and identifying differential contribution of participating variables using artificial neural networks. Frontline Learning Research, 1, 42-71. doi:10.14786/flr.v1i1.13
Nokelainen, P., Silander, T., Ruohotie, P., & Tirri, H. (2007). Investigating the Number of Non-linear and Multi-modal Relationships between Observed Variables Measuring a Growth-oriented Atmosphere. Quality & Quantity, 41(6), 869-890. doi:10.1007/s11135-006-9030-x
Nokelainen, P., & Ruohotie, P. (2009). Non-linear Modeling of Growth Prerequisites in a Finnish Polytechnic Institution of Higher Education. Journal of Workplace Learning, 21(1), 36-57. doi:10.1108/13665620910924907
Nokelainen, P., Tirri, K., Campbell, J. R., & Walberg, H. (2007). Factors that Contribute or Hinder Academic Productivity: Comparing two groups of most and least successful Olympians. Educational Research and Evaluation, 13(6), 483-500. doi:10.1080/13803610701785931
Schittenkopf, C., Deco, G., & Brauer, W. (1997). Two Strategies to Avoid Overfitting in Feedforward Networks. Neural Networks, 10(3), 505-516. doi:10.1016/s0893-6080(96)00086-x
Schneider, M., & Edelsbrunner, P. (2013). Modelling for Prediction vs. Modelling for Understanding: Commentary on Musso et al. (2013). Frontline Learning Research, 1(2), 99-101. doi:10.14786/flr.v1i2.74
Tirri, K., Nokelainen, P., & Komulainen, E. (2013). Multiple Intelligences: Can they be measured? Psychological Test and Assessment Modeling, 55(4), 438-461. doi:10.1007/978-94-6091-758-5_1
Villaverde, J. E., Godoy, D., & Amandi, A. (2006). Learning styles’ recognition in e-learning environments with feed-forward neural networks. Journal of Computer Assisted Learning, 22, 197–206. doi:10.1111/j.1365-2729.2006.00169.x
Xue, J-H., & Titterington, D. M. (2008). Comment on “On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes”. Neural Processing Letters, 28(3), 169-187. doi:10.1007/s11063-008-9088-7