The Rating Dilemma of Academic Management Journals: Attuning the Perceptions of Peer Rating

Authors

  • Ahmad Rahal University of Arkansas - Fort Smith
  • Mohamed Zainuba University of Arkansas - Fort Smith

Keywords:

Management

Abstract

The adoption of journal lists as proxies to scholarship quality has sparked an ongoing debate among academics over what is meant by quality, how it is perceived by the reviewers, and the thresholds for the rating, inclusion, or exclusion of journals from these lists. Given the insufficient transparencies in the processes of journal quality evaluation when composing such lists, this research explores the use of the revealed preference approach to attune the ratings in both the Australian Business Deans Council Journal Quality List and Academic Journal Guide, and approximate the rating of management journals if they were to be considered for inclusion in either of the two aforementioned lists.

Author Biographies

Ahmad Rahal, University of Arkansas - Fort Smith

Business Administration

Professor

Mohamed Zainuba, University of Arkansas - Fort Smith

Business Administration

Associate Professor

References

Abramo, G., & D’Angelo, C. (2011). Evaluating research: From informed peer review to bibliometrics. Scientometrics, 87(3), 499–514.

Adler, N., & Harzing, A. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education, 8(1), 72–95.

Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.

Batista, P., Campiteli, M., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179–189.

Bauer, K., & Bakkalbasi, N. (2005). An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(09). Retrieved from http://www.dlib.org/dlib/september05/bauer/09bauer. html.

Bauerly, R., & Johnson, D. (2005). An evaluation of journals used in doctoral marketing programs. Journal of the Academy of Marketing Science, 33(3), 313–329.

Baum, J. (2011). Free-riding on power laws: Questioning the validity of the impact factor as a measure of research quality in organization studies (SSRN Scholarly Paper No. ID 1860279). Retrieved from Social Science Research Network website: https://papers.ssrn.com/abstract=1860279.

Bornmann, L., & Daniel, H. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.

Brown, L. (2003). Ranking journals using social science research network downloads. Review of Quantitative Finance and Accounting, 20(3), 291–307.

Campanario, J. (2011). Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window. Scientometrics, 87(1), 189–204.

Cole, J., & Cole, S. (1973). Social stratification in science (First edition edition). Chicago: University of Chicago Press.

Colledge, L., De Moya-Anegon, F., Guerrero-Bote, V., Lopez-Illescas, C., El Aisati, M., & Moed, H. (2010). SJR and SNIP: two new journal metrics in Elsevier’s Scopus. Serials, 23(3), 215–221.

Dahlstrom, R., Nygaard, A., & Crosno, J. L. (2008). Strategic, metric, and methodological trends in marketing research and their implications for future theory and practice. Journal of Marketing Theory and Practice, 16(2), 139–152.

Durieux, V., & Gevenois, P. (2010). Bibliometric indicators: Quality measurements of scientific publication. Radiology, 255(2), 342–351.

Egghe, L., & Rousseau, R. (2008). An h-index weighted by citation impact. Information Processing & Management, 44(2), 770–780.

Gardner, S., & Eng, S. (2005). Gaga over Google? Scholar in the social sciences. Library Hi Tech News, 22(8), 42–45.

Garfield, E., & Sher, I. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195–201.

Glänzel, W., & Moed, H. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.

González-Pereira, B., Guerrero-Bote, V., & Moya-Anegón, F. (2009). The SJR indicator: A new indicator of journals’ scientific prestige. ArXiv: 0912.4141 [Physics]. Retrieved from http://arxiv.org/abs/0912.4141.

Groot, T., & García-Valderrama, T. (2006). Research quality and efficiency: An analysis of assessments and management issues in Dutch economics and business research programs. Research Policy, 35(9), 1362–1376.

Guerrero-Bote, V., & Moya-Anegón, F. (2012). A further step forward in measuring journals’ scientific prestige: The SJR2 indicator. Journal of Informetrics, 6(4), 674–688.

Haddad, K., Singh, G., Sciglimpaglia, D., & Chan, H. (2014). To what extent do articles published in other than “top journals” have impact on marketing? European Journal of Marketing, 48(1/2), 271–287.

Harzing, A. (2017, September). Journal quality list. Retrieved December 19, 2017, from Harzing.com website: https://harzing.com/resources/journal-quality-list

Harzing, A., & van der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41–46.

Hassan-Montero, Y., Guerrero-Bote, V., & De-Moya-Anegón, F. (2014). Graphical interface of the Scimago journal and country rank: An interactive approach to accessing bibliometric information. Interfaz Gráfica Del SCImago Journal & Country Rank: Un Enfoque Interactivo Para Acceder a La Información Bibliométrica., 23(3), 272–278.

Hirsch, J. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

Hoepner, A., & Unerman, J. (2012). Explicit and implicit subject bias in the ABS Journal Quality Guide. Accounting Education, 21(1), 3–15.

Holden, G., Rosenberg, G., & Barker, K. (2005). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions. Social Work in Health Care, 41(3/4), 67–92.

Hult, G., Reimann, M., & Schilke, O. (2009). Worldwide faculty perceptions of marketing journals: Rankings, trends, comparisons, and segmentations. Global Edge Business Review, 3(3), 1–23.

Hussain, S. (2015). Journal list fetishism and the ‘sign of 4’ in the ABS guide: A question of trust? Organization, 22(1), 119–138.

Jubb, M. (2016). Peer review: The current landscape and future trends. Learned Publishing, 29(1), 13–21.

Kelly, J., Sadeghieh, T., & Adeli, K. (2014). Peer review in scientific publications: Benefits, critiques, & a survival guide. EJIFCC, 25(3), 227–243.

Khabsa, M., & Giles, C. (2014). The number of scholarly documents on the public web. PLOS ONE, 9(5), e93949.

Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

Kostoff, R. (1998). The use and misuse of citation analysis in research evaluation. Scientometrics, 43(1), 27–43.

Lee, C., Sugimoto, C., Zhang, G., Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.

Lehmann, S., Jackson, A., & Lautrup, B. (2006, December 21). Measures for measures [Comments and Opinion]. https://doi.org/10.1038/4441003a

Li, J., Sanderson, M., Willett, P., Norris, M., & Oppenheim, C. (2010). Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments. Journal of Informetrics, 4(4), 554–563.

Lowry, P., Humphreys, S., Malwitz, J., & Nix, J. (2007). A Scientometric study of the perceived quality of business and technical communication journals (SSRN Scholarly Paper No. ID 1021608). Retrieved from Social Science Research Network website: http://papers.ssrn.com/abstract=1021608.

McGuigan, N. (2015). The impact of journal rankings on Australasian accounting education scholarship – A personal view. Accounting Education, 24(3), 187–207.

Mingers, J., & Harzing, A. (2007). Ranking journals in business and management: A statistical analysis of the Harzing data set. European Journal of Information Systems, 16(4), 303–316.

Mingers, J., & Willmott, H. (2013). Taylorizing business school research: On the ‘one best way’ performative effects of journal ranking lists. Human Relations, 66(8), 1051–1073.

Mingers, J., & Yang, L. (2017). Evaluating journal quality: A review of journal citation indicators and ranking in business and management. ArXiv: 1604.06685 [Cs]. Retrieved from http://arxiv.org/abs/1604.06685.

Moed, H. (2009). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277.

Moussa, S., & Touzani, M. (2010). Ranking marketing journals using the Google Scholar-based hg-index. Journal of Informetrics, 4(1), 107–117.

Orduna-Malea, E., Ayllón, J., Martín-Martín, A., & Delgado López-Cózar, E. (2015). Methods for estimating the size of Google Scholar. Scientometrics, 104(3), 931–949.

Ozbilgin, M. (2009). From journal rankings to making sense of the world. Academy of Management Learning & Education, 8(1), 113–121.

Page, L., Brin, S., Motwani, R., & Winograd, T. (1998). The PageRank citation ranking: Bringing order to the web. Technical Report. Stanford InfoLab.

Peters, K., Daniels, K., Hodgkinson, G., & Haslam, S. (2014). Experts’ judgments of management journal quality: An identity concerns model. Journal of Management, 40(7), 1785–1812.

Pidd, M., & Broadbent, J. (2015). Business and management studies in the 2014 research excellence framework. British Journal of Management, 26(4), 569–581.

Radicchi, F., Fortunato, S., & Castellano, C. (2008). Universality of citation distributions: Toward an objective measure of scientific impact. Proceedings of the National Academy of Sciences. 105(45), 17268–17272.

Rebne, D., & Davidson, N. (1992). Understanding patterns of publishing activity in academic research occupations. Decision Sciences, 23(4), 944–956.

Rowlinson, M., Harvey, C., Kelly, A., & Morris, H. (2011). The use and abuse of journal quality lists. Organization, 18(4), 443–446.

Sangster, A. (2015). You cannot judge a book by its cover: The problems with journal rankings. Accounting Education, 24(3), 175–186.

Schreiber, M. (2007). Self-citation corrections for the Hirsch index. EPL (Europhysics Letters), 78(3), 30002.

Schreiber, M. (2008). A modification of the h-index: The hm-index accounts for multi-authored manuscripts. Journal of Informetrics, 2(3), 211–216.

Seglen, P. (1992). The skewness of science. Journal of the American Society for Information Science, 43(9), 628–638.

Serenko, A., & Bontis, N. (2009). Global ranking of knowledge management and intellectual capital academic journals. Journal of Knowledge Management, 13(1), 4–15.

Tahai, A., & Meyer, M. (1999). A revealed preference study of management journals’ direct influences. Strategic Management Journal, 20(3), 279–296.

Taylor, J. (2011). The assessment of research quality in UK universities: Peer review or metrics? British Journal of Management, 22(2), 202–217.

Theoharakis, V., & Hirst, A. (2002). Perceptual differences of marketing journals: A worldwide perspective. Marketing Letters, 13(4), 389–402.

Tomer, C. (1986). A statistical assessment of two measures of citation: The impact factor and the immediacy index. Information Processing & Management, 22(3), 251–258.

Tourish, D., & Willmott, H. (2015). In defiance of folly: Journal rankings, mindless measures and the ABS guide. Critical Perspectives On Accounting, 26(1), 37–46.

Wainer, J., Goldenstein, S., & Billa, C. (2011). Invisible work in standard bibliometric evaluation of computer science. Commun. ACM, 54(5), 141–146.

Waltman, L., van Eck, N., van Leeuwen, T., & Visser, M. (2013). Some modifications to the SNIP journal impact indicator. Journal of Informetrics, 7(2), 272–285.

Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R., Kain, R., Kerridge, S., Thelwall, M., Tinkler, J., Viney, I., Wouters, P., Hill, J., & Johnson, B. (2015, July). The metric tide: Report of the independent review of the role of metrics in research assessment and management. Retrieved December 25, 2017, from Higher Education Funding Council for England website: http://www.hefce.ac.uk/pubs/rereports/year/2015/metrictide/#alldownloads.

Zainuba, M., & Rahal, A. (2015). Assessing the validity of business and management journals ranking list: An alternative approach for determining journal quality. Annals of Management Science. 4(2), 1-28.

Downloads

Published

2019-12-04

Issue

Section

ABR Journal Articles