Will the increase in publication volumes "dilute" prestigious journals' impact factors? A trend analysis of the FT50 journals.

In the wake of the unprecedented global scientific output boom, are top-tier journals such as the FT50 journals following suit? If these prestigious journals consistently increase their publication volumes, will their impact factors be affected? Drawing on the Mann-Kendall trend test method, this study analysed time series trends of the FT50 journals' annual publication volumes and impact factor ratios (IFR) over a 15 year period. The results indicate that half of the FT50 journals have consistently increased their publication volumes over the years. Although to increase publication volumes is riskier than to stay put, it has a significantly higher probability of increasing the IFR, and therefore keeping pace with other top journals. However, the expanding of publication volumes must be carried out cautiously, as the study also finds that growing too fast may lead to opposite effects. © Akadémiai Kiadó, Budapest, Hungary 2020.

Introduction

The global scientific output has grown considerably in recent years (Bhattacharya and Kaul 2015; Collyer 2018; Gui et al. 2019). This can be easily verified by checking the number of indexed scholarly publications in Scopus or Web of Science databases—most disciplines or topics are growing fast and continuing to mount. However, many prestigious journals seem less enthusiastic in expanding their publication volumes. For example, two of the world’s most prestigious journals, Science and Nature, both published fewer research articles in 2019 than in 1999. Some may argue that to expand publication volumes will reduce the quality of articles and thereby compromise journals’ reputation (Shen and Björk 2015). An extreme example is some predatory journals (Demir 2018; Xia et al. 2015) which exponentially increased their publication volumes in just a few years, but were later dropped from multiple databases due to various quality and ethical issues. Nevertheless, this study is not to discuss an exponential increase scenario like those predatory journals, on the contrary, it aims to investigate: If prestigious journals, i.e., those top-ranked, world-renowned journals in their respective areas or disciplines, gradually increase their publication volumes, will their impact factors be affected? In other words, will the increase in publication volumes “dilute” prestigious journals’ impact factors? To answer the question, the following analysis was conducted.

Data

The Financial Times’ top 50 journals list1(FT50) released by the Financial Times newspaper is a widely used and influential journal list among business schools worldwide (Vidgen et al. 2019). The number of publications in the FT50 journals is an important indicator to measure the performance of business schools, business and management disciplines as well as researchers. Previously 45 journals on the list, the Financial Times conducted a review in May 2016 and five new journals were added and the list has remained unchanged since then.

Journal impact factors (JIF) were obtained from Clarivate Analytics’s annual journal citation reports (JCR). Fifteen consecutive years’ (2005–2019) JIFs were retrieved. The publication volume data were obtained from JCRs’ “citable items” column which only includes research articles and reviews that a journal has published each year. Because a JIF of a certain year is calculated based on previous 2 years’ publications, to facilitate subsequent analysis, the citable items data were retrieved from 2004 to 2018. Note that eight journals’ JIFs are not complete, for example, Journal of Accounting Research’s JIFs are only available since 2007.

Methods

This study uses the Mann–Kendall (M–K) trend test (Kendall 1975; Mann 1945) to examine the increasing and decreasing trends of the time series-based publication volume/JIF data. The M–K test is a non-parametric test used to statistically assess if there is a monotonic upward or downward trends in Y values over time of the data collected. In an M–K test, a positive or negative Tau-value (τ) value (varies between  − 1 and 1) in conjunction with a statistically significant two-sided p value (< 0.05) indicate a consistent upward or downward trend of the observed data (Joseph et al. 2013). The τ value is approximately normal when the sample size n ≥ 8 (Kendall 1975; Mann 1945; Neeti and Eastman 2011), while some studies suggest 10 to be the optimal threshold (e.g., Gocic and Trajkovic 2013). The sample sizes of this study are all ≥ 10, such that the M–K test is suitable for this study. The M–K test was conducted using “Kendall” R package (McLeod 2015).

Considering most FT50 journals’ JIFs have grown considerably during the study period (mean = 293.8%, SD = 146.9%), to directly test the raw JIFs will give inaccurate results that most journals’ impact factors are rising despite the variation of their publication volumes. To tackle the problem, given the fact that the FT50 journals all belong to business or relevant disciplines, this study proposes a normalized impact factor ratio (IFR). The IFR is calculated by dividing a journal’s JIF by FT50 journals’ average JIF of the corresponding year. The IFR provides a cross-sequential perspective for the analysis, such that both longitudinal and cross-sectional variations of JIFs can be observed.

Results

The M–K test results for the FT50 journals are presented in Table 1, a summary is provided in Table 2. As is shown in Table 1, τ1 represents trends of journals’ publication volumes from 2004 to 2018, while τ2 suggests trends of journals’ IFRs from 2005 to 2019. Journals are ranked by the τ1 value, which means journals with a higher ranking exhibited a higher level of the upward trend in terms of annual publication volumes. Among the 50 journals, 25 journals (No.1 to No.25) have consistently increased their publication volumes, while only three journals (No.48 to No.50) show a significant decrease trend. This is somewhat against my intuition that prestigious journals tend to stabilize their publication volumes. Actually, the average publication volume of the FT50 journals has grown from 54.3 articles per journal in 2004 to 78.3 in 2018.

Table 1

Mann–Kendall trend test for the FT50 journals

No	Journal (abbreviation)	τ1	τ2	No	Journal (abbreviation)	τ1	τ2
1	Review of Accounting Studies (RAS)	0.89**	− 0.56**	26	Manufacturing & Service Operations Management (MSOP)	0.33	− 0.12
2	Strategic Management Journal (SMJ)	0.83**	0.00	27	Organizational Behavior and Human Decision Processes (OBHDP)	0.32	− 0.31
3	Review of Finance (ROF)	0.81**	− 0.28	28	Econometrica (ECO)	0.27	− 0.49*
4	Research Policy (RP)	0.78**	0.60**	29	Journal of Consumer Psychology (JCP)	0.25	− 0.29
5	Journal of Management (JOM)	0.77**	0.51**	30	Journal of Marketing Research (JMR)	0.24	− 0.26
6	Production and Operations Management (POM)	0.77**	− 0.35	31	Accounting Organizations and Society (AOS)	0.22	0.22
7	Contemporary Accounting Research (CAR)	0.75**	0.22	32	Journal of the Academy of Marketing Science (JAMS)	0.19	0.71**
8	Management Science (MAS)	0.72**	0.09	33	Entrepreneurship Theory and Practice (ETP)	0.13	0.68**
9	Administrative Science Quarterly (ASQ)	0.70**	0.40*	34	Organization Studies (OST)	0.08	0.33
10	Strategic Entrepreneurship Journal (SEJ)	0.68*	0.24	35	Journal of Political Economy (JPE)	0.05	− 0.33
11	Review of Economic Studies (RES)	0.66**	0.14	36	Journal of International Business Studies (JIBS)	0.03	0.62**
12	The Accounting Review (TAR)	0.64**	− 0.27	37	Marketing Science (MKS)	− 0.03	− 0.70**
13	Human Resource Management (HRM)	0.61**	0.46*	38	Journal of Consumer Research (JCR)	− 0.04	0.14
14	MIS Quarterly (MISQ)	0.59**	− 0.56**	39	Journal of Accounting Research (JAR)	− 0.09	− 0.08
15	Academy of Management Journal (AMJ)	0.57**	− 0.16	40	Journal of Management Information Systems (JMIS)	− 0.10	− 0.14
16	Journal of Financial and Quantitative Analysis (JFQA)	0.57**	− 0.18	41	Journal of Marketing (JM)	− 0.14	– 0.31
17	Information Systems Research (ISR)	0.56**	− 0.47*	42	Journal of Operations Management (JOPM)	− 0.20	0.31
18	American Economic Review (AER)	0.54**	0.46*	43	MIT Sloan Management Review (MSMR)	− 0.20	0.56**
19	Journal of Financial Economics (JFE)	0.52**	− 0.28	44	Quarterly Journal of Economics (QJE)	− 0.20	0.23
20	Journal of Accounting and Economics (JAE)	0.46*	− 0.48*	45	Journal of Applied Psychology (JAP)	− 0.24	− 0.28
21	Review of Financial Studies (RFS)	0.44*	0.07	46	Operations Research (OR)	− 0.35	− 0.45
22	Organization Science (OSC)	0.44*	− 0.41*	47	Harvard Business Review (HBR)	− 0.36	0.31
23	Human Relations (HR)	0.43*	0.77**	48	Academy of Management Review (AMR)	− 0.38*	− 0.04
24	Journal of Business Venturing (JBV)	0.41*	0.73**	49	Journal of Management Studies (JMS)	− 0.48*	0.37*
25	Journal of Business Ethics (JBE)	0.37*	0.79**	50	Journal of Finance (JOF)	− 0.63**	0.11

τ1: trends of publication volumes (2004–2018); τ1: trends of impact factor ratios (2005–2019). * p < 0.05, ** p < 0.01 (two-sided). Journals are ranked by τ1

Table 2

Summary of Mann–Kendall trend test for the FT50 journals

Publication volume trend	IFR upward	IFR downward	IFR n.s	Total
Upward (all)	8	5	12	25
Upward (τ1 > 0.6)	4	1	8	13
Downward	1	0	2	3
Not significant	4	2	16	22

n.s. = p value not significant

Among the 25 journals with the upward trend of publication volumes, eight journals (PR, JOM, ASQ, HRM, AER, HR, JBV, JBE) also exhibit an upward trend of IFRs, the mean τ2 value is 0.590. While five journals’ (RAS, MISQ, ISR, JAE, OSC) IFRs were on a downward trend (mean τ2 =  − 0.496). The remaining 12 journals did not show a significant upward nor downward trend of IFRs. If we take a closer look, to focus on the 13 high τ1 value journals (τ1 > 0.6), four of them were on an upward trend of IFRs, whereas only one journal (RAS) was on a downward trend. However, when τ1 > 0.8, none of the three journals (RAS, SMJ, ROF) has a positive τ2 value. Which implies that the increase of publication volumes cannot be too dramatic.

The results indicate that most of the FT50 journals which continually increased their publication volumes over the years, their positions among the prestigious journals have not been weakened in terms of impact factor ratios. The upward IFR journals outweighed the downward journals in terms of number and increasing/decreasing IFR trend. This advantage is more pronounced among the journals with mid to high upward trend of publication volumes (0.6 < τ1 < 0.8). Figure 1 shows two plots of an example journal (RP). The dotted trend lines indicate that both publication volumes and IFRs are on upward trends over the years, albeit a few fluctuations.

Fig.1

Publication volume and IFR trends of Research Policy journal

In addition, this study also finds a correlation between changes in publication volumes and IFRs. Among the 22 journals which did not show a significant upward/downward trend of publication volumes, six journals have their IFR trends changed (27.3%), only about half of those journals with an upward trend of publication volumes (52.0%).

Discussion

This study examines the impacts of prestigious journals which have gradually and consistently increased their publication volumes on their relative impact factors among the “elite squad” (FT50). The results indicate that although to increase publication volumes is riskier than to stay put, it has a significantly higher probability of increasing the impact factors. If the increase of publication volumes is consistent and significant (but not too dramatic), a decrease of impact factors, or, “dilute” is unlikely.

The results have some implications for top-tier journals which could cautiously consider increasing their publication volumes in order to better facilitate scholarly communication. Especially, during these extraordinary COVID-19 times, to publish in prestigious journals is an effective approach for disseminating new knowledge and tackling the pandemic “infodemic” (Rovetta and Bhagavathula 2020; Tangcharoensathien et al. 2020).

This study also has some limitations which can be a source of future studies. For example, this study only focuses on top-ranked journals in business and management areas, and for more generalisable conclusions, it should be extended to other disciplines such as sociology, engineering and medical science. Secondly, this study did not differentiate the subfields of the FT50 journals. Further analysis could calculate IFRs based on subfields such as marketing and accounting, the results may be different. Lastly, the 15 year study period is still limited for identifying further evidence. Twenty years or longer would be ideal for future research.