Impact factor JUMPS after the 2020 COVID-19 pandemic: a retrospective study in Dermatology journals.

BACKGROUND: The term "JUMPS" was used to describe the impact factor (IF) in an article published in PubMed in 2021, representing an increase of more than 40% of IF. AIMS: In this study, we aimed to compare the growth rate of IF JUMPS in Dermatology in the last 5 years, and particularly the effect of the 2020 COVID-19 pandemic.
METHODS: This study evaluated the growth rate (JUMP) in IF from 2016 to 2020. We used the Friedman and Wilcoxon signed ranks tests. We classified JUMPS in negative growth rate; Q1 to Q4 quartiles; and journals with > 100%. A 76-100% growth rate was observed in five (7%) journals, and twelve journals (17%) depicted a 51-75% percentage of change.
RESULTS: Several journals in the Dermatology category increased their IF by 50%. Repeated measures analyses showed a significant difference (p < .001).
CONCLUSION: Although we found journals with growth rates in the four quartiles, no journals depicted negative growth rates nor > 100% growth. Knowing the growing trends in this category might supplement the assessment of target journals for authors looking to submit their works.

Introduction

The interest in reporting on the impact factor (IF) in dermatology journals has been in the medical literature for almost 30 years [1, 2]. However, in the last years, when more technical articles have appeared commenting on self-citation and IF [3], the influence of article type [4], the impact factor as a double-edged sword (for editorial decisions) [5], and the percentage of growth of the IF for selected dermatology journals [6].

The number of citations an article achieves and the impact factor (IF) are the two most generally used metrics for evaluating a research paper’s quality in medical publications. In their decisions to submit their contributions, scientific authors continue to focus particularly value on the Journal IF [7]. The IF was established to let researchers compare journals of different sizes [8]. The Journal Citation Reports (JCR) introduced the journal IF for medical journals, administered by Clarivate Analytics via Web of Science and classified under the science citation index (SCI) and SCI Expanded (SCIE) [9].

The term “JUMPS,” applied to the IF, appeared published for the first time in PubMed in 2010, in the Int J Stroke; this journal experienced a 1-year increase of 43.6% (2.0 to 2.871, during 2008–2009) [10]. At the time of writing this manuscript (March–April 2022), the term IF JUMPS appeared again in PubMed in 2021, in an editorial of J Oral Pathol Med acknowledging that 2020 IF of that Journal had climbed to 4.235, equivalent to 69.7% (a record high for that journal and a significant increase from its 2019 IF of 2.495) [11].

Considering the information mentioned above, we aimed to compare the journals category of Dermatology, the IF, and their growing-rate percentage jump from 2015 to 2020; we believe knowing the growing trends of journals in this category might supplement the assessment of target journals for authors looking to submit their works.

Methods

Study design

The main objective of this study was to evaluate the achievement of journals in the Dermatology category on the Web of Knowledge [9] that recorded the IF values listed in the Journal Citations Reports (JCR) [12] for 5 years (2016–2020). Definitions of the selected bibliometrics from the Web of Knowledge have been recently published [7]. The report of the study followed the STROBE statement guidelines [13]. Because this retrospective study used public, historical data, it did not require the approval of an Institutional Review Board.

Search strategy

The search was performed on the Web Of Knowledge from the Clarivate platform, where the dermatology category was selected. The corresponding files were selected from 2016 to 2020, and an Excel file was downloaded.

Eligibility criteria

The journals included within the dermatology category and have an impact factor for more than three consecutive years were evaluated.

Data extraction

Database and percentages of change were calculated using Microsoft Excel v16.33 (Microsoft Corporation, Redmond, WA, USA). All analyses and graphical representations were performed using IBM® SPSS® Statistics software (version 27.0.1.0 IBM Corporation; Armonk, NY, USA). Boxplots and graphs were drawn with JMP Pro v16.1.0 (SAS Institute Inc., Cary, NC, USA). Statistical significance was indicated by p < 0.05 (two-tailed).

Impact factor measurement

The IF for a publication was calculated by dividing the number of times papers published in a journal were referenced in the previous 2 years by the number of articles published during the same period [14].

Journal selection and measured periods

We chose the IF values of journals in the JCR Science edition Dermatology category that coincidentally appeared between 2015 and 2020 (Table 1). Selected journals included at least three measures of IF in the selected period. The list of 69 chosen journals is available in the supplementary online-only file.

Table 1

IF for selected journals from 2016 to 2020 (the table represents all selected journals averaged together)

Year	N	Percentiles			IQR
	Valid	Median	75th	25th
2016 IF	62	1.696	2.627	1.109	1.518
2017 IF	63	1.925	2.788	1.327	1.461
2018 IF	64	2.090	3.087	1.365	1.722
2019 IF	67	2.156	3.164	1.380	1.784
2020 IF	69	2.875	4.015	1.841	2.174

IQR interquartile range; IF impact factor

Percentage of growing rate (IF JUMP)

We calculated each year IF percentage (%) change using the formula:

[(IF chosen year − IF previous)/ IF previous] × 100

We calculated five IF growth rates (2015–2016; 2016–2017; 2017–2018; 2018–2019; 2019–2020) for the selected journals; the growth rates of 69 selected journals during 2020 are available in the supplementary online-only file.

Statistical analysis

Changes in IF from 2015 to 2020

We assessed the IF and IF growing rates distribution using the Kolmogorov–Smirnov and Shapiro–Wilk tests; data sets with a non-normal distribution were reported using median, quartiles, and interquartile range (IQR). A significant difference between the IF values and IF growth rates for all the years was calculated using the Friedman test. The specific difference in 2019–2020 IF (to detect JUMPS during the COVID-19 pandemic) was assessed with the Wilcoxon signed ranks test.

Identification of IF JUMPS in growing rates

We calculated the % change in IF (growing rates) from 2015 to 2020 IF; after evidence of a non-normal distribution of IF in each year, the significant difference between years was explored with the Friedman Test, and the specific difference between 2019 and 2020 growing rate (to detect JUMPS during the COVID-19 pandemic) was assessed with the Wilcoxon signed ranks test.

Identification of journals with the highest growth rates

After demonstration of a significant difference in growing rates between 2019 and 2020, we grouped the growing rates (% change) of 2020 in 6 groups based on their quartile values: level 1 = negative grow; level 2 = 1–25%; level 3 = 26–50%; level 4 = 51–75%; and level 5 = 76–100%; and level 6 > 100%.

Software

All analyses and graphical representations were carried out using the IBM® SPSS® Statistics software (version 27.0.1.0 IBM Corporation; Armonk, NY, USA). The percentage of change was calculated using Microsoft Excel v16.33 (Microsoft Corporation, Redmon, WA, USA). Box-plot and graphs were drawn with JMP Pro v16.1.0 (SAS Institute Inc., Cary, NC, USA). Statistical significance was indicated by a p-value < 0.05 (two-tailed).

Results

Impact factor trends

The IF from 2015 to 2020 depicted non-normal distribution for all years, p < 0.001. There was a statistical difference between the impact factor values for all the years (2016–2020); χ2 = 106.295, p < 0.001. In addition, there was a significant difference in the IF of journals between 2019 and 2020; Z =  −6.990, p < 0.001. Table 1 shows the quartiles and IQR for selected journals; Fig. 1 shows the IF data distribution of selected journals.

Fig. 1

Comparison of box plots showing the Impact Factor evolution for journals in the Dermatology category from the Web of Knowledge from 2016 to 2020

Growing rates trends

The growing rates from 2016 to 2020 (Table 2) also depicted non-normal distribution for all years, p < 0.001, with a statistical difference between the rates values for all the years (2015–2020); χ2 = 36.8, p < 0.001. Also, there was a significant difference in the growth rates of journals, specifically between 2019 and 2020; Z =  − 5.189, p < 0.001. Table 2 shows the quartiles and IQR for selected journals; Fig. 2 shows the data distribution of the IF growing rates.

Table 2

IF growing rates of selected journals from 2016 to 2020 (the table represents all selected journals averaged together)

Year	Percentiles			IQR
	Median	75th	25th
GR-IF 2016	9.194	19.461	−4.470	23.931
GR-IF 2017	6.175	24.325	−6.879	31.204
GR-IF 2018	2.957	17.327	−11.452	28.779
GR-IF 2019	6.762	18.449	−7.652	26.101
GR-IF 2020	32.372*	52.041	17.577	34.464

IQR interquartile range, *Impact factor JUMP, GR-IF growth rate impact factor

Fig. 2

Comparison of box plots showing the growth rate in Impact Factor for journals in the Dermatology category from the Web of Knowledge from 2016 to 2020

Percentage of change for journals in 2020

From 69 journals evaluated, a negative growth was observed in 4 (6%) of journals; 23 (33%) journals showed 1–25% positive change; a 26–50% growth was observed in 23 (33%); 12 journals (17%) depicted a 51–75% percentage of change; a 76–100% growth rate was observed in 5 (7%) journals; and none of the journals had a growth > 100%. The complete list of journals grouped based on the % change of their 2020 IF is shown in Table 3.

Table 3

Classification of journals based on their IF growth rates in 2020 (total N = 67)

Negative growthN = 4	Indian Journal Of Dermatology Venereology & Leprology; Veterinary Dermatology; Indian Journal Of Dermatology; Leprosy Review
1–25% growth N = 23	Journal Of Investigative Dermatology; Journal Of The European Academy Of Dermatology And Venereology; Dermatitis; Journal Of Dermatological Science; Acta Dermato-Venereologica; Mycoses; Experimental Dermatology; Skin Pharmacology And Physiology; Dermatologic Clinics; European Journal Of Dermatology; International Wound Journal; Dermatology And Therapy; Journal Of Tissue Viability; Dermatologic Therapy; Skin Research And Technology; Journal Of Cutaneous Medicine And Surgery; Journal Of Wound Care; Journal Of Burn Care & Research; Journal Of Cutaneous Pathology; Wounds-A Compendium Of Clinical Research And Practice; Annals Of Dermatology; Acta Dermatovenerologica Croatica; Hautarzt
26–50% growth N = 23	Journal Of The American Academy Of Dermatology; JAMA Dermatology; British Journal Of Dermatology: American Journal Of Clinical Dermatology; Dermatology; Pigment Cell & Melanoma Research; Lasers In Surgery And Medicine; Journal Of Dermatology; Wound Repair And Regeneration; Melanoma Research; Clinics In Dermatology; Dermatologic Surgery; Photodermatology Photoimmunology & Photomedicine; Archives Of Dermatological Research; Burns; Clinical Cosmetic And Investigational Dermatology; Journal Of Drugs In Dermatology; International Journal Of Lower Extremity Wounds; Postepy Dermatologii I Alergologii; Pediatric Dermatology; American Journal Of Dermatopathology; Cutis; Dermatologica Sinica
51–75% growth N = 12	Contact Dermatitis; Journal Der Deutschen Dermatologischen Gesellschaft; Burns & Trauma; Advances In Wound Care; Journal Of Dermatological Treatment; Australasian Journal Of Dermatology; International Journal Of Dermatology; Journal Of Cosmetic Dermatology; Advances In Skin & Wound Care; Anais Brasileiros De Dermatologia; Annales De Dermatologie Et De Venereologie; Hong Kong Journal Of Dermatology & Venereology
76–100% growth N = 5	Clinical And Experimental Dermatology; International Journal Of Cosmetic Science; Journal Of Cosmetic And Laser Therapy; Giornale Italiano Di Dermatologia E Venereologia; Journal Of Cosmetic Science

Discussion

Even though it seems contradictory, this manuscript assessed the ability of the IF to detect JUMPS, not endorsing its usage. Several indexed journals compared dermatology journals in 2020 [6, 15, 16]. An unprecedented surge in scientific production has been observed for this speciality during the COVID-19 pandemic [17]. When authors decide where to submit their works, the IF rating of journals becomes a decisive factor. Readers should be aware that the IF has been misused as a proxy for article quality [18]. Usually, researchers look for journals with the highest impact factor instead of journals with the best audience for their research [19]. Our group has published before about the performance of bibliometrics in other specialities (Gastroenterology [20], Neurosciences [21] and Radiology [22]); because our first author will be a fellow of a dermatology program soon, we decided to explore the JUMPs in the IF in this category. For the dermatology community to recognize the new cutaneous associations in patients with coronavirus disease in 2019, dermatologists made significant contributions to COVID-19 and SARS-CoV-2 research during the pandemic [23]. Consequently, during the past 2 years, the trend in research publications in the dermatology field has been focused more on COVID-19 [24].

This study revealed significant IF JUMPS during the 2020 COVID-19 pandemic in selected Dermatology journals. We also grouped journals by quartiles based on their IF growth rate; we discovered that none journals during 2020 increased their IF growth rate by more than 100%.

Some journals have grown since 2016, but it has been much higher since 2020. In 2016, there was a journal with growth > 100%; in 2017, two journals with growth close to 100%, and in 2020, more than 50% of the journals grew > 50%.

Interestingly, the were only four journals with a negative growth rate in 2020; we do not have a confirmed explanation that explains the reason for their negative growth. We hypothesized that because these journals are highly specialized in the topics they published (veterinary dermatology or leprosy, as examples), these might not be common topics for most authors during the covid-19 pandemic. For that reason, those journals received fewer citations.

Academic writers generally agree that the general goal of scientific authors is to produce high-quality, high-impact research (basic or clinical), a task that necessitates endless after-hours work; nevertheless, only 10 to 14% of published information remains valid, and most articles are never cited again, except by the authors who originally published them [25]. Original articles, reviews, editorials, letters to the editor, and other scientific reports are published in the research process. Their influence on authors, journal profiles, and research institutions can be beneficial or harmful [26].

Although recent publications have displayed that the IF has been surpassed by other bibliometrics in journals in the Gastroenterology and Hepatology [20]; Neurosciences [21]; and Radiology, Nuclear Medicine & Medical Imaging [22] categories, we recognize that the success of researchers is nowadays judged by the number of publications they have published in high IF journals [27].

Authors interested in the IF value

The IF value is presently being pursued by several researchers: A) authors who are interested in how to choose a journal that will provide their articles more exposure; B) librarians who identify IF as a selection criterion for a title of greater scientific importance that should be included in the scientific collections of the institutions for which they work; C) medical editors regularly utilize it as a performance metric for their Journals; they agree that IF is an index of impact in fundraising, a way of rating their journals in comparison to their peers, and the attractiveness of high-quality papers to be published [28-31]. D) journals per se utilize the IF to “promote” their reputation and persuade potential writers to submit high-quality articles [32]. E) Funding organizations utilize IF to identify high-quality researchers and universities that can best address the needs of institutions. [33]. Academic institutions’ promotion committees frequently use the IF to assess the quality of applicants’ publications for promotion and tenure; departmental chairs may use it in the hiring and evaluation of recruits [34]. Journals with a more significant impact factor are often considered more prestigious than those with a lower impact factor [35].

Limitations

Several limitations of this study need to be addressed: a detailed explanation of each bibliometric or comparison between them is beyond the scope of this article; we only report the changes in IF. Other factors to consider: each medical speciality has its own IF threshold; for example, a publication in oncology can have an IF that is more than 30 times higher than the similar figure in forensic medicine [36]. Every journal has different citations, and even the most prestigious periodicals have some pieces that are never referenced [37]. Citations are not evenly distributed, with less than 20% of publications accounting for more than half of the total citations [38]. Despite these facts, the IF is still used to assess science's worth since it significantly rewards individual scientists and publications [39].

Although, if we recognize that in the 17 journals that we reported “JUMPS,” > 50% are journals that typically have an IF from 0.5 to 5, and regarding the periodicity of publication, we see that they are not monthly but bimonthly or quarterly. So there is no clearly defined pattern, and our number of observations is too small to apply statistical analysis. Interestingly, the journals also belong to free or open access journals, without a clear pattern or preference.

Additional elements influencing “citation” (self-citation, semi-mandatory, and mandatory citation) include more extended periods, the number of papers published in each issue, the circulation of each journal, and a slew of other variables that might influence citation calculations and, eventually, the IF. We did not include these possible confounding factors, as the Web of Knowledge does not consider them. We acknowledge that normalizing journal citations by their article count is desirable. However, we used the raw data from the JCR when writing this study. Other criteria that may influence where an author submits their work were mainly outside the scope of this study: topic, affiliation with society, geography, rejection by earlier submission, familiarity with the submission and revision process, turnaround time, and invitation by editors. Readers should be aware that the JCR includes approximately 171 categories in the sciences and 54 in the social sciences; future studies presenting IF JUMPS in other specialities would be desirable.

Future directions

Evaluating publications’ scientific effects (impact) using bibliometrics is a complex, multi-dimensional process. As a result, implementing a single bibliometric index to rank, assess, and value journals is ineffective. Readers should evaluate scientific publications on their merits rather than relying on the impact factor [40]. Preferably, multiple metrics with complementary features provide a comprehensive view of journals and their relative placements in their fields [41].

Conclusions

Knowing the growth rate of IF within each category and discipline can be used as a performance metric for dermatology authors. However, authors must first identify what metric(s) they will consider before submitting their manuscripts and which metric(s) they will use to evaluate and select the fittest journal and track their performance over the years. New metrics to assess the influence, growth, and publishing trends of acceptable research issues, and not only a high-IF growth rate might be a new trend in dermatology research.