Research on Covid-19: a disruptive phenomenon for bibliometrics.

The Covid-19 pandemic has been the highest disruptive event in the world recent history. Worldwide academic research on this topic has led to an explosion of scientific literature, never seen before. Bibliometrics provide methods to illustrate this exceptional phenomenon in academic publications. The objective of this paper is to analyze the Covid-19 research from a bibliometric perspective and to study the impact of the publication explosion on bibliometric indicators. The present study shows how an exceptional phenomenon has a disruptive impact on bibliometric indicators, such as the h-index and the Journal Impact Factor. The higher the specialization, the higher the possible impact of a disruptive phenomenon. In applied sciences, more important than the research or the discipline, the specific theme of the research is crucial for citations of articles and for their impact. The salience of the topic, the magnitude of the problem at study and the urgency to find solutions are drivers for citations. The study of the Covid-19 research illustrates the relativity of indicators and the need for context. The present study also confirms the plead for responsible metrics of the San Francisco Declaration on Research Assessment (DORA). © Akadémiai Kiadó, Budapest, Hungary 2021.

Introduction

The global crisis following the outbreak of the Covid-19 pandemic has been the highest disruptive event in the world recent history. It deeply affects the world economy and the daily life of all citizens. Many businesses have endured drastic consequences with sudden disruptive changes. People had to adopt new patterns of behaviour. In the search of solutions to this pandemic, this crisis has led to an unprecedent mobilization of the academic community for advancing knowledge on the virus and its cure (Haghani & Bliemer, 2020). According to the New York Times, never before have so many researchers all over the world focused so urgently on one single topic (Apuzzo & Kirkpatrick, 2020).

These research endeavors resulted in a multitude of scholarly publications in various aspects related to basic knowledge on the disease, clinical-healthcare and public health care measures (Belli et al., 2020). Academic research led to an unprecedent burgeoning of scientific publications on one single topic with all its aspects on different disciplines of medicine, immunology and public health. This uncommon phenomenon has been qualified in Science as one of “the biggest explosions of scientific literature ever” (Brainard, 2020). Just as for the race to a vaccine, also academic journals have reduced the time for review and increased the speed of publication schedules. The commitment of publishers to open the access to their publication and the expansion of preprints publications have resulted in a multiplication of citations as never before.

The objective of this paper is to analyze this exceptional phenomenon from a bibliometric perspective and to study the impact of the publication phenomenon on bibliometric indicators.

The structure of the paper is as follows. Following a brief literature overview and method section, the set of publications on Covid-19 are positioned and analyzed on the citation distribution of the major contributions in the literature. The bibliometric data of the main journals that have contributed in this Covid-19 research are compared. Then, the analysis is extended to a few bibliometric indicators, especially the h-index, the average ha-index, the journal impact factor (JIF) and the Immediacy Index.

Literature study

Despite the recent character of the phenomenon, a first few bibliometrics studies on Covid-19 research have already been performed only after a few months. Those studies recognize the magnitude of the scientific research in academic literature. Da Silva et al. (2021) analyze the publishing volumes in databases. Di Girolamo and Reynders (2020) attempt to establish a classification of the published articles in search of their characteristics. Belli et al. (2020) focus on the scientific collaboration and open access of Covid-19 research. Cheng, Cao and Liao (2020) reveal a network visualization to explore the main research themes of Covid-19. Tovstiga and Tovstiga (2020) study the Covid-19 pandemic from a knowledge and learning perspective. A few bibliometrics studies compare the research on Covid-19 pandemic with other coronavirus outbreaks as SARS (Haghani & Bliemer, 2020), the swine influenza pandemic (Di Girolamo & Reynders, 2020) or other international public health emergencies (Zhang et al., 2020). In the latter study, Zhang et al. (2020) confirm how the academic community always reacts quickly to public health emergencies resulting in a sharp increase in the number of publications.

Methods

In order to situate the phenomenon, the Web of Science data of the Covid-19 are compared with the data of another general topic ‘cancer’, on all-time bibliometric data and on the data for the year 2020. Cancer is one of the most important themes of research in medicine with a long tradition and application to a variety of medical subdisciplines. Cancer covers 2.5 million articles in the last 30 years, with 180.000 articles published in 2020. By the end of 2020, only 1 year after the appearance of the disease, 70.000 scientific articles on Covid-19 are listed in the Web of Science, with an estimate of 500.000 citations in less than one year time, compared to 100.000 citations for the 2020 cancer articles. Before 2020, only 13,500 articles have been published on the general theme of coronavirus.

Data of the citation distribution are assembled; first, on the topic ‘coronavirus’ over all years based on the keywords ‘coronavirus’, combined with 'SARS-cov-2' or 'Covid-19'; then for the coronavirus before 2020, so without the Covid-19 articles; then for the 2020 articles that cover mainly the Covid-19 topic; finally, for the search on keyword ‘cancer’ (see Table 1). A number of comparative data are collected for each of the four searches: the number of citations of the most cited article, of the articles ranked 10 and 100; then, the number of articles with more than respectively 1000, 500, 250 and 100 citations, the number of Highly cited papers (HCP) and hot papers in the Web of Science.

Table 1

Citation distributions

	n	n 2020	Max	m 2020	pl. 10	100	> 1000	> 500	> 250	> 100	HCP	Hot p
	n	n	cit	Cit	Cit	Cit	n	n	n	n	n	n
CORONA All	83,855	70,268	8488	8488	2541	636	50	159	468	1919	2080	520
CORONA n	13,576			2388	1236	373	15	59	193	1066	159	29
COVID-19		70,268		8488	2541	508	35	100	275	853	1921	491
Cancer	2,571,974	180,437	29,619	1446	188	55	2	3	5	44	958	49
NEJM	100,231	1524	17,183	5403	555	49	8	14	27	64	82	33
LAN	180,403	1473	34,828	8374	878	42	8	15	22	52	60	14
JAMA	96,464	1667	19,382	4991	477	44	5	10	18	44	37	9
NAT	204,293	2594	250,583	3492	250	31	2	4	10	24	156	27
SCI	139,933	2299	40,864	1287	234	25	1	3	9	21	107	30
BMJ	59,526	2999	44,187	756	125	10	0	2	2	12	17	3
LID	7050	577	2113	863	239	9	0	4	9	18	11	3
JMV	10,056	1013	740	551	270	21	0	2	11	29	68	1
JID	6744	469	571	350	102	5	0	0	1	11	21	0

All data have been retrieved on the Web of Science between 15th and 31st December 2020. It should be noted that not all citations for the current year 2020 are collected by then, as it usually takes a few months in the next year to collect all the data. The citation data for 2020 are thus somewhat underestimated.

The same data are looked up for the leading medical journals that have published much of the research on Covid-19: The Lancet, the New England Journal of Medicine (NEJM), JAMA Journal of the American Medical Association, the BMJ British Medical Journal, besides a few specialized journals as the Journal of Medical Virology (JMV), Lancet Infectious Diseases (LID) and the Journal of Infection (JI). Also the classic multidisciplinary journals Nature and Science have published a few important articles on Covid-19 (Table 1).

To put it in perspective, a further analysis is executed on a selection of three journals with different profile: NEJM as a leading medical journal, Science as a leading multidisciplinary journal, and JMV as a specialized journal in the field of the coronavirus.

Bibliometric indicators

The impact of the phenomenon can also be illustrated with bibliometric indicators such as the h-index, the average h-index ha, the journal impact factor and the immediacy-index. While the journal’s h-index has been defined as the highest number of papers a journal has published that have each received at least that number of citations (Braun et al., 2006, following Hirsch, 2005), the average ha-index is the largest number of papers published by a journal that have obtained at least ha citations per year on average (Fassin, 2020).

The journal impact factor has been presented by Garfield (Institute for Scientific Information ISI now Clarivate) as a measure of journal influence (Garfield, 1999). The 2-year journal impact factor is defined as the ratio between the number of citations received in that year for publications in that journal that were published in the two preceding years and the total number of "citable items" published in that journal during the two preceding years (Web of Science).

The immediacy index, also introduced by Garfield’s Institute for Scientific Information, is the ratio of the number of citations which a journal receives in its most recent complete year of publication to the number of source items published by that journal during the same interval. The immediacy index indicates “how rapidly a journal’s material is picked up and used” (Tomer, 1986, p. 251). It allows to evaluate the short term impact of the diffusion of scientific communications.

For each of the selected journals, the h-index is retrieved and the h-index for the 2020 articles; the number of publications on Covid-19 are complemented with the number of articles within the additional selections of the 10%-core and h-core—following the principles of the ghent-rating categorization proposed by Fassin (2018)—to allow to better distinguish the articles with impact (Table 2).

Table 2

h-index and indicators of medical journals

	h	h 2020	Covid	10%	h 259	covid	Place 1	JIF 2019	Rank JIF
NEJM	1079	75	309	132	25	7	31	74.699	2
LAN	806	69	435	159	16	10	4	60.390	5
JAMA	676	59	405	147	18	8	11	45.540	11
NAT	1346	52	334	97	9	2	204	42.779	14
SCI	1326	47	203	81	6	1	139	41.846	15
BMJ	354	34	1170	84	2	2	93	30.313	36
LID	221	37	179	75	9	10	16	24.446	59
JMV	132	56	67	20	0	12	2	5.022	1153
JID	102	10	763	177	10	11	2	2.021	5781

A few additional data are retrieved or calculated: the number of Covid-19 articles and information on their position in the ranking of that journal. The journal impact factor of the medical journals is looked up with the position in the Journal Citation Report (JCR) classification of the Web of Science. Their immediacy index is calculated.

A dynamic and prospective analysis

To illustrate the performance of the Covid-19 articles, a more detailed analysis is executed at the level of the individual journals, by comparison of the data for the last 5 years from 2015 to 2020 (as well as 2010 to have some more perspective) (Table 3). The evolution of the h-index and ha-index over those years is recalculated from the WoS data for the selected journals (Tables 4, 5). In order to further analyze the phenomenon, the h-indexes are compared for each cohort of citations of the last 5 years (Table 6).

Table 3

Evolution of citations

NEJM		max	max		y					y + 1					Average		y + 1
	n	cit	cit y + 1	> 1000	> 500	> 250	> 100	> 50	> 10	> 1000	> 500	> 250	> 100	> 50	> 500	> 250	> 100	> 50
2010	1693	97	367					8	117			8	46	127			14	48
2015	1551	333	886			4	16	39	162		16	32	89	164		16	41	89
2016	1606	316	682			1	7	18	139		4	13	61	156		4	18	61
2017	1494	238	804				4	16	132		4	13	78	153		4	18	78
2018	1530	291	888			1	10	25	122		7	22	79	148	1	10	31	79
2019	1462	461	1119			2	13	21	150	1	7	18	69	146	1	7	27	69
2020	1502	5403		8	14	27	64	93	248

Table 4

Evolution of the h-index

h	NJEM	Lancet	JAMA	JMV	Science	Nature
2010	721	529	445	95	926	925
2015	896	656	570	110	1109	1133
2016	929	684	589	114	1163	1176
2017	964	711	608	116	1203	1220
2018	1006	744	629	119	1243	1267
2019	1041	774	654	122	1289	1303
2020	1076	804	676	132	1326	1346

Table 5

Evolution of the ha-index

ha	NJEM	Lancet	JAMA	JMV	Science	Nature
2010	147	100	93	15	155	157
2015	176	122	100	15	180	187
2016	186	126	99	15	187	193
2017	190	131	103	14	194	202
2018	194	137	105	14	201	207
2019	200	143	109	14	208	213
2020	214	160	115	56	212	220

Table 6

Evolution of the h-index per cohort

h	NEJM	Science	JMV
2010	30	28	4
2015	44	33	3
2016	34	33	4
2017	34	30	6
2018	37	34	5
2019	34	32	4
2020	74	46	55

Based on the citation data, a simulation of the future JIF-factors of the three journals is performed in the coming period of 3 years. The data for the JIF of the last 3 years 2017, 2018 and 2019 are known and can be retrieved in Web of Science. Applying the formula of the definition of the JIF on the citation data of the Web of Science restricted to articles, review papers and proceeding papers, allows to approximately determine the JIF-figures for 2020 (Table 7). The number of citations of the articles of 2020 in 2020 are useful to extrapolate their number of citations in 2021. Three hypotheses are envisaged for the 2021 citations with a low, average and large growth, corresponding to a status quo, and an increase of 25% respectively 50%. For the number of citations of the 2019 papers, the same hypotheses are taken. For 2022, the projected number of articles is taken as the average of the period 2021. A decline in citation is assumed for 2022. This allows to forecast the range for the future JIF factors for 2021 to 2023.

Table 7

Evolution of the journal impact factor

	2017	2018	2019	2020e	2021e		2022e		2023e
NEJM	79.260	70.670	74.699	74	100	150	115	160	85
Lancet	53.254	59.102	60.390	59	88	130	100	120	70
JMV	1.988	2.049	2.021	2	11	16	13	20	10

Results: bibliometrics on Covid-19

The data of the number of publication and citations presented in Table 1 show the impact of the explosion of research on Covid-19. The highest number of citations for the 2020 articles reached 8488 on Covid-19, compared to 1446 for cancer. That places that most cited cancer article around position 1250 in the all-time ranking of articles on cancer, as the only 2020 articles that enters the h-core of the cancer articles. With 8488 citations, an article would be ranked at place 30 in the overall cancer ranking. In order to attain the top 1% in the citation distribution for the year 2020, an article on Covid-19 needs 116 citations, while only 11 for cancer. 10% of the articles on Covid-19 have at minimum 11 citations, compared to 3 for cancer.

The h-index for the 2020 sample climbs to 259 for Covid-19 and to 74 for cancer. 35 papers on Covid-19 gathered more than 1000 citations while only 2 articles on cancer realized this achievement. 15 articles on coronavirus before 2020 attained the 1000 citations. In fact, the figures of Table 1 effectively illustrate how fast the citation performance of the 2020 Covid-19 articles has overpassed the older coronavirus articles, and largely overpassed the 2020 articles on cancer.

Journals on Covid-19

The lower part of Table 1 provides the data of the 2020 citation distribution for the most important journals that have published on Covid-19. While these data give the counts for all their articles, nearly all articles with more than 500 citations are on Covid-19; multidisciplinary journals as Nature and Science have other highly-cited articles in 2020 on other topics or fields than on Covid-19.

The evolution of the total citations received each year shows a smooth, gradual growth at an average rate around 20–25% for Science and NEJM, from 300 to 400.000 citations per year for NEJM, and from 600.000 to 700.000 for Science, but more than a doubling for JMV from 9000 to 21.000 citations.

h-indexes and h-core

The next Table 2 presents the h-index for the selected journals and the h-index for the 2020 articles. Then follow, for each journal, the number of publications on Covid-19, complemented with the number of articles within the 10% (requiring for this sample at minimum 11 citations), and the number of articles within the h-core of Covid-19 (259).

The following column shows the number of Covid-19 articles that reach the journal’s h-core, and the place that the most cited Covid-19 article takes in the ranking of that journal obtained already within its incomplete first year of publication. Finally, the Journal Impact Factor is provided with the position of the medical journal in the Journal Citation Report1 (JCR) classification of the Web of Science. Three top medical journals are in the top 11 of the JCR, the multidisciplinary journals on places 14 and 15; two journals are situated around the 50th position, the two specialized journals in virology are placed in the range around the 1000th and the 5000th position.

Evolution of citations in the last 5 years

The data for the last 5 years from 2015 to 2020 (and 2010) are compared in a detailed analysis at the level of the individual journal. Table 3 shows the number of NEJM articles (of all type, including letters) per year, the number of citations for the highest cited article, the number of the maximum citations in the first 2 years. Then follow the number of NEJM articles, the number of NEJM articles in classes of more than respectively 1000 citations, 500, 250, 50, and 10 for the year of its publication, and for the year immediately after publication. It also shows the average number of citations per year for its first 2 years (for the first year, the average number of citations equals the total citations of the first year).

For the period from 2015 to 2019, the maximum citation for the first year reaches 300 for the first year, and 1100 for the first 2 years; only 8 articles reach more than 250 citations, while 27 articles already overpass that limit in 2020, all articles on Covid-19. In the second year, only one article has overpassed 1000 citations and 40 articles got more than 500 citations; 8 and respectively 14 articles have already reached those thresholds during their first year, and with the prudent hypotheses of doubling the number of citations during the second year, one can estimate that the first 2 years distribution may largely overpass all previous records.

A similar analysis can be executed for Science, where only two articles (of 2018) got more than 200 citations in their first year of publication, while already 8 articles have reached 250 citations in 2020, all Covid-19 articles. One article ends higher than 1000 in its first year (published on 13th March, so only 9 months) and 3 higher than 500 (published in March and in May). In previous years only 2 articles achieved more than 1000 citations by their second year and thus more than an average of 500 citations per year (the one 1107 in June 2015 and the other 1426 in June 2017 both on photovoltaic technology). Over the same period of 5 years, only 12 articles obtained more than 500 citations during their second year.

The comparative figures are even more impressive for the more specialized Journal of Virology. Until 2019, no JMV articles got more than 26 citations in their first year of publication, and only one JMV article obtained more than 50 citations after the second year, 59 exactly. For 2020, the most cited article reached 551 citations; 5 articles reached more than 250 citations and 40 papers more than 50 citations.

Evolution of h-indexes over the years

Table 4 exhibits the evolution of the h-index over the years from 2015 to 2020, recalculated from the WoS data for the selected journals, complemented with an older reference for the h-index in 2010.

In the period 2015–2019, the h-index of those journals raised by approximately 3% a year, and with a similar range in 2020, except for JMV where the rise of the h-index is around 6%. The steady growth is due to the global increase of the publications in the Web of Science (Hu, Leydesdorff & Rousseau, 2020) and to the intensified usage of larger lists of references (Varga, 2019), a practice encouraged by the easiness of export tools from databases.

NEJM, Lancet and JAMA, have high h-indexes, over 675; it is therefore extremely difficult to enter the h-core of the journal’s dataset before a few years, and none succeeded this performance during the last 5 years. The exceptional character of Covid-19 research with fast citation reactions has waived this limit: 7 Covid-19 articles entered the NEJM h-core (1076), 10 the Lancet h-core (804) and 7 the JAMA h-core (675). With its much lower h-index of 122, JMV had 12 articles that reached the journal’s h-core in their first year. With many more articles in the range just under the h-core, a higher growth of the h-index might be expected for 2021 and 2022, especially for the specialized journals.

Evolution ha-index over the years

The evolution of the ha-indexes of the selected journals is presented in Table 5. The ha-index has the advantage to be more stable. While the h-index of journals seems to rise linearly, the ha-index ‘s yearly increase is somewhat lower and it gradually and asymptotically approaches to a limit. The ha-index acknowledges an exemplary contribution at an earlier stage; highly-cited papers can join the ha-core much sooner than the h-core, in 2 or 3 years time.

35 NEJM articles join the journal’s ha-core (214) from their first year, 13 Lancet articles (163) and 14 in JAMA (116). 39 JMV articles joined and increased the ha-core from 14 to 56, while 76 other Covid-19 articles have the potential to join this ha-core for the next year if they receive double citations of 2020.

General rules of thumbs can change under exceptional circumstances. With huge increases of citations for the Covid-19 research, the higher stability of the ha-average comes under question, especially for specialized journals with a lower ha-index. The average ha-index can raise substantially, to set a new limit at a higher plateau. For the medical journals, the ha-average increased from approximately 3% to 5.5% for JAMA, to 7% for NEJM, to 12% for The Lancet. The increase is phenomenal for the Journal of Medical Virology where the ha-average had reached a plateau of 14–15 over the last 10 years, it suddenly quadrupled to 56.

As general journals, with multiple highly-cited articles from various fields, Nature and Science have not experienced the same impact from the Covid-19 publication on their h-indexes.

The h- and ha-indexes in cohorts per year

Table 6 gives, for each year cohort, the evolution of the h-index in the year of publication, for NEJM, Science and JMV.

The NEJM immediacy h-index2 (using the publication year and the same year as citation period) lies between 34 and 44 and climbs up to 74 for 2020, approaching the range the h-index normally attains by the second year. The impact on Science results in an increase by 50% to 46. Interesting is the calculation of the h-index for the 2020 Covid-19 articles in Science that reaches 41 compared to 31 for the non-Covid-19 articles in Science in 2020. But as already predicted by the quantum leap of JMV’s h-index in 2020, the immediacy h-index of JMV explodes from around 5 to 55. Remarkable is the fact that medical journals as NEJM largely overpass the immediacy h-index of Science that can benefit from all top articles in all sciences. The phenomenon reveals a tsunami for specialized journals, that suddenly experience much more attention and interest to their themes of research.

The impact on the journal impact factor JIF

It is interesting to further analyze the impact of the phenomenon on the JIF of a few medical journals and to examine its evolution and forecast.

According to the Journal Citation Report, the NEJM has the second highest JIF of all scientific articles, in recent years around 75. The impact of the highly-cited articles will be reflected in the 2021 JIF-factors, to be released early 2022. By that time, the high citations of 2020 will be integrated in the JIF calculation for the first time. In our simulation for NEJM, a raise is expected from 75 to 100 to 150 in 2021 and to 115 to 160 in 2022. Lancet with a JIF of around 60 can expect to increase its JIF to the range of 90 to 130 and to stabilize around 100–120 in 2022. The lower ranked JMV that has grown most with the Covid-19 publications will see its JIF sextupled from 2 to the range 11–16 by 2021 and even more in 2022. Depending from a stabilizing or declining interest in the theme, and possible lower amounts of publications once the vaccines have been implemented and the pandemic under control, one could see the journals ulteriorly return to a lower range of JIF.

In the following Table 7 the evolution is shown of the JIF of the three journals over the last 3 years (as retrieved in the JCR WoS) and a simulation of the future JIF-factors for 2021 and 2022 (to be released by mid 2022 and by mid 2023). The lower 2023 estimate is based on the hypotheses of a decline in new citations after the hype, but can occur later.

The simulation shows how NEJM and Lancet, two of the already best ranked journals in function of their JIF, on the 2nd and 5th place, may possibly double their current JIF which is an exceptional phenomenon in bibliometrics. With its explosion of JIF, the lower ranked JMV will raise in the JCR ranking from place 1353 to the range between 300 and 100. This simulation elucidates the disruptive character of the Covid-19 research for bibliometrics.

Science and Nature that have a larger and wide spread of articles in various research areas with a smoother distribution curve will not be affected that much by a few highly-cited papers on Covid-19. It is possible that a few highly-cited papers on Covid-19 will have a raise on the total number of citations in 2022 or 2023 but they fall within what can be expected by the law of large numbers. With 700–1000 articles per year, Science and Nature’s JIF factors will remain more stable.

The impact on the journal immediacy index

The Immediacy Indexes of the three journals explode thanks to the tsunami of citations within the year 2020. NEJM’s immediacy index jumps from around 20 to 127, Lancet from the range of 15 to 155 and JMV from 0.8 to 15.

Discussion

The evolution of the citations and indicators illustrate the disruptive nature of the citation explosion on this specific theme of Covid-19. The evolution since 2015 of two complementary indicators – the ha-index and the JIF factor—with their possible evolution as simulated, are visually represented in the two graphs in Figs. 1 and 2.

Fig. 1

Evolution of the ha-index

Fig. 2

Evolution of the JIF

For the general multidisciplinary journals, the impact is neutralized by the law of large numbers. For the general medical journals, the impact is visible with a quantum leap of the ha-index and a relatively small reduced and probably delayed increase of the h-index; but a high degree of disruption appears in the evolution of the JIF for the period following the high attention to the theme. For specialized journals, the severe disruption occurs for all indicators with a much higher magnitude. The Covid-19 pandemic is central for JMV’s special focus3: research concerning viruses affecting humans.

New developments in new themes or ‘hot’ topics lead to a raise in the number of publications, that often appear in cohorts, just as diffusion of innovation often operates in bandwagons (Abrahamson & Rosenkopf, 1993; Rogers, 2010). Fashionable topics in science4 (Crane, 1969) can also play a role in success for publication or funds raising, but in the case of Covid-19 research, it is a sense of urgency that drives the publication hype. In a very quick move, all efforts of the medical research worldwide have been directed to study all effects of the disease to find answers and remedies for this threat for humankind. JMV has published three special issues on the Novel Coronavirus, and as other publishers, it made its collections of journal articles and book chapters on Covid-19 online freely available to the global scientific community.

The magnitude of the explosion of the number of publications on one single topic is a disruptive element that no other theme in other sciences has ever achieved. 34 articles on Covid-19 have overpassed 1000 citations in their first year (or half year), a record management researchers can hardly achieve in the first 10 years of their career. 536 authors have contributed to these 34 highly-cited articles. Any of the authors of those top articles would reach immediately the top in other fields, even if it is their first or only article. Discipline and specialized themes have such a considerable impact that it remains very difficult to compare fields and sub-fields without normalization. In addition to field normalization, adjustment for multiple authorship is mandatory.

The quantum leap of the total citations in this Covid-19 research makes this indicator problematic as criterion for comparison of journal quality. H-indexes cover the whole lifetime or career and gradually but steadily increase over time. The disruption of the exceptional phenomenon forms a point of discontinuity in this growth pattern, but the ulterior growth continues at a comparable pace. For the ha-index, that normally evolves towards a plateau, the discontinuity is greater and the evolution continues towards a new plateau at a higher level. In such disruptive conditions, the more time sensitive JIF, however, can receive a forceful quantum leap that afterwards can decline a few years later once the hype is over. Contrarily to the JIF, the rocketing of the Immediacy Indexes is more in the logic of the index as it rapidly identifies forceful reactions.

Comparison across disciplines remains problematic. The JIF has been promoted as a measure of journal quality, but this suggestion has encountered a lot of criticism (a. o. Archambault & Larivière, 2009; Larivière, Kiermer, MacCallum et al., 2016). Journal impact factors are heavily influenced by a small number of highly cited papers. This is precisely what is going to happen with the avalanche of articles on Covid-19. With its impressive citations records, the Covid-19 provokes a huge disruption in the evolution of the JIF of specialized and general medical journals that publish on that ‘hot’ theme. A ranking of all scientific journals according to their JIF-factor as in the Journal Citation Report where all disciplines are mixed is thus misleading. If the JIF of a few specialized journals in the field raise to be propelled in the better ranking, it is not due to the quality of the journal, but due to the temporal interest in the theme under study. Other journals in other subdisciplines that do not publish on that theme, decline in ranking, despite the quality of that research in that given area. Bibliometric indicators need to be interpreted within the context.

Indicators of impact and quality need consistency and stability over time, with changes that can occur but gradually. The present analysis further questions the validity of the JIF, seen its short term perspective. It pleads for at least a replacement by the 5 years JIF that is formed on a larger window of time, and thus allows more averaging. In addition, this study confirms the critique of bibliometric scholars on the use of the journal’s JIF in the evaluation of individual researchers. Several Covid-19 articles in NEJM or in JMV have no citations, and thus no impact, despite the increased JIF of the journal.

The exceptional circumstances of the pandemic have also influenced the time for review which has impact on the more rapid growth of citations. And the review process is already much faster in life sciences than in social sciences. Many empirical studies in medicine or in applied sciences can be more focused and necessitate shorter study length than research in social sciences and results are published much quicker.

The Covid-19 research has benefitted from additional speed which has even another unprecedent consequence. Generally, it takes about one to two years to have the large expansion of the citations and the full impact of the article; it is therefore that the JIF takes the window of the next two years after the publication year into its calculation. However, in this exceptional case, the enormous amounts of citations will not be taken into account and the JIF-increase will only occur after two years. The disruptive upsurge of the JIF factor is thus delayed. However, the Immediacy Index fulfills this role of identifying short term disruptions, which is the case with the Covid-19 research with an acceleration of the publication process, online publication and preprints.

Comparison with other bibliometric Covid 19 research

The present research confirms Zhang et al. (2020)’s early detection of the swift reactivity of the academic community to public health emergencies. The results also confirm the first early bibliometric studies on Covid-19 research (da Silva et al., 2021; Di Girolamo & Reynders, 2020; Haghani & Bliemer, 2020; Zhang et al., 2020)5 that had already recognized the extraordinary upsurge of the academic publication output around this new theme of scientific research. All those early bibliometric studies make use of different perspectives (contribution per research area, country, institutions, journals) and different methods (descriptive, data mining, VOSviewer) for different objectives. A few of those studies concentrate on the comparison with other health emergencies. Our study performed nearly 1 year after the emergency, examines the impact of this exceptional phenomenon on bibliometric indicators.

The other concerns raised by the exceptional magnitude of the phenomenon of huge Covid-19 research are ethical considerations. First of all, the increased speed of publication process and possible lower criteria in the review process, has increased the possibility of errors and retractions, as already investigated by Moradi and Abdi (2021), Soltani and Patini (2020) and Yeo-Teh and Tang (2020). One can also question the effectiveness of such an inflation of publications at a speed no one can follow; the problem for researchers becomes a tough selection choice on what articles to read. And here again, bibliometrics play a role: time constrains place the focus to first read already highly-cited articles, and to cite them, which acts a self-fulfilling prophecy increasing the Matthew effect in science (Merton, 1968).

Conclusion

More than the research or the discipline, the theme of the research is important for impact and citations for articles in applied sciences. The salience of the topic, the magnitude of the problem at study and the urgency to find solutions are drivers for citations. An exceptional phenomenon has a disruptive impact on bibliometric indicators. The higher the specialization, the higher the possible impact of a disruptive phenomenon.

Bibliometrics offers a way to illustrate exceptional phenomena in publications. The study of the Covid-19 research illustrates the relativity of indicators and the need for context. The present study also confirms the plead for responsible metrics of the San Francisco Declaration on Research Assessment6 (DORA).