The influence of policies limiting author self-citations on journals impact factor and self-citation rate in respiratory system.

To assess the presence of journal policies discouraging inappropriate author's self-citation (A-SC) in "Respiratory System" journals, we evaluated submission guidelines of "Respiratory System" journals included in Journal-Citation Reports 2020 (Clarivate Analytics®) for the presence of policies on A-SC and its impact on journals' self-citation (J-SC) rate and impact factor (IF). We found that 14.3% of journals (n=8/56) reported policies on inappropriate A-SC. The median IF was not different in "Respiratory System" journals with (3.6; IQR:2.3) vs without A-SC policies (3.1; IQR:3.0; p=0.41). The J-SC rate was not influenced by the presence of A-SC policies (p=0.83). Fully open-access (n=14) and traditional (n=42) journals had no differences in IF (3.3; IQR:1.5 vs 3.1; IQR:3.4, respectively; p=0.77) and J-SC rate (4.5%; IQR:5.6 vs 6.2%; IQR:8.4, respectively; p=0.38). The majority of "Respiratory System" journals do not have policies discouraging A-SC. The presence of such policies is not associated with changes in IF or J-SC rates. ©Copyright: the Author(s).

Introduction

Inappropriate authors’ self-referencing is a growing form of scientific malpractice [1], and it is estimated that each self-citation generates almost four additional citations over ten years [2]. While efforts should be made to avoid inappropriate author self-citations (A-SCs), it is challenging to establish their appropriateness. Policies suggesting avoidance of inappropriate A-SC may represent an option, but not all self-referencing is synonymous with malpractice.

We surveyed submission guidelines for policies aiming to reduce A-SCs among Respiratory System journals [3] and analyzed journals’ self-citation (J-SC) practice, which may hinder an attempt to increase the impact factor (IF).

Methods

The Journal-Citation Reports 2020 (Clarivate Analytics®) [4] includes 96 “Respiratory System” journals but only 56 of them with an IF. On 22 August 2021, we evaluated the submission guidelines of journals with IF for the explicit presence of policies discouraging inappropriate A-SC. Subsequently, we analyzed the influence of these policies on the journal’s IF and the J-SC rate.

The latter was calculated as:

We also performed two secondary analyses. The first separate journals into sub-categories according to their focus of interest: 1) “multidisciplinary” (publishing also on other disciplines, i.e., intensive care, thoracic surgery); 2) “specific” (focusing only on Respiratory System). The second separate journals into “fully open-access” vs “traditional”. Continuous variables are presented as median and interquartile range [IQR] and categorical variables as numbers/percentages. A two-sided Mann-Whitney test for unrelated samples was performed; p<0.05 was considered statistically significant.

Results

The overall results and the analysis in groups according to the presence of A-SC policies, the focus of interest, and the publishing options are reported in Table 1. The included journals (Supplementary Table) had a median IF of 3.2 (IQR:2.9), while the J-SC rate was 5.5% (IQR:11.6) with high variability (0.4%- 22.3%). Only eight journals (14.3%) reported policies suggesting avoidance of inappropriate A-SCs (none proposed a cut-off). A post-hoc analysis revealed that only three journals (5.3%) discouraged Editors/Reviewers from asking authors to add citations of their papers without a solid rationale [4].

Table 1.

Overall results and analysis in groups according to presence of A-SC policies, focus of interest, and publishing options in Respiratory System journals with impact factor (IF). J-SC, journal’s self-citation.

	IF	J-SC rate (%)
Overall group (n=56)	3.2 [IQR:2.9]	5.5 [IQR:11.6]
A-SC policies
Present (n=8)	3.6 [IQR:2.3]	3.8 [IQR:8.4]
Absent (n=48)	3.1 [IQR:3.0]	5.9 [IQR:8.0]
p	0.41	0.83
Focus of interest
Multidisciplinary (n=25)	3.1 [IQR:3.6]	6.2 [IQR:7.7]
Specific (n=31)	3.4 [IQR:2.4]	3.9 [IQR:7.9]
p	0.93	0.28
Publishing options
Fully open-access (n=14)	3.3 [IQR:1.5]	4.5 [IQR:5.6]
Traditional (n=42)	3.1 [IQR:3.4]	6.2 [IQR:8.4]
p	0.77	0.38

Discussion

In this study, we found that Respiratory System journals had a low prevalence of A-SCs policies (~14%). This prevalence is identical to Anesthesiology journals [4] but lower than in Critical Care Medicine [5] and General Surgery [6] journals (22% and 25%, respectively), as shown in previously conducted analyses on A-SCs and J-SC rates in other medical categories.

Overall, the present study and the results pooled from previous analyses suggest that inappropriate A-SCs have not received enough consideration, with greater attention needed to stop such misbehavior. However, we strongly emphasize that not all A-SCs are synonymous with scientific malpractice or unethical behavior. Indeed, when introducing new studies, authors may need to quote their previous relevant work, pilot studies, or reviews, especially if conducted in a restricted research setting. Thus, self-referencing may be inevitable or anyhow appropriate. Therefore, it is challenging to define if an A-SC is appropriate or not [7]; moreover, a single cut-off for the percentage of A-SCs is unlikely to work. Still, it remains urgent to reduce the impact of self-referencing on scientific metrics as these are accounted for grants and exams and may influence academic competitions.

The other side of self-citation malpractice is represented by JSC, which may hinder the attempt to increase the journal IF. The J-SC rate was highly variable in Respiratory System journals, similarly to Anesthesiology (1-37%), Critical Care Medicine (0-35%) and General Surgery (0-31%) journals [4-6]. When comparing groups, we could not find a significant influence of A-SC policies on IF and J-SC rates. The IF and the J-SC rates were not different according to the presence of policies for A-SCs, discarding the assumption of higher editorial standards in journals with higher IF; however, the small sample size and the difference in median values warrant caution. Considering the results of previous investigations, an association between higher publishing standards and attention to A-SCs practice remains possible. A pooled analysis from different categories of journals may be warranted.

Similarly, we did not detect differences in IF and J-SC according to the journal’s focus of interest. This finding is partially different from previous investigations where J-SC was significantly different in Critical Care Medicine and General Surgery journals and had a trend toward significant differences in Anesthesiology [4-6]. More data from other branches are needed to better interpret factors influencing J-SCs practice [8].

Another finding highlighted by our study was that only 3 Respiratory System journals (number 11-13-27, Supplementary Table) suggested that Editors/Reviewers should not ask authors to add references to their studies without a strong rationale. However, this aspect of self-citation practice is difficult to investigate as it would require the availability of Editors/Reviewers’ reports with their identities. On reflection, whilst it still has to be demonstrated that the presence of A-SC policies ensures higher publishing standards, considering the difficulties in critically evaluating the appropriateness of author’s and journal’s self-referencing, a reasonable option might be to calculate all the scientific metrics excluding A-SCs since well-known databases (i.e., Scopus, Web of Science) allow to re-calculate them. Similarly, the journal IF without the contribution from J-SC can be easily calculated, as shown in the Supplementary Table. This approach would instantaneously make inappropriate authors’ and journals’ self-referencing harmless. Although abnormal authors’ and journals’ self-referencing might also be detected through automated algorithms, it is difficult to contain the problem with a restrictive and “one size fits all” procedure. In this context, the role of virtuous and competent reviewers is of utmost importance in judging the pertinence of A-SCs and avoiding inappropriate practices. Nevertheless, it is also difficult for reviewers to discriminate between all co-authors’ self-citations and fully appropriate or even necessary self-citations.

Moreover, sometimes peer reviewers request authors to quote their own work or editors give indications to add citations from their journal, but this is a difficult aspect to address.

As a limitation to our investigation, it should be noted that policies on self-referencing represent a complex issue, and the absence of A-SCs policies and a higher J-SC rate may be encountered in “predatory journals” [9,10], which were not considered by our study.

High-quality studies are urgently needed to analyze the impact of different types of inappropriate self-citation and to examine effective interventions to limit them.

Conclusions

The majority of “Respiratory System” journals do not have policies discouraging A-SC. The presence of such policies is not associated with significant differences in IF or J-SC rates, but the small sample size warrants caution in the interpretation of these results.