Variations in surgical peer-reviewed publications among editorial board members, associate editors and their respective journal: Towards maintaining academic integrity.

BACKGROUND: Physician scientists who are also Editorial Board members or Associate Editors may prefer publishing in their own journal and therefore create an environment for conflicts of interest to arise.
OBJECTIVES: To assess the relationship between the number of peer-reviewed publications in surgical journals in which authors serve as Editorial Board Members and Associate Editors and their total number of annual publications.
MATERIALS AND METHODS: A cross-sectional study utilizing PubMed was performed regarding the total annual number of peer-reviewed publications by Editorial Board Members/Associate Editors and the number published in their respective affiliated journals from 2016 to 2019. Significance defined as p < 0.05.
RESULTS: 80 Associate Editors and 721 Editorial Board Members (n = 801 total) were analyzed from 10 surgical journals. The mean number of total annual peer-reviewed publications varied from 5.19 to 17.18. The mean number of annual peer-reviewed publications in affiliated journals varied from 0.06 to 2.53. Multiple significant associations were discovered between the total number of annual peer-reviewed publications and number of peer-reviewed publications in affiliated journals for all authors/surgical journals evaluated, except for the International Journal of Surgery (p > 0.05).
CONCLUSIONS: We found significant associations between the total number of annual peer-reviewed publications by Editorial Board Members/Associate Editors and number of annual peer-reviewed publications by their affiliated surgical journals. The implementation and enforcement of a standardized double-blind review process and mandatory reporting of any potential conflicts of interest can reduce possible bias and promote a fair and high-quality peer-review process.

Introduction

The peer-review system of analyzing manuscripts submitted for publication was implemented over a century ago with the ultimate goal of maximizing the quality and relevance of articles published in journals [[1], [2], [3]]. The “publish or perish” dictum is a phenomenon that places pressure on physician scientists to publish research and contribute to the advancement of medical knowledge [[1], [2], [3]]. In many cases, publications and overall academic activity are considered promotional metrics used to evaluate physicians striving to advance their career, particularly in academia [4,5]. As a consequence of the pressure to produce publications, physician scientists who have high-ranking journal positions such as Editorial Board Member (EBM) or Associate Editor (AE) may prefer publishing in their own affiliated journal [6]. Such preferential publication may create opportunities for conflicts of interest (COI) to arise, as reviewers not blinded to submission authors may be less inclined to provide critical critique and to recommend manuscript acceptance based on merit alone [6].

Bias may also influence an editor's decision to accept a manuscript for publication [[7], [8], [9], [10], [11]]. An early study investigating this topic found that authors from prestigious universities replicated submitted studies with minor changes in order to publish pertinent conclusions first [7]. These findings have warranted the possibility that reviewer knowledge of author name and institutional affiliation plays a role in the acceptance or rejection of a manuscript. However, previous literature is inconclusive regarding the effect of blinding on the manuscript approval process [[8], [9], [10]]. While some studies reported an improved quality of manuscript reviews during a double-blinded review process, others have not identified any benefit [[8], [9], [10], [11]]. Therefore, the wide degree of variation in the efficacy of blinding reviewers may imply that the determination of manuscript quality may be subject to preferences for top institutions.

Author affiliation in an official journal position such as EBM or AE may also have a role in the number of their publications accepted [11,12]. A previous analysis of 4460 research publications from 20 medical specialty journals demonstrated a statistically significantly greater number of publications accepted from EBMs in 70% of journals evaluated [11]. Furthermore, these authors were nearly three times as likely to publish research in their affiliated journal compared to non-affiliated journals [11].

However, the frequency of surgical journal EBMs/AEs publishing in their own affiliated journal is not well described. We aim to investigate the ratio of annual peer-reviewed publications in affiliated journals in which authors serve as EBMs/AEs to the total annual number of peer-reviewed publications in 10 prominent surgical journals from 2016 to 2019.

Materials and Methods

Search strategy and selection criteria

A cross-sectional study utilizing PubMed data was performed regarding the total annual number of peer-reviewed publications by EBMs/AEs and the total number published to their respective affiliated journals from 2016 to 2019. We investigated the top two-hundred peer-reviewed surgical journals based on number of documents published according to the Scimago Journal & Country Rank [13]. We subsequently imported and randomized the top two-hundred surgical journals and followed a systemic random sampling by selecting every tenth journal for inclusion in our analysis until a total of ten journals were obtained. The following journals were included: 1) Trauma Surgery & Acute Care Open, 2) International Journal of Surgery, 3) British Journal of Surgery 4) Journal of the American College of Surgeons, 5) Annals of Surgery 6) Surgery, 7) Journal of Surgical Research, 8) JAMA Surgery, 9) Journal of Trauma and Acute Care Surgery and the 10) Journal of Pediatric Surgery. We manually searched the website of each journal to identify EBMs/AEs listed and used their name to obtain their number of annual peer-reviewed publications using the Advance Search feature of PubMed. This study was reported in line with the STROCSS 2019 guidelines [14]. This work was submitted to the Research registry (UIN 6067) which can be found via the following link: https://www.researchregistry.com/register-now#home/registrationdetails/5f73cc9e0a9e6e001596a0ec/

Inclusion & exclusion criteria

The inclusion criteria for this study were authors who hold a position as an AE or are EBMs for each surgical journal. Those AEs that also held EBM positions were only counted once in our analysis and represented as AE + EBM. The exclusion criteria for this study were authors holding a journal position other than AE or EBM, such as journal reviewer. Verification of single blind, double blind, or open review processes were verified with the Managing Editors from each surgical journal.

Statistical analysis

Descriptive statistics analyses were performed to determine the number of total annual peer-reviewed publications of AEs and EBMs of the 10 journals listed above. Additionally, this information was compared to the annual number of peer-reviewed publications these authors published in their own respective journals from 2016 to 2019. Linear regression was performed to determine any significant associations between the number of total annual peer-reviewed publications and number of annual peer-reviewed publications in the respective journals of AEs/EBMs. IBM SPSS Version 26.0 (Chicago, IL) was used for data analysis. Significance was defined as p < 0.05. This study was conducted in compliance with ethical standards and deemed exempt by our institutional review board.

Results

Trauma Surgery & Acute Care Open (TSACO)

A total of 10/10 AEs and 98/99 EBMs (n = 108 total; 1 AE + EBM) fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 7.49 in 2016, 7.97 in 2017, 8.64 in 2018, and 8.52 in 2019. The mean number of publications in TSACS for these authors was 0.08 in 2016, 0.07 in 2017, 0.28 in 2018, and 0.33 in 2019. All associations between the number of total publications and number of journal-specific publications per year can be found in Table 1.

Table 1

Summary of the Key Findings in Surgical Journals Evaluated. Shown are the Pearson correlations, 95% confidence intervals and p-values of linear regression analyses comparing the total number of annual publications for AEs/EBMs to the number of publications these authors published in their own affiliated journal. Abbreviations: JAMA = Journal of the American Medical Association.

Year	Pearson Coefficient	95% Confidence Interval	Significance (p-value)
Journal of Trauma Surgery and Acute Care Open (n = 108)
2016	0.041	(-0.005-0.008)	0.675
2017	0.145	(-0.001-0.010)	0.135
2018	0.156	(-0.002-0.024)	0.106
2019	0.308	(0.008–0.032)	0.001
International Journal of Surgery (n = 37)
2016	0.135	(-0.043-0.098)	0.433
2017	−0.107	(-0.016-0.009)	0.536
2018	−0.017	(-0.010-0.009)	0.921
2019	0.048	(-0.016-0.021)	0.782
British Journal of Surgery (n = 61)
2016	0.536	(0.021–0.051)	<0.001
2017	0.435	(0.011–0.038)	<0.001
2018	0.556	(0.018–0.042)	<0.001
2019	0.237	(-0.001-0.027)	0.066
Journal of the American College of Surgeons (n = 84)
2016	0.421	(0.016–0.046)	<0.001
2017	0.354	(0.008–0.029)	0.001
2018	0.411	(0.009–0.026)	<0.001
2019	0.595	(0.014–0.027)	<0.001
Annals of Surgery (n = 153)
2016	0.511	(0.027–0.047)	<0.001
2017	0.480	(0.029–0.053)	<0.001
2018	0.531	(0.036–0.061)	<0.001
2019	0.520	(0.046–0.079)	<0.001
Surgery (n = 97)
2016	0.454	(0.029–0.069)	<0.001
2017	0.538	(0.037–0.071)	<0.001
2018	0.368	(0.016–0.049)	<.001
2019	0.308	(0.011–0.050)	0.002
Journal of Surgical Research (n = 81)
2016	0.402	(0.031–0.097)	<0.001
2017	0.347	(0.028–0.113)	0.001
2018	0.499	(0.056–0.127)	<0.001
2019	0.473	(0.066–0.161)	<0.001
JAMA Surgery (n = 30)
2016	0.439	(0.010–0.090)	0.015
2017	0.206	(-0.028-0.095)	0.274
2018	0.601	(0.072–0.226)	<0.001
2019	0.603	(0.050–0.154)	<0.001
Journal of Trauma and Acute Care Surgery (n = 120)
2016	0.750	(0.233–0.322)	<0.001
2017	0.794	(0.238–0.316)	<0.001
2018	0.729	(0.187–0.265)	<0.001
2019	0.812	(0.224–0.291)	<0.001
Journal of Pediatric Surgery (n = 30)
2016	0.714	(0.150–0.333)	<0.001
2017	0.547	(0.069–0.269)	0.002
2018	0.564	(0.103–0.372)	0.001
2019	0.601	(0.110–0.343)	<.001

International Journal of Surgery (IJS)

A total of 11/11 AEs and 26/34 EBMs (n = 37 total; 8 AE + EBM) fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 5.86 in 2016, 5.78 in 2017, 5.61 in 2018, and 5.19 in 2019. The mean number of publications in IJS for these authors was 0.42 in 2016, 0.06 in 2017, 0.03 in 2018, and 0.11 in 2019. However, there were no significant associations between the number of total and journal-specific publications from 2016 to 2019 (Table 1).

British Journal of Surgery (BJS)

A total of 2/2 AEs and 59/60 EBMs (n = 61 total; 1 AE + EBM) fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 12.13 in 2016, 12.85 in 2017, 13.07 in 2018, and 14.82 in 2019. The mean number of publications in BJS for these authors was 0.36 in 2016, 0.36 in 2017, 0.33 in 2018, and 0.51 in 2019.

Journal of the American college of surgeons (JACS)

A total of 0/0 AEs and 84/84 EBMs (n = 84 total) fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 10.57 in 2016, 11.32 in 2017, 12.27 in 2018, and 11.93 in 2019. The mean number of publications in JACS for these authors was 0.67 in 2016, 0.50 in 2017, 0.45 in 2018, and 0.42 in 2019.

Annals of Surgery (AOS)

A total of 6/6 AEs and 147/153 (n = 153 total; 6 AE + EBM) EBMs fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 15.09 in 2016, 16.13 in 2017, 16.53 in 2018, and 17.18 in 2019. The mean number of publications in AOS for these authors was 0.64 in 2016, 0.95 in 2017, 1.12 in 2018, and 1.40 in 2019.

Surgery

A total of 0/0 AEs and 97/97 (n = 97 total) EBMs fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 13.89 in 2016, 14.98 in 2017, 15.31 in 2018, and 14.73 in 2019. The mean number of publications in Surgery for these authors was 1.18 in 2016, 1.14 in 2017, 1.04 in 2018, and 0.89 in 2019.

Journal of Surgical Research (JSR)

A total of 18/18 AEs and 63/64 (n = 81 total; 1 AE + EBM) EBMs fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 5.64 in 2016, 6.62 in 2017, 7.83 in 2018, and 8.16 in 2019. The mean number of publications in JSR for these authors was 0.42 in 2016, 0.64 in 2017, 0.90 in 2018, and 1.05 in 2019.

JAMA surgery

A total of 0/0 AEs and 30/30 (n = 30 total) EBMs fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 10.47 in 2016, 10.27 in 2017, 11.03 in 2018, and 12.27 in 2019. The mean number of publications in JAMA Surgery for these authors was 0.67 in 2016, 0.97 in 2017, 1.30 in 2018, and 1.83 in 2019.

Journal of trauma and Acute Care Surgery (JTACS)

A total of 3/3 AEs and 117/118 (n = 120 total; 1 AE + EBM) EBMs fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 6.83 in 2016, 7.87 in 2017, 8.06 in 2018, and 7.33 in 2019. The mean number of publications in JTACS for these authors was 1.78 in 2016, 2.32 in 2017, 2.04 in 2018, and 1.67 in 2019.

Journal of Pediatric Surgery (JPS)

A total of 30/30 AEs and 0/0 EBMs (n = 30 total) fit our inclusion criteria and were analyzed. The mean number of total publications for these authors was 5.70 in 2016, 6.17 in 2017, 7.17 in 2018, and 8.20 in 2019. The mean number of publications in JPS for these authors was 1.47 in 2016, 1.73 in 2017, 1.90 in 2018, and 2.53 in 2019.

Discussion

Significant associations were found between authors serving as AEs/EBMs and number of total annual and affiliated journal-specific peer-reviewed publications from 2016 to 2019 for 9 of the 10 journals evaluated, with the exception of IJS.

Our results supports previous literature, which demonstrates that AEs/EBMs publish a significantly greater number of publications in their affiliated journal [11]. However, our findings are also in contrast to literature which demonstrate significant heterogeneity in the number of articles published by EBMs in leading urologic journals, indicating that the propensity for AEs/EBMs to publish in their own affiliated journal may vary between fields and is specialty dependent [12]. However, while previous studies have analyzed AE/EBM publication activity in multiple journals in non-surgical fields such as pediatrics and psychiatry, to our knowledge this is the first study, which analyzes relevant and highly cited surgical journals [11,12].

One possible explanation for the AE/EBM publication trends found in this analysis may be as simple as the exceptional quality of manuscripts submitted. The journal EBM selection process is privy to several criteria including a surgeon's background, experience, academic faculty position, and research activity [15,16]. As engagement in surgical academia and publication count is a requirement for selection as an AE/EBM, it is reasonable to propose that these experienced authors submit high quality manuscripts which get accepted on the strict basis of merit rather than journal affiliation [15,16]. It is also possible that AEs/EBMs receive additional support from their own journal in the form of knowing relevant topics to submit in a timely fashion that have a substantial chance of being published. Therefore, the increased propensity for AEs/EBMs to publish in their own affiliated journal is likely multifactorial and may be dependent on both author skill and knowledge.

An additional factor, which may serve to explain the trends observed in this analysis, is the manuscript review process. In general, there are three main review processes: in a single-blind review process, author names are revealed to the editor, but the editors’ names are not revealed to the authors [17]. In contrast, a double-blind review process ensures that author and editor/reviewer names are not revealed to each other [17]. Finally, an open review process allows both the names of the authors and editors to be revealed to each other [17]. The single-blind approach has the potential to create a COI and influence the reviewer/editor decision to accept or reject a research submission [18]. A previous study found that knowledge of author name and institutional affiliation in a single-blind review process influenced the propensity for manuscript acceptance, with highly productive authors from top institutions benefiting the most [18]. Therefore, the possible bias in a single-blind or open manuscript review process may extend favor to AEs/EBMs and increase the chance of publishing in their own affiliated journal. In an analysis of 13 eligible institution ranking systems, 46% used research performance as the sole metric to rank institutions [19]. As more manuscripts from high-ranked institutions are accepted for publication, the divide between high and low-ranked institutions has the potential to continue widening. Therefore, the risk of implicit or explicit bias in selecting manuscripts from top ranked universities/hospitals is a potential COI which may prevent papers from being evaluated solely on their quality. This bias could be minimized by journals transitioning to a double-blind review process.

This notion of reviewer/editor blinding playing a crucial role in the manuscript review process is supported by our findings that of the 10 journals analyzed, a journal which adheres to a double-blind review process (IJS) did not display any statistically significant associations between the total number of annual peer-reviewed publications and number of peer-reviewed publications in their affiliated journal. In addition, AEs/EBMs for the IJS exhibited one of the lowest mean percentages of peer-reviewed publications in their affiliated journal from 2016 to 2019 (Fig. 1). Although not conclusive, it is possible that the double-blind review process has the potential to reduce bias, which may result from knowledge of author name/institution and could influence the trends in the number of papers published by AEs/EBMs in their affiliated journal (Fig. 2).

Fig. 1

Mean Percentage of Total Publications in Affiliated Journals for AEs/EBMs, 2016–2019. Shown are the mean proportion of publications accepted into affiliated journals compared to total publication activity per year for AEs/EBMs. The mean percentage of publications in the journal for which the author is an AE or EBM from 2016 to 2019 is the following: Journal of Trauma and Acute Care Open (2.3%), International Journal of Surgery (2.7%), British Journal of Surgery (2.9%), Journal of American College of Surgeons (4.5%), Annals of Surgery (6.3%), Surgery (7.2%), Journal of Surgical Research (10.4%), JAMA Surgery (10.6%), Journal of Trauma and Acute Care Surgery (25.9%), and the Journal of Pediatric Surgery (27.8%). Abbreviations: AEs = Associate Editors, EBMs = Editorial Board Members, JAMA = Journal of the American Medical Association.

Fig. 2

Trends of the Top 5 Journals with Highest Mean Percentage of AE/EBM Publications, 2016–2019. Shown are the annual mean percentage of publications AEs/EBMs published in their own affiliated journal compared to all publications. Journal of Trauma and Acute Care Surgery and Surgery display a downward trend whereas the remaining journals display an upward trend in mean percentage of publications in affiliated journals over the year range.

Opinions regarding the benefits of double-blind review have been shared by previous publications, so long that other methods to identify potential instances of COI otherwise known only by the author name alone are accounted for [20]. COIs have a possible role in the manuscript review process and can be minimized if guidelines regarding their disclosures in a standardized review process are established. Just as authors may possess financial COIs that may impact their judgement and results reported, editors have been shown to be vulnerable to the same pitfalls [21]. A previous analysis of 906 physician editors across fifteen orthopedic surgery journals revealed that 78% of EBMs received financial compensation with some receiving more than $950,000, therefore having potential for COI [21]. It is reasonable to propose that COIs may influence an editor to accept a manuscript similarly to the manner in which COIs may influence authors to publish results which favor the company/organization with which they have a COI [21]. Use of a standardized review process to mandate the disclosure of potential COIs by editors could improve transparency and accountability in the review process. In addition, policies advocating for their removal from the decision-making process for submissions at risk for unfair evaluation may help reduce this significant association.

Our study has several limitations. As we limited our analysis to investigating the publication productivity of AEs and EBMs, the trends described may not be generalizable to individuals who hold different journal positions such as reviewers. Additionally, the relatively small sample size of 10 surgical journals may not be representative for all journals and could over- or underestimate the trends observed. Future investigations with larger samples may help to validate our presented findings.

We offer serveral recommendations moving forward. It is critical for journals to implement and enforce a standardized double-blinded review process and mandatory reporting of any potential editor/author conflicts of interest in order to promote a fair and high-quality peer-review process. Further research is needed regarding journals of other surgical and medical fields to identify any potential disparities in the peer-review process and to promote academic integrity in scientific literature. These actions may serve to increase the quality and fairness of the surgical publication peer-review process to ensure evaluations are based entirely on manuscript merit.

Conclusions

Significant associations were found between the total number of annual peer-reviewed publications and number of annual peer-reviewed publications in affiliated journals for editorial board members and associate editors of prominent surgical journals from 2016 to 2019. The implementation and enforcement of a standardized double-blind review process and mandatory reporting of any potential conflicts of interest can reduce possible bias and promote a fair and high-quality peer-review process.

Provenance and peer review

Not commissioned, externally peer reviewed.

Ethical approval

This study does not involve human or animal subjects. The study was conducted in compliance with ethical standards, reviewed by our institutional review board and deemed exempt.

Source of Funding

None.

Author contribution

Study design and conception: Elkbuli.

Data collection: Sen-Crowe, Sutherland, Shir, Boneva, Elkbuli.

Data interpretation and analysis: Sutherland, Sen-Crowe, Elkbuli, McKenney.

Manuscript preparation: Sen-Crowe, Sutherland, Shir, Kinslow, Elkbuli, McKenney, Boneva.

Critical revisions of manuscript: Elkbuli, Kinslow, McKenney, Boneva, Sutherlands, Sen-Crowe.

All authors approved the final manuscript.

Trial registry number

This work was submitted to the Research registry (UIN 6067) which can be found via the following link: https://www.researchregistry.com/register-now#home/registrationdetails/5f73cc9e0a9e6e001596a0ec/

If you are submitting an RCT, please state the trial registry number – ISRCTN

Not applicable.

Guarantor

Mark McKenney.

Adel Elkbuli.

Declaration of competing interest

Authors declare no competing interests.