Comparison of Editor, Reviewer, and Author Demographics in The Journal of Hand Surgery.

Purpose: To determine whether demographic differences exist among editors, reviewers, and authors in The Journal of Hand Surgery (JHS). We aimed to test the null hypothesis that there would be no difference among these 3 groups with respect to gender, geographic location, academic productivity, and financial relationships with industry.
Methods: Editors, reviewers, and physician authors were identified for 2018 JHS. Gender and geographic location were recorded for each person. We used the Scopus database to determine the Hirsch index (h-index) as well as the number of publications and citations for members of each group. Industry payment information was obtained using the Open Payments Web site.
Results: The editor group contained 20% women compared with the author group (17% women). Authors (59%) were less likely to be from the United States compared with editors (91%) and reviewers (88%). Editors were found to have a higher h-index (16) compared with reviewers (14) and authors (12). Authors demonstrated significantly higher mean total payments from industry ($41,738) compared with editors ($13,712) and reviewers ($20,457). Conclusions: In 2018, there appeared to be an even distribution with respect to gender among editors, authors and reviewers in the JHS. International editors and reviewers are relatively under-represented compared to authors. Whereas editors and reviewers demonstrated higher h-indices compared with authors, JHS authors had significantly higher mean total payments in the Open Payments database. Clinical relevance: Defining demographics, academic productivity, and conflicts of interest for journal editors, reviewers, and authors may aid in identifying potential sources of both author and peer review bias.

Peer review emerged during the 17th century as a means of improving the quality of scientific scholarship. Although the concept has been in existence for over 200 years, systematic reviews have demonstrated that the process is largely unstudied., Within orthopedic surgery journals, there is substantial variation with respect to how peer review is used.

Because of the frequent relationship between industry and orthopedic research, substantial attention has been focused on financial disclosures and conflicts of interest. However, other forms of bias can affect the peer review process., In addition to peer-reviewed literature, prior authors demonstrated forms of bias within the grant review process. Although peer review implies the evaluation of scientific work by others working in the same field, it is uncertain whether demographic differences exist between authors producing manuscripts and reviewers contributing to the peer review process.

The purpose of this investigation was to determine whether demographic, academic productivity, and conflict of interest differences exist among editors, reviewers, and authors in The Journal of Hand Surgery (JHS). We aimed to test the null hypothesis that there would be no difference among these 3 groups with respect to gender, geographic location, academic productivity, and financial relationships with industry.

Materials and Methods

This study received exemption status from our institutional review board. Our investigation consisted of 3 groups (editors, reviewers, and authors) associated with JHS during 2018. A list of editors was created from the editorial board page on the JHS Web site (www.jhandsurg.org/content/edboard). The reviewer list was established using the list of consultant reviewers for 2018 published by JHS. To generate a list of authors, all JHS articles published in 2018 were reviewed. We included only articles under the headings of Editor’s Choice, Current Concepts, Hand Surgery Landscape, Surgical Techniques, and Scientific Articles. For the author group, only physicians (those with an MD, DO, or equivalent degree) were included in the analysis; those with an MD and PhD were included as well. Authors were excluded if they were identified as residents or trainees. The American Society for Surgery of the Hand (ASSH) Web site, Internet searches, and personal and/or professional Web pages were used to identify residents and trainees.

Employing a methodology similar to that described by Okike et al, we categorized editor, reviewer, and author gender as male or female. When gender could not be definitively determined, subjects were excluded. Demographic and geographic information for all editors, reviewers, and authors was obtained using the Find a Hand Surgeon link on the ASSH Web site, Internet searches, and personal and/or professional Web pages. In addition, the institutional affiliations for authors contained in the JHS articles were used.

To determine academic productivity, we utilized Scopus, an abstract and citation database for peer-reviewed literature. From the search feature within Scopus, we determined the Hirsch index (h-index), the number of publications and number of citations for each editor, reviewer, and author. The Scopus database was chosen over Google Scholar and Web of Science because it covers a broader range of articles. Scopus includes articles published between 1970 and 1996, which was a prior limitation of this database. Subjects were excluded when their name produced multiple results and either the practice location or subspecialty could not be confirmed within the database.

For financial disclosure information, we used the Open Payments Web site operated by the Centers for Medicare and Medicaid Services. This database contains information only for physicians in the United States. Using the results from the search feature, we recorded general payments as well as total payments, defined as the sum of the general payments, research payments, ownership and investment interests, and associated research funding. All financial information was from 2018.

We used descriptive statistics for baseline demographics. One-way analysis of variance and chi-square test to compare percentages or means between groups. Differences of P < .05 were considered statistically significant.

Results

We were able to identify 99% of JHS editors and reviewers as well as 95% of authors. Table 1 compares demographic information and academic productivity among the 3 groups. The editor group contained a higher percentage of women compared with authors (n = 14 [20%] vs n = 72 [17%]; P = .048). Authors were less likely to be from the United States compared with editors and reviewers (P < .001). Editors were found to have a higher h-index (16) compared with reviewers (14) and authors (12), and these results were statistically significant (P =.001). Differences in mean publications and citations among the 3 groups were not statistically significant (Table 1). Median publications varied among the groups, with editors demonstrating a median of 61 publications compared with 24 for authors.

Table 1

Comparison of Baseline Demographics for JHS Editors, Reviewers, and Authors for 2018

Baseline Demographics	Editors	Reviewers	Authors	P Value
Total, n	71	462	563
Identified, n (%)	70 (99)	458 (99)	537 (95)	.001
Male, n (%)	56 (80)	372 (81)	465 (83)	.048
Female, n (%)	14 (20)	86 (19)	72 (17)
MD or equivalent, n (%)	69 (99)	439 (96)	537 (100)
United States, n (%)	64 (91)	403 (88)	334 (59)	<.001
h-index
Mean (SD)	16 (10)	14 (14)	12 (12)	.001
Median	14	10	8
Range	2–52	0–126	0–73
Publications
Mean (SD)	77 (92)	62 (108)	57 (94)	.217
Median	61	32	24
Range	3–706	1–1,305	1–876
Citations
Mean (SD)	1,396 (2,133)	1,447 (4,218)	1,048 (2,427)	.142
Median	701	406	216
Range	8–10,985	0-65,905	0–21,360

Table 2 contains financial information from the Centers for Medicare and Medicaid Services (CMS) Open Payments Web site. Although there was no statistically significant difference with respect to General Payments among the 3 groups, authors had higher mean total payments ($41,738) compared with editors ($13,712) and reviewers ($20,457), and these results were statistically significant (P < .03). Median total payments were similar among the 3 groups. Nine percent of authors (26 of 286) had total payments above the mean, which suggested that a small group of authors receiving high payment amounts contributed to the increased mean value relative to the median. One of 60 editors (2%), 8 of 351 reviewers (2%), and 8 of 286 authors (3%) had total payments greater than $100,000.

Table 2

Comparison of 2018 Financial Relationships Reported in CMS Open Payments for JHS Editors, Reviewers, and Authors

	Editors	Reviewers	Authors	P Value
US physicians, n	63	387	334
Have Open Payments profile, n (% total)	60 (95)	351 (91)	286 (87)	.024
General payment
Mean (SD)	$8,110 (16,188)	$19,866 (110,101)	$36,818 (246,442)	.347
Median	$894	$760	$781
Range	$0 to $81,265	$0 to $1,497,187	$11 to $3,340,594
Total payments
Mean (SD)	$13,712 (43,332)	$20,457 (110,912)	$41,738 (255,278)	.027
Median	$945	$826	$928
Range	$11 to $316,028	$11 to $1,497,187	$11 to $3,445,886

Discussion

Although multiple forms of bias may affect the peer review process, the impact of these potential sources of bias has been infrequently analyzed within hand surgery.4, 5, 6 Gender appears to be equally distributed among editors, reviewers, and authors in JHS. Despite reaching statistical significance, the actual percentages of women in the 3 groups were relatively similar. As of 2016, 14% of US and 10% of international members of the ASSH were women. The percentage of female authors (17%), reviewers (19%), and editors (20%) in our investigation is comparable to those found in the ASSH membership. Within orthopedic journals, the percentage of female authors and journal editors increased from 1970 to 2007, with women representing 6.5% of first authors in 2007. Similar findings were noted within the Plastic and Reconstructive Surgery journal, which also noted an increase in female authorship. However, the authors found that increases in female authorship lagged behind those in other subspecialties. It remains uncertain whether gender discrepancies among editors, reviewers, or authors contribute to publication bias within orthopedic journals.

In contrast to gender, there are large geographic differences among authors, reviewers, and editors within JHS. A total of 59% of authors were from the United States compared with 91% of editors and 88% of reviewers. Although there is a paucity of evidence within the hand surgery literature with respect to geographic bias in the peer review process, Link demonstrated that for papers submitted to Gastroenterology, both US and non-US reviewers evaluate papers submitted by US authors more favorably. In addition, US reviewers appear to have a marked preference for US-authored papers. Considering these findings, there may be potential bias against articles produced by authors whose first language is not English.

Within the JHS, editors have a higher h-index compared with reviewers and authors. When viewed as groups, it may be beneficial for both editors and reviewers to have more academic and peer review experience compared with authors. Similarly, we note that median publications for editors were nearly twice as high as those for reviewers and 2.5 times higher than median author publications. Within sports medicine, the h-index of editorial board members functions as a significant predictor of journal impact factor. There are, however, some notable limitations to using the h-index as a measure of academic prowess. The Matthew effect describes a form of bias in which well-known authors tend to accrue more citations as opposed to less well-known authors, which can function to increase the h-index. Furthermore, self-citation can increase an author’s h-index., Despite these limitations, the h-index can be used as an indicator of both publication impact and academic advancement within orthopedic surgery.

The Journal of Hand Surgery authors had significantly higher mean total payments as reported in Open Payments compared with editors and reviewers. Previous investigations of orthopedic surgeons in the Open Payments database noted that approximately one-half of surgeons had a financial relationship with industry. Both Cvetanovich et al and Iyer et al found that a relatively small group of orthopedic surgeons received large royalties, accounting for most payments. Our results echo these findings. There were large differences between mean and median total payments in all 3 groups. For example, authors demonstrated mean total payments of $41,738, whereas the median was $928 (range, $11 to $3,445,886). Upon closer inspection, only 9% of authors (26 of 286) had total payments above the mean, which suggests that a small group of authors receiving high payment amounts contributed to the increased mean value relative to the median. In addition to potential bias from financial conflicts, it remains possible that intellectual bias could affect the peer review process. For example, previous authors demonstrated that studies involving total joint implants were influenced by initial results from institutions responsible for implant development, and that these initial results could be difficult to reproduce. We agree with the recent JHS editorial suggesting that although industry involvement in orthopedic research is unavoidable and can aid innovation, the effects of conflict of interest must continue to be addressed. A total of 41% of authors reside outside the United States and are not required to participate within the Open Payments database. This represents a substantial number of authors whose conflict of interest cannot easily be verified.

There are some notable limitations to our study. Our investigation was limited to a single journal and single calendar year, and thus our results may not be generalizable to other journals or time periods. We were unable to locate 13% of authors in the Open Payments database, and it is uncertain how these additional individuals would have affected our financial analysis. Also, we were unable to obtain information regarding age or academic rank. Furthermore, there are limitations to the information contained in the CMS Open Payments database; prior authors noted discrepancies between payment disclosures by authors and those found within the Open Payments database.

In 2018, there appeared to be an even distribution with respect to gender among editors, authors, and reviewers in JHS. Whereas 41% of authors were from outside the United States, international editors and reviewers were relatively underrepresented. Editors and reviewers demonstrated higher h-indices compared with authors. In the CMS Open Payments database, JHS authors had significantly higher mean total payments compared with both editors and reviewers. Defining demographics for journal editors, reviewers, and authors may aid in identifying potential sources of both author and peer review bias, and these results may serve as a foundation for future investigations into potential bias in the peer-reviewed hand surgery literature.