Female-Authored Articles Are More Likely to Include Methods-Trained Authors.

OBJECTIVE: Studies with authors trained in research methods are of higher quality than those without. We examined inclusion of authors with master's or doctoral degrees incorporating advanced research methods training on original research articles in high-impact journals, investigating differences between journals and by first-author sex.
METHODS: Using all original research articles from 1 issue of The New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), Annals of Internal Medicine (Annals), and JAMA-Internal Medicine/Archives of Internal Medicine (Archives) every alternate month, February 1994 to October 2016, we assessed the prevalence of articles listing authors with master's/doctoral research degrees and its adjusted associations with time of publication, journal, and first-author sex via multivariable logistic regression models (accounting for number of authors, study type, specialty/topic, and continent and for interactions between journal and time of publication, study type, and continent).
RESULTS: Of 3009 articles examined, 84.4% (n=2539) had authors listing research degrees. After adjustment, the prevalence of such articles increased from 1994 to 2016 (P<.001), but patterns differed among journals. Annals and NEJM increased to approximately100% by 2016; JAMA and Archives peaked around 2010 to 2011, then declined. Articles with female first authors were more likely to list authors with research degrees (adjusted odds ratio=1.66; 95% CI, 1.29-2.13; P<.001).
CONCLUSION: The prevalence of original research articles listing authors trained in research methods in high-impact journals increased significantly but is now declining at some journals, with potential effects on quality. The greater prevalence among female first-authored articles suggests possible sex differences in structuring/crediting research teams or subconscious sex bias during review.

Several studies have shown substantial increases over time in the number of authors per article in the peer-reviewed medical literature.1, 2, 3, 4, 5, 6, 7, 8 Contributing factors include the increasing complexity and interdisciplinary nature of research (leading to involvement and crediting of experts from multiple fields),3, 9, 10, 11, 12 the shift toward large multicenter studies (with leaders from each site included in the author list),7, 13, 14 increasing international publication (with academic traditions around the inclusion of students, mentors, or department chairs differing between countries),2, 7, 15 and “author inflation” (also known as gift or guest authorship).3, 16 A question that has received much less attention is how the composition of author lists has changed. This is important because published studies that include authors trained in research methods have been found to be of higher quality than those that do not. Of particular interest, therefore, is the inclusion of authors trained in research methods on original research articles in the high-impact publications most likely to influence clinical practice and health policy, and whether this differs according to journal or sex of the first author.

Methods

We assessed the prevalence of original research articles with authors trained in research methods—defined as at least 1 author holding a doctoral degree (doctor of philosophy [PhD], doctor of public health [DrPH], or doctor of science [ScD]) or a master's degree (master of science [MS] or master of public health [MPH]) that incorporated training in research methods—published in high-impact general medical journals for 1994 to 2016.

We also examined whether the sex of the first author was associated with inclusion of authors trained in research methods. Last, we assessed whether the prevalence of articles with authors trained in research methods changed over time or differed among the journals considered. Because only published data were collected, institutional review board approval was deemed unnecessary.

Inclusion and Exclusion Criteria

The article sampling method has been previously described. Briefly, we considered the 6 journals with the highest 2015 Journal Citation Reports impact factors in the category “medicine, general and internal”: Annals of Internal Medicine (Annals), Archives of Internal Medicine/JAMA-Internal Medicine (JAMA-IM), The BMJ, Journal of the American Medical Association (JAMA), The Lancet, and The New England Journal of Medicine (NEJM). The BMJ and The Lancet were excluded on discovery that they either do not list authors' academic credentials or list only 1 degree per author.

Data were collected for all original research articles (including meta-analyses) published in 1 issue per journal from even-numbered months (February, April, June, August, October, December) from February 1994 to October 2016. This strategy was designed to ensure the sample was robust to short-term variations of key study variables such as authors' degrees, and first authors' sex.

Variables of Interest

For each original research article we collected the first author's sex, authors' degrees, time of publication (year and month), journal name, total number of authors, study type (experimental or nonexperimental), specialty/topic (grouped into general medicine, cardiovascular disease/surgery, infectious disease, oncology, human immunodeficiency virus/acquired immunodeficiency syndrome, other), and country where the study was conducted (grouped into continents).

Author degrees were assessed using only those reported in the article heading. All listed degrees were included. Thus, for example, a single author listing an MD and a PhD was counted as contributing both a clinical degree (MD) and a degree incorporating research methods training (PhD). First-author sex was determined through inspection of the first name, followed by searching on institutional websites, social media, and Internet search engines to find photographs or biographical paragraphs. We previously reported approximately 97% success in identifying the first author's sex.

Data Quality and Cross-check

Using a standardized tool created for the study, data were collected by trained abstractors and cross-checked for accuracy using duplicate assessment of all NEJM articles. The proportion of discrepancies between abstractors across all data fields collected was 3.8%. Where discrepancies could not be easily resolved, 1 of us (G.F.) arbitrated a resolution through discussion with abstractors.

Statistical Analyses

All the article characteristics were tabulated and compared by inclusion of authors trained in research methods; Wilcoxon rank sum tests were used for continuous variables and Pearson χ2 tests for categorical variables.

A logistic regression approach was used to assess (1) the adjusted prevalence of articles listing authors trained in research methods (as described previously herein, doctoral or master's degree) and whether the prevalence changed over time or differed among the journals considered and (2) the association between the sex of the first author and the inclusion of authors trained in research methods. The model included the following independent variables: the first author's sex; journal; time of publication (incremental by month from February 1994 [month 1] to October 2016 [month 273]) modeled using a 5-knot restricted cubic spline21, 22; total number of listed authors (also modeled using cubic splines); study type; continent; specialty/topic; and interaction terms between journal and time of publication and between journal and study type. All fixed effects in this model, including potential interaction terms, were prespecified based on the nearly identical model we previously used to examine the association between female first authorship and journal. Additional 2-way interactions between first-author sex*time, first-author sex*journal, and journal*continent and a 3-way interaction between first-author sex*time*journal were tested, but none of these terms were included in the final model because they lacked significance at the P<.20 level.

Estimates from these models were used to calculate adjusted odds ratios for the measure of interest. Adjusted plots of these associations with time of publication, journal, and first-author sex were generated. All the analyses were performed using SAS software, version 9.4 (SAS Institute Inc).

Results

Of the 3009 sampled original research articles, 84.4% (n=2539) had an author holding a master's- or doctoral-level research degree. Of those, 93.2% (n=2367) had at least 1 author with a doctoral-level research degree, and 49.8% (n=1265) had at least 1 author with a master's-level research degree, indicating that it was not uncommon for multiple authors to be trained in research methods, with differing backgrounds or degrees of training.

Table shows the unadjusted comparison of characteristics between articles that did and did not include at least 1 author with a research methods degree. The unadjusted prevalence of original articles listing authors trained in research methods was higher when the first author was female (89.3% vs 81.9%; P<.001) or the articles were published in JAMA (92.1% vs NEJM=88.4% vs Annals=83.2% vs JAMA-IM=79.6%; P<.001) or in more recent years (73.4% of articles in 1994-1999 to 94.9% in 2012-2016; P<.001). Significant variation in the unadjusted prevalence of original articles listing authors trained in research methods was also seen across study types and topics.

Table

Characteristics of Manuscripts Published From 1994 to 2016 by Original Articles Listing Authors Trained in Research Methodsa

Characteristic	Articles including authors trained in research methodsb
	Total (N=3009)	No (n=470)	Yes (n=2539)	P value
First-author sex, No. (%)				<.0001
Male	1993 (66.2)	361 (18.1)	1632 (81.9)
Female	1016 (33.8)	109 (10.1)	907 (89.3)
Journal, No. (%)				<.0001
Annals	612 (20.3)	103 (16.8)	509 (83.2)
JAMA-IM	1243 (41.3)	254 (21.4)	989 (79.6)
JAMA	559 (18.6)	44 (7.9)	515 (92.1)
NEJM	595 (19.8)	69 (11.6)	526 (88.4)
Year of publication, No. (%)				<.0001
1994-1999	864 (28.7)	230 (26.6)	634 (73.4)
2000-2005	879 (29.2)	157 (17.9)	722 (82.1)
2006-2011	798 (26.5)	59 (7.4)	739 (92.6)
2012-2016	468 (15.6)	24 (5.1)	444 (94.9)
Continent of study, No. (%)				<.0001
Asia	79 (2.6)	22 (27.8)	57 (72.2)
Europe	328 (10.9)	63 (19.2)	265 (80.8)
Americas	1858 (61.8)	325 (17.5)	1533 (82.5)
Multicontinent	551 (18.3)	49 (8.9)	502 (91.1)
Unknown	193 (6.4)	11 (5.7)	182 (94.3)
Study type, No. (%)				<.0001
Experimental	689 (22.9)	65 (9.4)	624 (90.6)
Nonexperimental	1998 (66.4)	288 (14.4)	1710 (85.6)
Systematic review/meta-analysis	322 (10.7)	117 (36.3)	205 (63.7)
Study topic, No. (%)				<.01
Cardiology/thoracic surgery	853 (28.3)	154 (18.1)	699 (81.9)
General medicine	1199 (39.8)	175 (14.6)	1024 (85.4)
HIV/AIDS	105 (3.5)	10 (9.5)	95 (90.5)
Infectious disease	369 (12.3)	71 (19.2)	298 (80.8)
Oncology	279 (9.3)	31 (11.1)	248 (88.9)
Other	204 (6.8)	29 (14.2)	175 (85.8)
Total No. of authors listed, mean ± SD	7.9±5.6	4.9±4.3	8.5±5.6	<.0001

Annals = Annals of Internal Medicine; Archives = JAMA-Internal Medicine/Archives of Internal Medicine; HIV/AIDS = human immunodeficiency virus/acquired immunodeficiency syndrome; JAMA = Journal of the American Medical Association; NEJM = The New England Journal of Medicine.

Defined as 1 or more listed authors having a doctoral or master's degree that incorporates training in research methods (master of public health, master of science, doctor of philosophy, doctor of public health, or doctor of science).

After adjustment, the strong increasing prevalence of original articles listing authors trained in research methods from 1994 to 2016 persisted, but significant differences among journals emerged (Figure 1). Annals and NEJM increased to nearly 100% by 2016, whereas JAMA and JAMA-IM seemed to peak around 2010 to 2011 and then decline.

Figure 1

Manuscripts listing authors trained in research methods (≥1 listed author with a doctoral or master's degree that incorporates training in research methods [master of public health, master of science, doctor of philosophy, doctor of public health, or doctor of science]) among listed authors of original research articles from 1994 to 2016 in 4 high-impact general medical journals. Values are adjusted for month of publication (using restricted cubic spline function), journal, journal*month interaction, study type (experimental, nonexperimental, systematic review/meta-analysis), journal*study type interaction, sex of first author, total number of listed authors (using restricted cubic spline function), continent of study (Asia, North America, Europe, unknown, multiple continents), and topic of study (general medicine, cardiology, oncology, infectious disease, human immunodeficiency virus/acquired immunodeficiency syndrome, other). Annals = Annals of Internal Medicine; Archives = JAMA-Internal Medicine/Archives of Internal Medicine; JAMA = Journal of the American Medical Association; NEJM = The New England Journal of Medicine.

The adjusted probability of at least 1 author being trained in research methods increased over time for both female and male first-authored articles (Figure 2). However, the adjusted analysis confirmed the finding that articles with women as first authors were more likely to include at least 1 author with research methods training (odds ratio=1.66; 95% CI, 1.29-2.13; P<.001) (Figure 2).

Figure 2

Manuscripts listing authors trained in research methods (≥1 listed author with a doctoral or master's degree that incorporates training in research methods [master of public health, master of science, doctor of philosophy, doctor of public health, or doctor of science]) among listed authors of original research articles from 1994 to 2016 in 4 high-impact general medical journals by first-author sex. Values are adjusted for month of publication (using restricted cubic spline function), journal, journal*month interaction, study type (experimental, nonexperimental, systematic review/meta-analysis), journal*study type interaction, sex of first author, total number of listed authors (using restricted cubic spline function), continent of study (Asia, North America, Europe, unknown, multiple continents), and topic of study (general medicine, cardiology, oncology, infectious disease, human immunodeficiency virus/acquired immunodeficiency syndrome, other).

Discussion

These results show increasing inclusion of authors trained in research methods on original research articles published in high-impact general medical journals during the past 20 years. The data also revealed variation across study types, topics, and countries of origin. After accounting for these and other article characteristics, significant differences according to the first author's sex and journal were observed. These last 2 findings are particularly interesting when examined over time. The journals showed very different patterns of including authors trained in research methods over the study period: Annals increased fairly steadily, NEJM undulated but showed a general upward trend, and JAMA and JAMA-IM peaked around 2010 to 2011 and then declined. Both female and male first-authored original articles showed a gradual increase in articles with authors trained in research methods from 1994 to 2008, plateauing thereafter. Articles with women as first authors were significantly more likely to include at least 1 author with research methods training throughout the study period and across all journals.

Previous studies describing authorship patterns in high-impact general medical journals focused on the number of authors per article and the prevalence of group authorship.8, 16 Other studies have reported significant increases over time in the proportion of first authors or contributing authors with advanced research degrees in various specialty journals.2, 4, 6, 23 The present results are consistent with these previous findings but are not directly comparable because of the potential difference between specialty journals and general medical journals and because the previous studies looked at the proportion of authors holding these degrees whereas we examined the proportion of articles with at least 1 author holding such a degree.

We found no previous studies comparing the prevalence of original articles listing authors trained in research methods across high-impact general medical journals or according to first-author sex.

The overall increase in the prevalence of original articles listing authors trained in research methods is likely due to the combination of increasing numbers of physicians graduating with MD-PhD, MD-MPH, or other combined degrees that incorporate research methods training,24, 25 and the increased collaboration with nonclinician scientists with the expertise in study design, data management, and statistical analysis needed to conduct the large research studies and use the sophisticated analytic methods that increased computing power and the adoption of health information technology has made possible. The increasing use of more advanced statistical methods (such as multivariable regression, multilevel modeling, survival analysis, and sensitivity analysis), as well as increasing use of study designs that require solid statistical understanding (case-control study, cohort study, randomized controlled trial, meta-analysis), has previously been reported for articles published in JAMA. A similar trend was reported in a review of clinical studies published in 1990, 2000, and 2010 in NEJM, The Lancet, JAMA, and Nature Medicine. A separate study reported that more than 60% of the peer-reviewed articles assigned for an internal medicine residency journal club applied statistical methods taught only at the master's or doctoral level, which provides some insight into how frequently individuals with such training are likely to be needed on the research team and, thus, potentially eligible for authorship. In the present sample, 84.4% of articles had at least 1 master's- or doctoral-level researcher trained in methods among the listed authors, exceeding the rate with which expertise was judged to be needed in the journal club reading assignments. This may be due to the sample being limited to original research articles, which are more likely to involve sophisticated research methods than the case reports, clinical series, and narrative reviews included in the journal club readings and to the highest-impact general medical journals, which might attract or select for more complex study designs and analyses than the broader set of journals.

The differences we observed in the prevalence of articles listing at least 1 author trained in research methods among journals, after accounting for such factors as study type and topic and country of origin, is harder to explain. One possibility is different review practices between journals. Although we do not have information on how these might have changed over the study period, the 4 journals' current descriptions of their editorial review policies do show differences: NEJM states that after a manuscript passes peer review and if the editors are interested in publishing it, the manuscript is sent for review by a statistical consultant, implying that all published articles have undergone statistical review; Annals describes a selective process whereby “[q]uantitative or methods-focused papers that pass initial review are usually also reviewed by [their] statistical editors at a weekly statistical conference.” These practices may contribute to the high probabilities of original research articles in these journals having an author trained in research methods late in the study period. In contrast, the review processes described for JAMA and JAMA-IM make no specific mention of statistical review,29, 30 although these journals do list statistical reviewers or statistical editors among their editorial staff.31, 32

Finally, the finding that original research articles with women as first authors were more likely (after accounting for article characteristics) to have an author trained in research methods raises critical questions. First, this finding may indicate differences between men and women in how they structure their research teams (with women being more likely to include experts in research methods) or award credit to the members of those research teams (with women being more likely to credit team members with research methods training with authorship). Second, this finding may indicate subconscious sex bias in the editorial and peer review process, with articles with female first authors either being held to higher methodological standards that are best met by collaborating with experts on study design and analysis, or only being regarded as credible if the author list includes such an expert. Such subconscious bias has been previously demonstrated in various aspects of scientific review.33, 34, 35, 36 Support for the possibility that female first authors may be held to a higher standard can be found in previous reports that women generally have fewer but overall higher-quality peer-reviewed scientific publications than men. Investigating these potential underlying causes lies beyond the scope of these data, but such work could provide important insights into effective means of addressing the persistent sex gap in the production and impact of the medical research needed to drive evidence-based clinical practice and policy setting.

This study used methods to determine first-author sex similar to those reported previously in similar investigations,38, 39, 40, 41, 42, 43 but some misclassification may have occurred. The low rate of discrepancy between abstractors (2%) provides reassurance that such misclassifications should be rare; more importantly, they would be nondifferential. Some countries have no tradition of reporting multiple postgraduate degrees, and the many different degree types available internationally may result in variable ability to assign equivalents for aggregation. As such, the present results, which rely on the academic degrees reported in the author byline, may underestimate the number of articles for which an author had substantial training in research methods. However, having adjusted for country of origin in the analyses, this limitation should not significantly affect the differences observed over time among journals or by first-author sex.

We did not have access to data indicating how research teams determined who should be listed as authors. All 4 journals reference the International Committee of Medical Journal Editors guidelines for authorship criteria in their information for authors,28, 29, 30, 45 and all require each author to confirm that he or she meets those criteria, although the format does vary.28, 46, 47, 48 The consistency of the authorship criteria should ensure that variation in inclusion of research team members with methods training in the author list is nondifferential across journals.

The present results are based on a large sample of original research articles reporting on studies conducted in 50 countries and published over a more than 20-year study period. The sampling method was designed to ensure a sample that is both representative of the included journals and robust to short-term variations (as short as every 2 months) in the inclusion of authors trained in research methods. Furthermore, we adjusted for study type (which affects methods experts' involvement), specialty/topic, and continent, as well as number of authors to account for the possibility that, even within the defined group of high-impact general medical journals, there might be differences with respect to the type, size, and source of study that is prioritized for publication.

Overall, the present results confirm that the prevalence of original articles listing authors trained in research methods across high-impact general medical journals significantly increased over the past 20 years; we also show, however, that it differs significantly across those journals and for at least 2 journals is currently declining. Given that inclusion of authors trained in research methods is associated with the quality of the research methods in published studies, this finding has important implications for the quality of the evidence published.

We also found that articles on which women were the first authors were more likely to have a listed author trained in research methods. This finding warrants further study by investigators with access to the necessary data and expertise to determine whether it results from differences in how men and women structure/credit their research teams or from subconscious bias during review. If the latter was found to be the underlying cause of the difference we observed, it would need to be mitigated through strategies such as training for editors and reviewers, and use of double-blinded peer and editorial review, to lower the barrier to women's contributions to the research evidence that, being published in the highest-impact journals, is most likely to influence clinical practice and health policy.