Quantitative goals for research output and scholarly impact to enhance basic science R01 grant renewal for cardiothoracic surgeons.

Objectives: Cardiothoracic (CT) surgeons with National Institutes of Health (NIH) R01 funding face a highly competitive renewal process. The factors that contribute to successful grant renewal for CT surgeons remain poorly defined. We hypothesized that renewed basic science grants are associated with high research output and scholarly impact during the preceding award cycle.
Methods: Using a database of academic CT surgeons (n = 992) at accredited training institutions in 2018, we identified basic science R01 grants awarded to CT surgeon principal investigators since 1985. Data for each award were obtained from publicly available online sources. Scholarly impact was evaluated using the NIH-validated relative citation ratio (RCR), defined as an article's citation rate divided by that of R01-funded publications in the same field. Continuous data are presented as medians and analyzed using the Mann-Whitney test.
Results: We identified 102 basic science R01 award cycles, including 33 that were renewed (32.4%). Renewed and nonrenewed awards had a similar start year and funding period. Principal investigators of renewed versus nonrenewed awards were similar in surgical subspecialty, research training, attending experience, academic rank, and previous NIH funding. Renewed awards produced more publications per year over the funding cycle (3.4 vs 1.5; P = .0010) and exhibited a greater median RCR during the funding cycle (0.84 vs 0.66; P = .0183). Conclusions: CT surgery basic science R01 grants are associated with high research output and scholarly impact. At the 50th percentile among renewed grants, CT surgeons published 3.4 funded manuscripts per year with a median RCR of 0.84 during the previous award cycle.

Research output and impact for basic science R01s that were renewed versus not renewed.

At the 50th percentile among renewed basic science R01 grants, surgeons published 3.4 funded articles per year with a median relative citation ratio of 0.84 during the previous award cycle.

Cardiothoracic surgeons pursuing basic science research face a highly competitive extramural funding environment. Aside from supporting surgeons in the grant acquisition process, it is also important to optimize grant renewal. Herein, we quantify the research output and scholarly impact of renewed versus nonrenewed basic science R01 grants to help guide surgeons in the renewal process.

See Commentaries on pages 176 and 178.

Clinical advances in cardiothoracic (CT) surgery are driven by research and innovation, but concerns have been raised regarding the attrition of CT surgeon-scientists, due in part to an increasingly difficult extramural funding environment. Amidst a 16.5% decrease in National Institutes of Health (NIH) grants awarded to surgeons between 2003 and 2013, per capita NIH awards for CT surgeons in 2008 were more than 3 times less than that for other medical faculty. Furthermore, NIH R01 grants historically account for most research funding for cardiac surgeons, but the number of active cardiac surgery R01 grants has plateaued over the past 2 decades. In response, considerable attention has been focused on supporting CT surgeon-scientists endeavoring to achieve the extramural funding needed to maintain an independent research enterprise, for which the R01 grant remains the standard. To this end, the importance of research training,, mentored career development grants, departmental support and start-up funding, and academic development programs cannot be understated.

In addition to supporting surgeon-scientists on grant acquisition, it is also essential to optimize strategies for grant renewal to sustain funding in the long term. Indeed, the number of competitive renewal awards issued by the NIH to all surgeons declined by 60% between 2003 and 2013, representing a greater decrease than that for other specialties including medicine, pathology, pediatrics, and psychiatry. Furthermore, since 1980, only 17.3% of R01 grants awarded to thoracic surgeons were renewed for a subsequent cycle.

Successful grant renewal plays an important role in long-term funding longevity, but the factors affecting R01 renewal for CT surgeons have not been explored in detail. Herein, we quantify the research output and scholarly impact of renewed versus nonrenewed CT surgery basic science R01 grants to help guide surgeons in the renewal process. We hypothesized that renewed grants are associated with high research output and scholarly impact during the preceding award cycle.

Methods

In this study, we used a biographical database of 992 academic CT surgeons who were on faculty at the university hospital of the 77 accredited United States CT surgery training programs in 2018, as previously described.,10, 11, 12, 13 Emeritus professors, nonsurgical faculty (eg, PhD researchers), and surgeons working at affiliated satellite hospitals were excluded from our database. Data regarding each surgeon's training and professional career were obtained from department webpages, CTSNet (https://www.ctsnet.org), LinkedIn (https://www.linkedin.com), and other online sources. Each surgeon's career publication record was obtained using Scopus (https://www.scopus.com). The NIH funding rank of each surgeon's institution was determined using NIH Research Portfolio Online Reporting Tools (https://report.nih.gov).

We defined “grant” as the overarching R01 funding mechanism, encompassing 1 or more funding cycles, whereas “award” or “award cycle” refers to the individual funding cycles that comprise a grant. To identify R01 grants awarded to the CT surgeons in our database, each surgeon's NIH funding history was obtained using Grantome (https://grantome.com) and NIH Research Portfolio Online Reporting Tools Expenditures and Results (RePORTER; https://reporter.nih.gov). All basic science R01 grants awarded since 1985 (ie, the earliest year included in NIH RePORTER), in which a CT surgeon in our database served as principal investigator (PI) for the entire duration of the funding cycle, were included for analysis. Each grant was assessed for potential renewal through the year 2019. Grants with award cycles that had not been completed by the year 2019 or that had been terminated early (and therefore were not subject to potential renewal at the time of analysis) were excluded. Funding data and articles published during each award cycle were obtained from NIH RePORTER.

Scholarly impact was assessed using the relative citation ratio (RCR), a field-normalized metric developed and validated by the NIH. Briefly, the RCR represents the citation rate of an article divided by the citation rate of R01-funded articles in the same field. Thus, an article with an RCR of 1.00 has an equal number of citations per year as other R01-funded papers in the same field, whereas an article with an RCR of 2.00 has twice the number of citations per year, reflecting greater impact. The RCR of each individual publication was calculated using the NIH iCite database (https://icite.od.nih.gov), and the median and maximum RCR among the publications linked to each R01 award cycle was determined.

All data used in this study were obtained from publicly available online sources. Statistical analyses were performed using GraphPad Prism version 9.0.0 (Graph Pad Software). The D'Agostino–Pearson test was used to assess normality. Continuous data were non-normally distributed and presented as median with interquartile range and analyzed using the Mann–Whitney test. Categorical data are presented as counts with percentages and analyzed using Fisher exact test, or the χ2 test when more than 2 categories were involved.

Results

R01 Award Characteristics

A total of 76 basic science R01 grants were identified (Table E1), encompassing 102 award cycles, of which 33 (32.4%) were subsequently renewed and 69 (67.6%) were not renewed. Among the 992 CT surgeons in our database, 49 (4.9%) had completed a basic science R01 funding cycle as PI, among whom 20 (40.8%) had renewed a basic science R01 grant.

Table E1

List of 76 National Institutes of Health R01 grants included for analysis, with the corresponding cardiothoracic surgeon principal investigator(s)

	Grant identification number	Principal investigator(s)
1	R01AG036954	Ikonomidis, John S.
2	R01AI044078	Pierson, Richard N.
3	R01CA045187	Roth, Jack
4	R01CA090665	Luketich, James D.
5	R01CA093708	Jablons, David M.
6	R01CA131044	Colson, Yolonda L.
7	R01CA132566	Jablons, David M.
8	R01CA136705	Jones, David R.
9	R01CA149561	Colson, Yolonda L.
10	R01CA163256	Singhal, Sunil
11	R01CA176568	Roth, Jack
12	R01HL026640	Foker, John E.
13	R01HL029589	Miller, D. Craig
14	R01HL032257	Damiano, Ralph J.
15	R01HL037499	Miller, D. Craig
16	R01HL038078	Magovern, George J.
17	R01HL038791	Verrier, Edward Donald
18	R01HL041163	Spotnitz, Henry Michael
19	R01HL041281	Patterson, George Alexander
20	R01HL043357	Hanley, Frank
21	R01HL046207	Del Nido, Pedro
22	R01HL046242	Glower, Donald D.
23	R01HL047078	Mentzer, Steven J.
24	R01HL047191	Cameron, Duke
25	R01HL047604	Pasque, Michael K.
26	R01HL048091	Griffith, Bartley Perry
27	R01HL048109	Spotnitz, Henry Michael
28	R01HL051032	Damiano, Ralph J.
29	R01HL056227	Glower, Donald D.
30	R01HL057310	Jessen, Michael E.
31	R01HL057431	Cochran, Richard P.
32	R01HL058781	Bolling, Steven F.
33	R01HL060463	Mayer, John E.
34	R01HL061762	Verrier, Edward Donald
35	R01HL063095	Del Nido, Pedro
36	R01HL063159	Egan, Thomas M.
37	R01HL064950	Griffith, Bartley Perry
38	R01HL066015	Holman, William L.
39	R01HL066981	Rosengart, Todd K.
40	R01HL067025	Miller, D. Craig
41	R01HL067110	Allan, James S.
42	R01HL069949	Moon, Marc R.
43	R01HL070852	Thistlethwaite, Patricia A.
44	R01HL071128	Del Nido, Pedro
45	R01HL071541	Bull, David Andrew
46	R01HL072183	Milano, Carmelo A.
47	R01HL073647	Del Nido, Pedro
48	R01HL074150	Colson, Yolonda L.
49	R01HL075426	Mentzer, Steven J.
50	R01HL075488	Ikonomidis, John S.
51	R01HL080152	Spotnitz, Henry Michael
52	R01HL081106	Griffith, Bartley Perry
53	R01HL082631	Griffith, Bartley Perry
54	R01HL083118	Mann, Michael J.
55	R01HL085095	Rosengart, Todd K.
56	R01HL085341	Coselli, Joseph S.; Lemaire, Scott A.
57	R01HL089269	Del Nido, Pedro
58	R01HL089315	Woo, Y. Joseph
59	R01HL089592	Selzman, Craig Harold
60	R01HL090862	Chen, Frederick Y.
61	R01HL092088	Moon, Marc R.
62	R01HL093097	Mulligan, Michael Scott
63	R01HL094567	Mentzer, Steven J.
64	R01HL094601	Kreisel, Daniel
65	R01HL098182	Lawton, Jennifer S.
66	R01HL098353	Rodefeld, Mark D.
67	R01HL098634	Eghtesady, Pirooz
68	R01HL102121	Ikonomidis, John S.
69	R01HL109132	Gleason, Thomas Gillette
70	R01HL110997	Del Nido, Pedro
71	R01HL113931	Kreisel, Daniel; Krupnick, Alexander
72	R01HL118372	Griffith, Bartley Perry
73	R01HL118491	Kaushal, Sunjay
74	R01HL119543	Thistlethwaite, Patricia A.
75	R01HL124170	Griffith, Bartley Perry
76	R01NS039499	Kern, John A.

Grants are listed in alphabetical order according to identification number.

As presented in Table 1, the characteristics of R01 award cycles that were renewed were similar to those that were not renewed. Renewed and nonrenewed awards exhibited a similar median starting year (2001 vs 2003; P = .4680) and were disbursed over a similar funding period (4.0 years each; P = .7042). Renewed awards were associated with greater 2020 inflation-adjusted funding per year ($494,808 vs $453,676; P = .0799). Renewed awards were also associated predominantly with the National Heart, Lung, and Blood Institute (NHLBI; n = 31; 93.9%; Figure 1, A), and all other renewed awards were associated with the National Cancer Institute (n = 2; 6.1%). Among renewed awards in the NHLBI (Figure 1, B), the most common study sections were Surgery and Bioengineering (n = 12; 38.7%), Bioengineering, Technology, and Surgical Sciences (n = 9; 29.0%), and Surgery, Anesthesiology, and Trauma (n = 6; 19.4%). A similar distribution of NIH institutes (P = .6659) and NHLBI study sections (P = .4309) was observed for the renewed versus nonrenewed award cycles.

Table 1

Characteristics of cardiothoracic surgery basic science R01 awards that were renewed or not renewed

R01 award characteristic	Renewed (n = 33)	Not renewed (n = 69)	P value	95% CI of difference
Award start year	2001 (1996-2009)	2003 (2000-2009)	.4680	−5.0 to 2.0
Length of funding period, years	4.0 (4.0-5.0)	4.0 (4.0-5.0)	.7042	0.0-0.0
2020 Inflation-adjusted funding per year, $	494,808 (428,521-601,583)	453,676 (402,501-539,975)	.0799	−4088 to 107,708
NIH Institute			.6659
NHLBI	31 (93.9)	58 (84.1)
NCI	2 (6.1)	8 (11.6)
National Institute on Aging	0 (0)	1 (1.4)
National Institute of Allergy and Infectious Diseases	0 (0)	1 (1.4)
National Institute of Neurological Disorders and Stroke	0 (0)	1 (1.4)
NHLBI study section			.4309
Surgery and Bioengineering	12 (38.7)	22 (37.9)
Bioengineering, Technology and Surgical Sciences	9 (29.0)	11 (19.0)
Surgery, Anesthesiology, and Trauma	6 (19.4)	9 (15.5)
Special Emphasis Panel	2 (6.5)	9 (15.5)
Cardiac Contractility, Hypertrophy, and Failure	1 (3.2)	2 (3.4)
Lung Biology and Pathology	1 (3.2)	0 (0)
Clinical and Integrative Cardiovascular Sciences	0 (0)	3 (5.2)
Respiratory Integrative Biology and Translational Research	0 (0)	2 (3.4)
NCI study section			.5044
Clinical Oncology	1 (50.0)	0 (0)
Special Emphasis Panel	1 (50.0)	3 (37.5)
Medical Imaging	0 (0)	1 (12.5)
Biomaterials and Biointerfaces	0 (0)	1 (12.5)
Tumor Progression and Metastasis	0 (0)	1 (12.5)
Cancer Etiology	0 (0)	1 (12.5)
Experimental Therapeutics Subcommittee	0 (0)	1 (12.5)

Data are presented as median (interquartile range) or n (%). Percentages might not sum to 100% because of rounding.

NIH, National Institutes of Health; NHLBI, National Heart, Lung, and Blood Institute; NCI, National Cancer Institute.

Figure 1

Distribution of (A) National Institutes of Health (NIH) Institutes and (B) National Heart, Lung, and Blood Institute (NHLBI) study sections among renewed cardiothoracic surgery basic science R01 awards (n = 33). Renewed awards were predominantly associated with the NHLBI (n = 31), whereas the remainder were associated with the National Cancer Institute (NCI; n = 2). The NHLBI study sections among renewed R01 awards included the Surgery and Bioengineering study section (SB; n = 12), the Bioengineering, Technology and Surgical Sciences study section (BTSS; n = 9), the Surgery, Anesthesiology and Trauma study section (SAT; n = 6), the Cardiac Contractility, Hypertrophy, and Failure study section (CCHF; n = 1), the Lung Biology and Pathology study section (LBPA; n = 1), and Special Emphasis Panels (SEP; n = 2).

CT Surgeon PI Characteristics

The characteristics of CT surgeon PIs of award cycles that were renewed were similar to those of award cycles that were not renewed (Table 2). Men accounted for nearly all PIs in both groups (97.0% vs 91.3%; P = .4233, respectively (Figure 2, A), and only 1 woman was PI of a renewed basic science R01 grant. The distribution of cardiac (60.6% vs 56.5%), thoracic (24.2% vs 27.5%), and congenital surgeons (15.2% vs 15.9%) serving as PI was similar among the renewed and nonrenewed awards (P = .9209). The PIs of the renewed and nonrenewed awards had also served as attending surgeons for a similar duration (13.0 vs 11.0 years; P = .6495), and represented a similar distribution of full professors (69.7% vs 66.7%; P = .3976) and department/division chairs (42.4% vs 34.8%; P = .5144), respectively.

Table 2

CT surgeon PI characteristics for basic science R01 awards that were renewed or not renewed

CT surgeon PI characteristic	Renewed (n = 33)	Not renewed (n = 69)	P value	95% CI of difference
Sex			.4233
Male	32 (97.0)	63 (91.3)
Female	1 (3.0)	6 (8.7)
Surgeon Subspecialty			.9209
Cardiac	20 (60.6)	39 (56.5)
Thoracic	8 (24.2)	19 (27.5)
Congenital	5 (15.2)	11 (15.9)
Dedicated research training	20 (60.6)	47 (68.1)	.5072
PhD degree	5 (15.2)	12 (17.4)	>.9999
First author basic science publication during training	23 (69.7)	52 (75.4)	.5440
Start year as attending	1988 (1979-1999)	1990 (1984-1999)	.3413	−7.0 to 2.0
Total years as attending	13.0 (7.0-19.0)	11.0 (7.5-18.0)	.6495	−2.0 to 4.0
Academic rank			.3976
Unknown	4 (12.1)	15 (21.7)
Associate professor	6 (18.2)	8 (11.6)
Full professor	23 (69.7)	46 (66.7)
Department or division chair	14 (42.4)	24 (34.8)	.5144
Previous NIH K grant	5 (15.2)	6 (8.7)	.3282
Previous NIH R grant (non-R01)	7 (21.2)	20 (29.0)	.4777
Previous NIH R01 grant	21 (63.6)	37 (53.6)	.3964
Top 25 NIH-funded institution	23 (69.7)	37 (53.6)	.1228
Career first-author publications	33.0 (21.5-40.0)	27.0 (19.0-43.5)	.3348	−4.0 to 9.0
Career last-author publications	69.0 (28.5-116.5)	40.0 (20.5-95.0)	.1368	−4.0 to 36.0
Career total publications	170.0 (131.0-277.5)	140.0 (87.0-241.5)	.2385	−17.0 to 71.0
Publications per year as attending	10.0 (7.6-12.3)	7.7 (5.2-11.7)	.0745	−0.18 to 3.46
Changed institution during grant	3 (9.1)	7 (10.1)	>.9999
Co-PI	0 (0.0)	1 (1.4)	>.9999

Data are presented as median (interquartile range) or n (%). Percentages might not sum to 100% because of rounding.

CT, Cardiothoracic; PI, principal investigator; NIH, National Institutes of Health.

Figure 2

The characteristics of cardiothoracic surgeon principal investigators are compared for basic science R01 awards that were renewed (n = 33) versus not renewed (n = 69). Similar distributions were observed with regard to (A) surgeon gender (renewed n = 1/33 women vs nonrenewed n = 6/69 women), (B) history of previous R01 grant funding (renewed n = 21/33 vs nonrenewed n = 37/69), or (C) faculty position at a top-25 National Institutes of Health (NIH)-funded institution (renewed n = 23/33 vs nonrenewed n = 37/69).

A similar large percentage of PIs for the renewed and nonrenewed award cycles had pursued a dedicated research fellowship during training (60.6% vs 68.1%; P = .5072) and published a first-author basic science article during training (69.7% vs 75.4%; P = .5440), although only a small percentage in each group had earned a PhD degree (15.2% vs 17.4%, respectively; P > .9999). There was no difference in the proportion of PIs for renewed and nonrenewed awards who had previously received an NIH K grant (15.2% vs 8.7%; P = .3282), an NIH R grant other than an R01 (21.2% vs 29.0%; P = .4777), or a previous R01 grant (63.6% vs 53.6%; P = .3964; Figure 2, B). The PIs for renewed awards more often represented a top-25 NIH-funded institution (69.7% vs 53.6%; P = .1228; Figure 2, C). At the end of the funding cycle (ie, at the time of potential grant renewal), the PIs for renewed awards had been more prolific over their careers (170.0 vs 140.0 total publications; P = .2385) and published more frequently as an attending (10.0 vs 7.7 publications per year; P = .0745). Finally, a similar proportion of renewed and nonrenewed awards involved a CT surgeon PI who changed institutions during the funding cycle (9.1% vs 10.1%; P > .9999), and a similar proportion of renewed and nonrenewed awards were led by a co-PI in addition to the CT surgeon (0.0% vs 1.4%, respectively; P > .9999).

Research Output and Scholarly Impact During the R01 Funding Cycle

Research output and scholarly impact during the R01 funding cycle are presented in Table 3. Awards that were renewed produced more total publications over the funding cycle than awards that were not renewed (16.0 vs 8.0 publications; P = .0058; Figure 3, A), as well as more publications per year over the funding cycle (3.4 vs 1.5 publications per year; P = .0010; Figure 3, B). The publications linked to renewed awards also exhibited greater scholarly impact, in terms of median RCR for the funding cycle (0.84 vs 0.66; P = .0183; Figure 4, A) and maximum RCR for the funding cycle (3.22 vs 2.02; P = .0259; Figure 4, B).

Table 3

Research output and scholarly impact during the funding cycle of cardiothoracic surgery basic science R01 awards that were renewed or not renewed

Research output and scholarly impact during funding cycle	Renewed (n = 33)	Not renewed (n = 69)	P value	95% CI of difference
Total publications	16.0 (6.0-26.0)	8.0 (3.0-14.5)	.0058	2.00-11.00
Publications per year	3.4 (1.6-5.5)	1.5 (0.6-3.1)	.0010	0.58-2.50
Median RCR	0.84 (0.67-1.29)	0.66 (0.40-0.98)	.0183	0.03-0.39
Maximum RCR	3.22 (2.00-5.91)	2.02 (0.70-4.81)	.0259	0.20-1.99

Data are presented as median (interquartile range).

RCR, Relative citation ratio.

Figure 3

Research output for cardiothoracic surgery basic science R01 awards that were renewed (n = 33) versus not renewed (n = 69). A, Renewed awards yielded a greater number of total publications during the funding cycle compared with nonrenewed awards. B, Renewed awards yielded a greater number of publications per year of the funding cycle compared with nonrenewed awards. The upper and lower borders of the box define the interquartile range with the middle horizontal line representing the median. The upper and lower whiskers define the maximum and minimum values of nonoutliers, with additional dots representing outliers.

Figure 4

Scholarly impact of manuscripts published during the funding cycle of cardiothoracic surgery basic science R01 awards that were renewed (n = 33) or not renewed (n = 69). An article with a relative citation ratio (RCR) of 1.00 indicates equal impact as other R01-funded publications in the same field. A, Renewed awards exhibited a greater median RCR among publications linked to the funding cycle compared with nonrenewed awards. B, Renewed awards exhibited a greater maximum RCR among publications linked to the funding cycle compared with nonrenewed awards. The upper and lower borders of the box define the interquartile range with the middle horizontal line representing the median. The upper and lower whiskers define the maximum and minimum values of nonoutliers, with additional dots representing outliers.

Discussion

In this study, we compared the award characteristics, the surgeon PIs, and the publications funded by basic science CT surgery R01 awards that were renewed versus those that were not renewed. We observed similar attributes among the CT surgeon PIs of renewed versus nonrenewed awards, including subspecialty type, research training, clinical experience, academic rank, and history of previous NIH funding, although renewed awards tended to be associated with a PI with greater career research output at a top NIH-funded institution. We also observed, however, that renewed awards were associated with more publications during the funding period, and that these funded publications also exhibited a greater scholarly impact. These findings suggest that research output and scholarly impact during the preceding award cycle might represent important factors for R01 grant renewal (Figure 5).

Figure 5

On the basis of data regarding research output and scholarly impact for 102 National Institutes of Health (NIH) basic science R01 funding cycles awarded to cardiothoracic (CT) surgeons, we observed that surgeon-scientists should aim to publish 3.4 articles per year during the R01 funding cycle while maintaining a goal median relative citation ratio (RCR) of 0.84 to be at the 50th percentile among CT surgeons in the renewal process. A publication with an RCR of 1.00 indicates equal impact as a R01-funded publication in the same field. The upper and lower borders of the box define the interquartile range with the middle horizontal line representing the median. The upper and lower whiskers define the maximum and minimum values of nonoutliers, with additional dots representing outliers.

The evaluation of R01 grant applications centers on peer review, involving a standardized scoring system and 2 levels of assessment. In the first round, reviewers from a scientific review group, also known as a study section, evaluate the scientific and technical merit of the proposal and assign a criterion score in each of 5 areas: significance, investigator, innovation, approach, and environment. Renewal status (as opposed to a first-time application) is considered as an additional review criteria at this stage. An overall impact score is then determined for the grant application. Subsequently, a second round of review is conducted by the appropriate NIH institute, which considers the relevance of each application according to the institute's mission, goals, and priorities when determining the final funding decision.

Analyses conducted by the NIH have shown that each of the 5 criterion scores is an important contributor to an R01 application's overall impact score. Interestingly, the approach score appears to be the most important predictor of the overall impact score and the likelihood of funding, followed by the significance and innovation scores. In contrast, the environment score had the lowest association, followed by the investigator score. A follow-up analysis by the NIH, focusing exclusively on R01 renewal applications, confirmed that the approach and significance scores were the strongest predictors of the overall impact score and the ultimate renewal result, and that PI characteristics (eg, age, gender, research training background) were not correlated with success. On the basis of these data, the NIH has emphasized that a well designed and clearly described experimental strategy for continued high-impact research is central to a successful R01 grant renewal application.

In parallel with the NIH data showing that the investigator and environment scores might be least influential among the 5 criterion scores, we noted similar CT surgeon PI characteristics among the renewed versus nonrenewed award cycles. However, we acknowledge the small sample size of our study, and we nevertheless observed that PIs of renewed grants tended to have a higher career publication rate and may be more likely to represent a top-25 NIH-funded institution. As such, the surgeon's academic record and the strength of the institution's research environment might be less influential than the overall impact and strategy of the proposed research, but they are nevertheless important.

Notably, we identified only 1 renewed basic science R01 grant for which a woman in our database served as PI. Krebs and colleagues recently showed that women constitute a greater than anticipated proportion of surgeon-scientists with R01 funding, and that female surgeons with R01 grants were more likely to be first-time awardees with no previous NIH funding. Although women obtain new R01 or equivalent grants with comparable success rates as men, women historically have experienced lower success rates for renewal applications, in part because of lower approach, significance, and overall impact scores than men during peer review. To sustain the women who represent an essential, enlarging group of surgeon-scientists, additional support and attention must be directed toward facilitating research training opportunities and expanding mentorship and sponsorship networks for women in CT surgery. In addition, women remain under-represented in CT surgery department leadership, and previous reports indicate that female surgeons with R01 funding are significantly more likely to be from surgery departments led by women. Women are also under-represented among the leadership of our specialty's national societies, as well as on the NIH study sections that evaluate R01 grant applications. The correction of these gender disparities at the level of local and national leadership might further help female surgeon-scientists maintain long-term research funding.

Importantly, our analysis of publications linked to each R01 funding cycle provides a new quantitative dimension of the R01 renewal application not captured by the 5 criterion scores. Awards that were renewed produced a median of 3.4 publications per year during the funding cycle, translating to a median of 16.0 publications over the course of the award. Although nonrenewed awards were also highly productive in terms of research output, our data suggest that CT surgeons should aim to publish 3.4 funded manuscripts per year during the R01 funding cycle to be at the 50th percentile in the renewal process.

In addition to quantity, we also observed that the impact of research published during an R01 funding cycle might be an important factor distinguishing renewed and nonrenewed awards. Compared with a reference RCR of 1.00 for other R01-funded articles in the same field, we calculated a median RCR of 0.84 among publications linked to each renewed CT surgery R01 award. This result indicates that the scholarly impact of articles funded by renewed CT surgery basic science R01 grants was approximately on par with the expected impact of R01-funded articles in the field, and that CT surgeons should aim to maintain a median RCR at least 0.84 during the R01 funding cycle to be at the 50th percentile in the renewal process.

In our study, the overall R01 renewal rate for CT surgeons was 32.4%, whereas the success rate for renewal applications across the entire NIH was as high as 45% in recent years. However, of the 14 CT surgery R01 awards that published at least 3.4 articles per year with a median RCR at least 0.84, 10 were renewed, yielding an impressive renewal rate of 71.4%. In contrast, the renewal rate for awards with at least 3.4 papers per year but median RCR <0.84 was 7 of 15 (46.7%), on par with the NIH average, whereas the renewal rate for awards with <3.4 articles per year but median RCR of at least 0.84 was 7 of 25 (28.0%). Among the remaining 48 awards with research output of <3.4 articles per year and a median RCR of <0.84, only 9 were renewed (18.8%). These data suggest that high research output alone might be insufficient to enhance the likelihood of renewal compared with the NIH average. Instead, both research output and scholarly impact appear to be important, as high-impact research proposals might have stronger significance, innovation, and approach criterion scores. Awareness of how research output and scholarly impact relate to R01 grant renewal might help CT surgeon-scientists develop a multiyear research plan that optimizes the balance of output and impact. Such planning might be particularly important in basic science research, in which the highest-impact studies might require substantial time to complete supporting experiments to confirm an initial discovery. As a result, pursuing only the highest-impact experiments might result in fewer publications, whereas aiming to publish prolifically but with lower scholarly impact might risk an unfavorable overall impact score.

It is important to note that the research produced by CT surgery basic science R01 grants was highly impactful regardless of renewal status, as the 50th percentile of maximum RCR among nonrenewed award cycles was 2.02, which compares favorably to the reference value of 1.00 for other R01-funded publications in the same field. Renewed awards exhibited an even greater maximum RCR of 3.22 at the 50th percentile. To maintain the high impact of CT surgery basic science research in the future, support for residents and early-career surgeons who represent the next generation of independent surgeon-scientists must be a top priority, at the institutional and national levels. Indeed, in our database, only 7.4% of CT surgery faculty are currently leading basic science research. Aside from providing laboratory training, mentorship, protected research time, career advancement incentives, and recognition of research accomplishments, institutional leaders should facilitate multidisciplinary collaborations to encourage the exchange of innovative ideas and novel experimental techniques. In addition, through scholarships and fellowships, national organizations such as the American Association for Thoracic Surgery (AATS; https://www.aats.org/aatsimis/AATSWeb/Scholarships/AATSWeb/Scholarships/Scholarship_Overview.aspx) and the Society of Thoracic Surgeons and Thoracic Surgery Foundation (https://thoracicsurgeryfoundation.org/awards) are actively supporting high-impact CT surgery research and are working to further expand funding for CT surgeon-scientists by hosting communications with the NIH and other funding sources (eg, through the AATS Scientific Affairs and Government Relations Committee), and by continuing to feature academic development programs (eg, AATS Grant Writing Workshop, Clinical Trials Methods Course, and Innovation Summit) and dedicated conference sessions focused on academic career development during the annual AATS and Society of Thoracic Surgeons meetings.

Limitations

Our study has several limitations that must be carefully considered. First, because our analysis is focused only on NIH basic science R01 funding, our study does not represent a comprehensive assessment of CT surgery research funding, because other funding sources (eg, National Science Foundation, American Cancer Society, Department of Defense, Veterans Affairs) were not included. Future studies will endeavor to incorporate non-NIH funding sources as data become available. Next, our database of CT surgeons is derived from the university faculty at accredited CT surgery training hospitals, excluding surgeons working at affiliated satellite hospitals (eg, Veterans Affairs hospitals, county/community hospitals) and those working at university hospitals without accredited CT surgery training programs. As a result, our database does not include all of the academic CT surgeons in the country, does not include a control group of other surgeons or physician-scientists to which to compare our results, and does not capture every NIH R01 grant awarded to CT surgeons. Because clinical research grants often are not renewed after project completion, we chose to focus our analysis only on basic science R01 grants. With 102 award cycles identified, of which only 33 were renewed, it is possible that our sample size limited the statistical power of some analyses, although our findings regarding the association of high research output and scholarly impact to grant renewal would not be expected to change. Nevertheless, we recognize that during the NIH's evaluation process, the determination of which grants are renewed is on the basis of numerous factors beyond simply the research output and scholarly impact of the previous funding cycle. Thus, further studies with multivariable models including the criterion scores from peer review are required to clarify the relative weight of these factors during R01 renewal evaluation. In addition, we assumed that all R01 awards were intended for renewal, whereas some might have been intended only for a single cycle, leading to an underestimated overall renewal rate. Unfortunately, with the publicly available resources at our disposal, we have access only to data for funded grants and not the full list of grant applications. Finally, we determined the research output of each R01 award cycle using NIH RePORTER, which links publications to the grant on the basis of documentation of funding in each article or by direct PI reporting. Thus, it is possible that some publications supported by an R01 grant were not linked to the funding cycle, leading to underestimates for research output.

Conclusions

Overall, we observed that CT surgery basic science R01 awards are associated with high research output and scholarly impact. At the 50th percentile among renewed basic science R01 grants, CT surgeons published 3.4 funded manuscripts per year with a median RCR of 0.84 during the previous award cycle. These goals for research output and scholarly impact might help guide CT surgeon-scientists aiming to renew a basic science R01 grant.

Webcast

You can watch a Webcast of this AATS meeting presentation by going to: https://aats.blob.core.windows.net/media/21%20AM/AM21_A10/AM21_A10_07.mp4.

Conflict of Interest Statement

The authors reported no conflicts of interest.

The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest.