Narasimhan Danthi1, Colin O Wu, Peibei Shi, Michael Lauer. 1. From the Advanced Technologies and Surgery Branch (N.D.), the Office of Biostatistics Research (C.O.W., P.S.), and the Office of the Director (M.L.), Division of Cardiovascular Sciences of the National Heart, Lung, and Blood Institute (NHLBI), Bethesda, MD.
Abstract
RATIONALE: Funding decisions for cardiovascular R01 grant applications at the National Heart, Lung, and Blood Institute (NHLBI) largely hinge on percentile rankings. It is not known whether this approach enables the highest impact science. OBJECTIVE: Our aim was to conduct an observational analysis of percentile rankings and bibliometric outcomes for a contemporary set of funded NHLBI cardiovascular R01 grants. METHODS AND RESULTS: We identified 1492 investigator-initiated de novo R01 grant applications that were funded between 2001 and 2008 and followed their progress for linked publications and citations to those publications. Our coprimary end points were citations received per million dollars of funding, citations obtained <2 years of publication, and 2-year citations for each grant's maximally cited paper. In 7654 grant-years of funding that generated $3004 million of total National Institutes of Health awards, the portfolio yielded 16 793 publications that appeared between 2001 and 2012 (median per grant, 8; 25th and 75th percentiles, 4 and 14; range, 0-123), which received 2 224 255 citations (median per grant, 1048; 25th and 75th percentiles, 492 and 1932; range, 0-16 295). We found no association between percentile rankings and citation metrics; the absence of association persisted even after accounting for calendar time, grant duration, number of grants acknowledged per paper, number of authors per paper, early investigator status, human versus nonhuman focus, and institutional funding. An exploratory machine learning analysis suggested that grants with the best percentile rankings did yield more maximally cited papers. CONCLUSIONS: In a large cohort of NHLBI-funded cardiovascular R01 grants, we were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.
RATIONALE: Funding decisions for cardiovascular R01 grant applications at the National Heart, Lung, and Blood Institute (NHLBI) largely hinge on percentile rankings. It is not known whether this approach enables the highest impact science. OBJECTIVE: Our aim was to conduct an observational analysis of percentile rankings and bibliometric outcomes for a contemporary set of funded NHLBI cardiovascular R01 grants. METHODS AND RESULTS: We identified 1492 investigator-initiated de novo R01 grant applications that were funded between 2001 and 2008 and followed their progress for linked publications and citations to those publications. Our coprimary end points were citations received per million dollars of funding, citations obtained <2 years of publication, and 2-year citations for each grant's maximally cited paper. In 7654 grant-years of funding that generated $3004 million of total National Institutes of Health awards, the portfolio yielded 16 793 publications that appeared between 2001 and 2012 (median per grant, 8; 25th and 75th percentiles, 4 and 14; range, 0-123), which received 2 224 255 citations (median per grant, 1048; 25th and 75th percentiles, 492 and 1932; range, 0-16 295). We found no association between percentile rankings and citation metrics; the absence of association persisted even after accounting for calendar time, grant duration, number of grants acknowledged per paper, number of authors per paper, early investigator status, human versus nonhuman focus, and institutional funding. An exploratory machine learning analysis suggested that grants with the best percentile rankings did yield more maximally cited papers. CONCLUSIONS: In a large cohort of NHLBI-funded cardiovascular R01 grants, we were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.
Entities:
Keywords:
National Heart, Lung, and Blood Institute (U.S.); bibliometrics
Authors: Nancy E Mayo; James Brophy; Mark S Goldberg; Marina B Klein; Sydney Miller; Robert W Platt; Judith Ritchie Journal: J Clin Epidemiol Date: 2006-03-27 Impact factor: 6.437
Authors: Jonathan R Kaltman; Frank J Evans; Narasimhan S Danthi; Colin O Wu; Donna M DiMichele; Michael S Lauer Journal: Circ Res Date: 2014-09-12 Impact factor: 17.367
Authors: Adishesh K Narahari; J Hunter Mehaffey; Robert B Hawkins; Eric J Charles; Pranav K Baderdinni; Anirudha S Chandrabhatla; Joseph W Kocan; R Scott Jones; Gilbert R Upchurch; Irving L Kron; John A Kern; Gorav Ailawadi Journal: J Am Coll Surg Date: 2018-02-23 Impact factor: 6.113
Authors: Richard R Fabsitz; George J Papanicolaou; Phyliss Sholinsky; Sean A Coady; Cashell E Jaquish; Cheryl R Nelson; Jean L Olson; Mona A Puggal; Kevin L Purkiser; Pothur R Srinivas; Gina S Wei; Michael Wolz; Paul D Sorlie Journal: Circulation Date: 2015-10-27 Impact factor: 29.690