Jonathan R Kaltman1, Frank J Evans1, Narasimhan S Danthi1, Colin O Wu1, Donna M DiMichele1, Michael S Lauer2. 1. From the Heart Development and Structural Disease Branch (J.R.K., F.J.E.), Advanced Technologies and Surgery Branch (N.S.D.), Office of Biostatistics Research (C.O.W.), and the Office of the Director (M.S.L.), Division of Cardiovascular Sciences of the National Heart, Lung, and Blood Institute (NHLBI), Bethesda, MD; and Office of the Director, Division of Blood Diseases and Resources of the National Heart, Lung, and Blood Institute, Bethesda, MD (D.M.D.). 2. From the Heart Development and Structural Disease Branch (J.R.K., F.J.E.), Advanced Technologies and Surgery Branch (N.S.D.), Office of Biostatistics Research (C.O.W.), and the Office of the Director (M.S.L.), Division of Cardiovascular Sciences of the National Heart, Lung, and Blood Institute (NHLBI), Bethesda, MD; and Office of the Director, Division of Blood Diseases and Resources of the National Heart, Lung, and Blood Institute, Bethesda, MD (D.M.D.). lauerm@nhlbi.nih.gov.
Abstract
RATIONALE: We previously demonstrated absence of association between peer-review-derived percentile ranking and raw citation impact in a large cohort of National Heart, Lung, and Blood Institute cardiovascular R01 grants, but we did not consider pregrant investigator publication productivity. We also did not normalize citation counts for scientific field, type of article, and year of publication. OBJECTIVE: To determine whether measures of investigator prior productivity predict a grant's subsequent scientific impact as measured by normalized citation metrics. METHODS AND RESULTS: We identified 1492 investigator-initiated de novo National Heart, Lung, and Blood Institute R01 grant applications funded between 2001 and 2008 and linked the publications from these grants to their InCites (Thompson Reuters) citation record. InCites provides a normalized citation count for each publication stratifying by year of publication, type of publication, and field of science. The coprimary end points for this analysis were the normalized citation impact per million dollars allocated and the number of publications per grant that has normalized citation rate in the top decile per million dollars allocated (top 10% articles). Prior productivity measures included the number of National Heart, Lung, and Blood Institute-supported publications each principal investigator published in the 5 years before grant review and the corresponding prior normalized citation impact score. After accounting for potential confounders, there was no association between peer-review percentile ranking and bibliometric end points (all adjusted P>0.5). However, prior productivity was predictive (P<0.0001). CONCLUSIONS: Even after normalizing citation counts, we confirmed a lack of association between peer-review grant percentile ranking and grant citation impact. However, prior investigator publication productivity was predictive of grant-specific citation impact.
RATIONALE: We previously demonstrated absence of association between peer-review-derived percentile ranking and raw citation impact in a large cohort of National Heart, Lung, and Blood Institute cardiovascular R01 grants, but we did not consider pregrant investigator publication productivity. We also did not normalize citation counts for scientific field, type of article, and year of publication. OBJECTIVE: To determine whether measures of investigator prior productivity predict a grant's subsequent scientific impact as measured by normalized citation metrics. METHODS AND RESULTS: We identified 1492 investigator-initiated de novo National Heart, Lung, and Blood Institute R01 grant applications funded between 2001 and 2008 and linked the publications from these grants to their InCites (Thompson Reuters) citation record. InCites provides a normalized citation count for each publication stratifying by year of publication, type of publication, and field of science. The coprimary end points for this analysis were the normalized citation impact per million dollars allocated and the number of publications per grant that has normalized citation rate in the top decile per million dollars allocated (top 10% articles). Prior productivity measures included the number of National Heart, Lung, and Blood Institute-supported publications each principal investigator published in the 5 years before grant review and the corresponding prior normalized citation impact score. After accounting for potential confounders, there was no association between peer-review percentile ranking and bibliometric end points (all adjusted P>0.5). However, prior productivity was predictive (P<0.0001). CONCLUSIONS: Even after normalizing citation counts, we confirmed a lack of association between peer-review grant percentile ranking and grant citation impact. However, prior investigator publication productivity was predictive of grant-specific citation impact.