Jiantao Bian1, Samir Abdelrahman2, Jianlin Shi2, Guilherme Del Fiol3. 1. Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States; VA Salt Lake City Health Care System, Salt Lake City, UT, United States; Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, UT, United States. 2. Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States. 3. Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States. Electronic address: guilherme.delfiol@utah.edu.
Abstract
OBJECTIVES: Finding recent clinical studies that warrant changes in clinical practice ("high impact" clinical studies) in a timely manner is very challenging. We investigated a machine learning approach to find recent studies with high clinical impact to support clinical decision making and literature surveillance. METHODS: To identify recent studies, we developed our classification model using time-agnostic features that are available as soon as an article is indexed in PubMed®, such as journal impact factor, author count, and study sample size. Using a gold standard of 541 high impact treatment studies referenced in 11 disease management guidelines, we tested the following null hypotheses: (1) the high impact classifier with time-agnostic features (HI-TA) performs equivalently to PubMed's Best Match sort and a MeSH-based Naïve Bayes classifier; and (2) HI-TA performs equivalently to the high impact classifier with both time-agnostic and time-sensitive features (HI-TS) enabled in a previous study. The primary outcome for both hypotheses was mean top 20 precision. RESULTS: The differences in mean top 20 precision between HI-TA and three baselines (PubMed's Best Match, a MeSH-based Naïve Bayes classifier, and HI-TS) were not statistically significant (12% vs. 3%, p = 0.101; 12% vs. 11%, p = 0.720; 12% vs. 25%, p = 0.094, respectively). Recall of HI-TA was low (7%). CONCLUSION: HI-TA had equivalent performance to state-of-the-art approaches that depend on time-sensitive features. With the advantage of relying only on time-agnostic features, the proposed approach can be used as an adjunct to help clinicians identify recent high impact clinical studies to support clinical decision-making. However, low recall limits the use of HI-TA for literature surveillance.
OBJECTIVES: Finding recent clinical studies that warrant changes in clinical practice ("high impact" clinical studies) in a timely manner is very challenging. We investigated a machine learning approach to find recent studies with high clinical impact to support clinical decision making and literature surveillance. METHODS: To identify recent studies, we developed our classification model using time-agnostic features that are available as soon as an article is indexed in PubMed®, such as journal impact factor, author count, and study sample size. Using a gold standard of 541 high impact treatment studies referenced in 11 disease management guidelines, we tested the following null hypotheses: (1) the high impact classifier with time-agnostic features (HI-TA) performs equivalently to PubMed's Best Match sort and a MeSH-based Naïve Bayes classifier; and (2) HI-TA performs equivalently to the high impact classifier with both time-agnostic and time-sensitive features (HI-TS) enabled in a previous study. The primary outcome for both hypotheses was mean top 20 precision. RESULTS: The differences in mean top 20 precision between HI-TA and three baselines (PubMed's Best Match, a MeSH-based Naïve Bayes classifier, and HI-TS) were not statistically significant (12% vs. 3%, p = 0.101; 12% vs. 11%, p = 0.720; 12% vs. 25%, p = 0.094, respectively). Recall of HI-TA was low (7%). CONCLUSION:HI-TA had equivalent performance to state-of-the-art approaches that depend on time-sensitive features. With the advantage of relying only on time-agnostic features, the proposed approach can be used as an adjunct to help clinicians identify recent high impact clinical studies to support clinical decision-making. However, low recall limits the use of HI-TA for literature surveillance.
Authors: Yindalon Aphinyanaphongs; Ioannis Tsamardinos; Alexander Statnikov; Douglas Hardin; Constantin F Aliferis Journal: J Am Med Inform Assoc Date: 2004-11-23 Impact factor: 4.497
Authors: Salimah Z Shariff; Shayna A D Bejaimal; Jessica M Sontrop; Arthur V Iansavichus; Matthew A Weir; R Brian Haynes; Mark R Speechley; Amardeep Thind; Amit X Garg Journal: J Nephrol Date: 2011 Nov-Dec Impact factor: 3.902
Authors: L L Leape; D W Bates; D J Cullen; J Cooper; H J Demonaco; T Gallivan; R Hallisey; J Ives; N Laird; G Laffel Journal: JAMA Date: 1995-07-05 Impact factor: 56.272