Allison B McCoy1, Adam Wright2, Deevakar Rogith3, Safa Fathiamini4, Allison J Ottenbacher5, Dean F Sittig6. 1. The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: amccoy1@tulane.edu. 2. Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont St., Boston, MA 02115, USA. Electronic address: awright5@partners.org. 3. The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: deevakar.rogith@uth.tmc.edu. 4. The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: safa.fathiamini@uth.tmc.edu. 5. The University of Texas Medical School at Houston, 6410 Fannin St., Ste. 1100, Houston, TX 77030, USA. Electronic address: allison.ottenbacher@nih.gov. 6. The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: dean.f.sittig@uth.tmc.edu.
Abstract
BACKGROUND: Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements. OBJECTIVE: We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review. APPROACH: We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs. RESULTS: Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases. CONCLUSION: A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.
BACKGROUND: Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements. OBJECTIVE: We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review. APPROACH: We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs. RESULTS: Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases. CONCLUSION: A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.
Authors: Joan S Ash; Dean F Sittig; Eric G Poon; Kenneth Guappone; Emily Campbell; Richard H Dykstra Journal: J Am Med Inform Assoc Date: 2007-04-25 Impact factor: 4.497
Authors: Ross Koppel; Joshua P Metlay; Abigail Cohen; Brian Abaluck; A Russell Localio; Stephen E Kimmel; Brian L Strom Journal: JAMA Date: 2005-03-09 Impact factor: 56.272
Authors: Juan Antonio Lossio-Ventura; William Hogan; François Modave; Yi Guo; Zhe He; Xi Yang; Hansi Zhang; Jiang Bian Journal: BMC Med Inform Decis Mak Date: 2018-07-23 Impact factor: 2.796