Liwen Ouyang1, Daniel W Apley2, Sanjay Mehrotra1. 1. Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL 60208, USA. 2. Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL 60208, USA apley@northwestern.edu.
Abstract
BACKGROUND AND OBJECTIVE: Electronic medical record (EMR) databases offer significant potential for developing clinical hypotheses and identifying disease risk associations by fitting statistical models that capture the relationship between a binary response variable and a set of predictor variables that represent clinical, phenotypical, and demographic data for the patient. However, EMR response data may be error prone for a variety of reasons. Performing a manual chart review to validate data accuracy is time consuming, which limits the number of chart reviews in a large database. The authors' objective is to develop a new design-of-experiments-based systematic chart validation and review (DSCVR) approach that is more powerful than the random validation sampling used in existing approaches. METHODS: The DSCVR approach judiciously and efficiently selects the cases to validate (i.e., validate whether the response values are correct for those cases) for maximum information content, based only on their predictor variable values. The final predictive model will be fit using only the validation sample, ignoring the remainder of the unvalidated and unreliable error-prone data. A Fisher information based D-optimality criterion is used, and an algorithm for optimizing it is developed. RESULTS: The authors' method is tested in a simulation comparison that is based on a sudden cardiac arrest case study with 23 041 patients' records. This DSCVR approach, using the Fisher information based D-optimality criterion, results in a fitted model with much better predictive performance, as measured by the receiver operating characteristic curve and the accuracy in predicting whether a patient will experience the event, than a model fitted using a random validation sample. CONCLUSIONS: The simulation comparisons demonstrate that this DSCVR approach can produce predictive models that are significantly better than those produced from random validation sampling, especially when the event rate is low.
BACKGROUND AND OBJECTIVE: Electronic medical record (EMR) databases offer significant potential for developing clinical hypotheses and identifying disease risk associations by fitting statistical models that capture the relationship between a binary response variable and a set of predictor variables that represent clinical, phenotypical, and demographic data for the patient. However, EMR response data may be error prone for a variety of reasons. Performing a manual chart review to validate data accuracy is time consuming, which limits the number of chart reviews in a large database. The authors' objective is to develop a new design-of-experiments-based systematic chart validation and review (DSCVR) approach that is more powerful than the random validation sampling used in existing approaches. METHODS: The DSCVR approach judiciously and efficiently selects the cases to validate (i.e., validate whether the response values are correct for those cases) for maximum information content, based only on their predictor variable values. The final predictive model will be fit using only the validation sample, ignoring the remainder of the unvalidated and unreliable error-prone data. A Fisher information based D-optimality criterion is used, and an algorithm for optimizing it is developed. RESULTS: The authors' method is tested in a simulation comparison that is based on a sudden cardiac arrest case study with 23 041 patients' records. This DSCVR approach, using the Fisher information based D-optimality criterion, results in a fitted model with much better predictive performance, as measured by the receiver operating characteristic curve and the accuracy in predicting whether a patient will experience the event, than a model fitted using a random validation sample. CONCLUSIONS: The simulation comparisons demonstrate that this DSCVR approach can produce predictive models that are significantly better than those produced from random validation sampling, especially when the event rate is low.
Authors: V Jagannathan; Charles J Mullett; James G Arbogast; Kevin A Halbritter; Deepthi Yellapragada; Sushmitha Regulapati; Pavani Bandaru Journal: Int J Med Inform Date: 2008-10-05 Impact factor: 4.046
Authors: Serguei Pakhomov; Susan A Weston; Steven J Jacobsen; Christopher G Chute; Ryan Meverden; Véronique L Roger Journal: Am J Manag Care Date: 2007-06 Impact factor: 2.229
Authors: Robert H Lyles; Li Tang; Hillary M Superak; Caroline C King; David D Celentano; Yungtai Lo; Jack D Sobel Journal: Epidemiology Date: 2011-07 Impact factor: 4.822
Authors: Krishnan Radhakrishnan; Mihaela Aslan; Kelly M Harrington; Robert H Pietrzak; Grant Huang; Sumitra Muralidhar; Kelly Cho; Rachel Quaden; David Gagnon; Saiju Pyarajan; Ning Sun; Hongyu Zhao; Michael Gaziano; John Concato; Murray B Stein; Joel Gelernter Journal: Int J Methods Psychiatr Res Date: 2019-02-14 Impact factor: 4.035