Majid Afshar1,2, Brihat Sharma3, Sameer Bhalla4, Hale M Thompson3, Dmitriy Dligach5, Randy A Boley3, Ekta Kishen6, Alan Simmons6, Kathryn Perticone3, Niranjan S Karnik3. 1. Division of Health Informatics and Data Science, Loyola University Chicago, Maywood, IL, USA. majid.afshar@wisc.edu. 2. Department of Medicine, University of Wisconsin, 1685 Highland Avenue, Madison, WI, 53705, USA. majid.afshar@wisc.edu. 3. Department of Psychiatry and Behavioral Sciences, Rush University Medical Center, Chicago, IL, USA. 4. Rush Medical College, Rush University, Chicago, IL, USA. 5. Department of Computer Science, Loyola University Chicago, Chicago, IL, USA. 6. Clinical Research Analytics, Research Core, Rush University Medical Center, Chicago, IL, USA.
Abstract
BACKGROUND: Opioid misuse screening in hospitals is resource-intensive and rarely done. Many hospitalized patients are never offered opioid treatment. An automated approach leveraging routinely captured electronic health record (EHR) data may be easier for hospitals to institute. We previously derived and internally validated an opioid classifier in a separate hospital setting. The aim is to externally validate our previously published and open-source machine-learning classifier at a different hospital for identifying cases of opioid misuse. METHODS: An observational cohort of 56,227 adult hospitalizations was examined between October 2017 and December 2019 during a hospital-wide substance use screening program with manual screening. Manually completed Drug Abuse Screening Test served as the reference standard to validate a convolutional neural network (CNN) classifier with coded word embedding features from the clinical notes of the EHR. The opioid classifier utilized all notes in the EHR and sensitivity analysis was also performed on the first 24 h of notes. Calibration was performed to account for the lower prevalence than in the original cohort. RESULTS: Manual screening for substance misuse was completed in 67.8% (n = 56,227) with 1.1% (n = 628) identified with opioid misuse. The data for external validation included 2,482,900 notes with 67,969 unique clinical concept features. The opioid classifier had an AUC of 0.99 (95% CI 0.99-0.99) across the encounter and 0.98 (95% CI 0.98-0.99) using only the first 24 h of notes. In the calibrated classifier, the sensitivity and positive predictive value were 0.81 (95% CI 0.77-0.84) and 0.72 (95% CI 0.68-0.75). For the first 24 h, they were 0.75 (95% CI 0.71-0.78) and 0.61 (95% CI 0.57-0.64). CONCLUSIONS: Our opioid misuse classifier had good discrimination during external validation. Our model may provide a comprehensive and automated approach to opioid misuse identification that augments current workflows and overcomes manual screening barriers.
BACKGROUND: Opioid misuse screening in hospitals is resource-intensive and rarely done. Many hospitalized patients are never offered opioid treatment. An automated approach leveraging routinely captured electronic health record (EHR) data may be easier for hospitals to institute. We previously derived and internally validated an opioid classifier in a separate hospital setting. The aim is to externally validate our previously published and open-source machine-learning classifier at a different hospital for identifying cases of opioid misuse. METHODS: An observational cohort of 56,227 adult hospitalizations was examined between October 2017 and December 2019 during a hospital-wide substance use screening program with manual screening. Manually completed Drug Abuse Screening Test served as the reference standard to validate a convolutional neural network (CNN) classifier with coded word embedding features from the clinical notes of the EHR. The opioid classifier utilized all notes in the EHR and sensitivity analysis was also performed on the first 24 h of notes. Calibration was performed to account for the lower prevalence than in the original cohort. RESULTS: Manual screening for substance misuse was completed in 67.8% (n = 56,227) with 1.1% (n = 628) identified with opioid misuse. The data for external validation included 2,482,900 notes with 67,969 unique clinical concept features. The opioid classifier had an AUC of 0.99 (95% CI 0.99-0.99) across the encounter and 0.98 (95% CI 0.98-0.99) using only the first 24 h of notes. In the calibrated classifier, the sensitivity and positive predictive value were 0.81 (95% CI 0.77-0.84) and 0.72 (95% CI 0.68-0.75). For the first 24 h, they were 0.75 (95% CI 0.71-0.78) and 0.61 (95% CI 0.57-0.64). CONCLUSIONS: Our opioid misuse classifier had good discrimination during external validation. Our model may provide a comprehensive and automated approach to opioid misuse identification that augments current workflows and overcomes manual screening barriers.
Entities:
Keywords:
Computable phenotype; Heroin; Machine learning; Natural language processing; Opioid misuse; Opioid use disorder
Authors: Sabrina Siregar; Daan Nieboer; Michel I M Versteegh; Ewout W Steyerberg; Johanna J M Takkenberg Journal: Interact Cardiovasc Thorac Surg Date: 2019-03-01
Authors: Nihar R Desai; Joseph S Ross; Ji Young Kwon; Jeph Herrin; Kumar Dharmarajan; Susannah M Bernheim; Harlan M Krumholz; Leora I Horwitz Journal: JAMA Date: 2016-12-27 Impact factor: 56.272
Authors: Bridget F Grant; Tulshi D Saha; W June Ruan; Risë B Goldstein; S Patricia Chou; Jeesun Jung; Haitao Zhang; Sharon M Smith; Roger P Pickering; Boji Huang; Deborah S Hasin Journal: JAMA Psychiatry Date: 2016-01 Impact factor: 21.596
Authors: Wei-Hsuan Lo-Ciganic; James L Huang; Hao H Zhang; Jeremy C Weiss; Yonghui Wu; C Kent Kwoh; Julie M Donohue; Gerald Cochran; Adam J Gordon; Daniel C Malone; Courtney C Kuza; Walid F Gellad Journal: JAMA Netw Open Date: 2019-03-01
Authors: Chelsea Canan; Jennifer M Polinski; G Caleb Alexander; Mary K Kowal; Troyen A Brennan; William H Shrank Journal: J Am Med Inform Assoc Date: 2017-11-01 Impact factor: 4.497
Authors: David Goodman-Meza; Amber Tang; Babak Aryanfar; Sergio Vazquez; Adam J Gordon; Michihiko Goto; Matthew Bidwell Goetz; Steven Shoptaw; Alex A T Bui Journal: Open Forum Infect Dis Date: 2022-09-12 Impact factor: 4.423
Authors: Hale M Thompson; Brihat Sharma; Sameer Bhalla; Randy Boley; Connor McCluskey; Dmitriy Dligach; Matthew M Churpek; Niranjan S Karnik; Majid Afshar Journal: J Am Med Inform Assoc Date: 2021-10-12 Impact factor: 7.942