Samah Jamal Fodeh1, Mohammed Al-Garadi2, Osama Elsankary3, Jeanmarie Perrone4, William Becker5, Abeed Sarker2. 1. Department of Emergency Medicine, Yale School of Medicine, Yale University, New Haven, CT 06510, USA; VA Connecticut Healthcare System, West Haven, CT 06516, USA. Electronic address: samah.fodeh@yale.edu. 2. Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA 30322, USA. 3. Frank Netter M.D. School of Medicine, Quinnipiac University, North Haven, CT 06473, USA. 4. Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA. 5. VA Connecticut Healthcare System, West Haven, CT 06516, USA.
Abstract
BACKGROUND: Opioid misuse (OM) is a major health problem in the United States, and can lead to addiction and fatal overdose. We sought to employ natural language processing (NLP) and machine learning to categorize Twitter chatter based on the motive of OM. MATERIALS AND METHODS: We collected data from Twitter using opioid-related keywords, and manually annotated 6988 tweets into three classes-No-OM, Pain-related-OM, and Recreational-OM-with the No-OM class representing tweets indicating no use/misuse, and the Pain-related misuse and Recreational-misuse classes representing misuse for pain or recreation/addiction. We trained and evaluated multi-class classifiers, and performed term-level k-means clustering to assess whether there were terms closely associated with the three classes. RESULTS: On a held-out test set of 1677 tweets, a transformer-based classifier (XLNet) achieved the best performance with F1-score of 0.71 for the Pain-misuse class, and 0.79 for the Recreational-misuse class. Macro- and micro-averaged F1-scores over all classes were 0.82 and 0.92, respectively. Content-analysis using clustering revealed distinct clusters of terms associated with each class. DISCUSSION: While some past studies have attempted to automatically detect opioid misuse, none have further characterized the motive for misuse. Our multi-class classification approach using XLNet showed promising performance, including in detecting the subtle differences between pain-related and recreation-related misuse. The distinct clustering of class-specific keywords may help conduct targeted data collection, overcoming under-representation of minority classes. CONCLUSION: Machine learning can help identify pain-related and recreational-related OM contents on Twitter to potentially enable the study of the characteristics of individuals exhibiting such behavior.
BACKGROUND: Opioid misuse (OM) is a major health problem in the United States, and can lead to addiction and fatal overdose. We sought to employ natural language processing (NLP) and machine learning to categorize Twitter chatter based on the motive of OM. MATERIALS AND METHODS: We collected data from Twitter using opioid-related keywords, and manually annotated 6988 tweets into three classes-No-OM, Pain-related-OM, and Recreational-OM-with the No-OM class representing tweets indicating no use/misuse, and the Pain-related misuse and Recreational-misuse classes representing misuse for pain or recreation/addiction. We trained and evaluated multi-class classifiers, and performed term-level k-means clustering to assess whether there were terms closely associated with the three classes. RESULTS: On a held-out test set of 1677 tweets, a transformer-based classifier (XLNet) achieved the best performance with F1-score of 0.71 for the Pain-misuse class, and 0.79 for the Recreational-misuse class. Macro- and micro-averaged F1-scores over all classes were 0.82 and 0.92, respectively. Content-analysis using clustering revealed distinct clusters of terms associated with each class. DISCUSSION: While some past studies have attempted to automatically detect opioid misuse, none have further characterized the motive for misuse. Our multi-class classification approach using XLNet showed promising performance, including in detecting the subtle differences between pain-related and recreation-related misuse. The distinct clustering of class-specific keywords may help conduct targeted data collection, overcoming under-representation of minority classes. CONCLUSION: Machine learning can help identify pain-related and recreational-related OM contents on Twitter to potentially enable the study of the characteristics of individuals exhibiting such behavior.
Authors: R H Perlis; D V Iosifescu; V M Castro; S N Murphy; V S Gainer; J Minnier; T Cai; S Goryachev; Q Zeng; P J Gallagher; M Fava; J B Weilburg; S E Churchill; I S Kohane; J W Smoller Journal: Psychol Med Date: 2011-06-20 Impact factor: 7.723