Chi-Shin Wu1, Chian-Jue Kuo2, Chu-Hsien Su3, Shi-Heng Wang4, Hong-Jie Dai5. 1. Department of Psychiatry, National Taiwan University Hospital, Taipei, Taiwan R.O.C; College of Medicine, National Taiwan University, Taipei, Taiwan R.O.C. 2. Taipei City Psychiatric Center, Taipei City Hospital, Taipei, Taiwan R.O.C; Department of Psychiatry, School of Medicine, College of Medicine, Taipei Medical University, Taiwan R.O.C. 3. Department of Psychiatry, National Taiwan University Hospital, Taipei, Taiwan R.O.C. 4. Department of Public Health and Department of Occupational Safety and Health, China Medical University, Taichung, Taiwan R.O.C. 5. Department of Electrical Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan R.O.C; School of Post-Baccalaureate Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan R.O.C. Electronic address: hjdai@nkust.edu.tw.
Abstract
BACKGROUND: Many studies have used Taiwan's National Health Insurance Research database (NHIRD) to conduct psychiatric research. However, the accuracy of the diagnostic codes for psychiatric disorders in NHIRD is not validated, and the symptom profiles are not available either. This study aimed to evaluate the accuracy of diagnostic codes and use text mining to extract symptom profile and functional impairment from electronic health records (EHRs) to overcome the above research limitations. METHODS: A total of 500 discharge notes were randomly selected from a medical center's database. Three annotators reviewed the notes to establish gold standards. The accuracy of diagnostic codes for major psychiatric illness was evaluated. Text mining approaches were applied to extract depressive symptoms and function profiles and to identify patients with major depressive disorder. RESULTS: The accuracy of the diagnostic code for major depressive disorder, schizophrenia, and dementia was acceptable but that of bipolar disorder and minor depression was less satisfactory. The performance of text mining approach to recognize depressive symptoms is satisfactory; however, the recall for functional impairment is lower resulting in lower F-scores of 0.774-0.753. Using the text mining approach to identify major depressive disorder, the recall was 0.85 but precision was only 0.69. CONCLUSIONS: The accuracy of the diagnostic code for major depressive disorder in discharge notes was generally acceptable. This finding supports the utilization of psychiatric diagnoses in claims databases. The application of text mining to EHRs might help in overcoming current limitations in research using claims databases.
BACKGROUND: Many studies have used Taiwan's National Health Insurance Research database (NHIRD) to conduct psychiatric research. However, the accuracy of the diagnostic codes for psychiatric disorders in NHIRD is not validated, and the symptom profiles are not available either. This study aimed to evaluate the accuracy of diagnostic codes and use text mining to extract symptom profile and functional impairment from electronic health records (EHRs) to overcome the above research limitations. METHODS: A total of 500 discharge notes were randomly selected from a medical center's database. Three annotators reviewed the notes to establish gold standards. The accuracy of diagnostic codes for major psychiatric illness was evaluated. Text mining approaches were applied to extract depressive symptoms and function profiles and to identify patients with major depressive disorder. RESULTS: The accuracy of the diagnostic code for major depressive disorder, schizophrenia, and dementia was acceptable but that of bipolar disorder and minor depression was less satisfactory. The performance of text mining approach to recognize depressive symptoms is satisfactory; however, the recall for functional impairment is lower resulting in lower F-scores of 0.774-0.753. Using the text mining approach to identify major depressive disorder, the recall was 0.85 but precision was only 0.69. CONCLUSIONS: The accuracy of the diagnostic code for major depressive disorder in discharge notes was generally acceptable. This finding supports the utilization of psychiatric diagnoses in claims databases. The application of text mining to EHRs might help in overcoming current limitations in research using claims databases.
Authors: George Karystianis; Annabeth Simpson; Armita Adily; Peter Schofield; David Greenberg; Handan Wand; Goran Nenadic; Tony Butler Journal: J Med Internet Res Date: 2020-12-24 Impact factor: 5.428