Comparing Deep Learning and Conventional Machine Learning Models for Predicting Mental Illness from History of Present Illness Notations.

Mental illness, a serious problem across the globe, requires multi-pronged solutions including effective computational models to predict illness. Mental illness diagnosis is complicated by the pronounced sharing of symptoms and mutual pre-dispositions. Set in this context we offer a systematic comparison of seven deep learning and two conventional machine learning models for predicting mental illness from the history of present illness free-text descriptions in patient records. The models tested include a new architecture CB-MH which ranks best for F1 (0.62) while another attention model is best for F2 (0.71). We also explore model decisions using Integrated Gradients interpretability method which we use to identify key influential features. Overall, the majority of true positives have key features appearing in meaningful contexts. False negatives are most challenging with most key features appearing in unclear contexts. False positives are mostly true positives in actuality as supported by a small-scale clinician-based user judgement study. ©2021 AMIA - All rights reserved.