Gregory E Simon1, Eric Johnson1, Jean M Lawrence1, Rebecca C Rossom1, Brian Ahmedani1, Frances L Lynch1, Arne Beck1, Beth Waitzfelder1, Rebecca Ziebell1, Robert B Penfold1, Susan M Shortreed1. 1. From the Kaiser Permanente Washington Health Research Institute, Seattle; the Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena; the HealthPartners Institute, Minneapolis; the Center for Health Services Research, Henry Ford Health System, Detroit; the Center for Health Research, Kaiser Permanente Northwest, Portland, Oreg.; the Institute for Health Research, Kaiser Permanente Colorado, Denver; and the Center for Health Research, Kaiser Permanente Hawaii, Honolulu.
Abstract
OBJECTIVE: The authors sought to develop and validate models using electronic health records to predict suicide attempt and suicide death following an outpatient visit. METHOD: Across seven health systems, 2,960,929 patients age 13 or older (mean age, 46 years; 62% female) made 10,275,853 specialty mental health visits and 9,685,206 primary care visits with mental health diagnoses between Jan. 1, 2009, and June 30, 2015. Health system records and state death certificate data identified suicide attempts (N=24,133) and suicide deaths (N=1,240) over 90 days following each visit. Potential predictors included 313 demographic and clinical characteristics extracted from records for up to 5 years before each visit: prior suicide attempts, mental health and substance use diagnoses, medical diagnoses, psychiatric medications dispensed, inpatient or emergency department care, and routinely administered depression questionnaires. Logistic regression models predicting suicide attempt and death were developed using penalized LASSO (least absolute shrinkage and selection operator) variable selection in a random sample of 65% of the visits and validated in the remaining 35%. RESULTS: Mental health specialty visits with risk scores in the top 5% accounted for 43% of subsequent suicide attempts and 48% of suicide deaths. Of patients scoring in the top 5%, 5.4% attempted suicide and 0.26% died by suicide within 90 days. C-statistics (equivalent to area under the curve) for prediction of suicide attempt and suicide death were 0.851 (95% CI=0.848, 0.853) and 0.861 (95% CI=0.848, 0.875), respectively. Primary care visits with scores in the top 5% accounted for 48% of subsequent suicide attempts and 43% of suicide deaths. C-statistics for prediction of suicide attempt and suicide death were 0.853 (95% CI=0.849, 0.857) and 0.833 (95% CI=0.813, 0.853), respectively. CONCLUSIONS: Prediction models incorporating both health record data and responses to self-report questionnaires substantially outperform existing suicide risk prediction tools.
OBJECTIVE: The authors sought to develop and validate models using electronic health records to predict suicide attempt and suicide death following an outpatient visit. METHOD: Across seven health systems, 2,960,929 patients age 13 or older (mean age, 46 years; 62% female) made 10,275,853 specialty mental health visits and 9,685,206 primary care visits with mental health diagnoses between Jan. 1, 2009, and June 30, 2015. Health system records and state death certificate data identified suicide attempts (N=24,133) and suicide deaths (N=1,240) over 90 days following each visit. Potential predictors included 313 demographic and clinical characteristics extracted from records for up to 5 years before each visit: prior suicide attempts, mental health and substance use diagnoses, medical diagnoses, psychiatric medications dispensed, inpatient or emergency department care, and routinely administered depression questionnaires. Logistic regression models predicting suicide attempt and death were developed using penalized LASSO (least absolute shrinkage and selection operator) variable selection in a random sample of 65% of the visits and validated in the remaining 35%. RESULTS: Mental health specialty visits with risk scores in the top 5% accounted for 43% of subsequent suicide attempts and 48% of suicide deaths. Of patients scoring in the top 5%, 5.4% attempted suicide and 0.26% died by suicide within 90 days. C-statistics (equivalent to area under the curve) for prediction of suicide attempt and suicide death were 0.851 (95% CI=0.848, 0.853) and 0.861 (95% CI=0.848, 0.875), respectively. Primary care visits with scores in the top 5% accounted for 48% of subsequent suicide attempts and 43% of suicide deaths. C-statistics for prediction of suicide attempt and suicide death were 0.853 (95% CI=0.849, 0.857) and 0.833 (95% CI=0.813, 0.853), respectively. CONCLUSIONS: Prediction models incorporating both health record data and responses to self-report questionnaires substantially outperform existing suicide risk prediction tools.
Authors: Ronald C Kessler; Christopher H Warner; Christopher Ivany; Maria V Petukhova; Sherri Rose; Evelyn J Bromet; Millard Brown; Tianxi Cai; Lisa J Colpe; Kenneth L Cox; Carol S Fullerton; Stephen E Gilman; Michael J Gruber; Steven G Heeringa; Lisa Lewandowski-Romps; Junlong Li; Amy M Millikan-Bell; James A Naifeh; Matthew K Nock; Anthony J Rosellini; Nancy A Sampson; Michael Schoenbaum; Murray B Stein; Simon Wessely; Alan M Zaslavsky; Robert J Ursano Journal: JAMA Psychiatry Date: 2015-01 Impact factor: 21.596
Authors: Christine Y Lu; Christine Stewart; Ameena T Ahmed; Brian K Ahmedani; Karen Coleman; Laurel A Copeland; Enid M Hunkeler; Matthew D Lakoma; Jeanne M Madden; Robert B Penfold; Donna Rusinak; Fang Zhang; Stephen B Soumerai Journal: Pharmacoepidemiol Drug Saf Date: 2014-02 Impact factor: 2.890
Authors: Yuval Barak-Corren; Victor M Castro; Solomon Javitt; Alison G Hoffnagle; Yael Dai; Roy H Perlis; Matthew K Nock; Jordan W Smoller; Ben Y Reis Journal: Am J Psychiatry Date: 2016-09-09 Impact factor: 18.112
Authors: Gregory K Brown; Thomas Ten Have; Gregg R Henriques; Sharon X Xie; Judd E Hollander; Aaron T Beck Journal: JAMA Date: 2005-08-03 Impact factor: 56.272
Authors: Thomas H McCoy; Victor M Castro; Ashlee M Roberson; Leslie A Snapper; Roy H Perlis Journal: JAMA Psychiatry Date: 2016-10-01 Impact factor: 21.596
Authors: Julie E Richards; Susan M Shortreed; Greg E Simon; Robert B Penfold; Joseph E Glass; Rebecca Ziebell; Emily C Williams Journal: Gen Hosp Psychiatry Date: 2019-12-18 Impact factor: 3.238
Authors: Justin T Baker; Laura T Germine; Kerry J Ressler; Scott L Rauch; William A Carlezon Journal: Neuropsychopharmacology Date: 2018-08-02 Impact factor: 7.853
Authors: Emily E Haroz; Colin G Walsh; Novalene Goklish; Mary F Cwik; Victoria O'Keefe; Allison Barlow Journal: Suicide Life Threat Behav Date: 2019-11-06
Authors: Brian K Ahmedani; Joslyn Westphal; Kirsti Autio; Farah Elsiss; Edward L Peterson; Arne Beck; Beth E Waitzfelder; Rebecca C Rossom; Ashli A Owen-Smith; Frances Lynch; Christine Y Lu; Cathrine Frank; Deepak Prabhakar; Jordan M Braciszewski; Lisa R Miller-Matero; Hsueh-Han Yeh; Yong Hu; Riddhi Doshi; Stephen C Waring; Gregory E Simon Journal: Prev Med Date: 2019-08-07 Impact factor: 4.018
Authors: Gregory E Simon; Bobbi Jo Yarborough; Rebecca C Rossom; Jean M Lawrence; Frances L Lynch; Beth E Waitzfelder; Brian K Ahmedani; Susan M Shortreed Journal: Psychiatr Serv Date: 2018-12-10 Impact factor: 3.084
Authors: Carrie Reale; Laurie L Novak; Katelyn Robinson; Christopher L Simpson; Jessica D Ribeiro; Joseph C Franklin; Michael Ripperger; Colin G Walsh Journal: AMIA Annu Symp Proc Date: 2021-01-25