Ronald C Kessler1, Irving Hwang1, Claire A Hoffmire2, John F McCarthy3, Maria V Petukhova1, Anthony J Rosellini4, Nancy A Sampson1, Alexandra L Schneider2, Paul A Bradley5, Ira R Katz6, Caitlin Thompson7,8, Robert M Bossarte9,10. 1. Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, USA. 2. VISN 19 Mental Illness Research, Education and Clinical Care Center, Denver, Colorado, USA. 3. Office of Mental Health Operations, VA Center for Clinical Management Research, Serious Mental Illness Treatment Resource and Evaluation Center, Ann Arbor, Michigan, USA. 4. Center for Anxiety and Related Disorders, Boston University, Boston, Massachusetts, USA. 5. PricewaterhouseCoopers PS LLP, Washington, District of Columbia, USA. 6. Office of Mental Health Operations, Veterans Health Administration, Washington, District of Columbia, USA. 7. Office of Suicide Prevention, Veterans Health Administration, Washington, District of Columbia, USA. 8. Department of Psychiatry, University of Rochester, Rochester, New York, USA. 9. West Virginia University Injury Control Research Center and Department of Behavioral Medicine and Psychiatry, West Virginia University School of Medicine, Morgantown, West Virginia, USA. 10. Office of Suicide Prevention and VISN 2 Center of Excellence for Suicide Prevention, Veterans Health Administration, Washington, District of Columbia, USA.
Abstract
OBJECTIVES: The US Veterans Health Administration (VHA) has begun using predictive modeling to identify Veterans at high suicide risk to target care. Initial analyses are reported here. METHODS: A penalized logistic regression model was compared with an earlier proof-of-concept logistic model. Exploratory analyses then considered commonly-used machine learning algorithms. Analyses were based on electronic medical records for all 6,360 individuals classified in the National Death Index as having died by suicide in fiscal years 2009-2011 who used VHA services the year of their death or prior year and a 1% probability sample of time-matched VHA service users alive at the index date (n = 2,112,008). RESULTS: A penalized logistic model with 61 predictors had sensitivity comparable to the proof-of-concept model (which had 381 predictors) at target thresholds. The machine learning algorithms had relatively similar sensitivities, the highest being for Bayesian additive regression trees, with 10.7% of suicides occurred among the 1.0% of Veterans with highest predicted risk and 28.1% among the 5.0% of with highest predicted risk. CONCLUSIONS: Based on these results, VHA is using penalized logistic regression in initial intervention implementation. The paper concludes with a discussion of other practical issues that might be explored to increase model performance.
OBJECTIVES: The US Veterans Health Administration (VHA) has begun using predictive modeling to identify Veterans at high suicide risk to target care. Initial analyses are reported here. METHODS: A penalized logistic regression model was compared with an earlier proof-of-concept logistic model. Exploratory analyses then considered commonly-used machine learning algorithms. Analyses were based on electronic medical records for all 6,360 individuals classified in the National Death Index as having died by suicide in fiscal years 2009-2011 who used VHA services the year of their death or prior year and a 1% probability sample of time-matched VHA service users alive at the index date (n = 2,112,008). RESULTS: A penalized logistic model with 61 predictors had sensitivity comparable to the proof-of-concept model (which had 381 predictors) at target thresholds. The machine learning algorithms had relatively similar sensitivities, the highest being for Bayesian additive regression trees, with 10.7% of suicides occurred among the 1.0% of Veterans with highest predicted risk and 28.1% among the 5.0% of with highest predicted risk. CONCLUSIONS: Based on these results, VHA is using penalized logistic regression in initial intervention implementation. The paper concludes with a discussion of other practical issues that might be explored to increase model performance.
Authors: Ronald C Kessler; Christopher H Warner; Christopher Ivany; Maria V Petukhova; Sherri Rose; Evelyn J Bromet; Millard Brown; Tianxi Cai; Lisa J Colpe; Kenneth L Cox; Carol S Fullerton; Stephen E Gilman; Michael J Gruber; Steven G Heeringa; Lisa Lewandowski-Romps; Junlong Li; Amy M Millikan-Bell; James A Naifeh; Matthew K Nock; Anthony J Rosellini; Nancy A Sampson; Michael Schoenbaum; Murray B Stein; Simon Wessely; Alan M Zaslavsky; Robert J Ursano Journal: JAMA Psychiatry Date: 2015-01 Impact factor: 21.596
Authors: Mark A Reger; Derek J Smolenski; Nancy A Skopp; Melinda J Metzger-Abamukang; Han K Kang; Tim A Bullman; Sondra Perdue; Gregory A Gahm Journal: JAMA Psychiatry Date: 2015-06 Impact factor: 21.596
Authors: John F McCarthy; Robert M Bossarte; Ira R Katz; Caitlin Thompson; Janet Kemp; Claire M Hannemann; Christopher Nielson; Michael Schoenbaum Journal: Am J Public Health Date: 2015-06-11 Impact factor: 9.308
Authors: Ronald C Kessler; Irving Hwang; Claire A Hoffmire; John F McCarthy; Maria V Petukhova; Anthony J Rosellini; Nancy A Sampson; Alexandra L Schneider; Paul A Bradley; Ira R Katz; Caitlin Thompson; Robert M Bossarte Journal: Int J Methods Psychiatr Res Date: 2017-07-04 Impact factor: 4.035
Authors: R C Kessler; M B Stein; M V Petukhova; P Bliese; R M Bossarte; E J Bromet; C S Fullerton; S E Gilman; C Ivany; L Lewandowski-Romps; A Millikan Bell; J A Naifeh; M K Nock; B Y Reis; A J Rosellini; N A Sampson; A M Zaslavsky; R J Ursano Journal: Mol Psychiatry Date: 2016-07-19 Impact factor: 15.992
Authors: Gregory E Simon; Eric Johnson; Jean M Lawrence; Rebecca C Rossom; Brian Ahmedani; Frances L Lynch; Arne Beck; Beth Waitzfelder; Rebecca Ziebell; Robert B Penfold; Susan M Shortreed Journal: Am J Psychiatry Date: 2018-05-24 Impact factor: 18.112
Authors: Julie E Richards; Susan M Shortreed; Greg E Simon; Robert B Penfold; Joseph E Glass; Rebecca Ziebell; Emily C Williams Journal: Gen Hosp Psychiatry Date: 2019-12-18 Impact factor: 3.238
Authors: John Torous; Mark E Larsen; Colin Depp; Theodore D Cosco; Ian Barnett; Matthew K Nock; Joe Firth Journal: Curr Psychiatry Rep Date: 2018-06-28 Impact factor: 5.285
Authors: Sarah Graham; Colin Depp; Ellen E Lee; Camille Nebeker; Xin Tu; Ho-Cheol Kim; Dilip V Jeste Journal: Curr Psychiatry Rep Date: 2019-11-07 Impact factor: 5.285
Authors: Emily E Haroz; Colin G Walsh; Novalene Goklish; Mary F Cwik; Victoria O'Keefe; Allison Barlow Journal: Suicide Life Threat Behav Date: 2019-11-06
Authors: Carrie Reale; Laurie L Novak; Katelyn Robinson; Christopher L Simpson; Jessica D Ribeiro; Joseph C Franklin; Michael Ripperger; Colin G Walsh Journal: AMIA Annu Symp Proc Date: 2021-01-25