Kasper van Mens1, Cwm de Schepper2, Ben Wijnen3, Saskia J Koldijk2, Hugo Schnack2, Peter de Looff4, Joran Lokkerbol3, Karen Wetherall5, Seonaid Cleare5, Rory C O'Connor5, Derek de Beurs6. 1. Altrecht mental health Centre, Utrecht, the Netherlands. 2. University medical center Utrecht, departement of psychiatry, Utrecht, the Netherlands. 3. Trimbos-Institute, Netherlands Institute for Mental Health and Addication, Utrecht, the Netherlands. 4. National expert Centre Specialized and Forensic Care, Den Dolder, The Netherlands. 5. Glasgow university, suicidal behaviour research laboratory, Glasgow, United Kingdom. 6. Trimbos-Institute, Netherlands Institute for Mental Health and Addication, Utrecht, the Netherlands. Electronic address: dbeurs@trimbos.nl.
Abstract
BACKGROUND: The predictive accuracy of suicidal behaviour has not improved over the last decades. We aimed to explore the potential of machine learning to predict future suicidal behaviour using population-based longitudinal data. METHOD: Baseline risk data assessed within the Scottish wellbeing study, in which 3508 young adults (18-34 years) completed a battery of psychological measures, were used to predict both suicide ideation and suicide attempts at one-year follow-up. The performance of the following algorithms was compared: regular logistic regression, K-nearest neighbors, classification tree, random forests, gradient boosting and support vector machine. RESULTS: At one year follow up, 2428 respondents (71%) finished the second assessment. 336 respondents (14%) reported suicide ideation between baseline and follow up, and 50 (2%) reported a suicide attempt. All performance metrics were highly similar across methods. The random forest algorithm was the best algorithm to predict suicide ideation (AUC 0.83, PPV 0.52, BA 0.74) and the gradient boosting to predict suicide attempt (AUC 0.80, PPV 0.10, BA 0.69). LIMITATIONS: The number of respondents with suicidal behaviour at follow up was small. We only had data on psychological risk factors, limiting the potential of the more complex machine learning algorithms to outperform regular logistical regression. CONCLUSIONS: When applied to population-based longitudinal data containing multiple psychological measurements, machine learning techniques did not significantly improve the predictive accuracy of suicidal behaviour. Adding more detailed data on for example employment, education or previous health care uptake, might result in better performance of machine learning over regular logistical regression.
BACKGROUND: The predictive accuracy of suicidal behaviour has not improved over the last decades. We aimed to explore the potential of machine learning to predict future suicidal behaviour using population-based longitudinal data. METHOD: Baseline risk data assessed within the Scottish wellbeing study, in which 3508 young adults (18-34 years) completed a battery of psychological measures, were used to predict both suicide ideation and suicide attempts at one-year follow-up. The performance of the following algorithms was compared: regular logistic regression, K-nearest neighbors, classification tree, random forests, gradient boosting and support vector machine. RESULTS: At one year follow up, 2428 respondents (71%) finished the second assessment. 336 respondents (14%) reported suicide ideation between baseline and follow up, and 50 (2%) reported a suicide attempt. All performance metrics were highly similar across methods. The random forest algorithm was the best algorithm to predict suicide ideation (AUC 0.83, PPV 0.52, BA 0.74) and the gradient boosting to predict suicide attempt (AUC 0.80, PPV 0.10, BA 0.69). LIMITATIONS: The number of respondents with suicidal behaviour at follow up was small. We only had data on psychological risk factors, limiting the potential of the more complex machine learning algorithms to outperform regular logistical regression. CONCLUSIONS: When applied to population-based longitudinal data containing multiple psychological measurements, machine learning techniques did not significantly improve the predictive accuracy of suicidal behaviour. Adding more detailed data on for example employment, education or previous health care uptake, might result in better performance of machine learning over regular logistical regression.
Authors: Marie C Navarro; Isabelle Ouellet-Morin; Marie-Claude Geoffroy; Michel Boivin; Richard E Tremblay; Sylvana M Côté; Massimiliano Orri Journal: JAMA Netw Open Date: 2021-03-01