Michael Sanderson1, Andrew Gm Bulloch2, JianLi Wang3, Tyler Williamson4, Scott B Patten5. 1. Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, TRW, 4th Floor, Room 4D66, 3280 Hospital Drive NW, Calgary, Alberta, Canada. Electronic address: michael.sanderson@gov.ab.ca. 2. Hotchkiss Brain Institute, Department of Psychiatry, Cumming School of Medicine, University of Calgary, TRW, 4th Floor, Room 4D67, 3280 Hospital Drive NW, Calgary, Alberta, Canada. 3. School of Epidemiology, Public Health and Preventive Medicine, Department of Psychiatry, Faculty of Medicine, University of Ottawa, Royal Ottawa Mental Health Centre, 1145 Carling Avenue, Ottawa, Ontario, Canada. 4. Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, TRW, 3rd Floor, Room 3D15, 3280 Hospital Drive NW, Calgary, Alberta, Canada. 5. Department of Community Health Sciences, Department of Psychiatry, Cumming School of Medicine, University of Calgary, TRW, 4th Floor, Room 4D66, 3280 Hospital Drive NW, Calgary, Alberta, Canada.
Abstract
BACKGROUND: Suicide is a leading cause of death, particularly in younger persons, and this results in tremendous years of life lost. OBJECTIVE: To compare the performance of recurrent neural networks, one-dimensional convolutional neural networks, and gradient boosted trees, with logistic regression and feedforward neural networks. METHODS: The modeling dataset contained 3548 persons that died by suicide and 35,480 persons that did not die by suicide between 2000 and 2016. 101 predictors were selected, and these were assembled for each of the 40 quarters (10 years) prior to the quarter of death, resulting in 4040 predictors in total for each person. Model configurations were evaluated using 10-fold cross-validation. RESULTS: The optimal recurrent neural network model configuration (AUC: 0.8407), one-dimensional convolutional neural network configuration (AUC: 0.8419), and XGB model configuration (AUC: 0.8493) all outperformed logistic regression (AUC: 0.8179). In addition to superior discrimination, the optimal XGB model configuration also achieved superior calibration. CONCLUSIONS: Although the models developed in this study showed promise, further research is needed to determine the performance limits of statistical and machine learning models that quantify suicide risk, and to develop prediction models optimized for implementation in clinical settings. It appears that the XGB model class is the most promising in terms of discrimination, calibration, and computational expense. LIMITATIONS: Many important predictors are not available in administrative data and this likely places a limit on how well prediction models developed with administrative data can perform.
BACKGROUND: Suicide is a leading cause of death, particularly in younger persons, and this results in tremendous years of life lost. OBJECTIVE: To compare the performance of recurrent neural networks, one-dimensional convolutional neural networks, and gradient boosted trees, with logistic regression and feedforward neural networks. METHODS: The modeling dataset contained 3548 persons that died by suicide and 35,480 persons that did not die by suicide between 2000 and 2016. 101 predictors were selected, and these were assembled for each of the 40 quarters (10 years) prior to the quarter of death, resulting in 4040 predictors in total for each person. Model configurations were evaluated using 10-fold cross-validation. RESULTS: The optimal recurrent neural network model configuration (AUC: 0.8407), one-dimensional convolutional neural network configuration (AUC: 0.8419), and XGB model configuration (AUC: 0.8493) all outperformed logistic regression (AUC: 0.8179). In addition to superior discrimination, the optimal XGB model configuration also achieved superior calibration. CONCLUSIONS: Although the models developed in this study showed promise, further research is needed to determine the performance limits of statistical and machine learning models that quantify suicide risk, and to develop prediction models optimized for implementation in clinical settings. It appears that the XGB model class is the most promising in terms of discrimination, calibration, and computational expense. LIMITATIONS: Many important predictors are not available in administrative data and this likely places a limit on how well prediction models developed with administrative data can perform.
Authors: Dawid Majcherek; Arkadiusz Michał Kowalski; Małgorzata Stefania Lewandowska Journal: Int J Environ Res Public Health Date: 2022-09-21 Impact factor: 4.614
Authors: Michael Sanderson; Andrew Gm Bulloch; JianLi Wang; Kimberly G Williams; Tyler Williamson; Scott B Patten Journal: EClinicalMedicine Date: 2020-02-18