The Development of a Suicidal Ideation Predictive Model for Community-Dwelling Elderly Aged >55 Years.

PURPOSE: Suicide is an important health and social concern worldwide. Both suicidal ideation and suicide rates are higher in the elderly population than in other age groups; thus, more careful attention and targeted interventions are required. Therefore, we have developed a model to predict suicidal ideation in the community-dwelling elderly aged of >55 years. PATIENTS AND METHODS: A random forest algorithm was applied to those who participated in the Korea Welfare Panel. We used a total of 26 variables as potential predictors. To resolve the imbalance in the dataset resulting from the low frequency of suicidal ideation, training was performed by applying the synthetic minority oversampling technique. The performance index was calculated by applying the predictive model to the test set, which was not included in the training process.
RESULTS: A total of 6410 elderly Korean aged of >55 (mean, 71.48; standard deviation, 9.56) years were included in the analysis, of which 2.7% had suicidal ideation. The results for predicting suicidal ideation using the 26 chosen variables showed an AUC of 0.879, accuracy of 0.871, sensitivity of 0.750, and specificity of 0.874. The most significant variable in the predictive model was the severity of depression, followed by life satisfaction and self-esteem factors. Basic demographic variables such as age and gender demonstrated a relatively small effect.
CONCLUSION: Machine learning can be used to create algorithms for predicting suicidal ideation in community-dwelling elderly. However, there are limitations to predicting future suicidal ideation. A predictive model that includes both biological and cognitive indicators should be created in the future.

Introduction

As people age, their close relationships are gradually lost and their physical functions decline.1 Particularly when depression and personal vulnerability, such as alcohol, are present together, the risk of suicide increases.2,3 In fact, old age is one of the most prevalent risk factors for suicide.4,5 The global aging population is on the rise as fertility and mortality rates continue to decline.6 The population is rapidly aging, especially in East Asian countries such as Korea, China, and Japan.7 In 2017, people aged > 65 years accounted for 14% of the total population of South Korea.8

Suicide is a global public health concern. In 2016, 793,000 people died from suicide, which is equivalent to 10.5/100,000 people.9 Suicide is also very costly to society. According to a study conducted in the United States, the socioeconomic costs associated with suicide and suicide attempts reached USD 93.5 billion.10

Suicide rates in the elderly are particularly severe. The suicide rate in the elderly in Korea in 2019 was 46.6/100,000 people, which was the highest among all age groups.11 In addition, according to the OECD, the average elderly suicide rate is 17.2/100,000 people, which is significantly higher than the overall suicide rate of 11.2/100,000 people.12,13

Therefore, many studies have been conducted to clarify the factors that influence suicidal ideation in the elderly. In several studies, depression is consistently confirmed as the most important factor in suicidal ideation in the elderly, and the more severe the depressive symptoms, the higher the suicidal ideation.14,15 Representative examples of these factors include age, gender, economic status, social support, smoking, sleep, chronic diseases, life satisfaction, and psychosocial well-being.16 However, predicting whether a particular person will develop suicidal ideation even if the risk factors above are identified is difficult. Machine learning is not constrained by the limitations of conventional group-level and can be used for individual-level analyses. Therefore, the application of machine learning to predict high-risk groups in the medical field is continuously increasing,17 and psychiatry is no exception.

Suicidal ideation is a stage in which one has a clear thought of wanting to die, but has not yet put it into action.18 Suicidal ideation is categorized into active and passive, depending on whether the individual has an actual suicide plan.19,20 In this study, suicidal ideation included passive and active suicide and was evaluated as having occurred during the last 1 year. People with suicidal ideation are at a higher risk of dying from suicide due to the interaction of various factors such as stressful life events, mental and physical comorbidities, and social support. Thus, if those who have suicidal ideation are identified before they act, suicide and suicide attempts can be prevented through early intervention.

Several studies have attempted to develop a model that predicts suicidal ideation in adolescents,21 soldiers,22 and patients using primary care23 through machine learning. In a previous paper that developed a model for predicting suicide of the elderly using the medical check-up cohort DB, the importance of variables was the highest in the order of benzodiazepine history, BMI, age, and sleeping pill history.24 However, very few studies have focused on the elderly in the community. We, therefore, attempted to create a model to predict suicidal ideation in the elderly in the community using only non-invasive and basic patient information. In addition, we assessed whether suicidal ideation one year later could be predicted using the same variables.

Materials and Methods

Subjects

The Korean Welfare Panel Study (KoWePS), which conducted its first survey on 7072 households in 2006, is one of the largest panel surveys conducted in Korea.25 The population surveyed by the KoWePS includes the 90% enumeration districts from the 2005 census. The extraction frame of the KoWePS includes a total of 230,000 enumeration districts (excluding islands and special facilities) from 90% of the 2005 Population and Housing Census. In the first stage, 517 enumeration districts are sampled from 90% of the population census data, and the household income and economic activity status of household members are investigated. In the second stage, 3500 ordinary and low-income households are extracted from the survey data from the first stage, totaling 7000 households. Finally, panel households are selected using the stratified double extraction method. The KoWePS has been conducting a survey every year. For this study, we used the data from the twelfth survey from 2017 and the thirteenth survey from 2018. In the thirteenth survey, 4266 households participated, and the retention rate of the originally sampled households compared to the initial 7072 households was 60.32%. The decrease in the original sample household retention rate compared to the twelfth survey (the previous year) was 1.9%.

The data accessed from the official homepage () of KoWePS is freely available. The KoWePS obtains approval from the institutional review board of the Korea Institute for Health and Social Affairs (KIHASA) before conducting the annual survey. After receiving a full explanation of the survey, all survey subjects sign consent to participate in the survey and agree to the use of their data. All data included in this survey are anonymized and all personally identifiable information is removed.

Variables

We used a total of 26 variables as potential predictors as follows: age, sex, religion, marital status, education level, chronic diseases, disability status, number of private medical insurance subscriptions, number of outpatient visits, number of hospitalizations, number of days spent in the hospital, average monthly premium paid for private health insurance, education and entertainment expenses, taxes paid, total cost of living, internet use, health satisfaction, satisfaction with family income, satisfaction with residential environment, satisfaction with family relationships, satisfaction with social relationships, satisfaction with leisure life, alcohol intake, high-risk drinking (Alcohol Use Disorders Identification Test (AUDIT)),26,27 depression severity (Center for Epidemiologic Studies Depression Scale (CES-D)),28 self-esteem level (Rosenberg Self-Esteem Scale),29 and suicidal ideation. For suicidal ideation, the following question was asked: “Have you seriously considered suicide in the past year?”

To compare the sociodemographic and clinical characteristics between the two groups divided by the presence or absence of suicidal ideation, statistical tests were performed using independent t-test for continuous variables or Chi-square test for categorical and ordinal variables.

Data Preprocessing and Machine Learning

All data preprocessing and machine learning processes were performed using scikit-learn, the machine learning library for Python 3.8. The entire dataset was largely divided into a training set and a test set. For the training set, the entire dataset was first divided into a training set and a test set at a ratio of 7:3. For the validation set, hyperparameters were tuned in the direction of maximizing the AUC by using 10-fold cross-validation. Due to the low frequency of suicidal ideation, the observed value of the group with suicidal ideation increased the same as the group without suicidal ideation by applying a synthetic minority over-sampling technique (SMOTE) to the training set.30 No structural transformations were applied to the data in the test set. Subjects with missing values in any of the 26 predictors were excluded.

In general, the advantage of the decision tree model is that variable importance and classification mechanisms can be intuitively identified, and the computing cost is low. Crucially, however, the decision tree is vulnerable to overfitting and is, therefore, rarely used as an algorithm for solving problems in practice. Random forest and gradient boosting algorithms are ensemble models used to compensate for the shortcomings of the decision tree. Both maximize the predictive power by providing new data (ie, the test set) rather than the data used during training to minimize overfitting of the decision tree. We chose the random forest model31 for this study because it operates in parallel, making the computing cost relatively lower, while the gradient boosting algorithm operates sequentially.32 Accordingly, the random forest model is widely used in big data analyses.33

The performance evaluation was calculated using a test set that was not used to create the predictive model. Accuracy, specificity, sensitivity, negative predictive value, and positive predictive value were calculated along with the AUC as performance evaluation indicators.

Results

Of the total subjects aged > 55 years, 6410 had no missing values for any of the 26 predictors and were thus included in the machine learning analysis. Among them, 2532 (39.5%) were male and 3878 (60.5%) were female. The mean age of the subjects was 71.4 (SD = 9.6), and 2.7% (173 out of 6410) of all subjects reported having serious ideation of suicide during the previous year.

Table 1 illustrates the differences in variables such as sociodemographic and economic characteristics, healthcare use, life satisfaction, alcohol, depression, and self-esteem in participants with and without suicidal ideation. Based on suicidal ideation status, a significant difference was found in the marital status between the two groups (χ2 = 38.5557, p < 0.001). Specifically, 35.26% of those who had suicidal ideation were widowed, which was higher than that in the group without suicidal ideation (27.58%). Significantly, more of those in the group with suicidal ideation were divorced or separated than of those in the group without suicidal ideation (16.76% vs 6.49%).

Table 1

Comparison of Sociodemographic Features, Use of Healthcare, Satisfaction, and Clinical Variables Between the Two Groups According to Suicidal Ideation Status

Variables	Suicidal Ideationc (n = 173)	No suicidal Ideation (n = 6237)	Statistics
Age, yearsa	72.25 ± 9.21	71.46 ± 9.57	t = −0.10756, p = 0.2821
Sex, femaleb	117 (67.63%)	3761 (60.30%)	χ2 = 3.7832, p = 0.052
Religionb (Yes)	97 (56.74%)	3539 (56.74%)	χ2 = 0.0310, p = 0.860
Marital statusb			χ2 = 38.5548, p < 0.001***
Married	80 (46.24%)	4018 (64.42%)
Widowed	61 (35.26%)	1720 (27.58%)
Divorced, separated, other	29 (16.76%)	405 (6.49%)
Single (including single mothers)	3 (1.73%)	94 (1.51%)
Chronic diseasesb			χ2 = 25.0903, p < 0.001***
None	9 (5.20%)	1109 (17.78%)
Chronic disease with medications < 3 mo	1 (0.58%)	150 (2.41%)
Chronic disease with medications 3–6 mo	1 (0.58%)	133 (2.13%)
Chronic disease with medications > 6 mo	162 (93.64%)	4845 (77.68%)
Disability statusb (Yes)	45 (26.01%)	897 (14.38%)	χ2 = 18.1607, p < 0.001***
Number of private health insurance subscriptionsb			χ2 = 34.4967, p < 0.001***
None	137 (79.19%)	3633 (58.25%)
1	25 (14.45%)	1316 (21.10%)
2	3 (1.73%)	751 (12.04%)
3	5 (2.89%)	310 (4.97%)
> 4	3 (1.73%)	227 (3.64%)
Number of outpatient visitsa	49.50 ± 52.43	26.68 ± 33.43	t = −8.6880, p < 0.001***
Number of hospitalizationsa	0.54 ± 1.06	0.22 ± 0.63	t = −6.3088, p < 0.001***
Number of days spent in the hospitala	13.40 ± 36.93	3.68 ± 15.61	t = −7.6240, p < 0.001***
Average monthly private health insurance premiuma (KRW 10,000)	3.82 ± 10.50	11.81 ± 19.43	t = 5.3835, p < 0.001***
Education and entertainment expensesa	3.40 ± 7.27	8.36 ± 17.81	t = 3.6527, p < 0.001***
Taxa	2.88 ± 10.33	9.64 ± 45.23	t = 1.9626, p = 0.0497*
Average monthly cost of livinga (KRW 10,000)	150.66 ± 115.97	250.41 ± 217.68	t = 6.0027, p < 0.001***
Internet useb	25 (14.45%)	1445 (23.17%)	χ2 = 7.2377, p = 0.007**
Health satisfactiona	2.01 ± 0.82	2.91 ± 0.91	t = 12.9140, p < 0.001***
Family income satisfactiona	2.35 ± 0.91	2.93 ± 0.85	t = 8.7312, p < 0.001***
Satisfaction with residential environmenta	3.35 ± 0.79	3.63 ± 0.71	t = 5.0877, p < 0.001***
Satisfaction with family relationshipsa	3.30 ± 0.84	3.81 ± 0.63	t = 10.3878, p < 0.001***
Job satisfactiona	2.88 ± 0.88	3.40 ± 0.73	t = 9.0754, p < 0.001***
Satisfaction with social relationshipsa	3.18 ± 0.87	3.66 ± 0.64	t = 9.5446, p < 0.001***
Satisfaction with leisure lifea	2.86 ± 0.79	3.31 ± 0.76	t = 7.6007, p < 0.001***
Overall satisfactiona	2.82 ± 0.72	3.51 ± 0.64	t = 14.0470, p < 0.001***
High-risk drinking (AUDIT)a	3.36 ± 7.08	3.91 ± 6.29	t = 1.1401, p = 0.2543
Depression (CES-D)a	13.17 ± 7.19	4.01 ± 4.78	t = −24.4482, p < 0.001***
Self-esteem (RSES)a	24.28 ± 2.22	23.14 ± 1.84	t = −7.9242, p < 0.001***

Notes: Data are presented as means ± standard deviation or number (percentage of the group); statistics were performed using independent sample t-testsa and chi-square testsb. *p < 0.05, **p < 0.01, ***p < 0.001. Values in bold indicate a statistically significant difference (p<0.05). cThe group that answered “yes” to the question “Have you seriously considered suicide in the past year?”.

Abbreviations: AUDIT, alcohol use disorders identification test; CES-D, center for epidemiologic studies depression scale; RSES, Rosenberg self-esteem scale.

Moreover, participants with suicidal ideation overall had a higher rate of illness, greater economic difficulties, and lower quality of life. Having a chronic disease and taking medications for >6 months were significantly more common in the group with suicidal ideation (93.64% vs 77.68%) (χ2 = 25.0903, p < 0.001). Furthermore, a total of 26.01% of those in the group with suicidal ideation had a disability, which was significantly more than the 14.38% who reported a disability in the group without suicidal ideation (χ2 = 18.1607, p < 0.001). For the group without suicidal ideation, spending on education and entertainment (t = 3.6527, p < 0.001) and total cost of living (t = 6.0027, p < 0.001) were significantly higher.

Satisfaction with health, family income, residential environment, family relationships, job, social relationships, and leisure life and overall satisfaction were all significantly lower in the group with suicidal ideation. The variable with the largest statistical difference between the two groups was the depression scale score. The average score on the Center for Epidemiologic Studies Depression (CES-D) scale for the group with suicidal ideation was 13.17, while the average score for the group without suicidal ideation was 4.01. Therefore, depression was found to be significantly more common in the group with suicidal ideation (t = −7.253, p < 0.001).

Performance Evaluation

The main hyperparameters of the optimized random forest algorithm are as follows: “n_estimators” = 500, “max_depth” = 6, “max_features” = 7, “criterion” = “gini.” The performance evaluation results for the random forest algorithm showed an AUC of 0.879 (Figure 1) and an accuracy of 0.871. Table 2 shows the true-positive, true-negative, false-positive, and false-negative values through the confusion matrix. The sensitivity, that is, the proportion of the participants who were found to be positive for suicidal ideation among all participants with suicidal ideation, was 0.750. The specificity, that is, the proportion of participants who were found to be negative for suicidal ideation among all participants without suicidal ideation, was 0.874. The positive-predictive value, that is, the probability that a participant who was found to be positive actually would have suicidal ideation, was 0.142. The negative-predictive value, that is, the probability that a participant who was found to be negative for suicidal ideation actually would not have suicidal ideation, was 0.992 (Table 3). For the model predicting suicidal ideation one year later, the AUC and accuracy were 0.701 and 0.725, respectively.

Figure 1

Receiver operating characteristic curve for the model predicting suicidal ideation in the elderly.

Table 2

Confusion Matrix for the Model Predicting Suicidal Ideation in the Elderly

		True
		Suicidal Ideation	Non-Suicidal Ideation
Predicted	Suicidal ideation	39a	235b
	Non-suicidal ideation	13c	1639d

Notes: aTrue positive: an individual with suicidal ideation correctly identified to have suicidal ideation; bFalse positive: an individual without suicidal ideation incorrectly identified to have suicidal ideation; cFalse negative: an individual with suicidal ideation incorrectly identified to have no suicidal ideation; dTrue negative: an individual without suicidal ideation correctly identified to have no suicidal ideation.

Table 3

Performance Metrics for the Model Predicting Suicidal Ideation in the Elderly

Sensitivitya	Specificityb	Accuracy	AUC	PPV	NPV
0.750	0.874	0.871	0.879	0.142	0.992

Notes: aSensitivity=TP/(TP + FN); bSpecificity=TN/(FP + TN).

Abbreviations: AUC, area under the receiver operating characteristic curve; PPV, positive-predictive value; NPV, negative-predictive value; TP, true-positive; TN, true-negative; FP, false-positive; FN, false-negative.

Variable Importance

Figure 2 shows how the 26 variables in the model affected the predictive power of the algorithm. Depression had the greatest impact, with a marked difference from other variables, followed by satisfaction with leisure life, health satisfaction, and satisfaction with family relationships. Self-esteem was the fifth most influential variable. Basic demographic variables such as age and gender had a relatively small impact.

Figure 2

Variable importance in the model predicting suicidal ideation in the elderly.

Discussion

In this study, the model for predicting suicidal ideation in the same year performed relatively well, with an AUC of 0.865 and accuracy of 0.871. The excellence criterion for the AUC value has not been clearly established, and the significance of the AUC value may vary according to the context and distribution of the data. A value > 0.7 is considered acceptable, a value of > 0.8 is excellent, and a value of > 0.9 is considered outstanding.34 The experimental model that was meant to predict suicidal ideation a year later had lower performance than the model predicting suicidal ideation in the same year. This was somewhat expected given the low prevalence of suicidal ideation and the follow-up design. One previous study compared effect sizes in several studies that carried out follow-up analyses and suggested that an AUC of 0.714 corresponds to a Cohen’s d of 0.8.35 Cohen stated that a Cohen’s d above 0.8 is a high effect size.36

Considering the predictive value of each variable for suicidal ideation, depression severity was found to have the greatest influence on suicidal ideation in the elderly, which is similar to findings of previous studies.37–39 Indeed, the significance of depression was more than double that of satisfaction with leisure life, the second most influential predictor. However, the fact that other variables also had an impact on the predictive model cannot be overlooked. In fact, many previous studies have found that various sociodemographic and economic factors contribute to the development and course of depression.40

We predicted suicidal ideation in the elderly by using community samples. Some studies have predicted suicidal ideation in a group of participants with diagnosed mental disorders such as major depressive disorder and bipolar disorder.39,41,42 However, suicidal ideation and these major mental disorders do not necessarily share the same risk factors. This is especially true in East Asian countries such as South Korea, China, and Japan, where significantly fewer of those who die by suicide have major psychiatric disorders, such as depressive disorder, than in North America or Europe.43 This suggests that aspects of suicidality, such as suicidal ideation, need to be considered in a more comprehensive framework and not necessarily in a clinical group of patients affected by mental illness.

Another aspect that differentiates our study from previous studies is that we utilized machine learning. Many previous studies have conducted group-level analysis.37,44 This is effective in identifying the risk factors for major target variables such as suicidal ideation; however, predicting individually who is at risk of suicidal ideation is not easy. Furthermore, none of the multiple risk factors reported in previous studies have been sufficient to predict future suicidality.45 It can, therefore, be assumed that using a set of various risk factors rather than individual risk factors can increase predictive power. From this perspective, machine learning is an optimal means to increase predictive power by comprehensively utilizing numerous variables.46

Lastly, while past studies have mainly relied on cross-sectional analyses to predict suicidal ideation,47 in this study, we even attempted to predict suicidal ideation after one year. Predicting one-sided suicidal ideation can have higher predictive power because it utilizes predictors that are closely related in time. If the combination of current variables could predict suicidal ideation after at least one year, it would be much more useful for preemptively intervening and treating patients.

There are several limitations to this study. First, biological indicators were not included in the predictors. The KoWePS has excluded biological indicators from the initial design. This is because the main purpose of the KoWePS was to dynamically identify changes in the class size and living conditions of the poor, working poor, and near poor in the context of rapidly changing household types, income levels, and employment status after Korea received an IMF bailout in 1997 due to the foreign exchange crisis. As a result, we were able to collect very detailed information on the economic level, use of healthcare, and life satisfaction of the subjects. This could be considered a strength of this study.

Second, many previous studies have suggested that psychopathologies such as hopelessness and rumination play an important role in predicting depression.37–39 Our study reflected various aspects of depression using the CES-D. However, considering that hopelessness and rumination are also important variables worth measuring more comprehensively and systematically using separate scales with proven validity, it is unfortunate that these factors were not included.

Third, older people often experience cognitive decline, which may have an impact on predicting suicidal ideation. Previous studies have reported that depression may contribute to cognitive decline, and conversely, cognitive function may affect the course of depression.48,49 However, one study reported that cognitive decline is not associated with suicide,50 although it significantly increases accidental death. Thus, further studies on the relationship between cognitive decline and suicidality in the elderly are needed.

Fourth, in our study, we included subjects over the age of 55. In most modern societies, age 65 years is regarded as the standard of the elderly population. However, elderly suicide is not only related to biological aging but also to various aspects such as economic status and social role. Thus, the Canadian Toronto provincial government has set the standard for “older adults” who apply the mental health promotion program to age 55 years or older.51 In the Republic of Korea, the Employment Promotion for the Aged Act also defines “the aged” as those aged 55 years or older. Several studies have set age 55 years as the standard for the elderly, depending on the purpose and context of the study.52–54 A study in Australia, for example, developed a model that predicted suicidal ideation for 2160 people over the age of 55.55

Finally, it is difficult to clearly distinguish suicidal ideation, the main variable of our study, only through the reporting of the participants. Suicidal ideation is a broad term that includes wishes, desires, and preoccupations regarding suicide and death and is a continuous and complex concept of suicide plans and attempts. Other than the rare event of suicidal ideation, the decrease in the predicting power of the test model in comparison with the training model could be also due to the reasons that suicidal ideation could be transient in the course of a lifetime.

Conclusion

A suicidal ideation predictive model for older adults in local communities in the same year was developed using big data from the Korea Welfare Panel. The predictive model showed excellent performance with an AUC of 0.865, and the degree of depression had the greatest influence on suicidal ideation, followed by life satisfaction, among the other variables included in this study. Future studies should consider the following: using more predictive variables, such as biomarkers; employing several systematic measures that can identify various psychopathologies; and examining the effect of cognitive functions on suicidal ideation predicted for a longer period of time. Therefore, effective multilateral interventions that consider the special circumstances of individuals exposed to suicide risk, such as sex, age, class, and motivation, should be developed to reduce suicidal ideation among older adults.