Psychometric Properties and the Cut-Off Point of the English Version of the Yatt Suicide Attitude Scale.

Background: The Malay version of the Yatt Suicide Attitude Scale (YSAS) was found to be reliable and valid for use in the Malaysian context. This study aims to validate and determine the cut-off points of the English version of the YSAS for a wider application by English-speaking researchers and practitioners.
Methods: The English version of the YSAS, the Suicidal Behaviors Questionnaire-Revised (SBQ-R), and Kessler's K10 Psychological Distress Scale were distributed to 527 university students. Factor analysis was employed in the evaluation of its construct validity. The determination of the convergent and concurrent validity was determined by way of a bivariate correlation with the SBQ-R and Kessler's K10 scales. Cronbach's alpha and Receiver Operating Characteristic-analysis were also used to measure the internal consistency reliability as well as evaluate the screening properties of the scale.
Results: The English YSAS and its ideation and attempt subdomains exhibited a high internal consistency value (>.8). The scale also had a cumulative variance of 76.39% and acceptable convergent and concurrent validity when compared to the SBQ-R and K10. The instrument demonstrated a better specificity in terms of the total score and suicide attempt and a better sensitivity trait on the suicidal ideation score.
Conclusion: The English YSAS is a valid and reliable scale to assess suicidality, suicidal ideation, and suicide attempt among Malaysian university students.

Introduction

Suicide has become a leading cause of death globally as well as among Malaysian youth. More than 700 000 people die by suicide every year; more specifically, in every 40 seconds, one person dies of suicide. A study revealed that there is a significant rise in suicide rates in Malaysia since 2013, the suicide rate had increased significantly for males, and an average suicide rate of 5.6 per 100 000 population was recorded for both sexes from 2017 to 2019. According to the National Health and Morbidity Survey, the prevalence of suicidality among Malaysian adolescents was 10.0% and 6.9% for suicidal ideation and attempt respectively, which was higher than the neighboring countries of Indonesia and Brunei. There are diverse risk factors leading to suicidality, such as demographic (e.g., gender) and social (e.g., loneliness) factors, internalizing and externalizing psychological factors (e.g., hopelessness and psychache), and prior suicidal thoughts and behaviors.[4,5] A systematic review in Malaysia also revealed that debt was associated with suicidal ideation among Asians. However, a meta-analysis by Franklin et al. showed that the power of risk factors in predicting suicide outcomes through research from the past 50 years was poor. Therefore, there is a need to continuously improve preventive and screening methods that seek to detect and stop one's progression on the suicidality continuum.

It is important to note that most people who experience suicidal ideation do not always follow through with it, although some may have made attempts. Whilst suicide risk and ideation are often associated with specific traumas, depression, and substance abuse,[7-9] a suicidal process may, however, take weeks to years for the fatal transition to occur from suicidal ideation and attempts to dying by suicide. From the community surveys that were conducted in 21 countries (n >100 000 individuals), the World Health Organization found that the 12-month prevalence of suicidal ideation was approximately 2%, a lifetime prevalence of 9%, and 33 and 30% of those who had suicidal ideation progressed to suicide planning and attempt, respectively. Among university students, the lifetime prevalence of suicidal ideation and suicide attempt was 22.3% and 3.2% respectively.

Uncontrolled suicidal ideation may lead to a suicide attempt, defined as a self-inflicted, potentially harmful behavior resulting in a non-fatal outcome to which there is either explicit or implicit evidence concerning the intent to die. It is estimated that for every suicide, 50 suicide attempts are made . A study among Chinese university students found that individuals who had a lifetime history of attempted suicide continued to suffer from worse mental health and lower social support. For this reason, a standardized tool screening individuals for suicidal ideation and attempts would be deemed useful so that they could be directed to further diagnosis and treatment. Although a number of studies in Malaysia has made use of different scales for measuring suicidal ideation and attempt, such as the Reasons for Living Inventory and the Suicide Intent Scale, there is no scale which is developed in Malaysia to screen for suicidality in the local context. As such, the Yatt Suicide Attitude Scale (YSAS) was developed following a review on the Suicidal Behaviors Questionnaire-Revised (SBQ-R), Columbia Suicide Severity Rating Scale (C-SSRS), Suicidal Ideation Attribute Scale (SIDAS), Suicidal Affect Cognition Scale (SABCS) and the Beck Scale for Suicide Ideation (BSS) assessment tools.

The original YSAS, developed in the Malay language, has now been translated into the English Language (Malaysia’s second language) through the use of forward-backward translation. The aim of validating the English version was to allow the scale to be utilized by the Malaysian English-speaking populace. Although Malaysia is a multicultural country with diverse ethnic groups such as Malays, Chinese, Indians, Kadazan, and Dusun, and Malay being the official language of the country, there are still many who prefer to communicate in their own dialects or languages, such as the Chinese using either Hokkien, Hakka, or Mandarin and the Indians using Tamil or Malayali. Thus, the English version of the YSAS will not only be utilized among Malaysians who have limited knowledge of the Malay language, but could be used among English-speaking individuals. In addition, as English is a second language, the scale was constructed using short and simple sentences in order to facilitate a better understanding of the items. The validation of the scale in Malaysia may further reduce the translation and cross-cultural issues observed in previous studies.[17,18] This study has therefore attempted to record the validation process of the validity, reliability, and cut-off points of the English-translated version of the Yatt Suicide Attitude Scale.

The Malay version of the original YSAS has been tested for its suitability as a screening tool in the Malaysian context. The psychometric properties of the original YSAS were evaluated based on data from university students and was found to yield two components (Suicidal Ideation and Suicide Attempt) with each containing a total of 5 items. Apart from showing a good internal consistency reliability (.89 and .86 respectively for ideation and attempt) that accounted for 67.8% of the total variance, the YSAS was also revealed to have corresponded to the Suicide Ideation Scale (SIS) measurement of the same constructs. There was however no investigation on the Malay YSAS on its yield of an optimal cut-off for predicting the risk of suicidal ideation and attempt. This study therefore aimed to evaluate the reliability and validity of the YSAS as well as to determine an appropriate cut-off threshold for overall suicide risk, suicidal ideation, and suicide attempt for the English version of the YSAS among university students.

Materials and Methods

Participants

The recruitment of participants was carried out face-to-face from March to April 2019, and the students were selected using convenience sampling from various courses and years of study. The questionnaire was self-administered using paper-and-pen format in person. They were approached before or after their classes.

Instruments

The translated version of the YSAS questionnaire was validated with the use of two instruments, namely the SBQ-R and Kessler’s K10 Psychological Distress Scale.

Yatt Suicide Attitude Scale

The Malay version of the Yatt Suicide Attitude Scale (YSAS) had been developed by a research team from Universiti Kebangsaan Malaysia. Consisting of 10 items scored on a 5-point Likert Scale, this questionnaire had been constructed to gauge suicidal ideation and suicide attempt among youth between 18 and 25 years of age. Apart from indicating a reliability level with a Cronbach’s alpha value of above .80 for both suicidal ideation and suicide attempt, the Malay version of the Yatt Suicide Attitude Scale was also found to have demonstrated a convergent and concurrent validity as shown by the significant correlation of the Suicide Ideation Scale (r = .64) and Psychological Distress (r = .38). It is important to note that the original Malay version of the YSAS had been first subjected to a forward-backward translation to English by experienced bilingual psychologists and was also examined by an independent three-panel expert prior to its use in the study.

Kessler’s K10 Psychological Distress Scale

The K10 scale is a short instrument that is used to measure anxiety and depression through a 10-item questionnaire. This scale can be administered to both the general and clinical population and since it consists of ten questions pertaining to the respondent’s emotional state within the last month, a cut-off score of 20 would then be used to determine the likelihood of distress among the respondents. This scale exhibited a reliability value of .93 from a sample of caregivers of cancer patients in Guam, USA, and a respective .87 and .91 value among Malaysian caregivers of schizophrenia patients and first semester university students.

Suicide Behavior Questionnaire-Revised

The SBQ-R is an abbreviated version of 4 items, where the Likert-type questions are used to gauge the frequency of suicidal ideation, communication of suicidal thoughts to others, as well as the attitudes and expectations about the current suicide attempt. Since this questionnaire composes of items that are related to both past and future suicidal thoughts and behaviors, this scale can therefore be used to measure past suicidal thoughts and attempts in predicting future suicidal behavior risk.

Data Analysis

Descriptive and exploratory factor analyses was conducted using SPSS (v21) to analyze demographic data, while the reliability and content validity of the questionnaire were confirmed from the Cronbach’s alpha value and Pearson Correlation analysis.

Apart from examining the concurrent and convergent validity of the instrument through a comparison of the K-10 and SBQ-R scores, the construct validity and the internal consistencies (Cronbach’s α) of the extracted components were also determined on the university student sample using a principal component analysis (PCA) with a varimax rotation.

The screening properties of the translated YSAS version were then evaluated with the use of an Receiver Operating Characteristic (ROC)-analysis, where the cut-off threshold for the instrument was defined by its optimal sensitivity and specificity trade-offs (Youden’s index) and compared with the cases with suicide risk, suicidal ideation, and suicide attempt that had been measured by the SBQ-R standard. The binary logistic regression was also used to determine the sensitivity, specificity and under-the-curve area for the different cut-offs of the total YSAS, suicidal ideation and suicide attempt scores.

Ethical Approval

This study obtained institutional ethical approval from the Universiti Kebangsaan Malaysia Research Ethics Committee (approval number NN-2018-060).

Results

As shown in Table 1, from the 527 university students who had received the questionnaires, 75.9% of them were female, and 56.2% were between 21 and 23 years of age. The majority of the respondents were Malays and Muslims (65.7%) (Table 1).

Table 1.

Demographic Characteristics of the Study Participants (N=527).

Characteristic	Frequency	Percentage (%)
Gender
Male	127	24.1
Female	400	75.9
Age
18-20 years old	174	33.0
21-23 years old	296	56.2
24 years old and above	57	10.8
Race
Malay	346	65.7
Chinese	126	23.9
Indian	46	8.7
Others	9	1.7
Religion
Muslim	346	65.7
Christian	33	6.3
Buddhist	104	19.7
Hindu	42	8.0
Others	2	0.4

The sample that had been subjected to a Varimax rotation under the principal components analysis obtained a value of .892 in the Kaiser-Meyer-Olkin for sampling adequacy, which is above the recommended value of .60. The Bartlett’s test of sphericity was significant at P < .001. This implied that the collection of responses was suitable to be used for this research. As shown in Table 2, the factor loading values generated for Suicidal Ideation, with a total explained variance of 70.1%, and Suicide Attempt, with a total explained variance of 56.7%, indicated these factors as having extracted a sufficient variance from the variable. On the whole, the two factors were found to have generated a cumulative variance of 76.4% (Table 2).

Table 2.

Exploratory Factor Analysis of the Yatt Suicide Attitude Scale, Internal Consistency Reliability and Convergent/Concurrent Validity.

Factors	Factor Loading		Explained Variance %	Cronbach’s Alpha	Convergent Validity with K10	Concurrent Validity with Suicidal Behaviors Scale-Revised
	Component 1	Component 2
10 items of Yatt suicide attitude scale
Suicidal ideation section
Component 1			70.051	.884	.497	.714
B1. I have no will to continue…	.177	.840
B2. I feel like there is no reason…	.129	.845
B3. It has crossed my mind to end my life when I…	.338	.745
B4. I have once thought to end ……	.373	.759
B5. It has crossed my mind to end my life, but I am…	.457	.686
Suicide attempt section
Component 2			56.652	.866	.282	.547
B6. I have hurt myself for the purpose…	.761	.288
B7. I have tried certain methods to end…	.858	.201
B8. I have tried to end my life but ceased to do…	.700	.447
B9. I have tried to end my life, but it did not…	.832	.145
B10. I have attempted to end life, but I actually…	.729	.318
Total			76.392	.905	.451	.708

The overall Yatt Suicide Attitude Scale was found to have a concurrent and convergent validity as depicted by its significant correlation with psychological distress and suicidal thoughts and behaviors (r = .451, P < .001; r = .708, P < .001). Both Suicidal Ideation and Suicide Attempt sub-constructs demonstrated a significant and positive correlation with the K10 and SBQ-R (Table 2).

For the internal consistency reliability, the Cronbach’s alpha for each of the five items under the Suicide Ideation and Suicide Attempt domains and the Yatt Suicide Attitude Scale had exceeded the recommended .80 value at .884, .866 and .905 respectively. This suggested that the items had a good level of internal consistency (Table 2).

Validity as a Screening Instrument for Suicide Risk

The ROC analysis (Figure 1) of the YSAS total score against the SBQ-confirmed suicide risk for the whole sample yielded an area under-curve (AUC) of .95, with the optimal cut-off threshold at 14/15, showing a 90.0% and 88.0% sensitivity and specificity levels respectively. The optimal trade-off that occurred between the sensitivity and specificity levels (Youden index = .78) and detected at a cut-off score of 14/15, was also discovered to have corresponded with the results that were obtained via the binary logistic regression, where at the different cut-offs in the YSAS total score, the optimal cut-off occurred at 14/15 with a corresponding sensitivity and specificity levels of 83.3% and 92.3% and a Youden’s index of .76 (Table 3). By using this cut-off point, 85.4% of the respondents were found to not have exhibited suicidal ideation or attempt risks.

Figure 1.

The receiver operating characteristic analysis on the detection of suicidal behaviour from the English version of the yatt suicide attitude scale against the gold-standard suicidal behaviors scale-revised in a sample of 527 university undergraduates.

Table 3.

The Validity Coefficients of the Different Yatt Suicide Attitude Scale cut-offs Against the Suicidal Behavior, Suicidal Ideation, and Suicide Attempt of the University Students as Measured on the Suicidal Behaviors Scale-Revised.

Yatt Suicide Attitude Scale Cut-Off (Total Score)	12/13	13/14	14/15	15/16	16/17	17/18	18/19	19/20	20/21	21/22
Sensitivity (%)	93.8	89.6	83.3	70.8	58.3	54.2	50.0	41.7	35.4	35.4
Specificity (%)	82.9	88.1	92.3	94.8	96.7	98.3	99.0	99.2	99.2	99.2
Youden index	.77	.78	.76	.66	.55	.53	.49	.41	.35	.35
AUC (%)	88.3	88.9	95.1	92.6	93.2	94.3	94.5	93.9	93.4	93.4

Note. AUC: Area Under the Curve. p<0.05.

Optimal Cut-Off for Suicidal Ideation Risk

The ROC analysis for Suicidal Ideation clearly showed the area under the curve to be .943. By referring to the ROC analysis, the optimal cut-off was found to be 7/8 with a corresponding 89.6% and 85.4% sensitivity and specificity levels, and a Youden index of .75 (Table 3), which was also supported by the results that were obtained from the binary logistic regression. As such, the use of this cut-off point indicates that 56.6% of the respondents do not demonstrate suicidal ideation risk.

Optimal Cut-Off for Suicidal Attempt Risk

The ROC analysis for Suicide Attempt clearly showed the area under the curve to be .816. By referring to the ROC analysis, the optimal cut-off was found to be 5/6 with corresponding sensitivity and specificity levels of 70.8% and 88.1% respectively, and a Youden index of .59. The use of this cut-off point thus indicates a majority of the respondents (82.8%) as having a score of less than 5 and screening negative for suicide attempt risk (Table 3).

Discussion

This study was conducted to determine the reliability and validity of the English version of the YSAS, which had been previously developed in the Malay language. Although Malay is the official language of Malaysia, the English language is still widely used and spoken among the various communities in the country. English Language was institutionalized as the second language by the Education Ordinance in 1957 and was reaffirmed in both the Education Act[27-29] and the National Education Policy in 1970. Thus, this study is vital for establishing the reliability and validity of the YSAS in a language that is readily used in Malaysia. In addition, the 10-item scale is shorter than a number of scales that measured suicidal ideation or suicide attempt (e.g., the Beck Scale for Suicidal Ideation), and employs short and simple sentences suitable for users of the English language as a second language.

As seen from the findings in this study, the items were segregated into factors that were similar to the original version of the YSAS. The 10 items that were separated equally under the suicidal ideation and suicide attempt components had desirable loading factors of above .60. The separation of the scale into two separate factors consisting of suicidal ideation and suicide attempt concur with current understanding of suicidal ideation and suicide attempt as two separate concepts. The former refers to thoughts of attempting suicide with or without the intention to engage in suicidal behaviors, whilst the latter refers to suicidal behavior with intent to die. The Cronbach’s alpha values exceeded .80, thus showing that the internal consistency reliability of the English version of the YSAS was acceptable. As such, the replication of the results from the original Malay version was found to have corresponded with the statements made by Chakrabartty and Twycross and Shields, where reliability is the ability of one’s research or instrument to produce a uniform result under identical situations but within different circumstances. As both the YSAS English and Malay versions were found to be consistent when used on a similar population of university students, the instrument’s reliability is reaffirmed.

Since validity indicates the extent to which the instrument measures what it was designed to measure,[35,36] a cumulative percentage of the variance can then be used to indicate the extent of the variance that is explained by the variable. Humanities and social science fields adopt a cumulative variance of 50-60% ; with that, the English version of the YSAS is deemed acceptable, as it has a high cumulative variance of 76.4%. The scale had also demonstrated medium to high correlation with other developed and established suicide instruments (i.e., the SBQ-R and the K10) in terms of concurrent and convergent validity. Also, there has been a general consensus that even if the measures are theoretically related with an insubstantial correlation in concurrent validity, the validity measure would still be considered as significant and acceptable.

The larger sample size that was used in this study was also found to have produced a more convincing result than what had been done in the previous research. Our sample size corresponded to the sample size guideline provided by Comrey and Lee (50 – very poor; 100 – poor; 200 – fair; 300 – good; 500 – very good and 1000 or more – excellent). This is in agreement with Lenth, who stated that it is important for the sample to be large enough such that the effect from the expected magnitude of scientific significance would render it to be statistically significant, while those with very large sample sizes results in an effect of little scientific importance that are nevertheless statistically detectable.

Since the choice of criteria in determining an optimal cut-off value has become a matter of concern in quantitative diagnostic tests, many methods have thus been proposed in the literature for achieving this objective; one of them is by estimating the area under the ROC curve, or the AUC. Under this approach, the optimal cut-point value is defined as the value whose sensitivity and specificity are the closest to the value of the area under the ROC curve with a minimum absolute value of the difference between the sensitivity and the specificity values. In this case, a biomarker with AUC = 1 would discriminate the individuals as diseased or healthy, while an AUC = .5 would denote the absence of apparent distributional difference between the biomarker values of the two groups. As such, the identification of a cut-point value will not only require a simultaneous sensitivity and specificity assessment, but will also be considered optimal when the point has classified most of the individuals correctly. While the AUC, sensitivity and specificity values are useful for the evaluation of a marker, they however, do not specify the “optimal” cut-points directly as opposed to the Youden index, where the optimal cut-off has been defined as the point that maximizes the Youden function.[44,45] Although this research had shown the 13/14 cut-off point as having a higher Youden index, the optimal cut-off point of 14/15 for the YSAS total score was however chosen because of the highest .95 AUC percentage that was obtained from the ROC analysis and binary logistic regression; and as such, implied the points lower than 14 as having no suicidal behaviors and those with 15 and above as denoting otherwise. In addition, the cut-off of 13/14 would preclude the suicide attempt domain, and thus may diminish the validity of the scale. As for the suicidal ideation and suicide attempt sub-constructs, the scores below 7 and 5 are then seen as indicating the absence of suicidal ideation and attempt risks, while scores above 8 and 6 would be deemed as indicating otherwise.

In this research, the English version of the YSAS instrument was not only found to have demonstrated a better specificity for the total score and suicide attempt, but also showed a better sensitivity characteristic for the suicidal ideation score. There was a high percentage of participants who screened positive for suicidal ideation (43.4%) and suicide attempt (17.2%) risks, considering the prevalence of suicidal ideation and suicide attempt among college students was 22.3% and 3.2% respectively. There needs to be further studies to establish the value of these cutoffs for screening of suicidal ideation and suicide attempt risk in the clinical context. Until then, caution needs to be exercised when utilizing these cut-offs.

Although the researchers managed to achieve the minimal requirements for both the reliability and validity, this study is not without its limitations. The English version of the YSAS was validated only among Malaysian university students. Hence the usage of the instrument for other populations needs to be further validated. Level of education may also influence the understanding of the instrument. This instrument was also specifically developed for youth between 18 and 25 years of age. It is therefore advisable to further validate the instrument among individuals from other age ranges, such as among older adults and adolescents, and educational levels. The two versions of the YSAS can also be tested on a clinical sample of psychiatric inpatients with a history of suicidal ideation and suicide attempt, in order to gauge their clinical utility. This study contributed to existing body of knowledge by offering an alternative for assessing the level of suicidal ideation and suicide attempt together, using a relatively short scale (10 items) with two sub-scales. In the Malaysian context, usage of the YSAS could be more compatible with the Malaysian population and minimizes the translation and cross-cultural issues that may arise.[17,18]

Conclusions

The English version of the YSAS not only demonstrated a high reliability and validity, but also displayed favorable specificity and sensitivity traits. Hence, these qualities render it a useful tool in the evaluation of suicidal thoughts and behaviors. Since this instrument can be used as a screening tool for aiding researchers, health practitioners and educators in the early detection of suicidal behaviors, a further validation of this scale, particularly with regards to clinical subjects, should therefore be conducted. This will ensure that the assessment tool is functional and applicable across both the general population and clinical settings so as to produce more accurate results which can then lead to a better understanding of this serious issue.