Application of Short Screening Tools for Post-Traumatic Stress Disorder in the Korean Elderly Population.

OBJECTIVE: Post-traumatic stress disorder (PTSD) is often missed or incorrectly diagnosed in primary care settings. Although brief screening instruments may be useful in detecting PTSD, an adequate validation study has not been conducted with older adults. This study aimed to evaluate the reliability and validity of the Korean version of the primary care PTSD screen (PC-PTSD) and single-item PTSD screener (SIPS) in elderly veterans.
METHODS: The PC-PTSD and SIPS assessments were translated into Korean, with a back-translation to the original language to verify accuracy. Vietnamese war veterans [separated into a PTSD group (n=41) and a non-PTSD group (n=99)] participated in several psychometric assessments, including the Korean versions of the PC-PTSD (PC-PTSD-K), SIPS (SIPS-K), a structured clinical interview from the Diagnostic and Statistical Manual of Mental Disorders-IV(SCID), and PTSD checklist(PCL).
RESULTS: The PC-PTSD-K showed high internal consistency (Cronbach α=0.76), and the test-retest reliability of the PC-PTSD-K and SIPS-K were also high (r=0.97 and r=0.91, respectively). A total score of 3 from the PC-PTSD-K yielded the highest diagnostic efficiency, with sensitivity and specificity values of 0.90 and 0.86, respectively. The 'bothered a lot' response level from the SIPS-K showed the highest diagnostic efficiency, with sensitivity and specificity values of 0.85 and 0.89, respectively.
CONCLUSION: Our findings suggest that both PC-PTSD-K and SIPS-K have good psychometric properties with high validity and reliability for detecting PTSD symptoms in elderly Korean veterans. However, further research will be necessary to increase our understanding of PTSD characteristics in diverse groups with different types of trauma.

INTRODUCTION

Post-traumatic stress disorder (PTSD) develops in response to a severe stress event or trauma, and induces clinically severe pain and impairments in social, occupational, or other important daily functions.1 In the United States, over 50% of men and women experience a severe trauma event within their lifetime and at least 10–20% of the victims of traumatic events develop PTSD, with a lifetime prevalence of about 7.8%.2 Despite this high prevalence, identifying PTSD in a primary care environment is difficult. Although 12–39% of patients in primary care clinics satisfy the diagnostic criteria for PTSD,34 only half of veterans diagnosed with PTSD are detected through primary care services.5 The probability that patients with PTSD are diagnosed by their primary care physician is only about 2%.56 These finding suggest that PTSD may be underestimated within the primary care environment, and that it is highly likely that patients are not receiving adequate treatment.7

Screening is the process by which a healthy population is examined to identify persons likely or unlikely to have a particular condition.8 Screening is not meant to replace assessments or diagnoses, rather to be used as a tool to determine whether further evaluation is necessary. Positive results of screening should have a clinical estimation. Of the many PTSD screening instruments developed thus far, the following self-report scales used within a primary care setting were reviewed for their potential standardization in Korea: the PTSD checklist (PCL),9 the post-traumatic diagnostic scale (PDS),10 the Davidson trauma scale (DTS),11 the impact of event scale-revised (IES-R),12 and the Keane MMPI-PTSD (MMPI-PK).13 However, these scales are primarily focused on the severity of symptoms after trauma, and many of the examination questions are not easy to use for patients in a primary care setting. Additionally, screening tools should be simple and familiar for the clinician to use, which highlights the need for an easy scoring system and educational training for managers. The scales should also consist of a minimal set of items in order to maintain sensitivity and simultaneously specificity.

Short-version PTSD screeners with less than 10 items meet the minimal requirements for accuracy and sensitivity.1415 Although the SPAN (startle, physiological arousal, anger, and numbness) screen,16 short form of PTSD checklist screen (SF-PCL),17 primary care PTSD screen (PC-PTSD)18 and single-item PTSD screener (SIPS)19 are the only self-report scales with fewer than 5 items, these instruments have not been standardized in the Korean language. Furthermore, to date no studies pertaining to the reliability or validity of these PTSD screening tools have been performed for use in an elderly Korean population. Therefore, this study aimed to assess the diagnostic efficiency of the PC-PTSD and SIPS by translating them into Korean to use as screening tools for Vietnamese war veterans within the elderly Korean population.

METHODS

Participants

We conducted this study from August 2014 to December 2014 with Vietnamese war veterans. We enrolled 174 male participants from the psychiatry, orthopedic surgery, or internal medicine wards or psychiatric outpatient clinics of the Veterans Health Service (VHS) Medical Center. Exclusion criteria included: a history of organic disorders such as head trauma, cerebral infarction, or cerebral hemorrhage; and/or a Korean Mini Mental Status Examination (K-MMSE) score below 24 points (which indicates suspected cognitive disorders). Information was obtained through history taking, psychiatric interviews, and neurological examinations. Four participants did not finish the questionnaire survey and thirty patients were excluded for cognitive disorders confirmed by the K-MMSE. In total, 140 participants were analyzed in this study. The Institutional Review Board of the VHS Medical Center approved this study, and all participants were given a thorough explanation about the purposes and procedure of this research before completing the written consent form.

Measures

The Korean Mini Mental State Examination

The original MMSE was slightly modified during the development of the K-MMSE to adapt it to the cultural background of Korea.2021 Temporal orientation was assessed according to the two methods used to calculate year and time in Korea: the solar and lunar years. The words "airplane," "pencil," and "pine tree" were used in the memory assessment with their associated sounds ("bee-haeng-gi," "yeon-pil," and "so-na-mu"). Serial sevens (sequential subtraction of 7 from 100) were performed to assess the attention of each patient. The repetition phrase item, "no ifs, ands, or buts," was modified to "seeing is believing" with vowel sounds ("baek-mun-yi-bul-yeo-il-gyen"). The three-stage command was modified to "turn the paper over, fold it in half, and give it to me." The total K-MMSE score was calculated by summing the correct responses to all the K-MMSE sub items. The test scores ranged from 0 to 30.

The Korean version of the Primary Care Post-traumatic Stress Disorder Screen

The PC-PTSD is a four item screen designed for use in primary care clinics.18 Questions are scored dichotomously as either 0 or 1 (0=No; 1=Yes). Each item corresponds to one of the four factors associated with PTSD including 1) re-experiencing a traumatic event, 2) emotional numbing, 3) avoidance, and 4) hyperarousal.22 A score of 3 points is a recommended reference point for further evaluation in order to achieve an effective screening for PTSD, but often 2 points is sufficient to use as a reference point to maintain the proper amount of sensitivity.23

The Korean version of the Single-item Post-traumatic Stress Disorder Screener

The SIPS had one question, which was chosen from of a large number of candidate questions from repeated reviews by investigators based on feedback from respondents.19 This question read as follows: "Were you recently bothered by a past experience that caused you to believe you would be injured or killed?" Response options were "not bothered at all," "bothered a little," and "bothered a lot."

The Korean version of the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders-IV

The SCID is a comprehensive and standardized structured diagnosis that follows the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders 4th edition-revised (DSM-IV-TR), and evaluates all of the Axis I disorders.24 For PTSD, questions standardized for 17 symptoms are given to the subject, and each question is answered one of "?=Insufficient information," "1=None or Not applicable," "2=Below threshold," and "3=Threshold or applicable." The scale also evaluates activity, daily living, disability, onset, etc. In Korea, the standardized SCID has been tested in several studies.25 Of all SCIDs, only the PTSD module consisting of standardized questions regarding 17 symptoms was applied in this study and the result was used as the standard for PTSD diagnosis.

The Korean version of the Post-traumatic Stress Disorder Checklist

The PCL, developed by Weathers et al. in 1990, is a self-reporting rating scale for assessing the 17 DSM-IV symptoms of PTSD. The PCL classifies the subjects into three groups based on re-experience and evasion, although the differences are very subtle. The subjects answer on a 5-point Likert scale, from "1=not at all" to "5=absolutely yes", about their individual symptoms during the previous month. The total scores range from 17 to 85.26 When the scale was applied to adults traumatized by an accident or sexual abuse, its diagnostic performance was highest with a cut-off score of 4426 with 50 is considered an adequate cut-off score for the PCL in general.27

Procedure

The PC-PTSD and SIPS were first translated into Korean by two psychiatrists and two physicians, and any English phrases that were difficult to understand were translated into Korean after consulting a Korean professor of English literature. Next, the Korean PC-PTSD and SIPS were back-translated to English by a medical doctor to validate the Korean translation, and then the back-translated version was reviewed. Preliminary data were collected from the normal elderly population and from patients with PTSD. The content of the final PC-PTSD-K and SIPS-K versions were further verified by the back-translation procedure until the meaning of each item matched that of the original item in the PC-PTSD and SIPS.

A psychiatrist conducted all participant interviews. Demographic data included age, gender, education, marital status, annual income, and military branch of service. The MMSE, SCID, PCL, PC-PTSD-K, and SIPS-K were performed through interviews and written evaluations. A psychiatrist applied the PCL and SCID to screen and diagnose PTSD, respectively. The SCID is a comprehensive and standardized diagnosis tool evaluating all Axis I disorders,24 therefore a correlation coefficient in conjunction with this scale was used as an indicator of the "criterion validity" of the PCL. All subjects were retested with the PC-PTSD-K and SIPS-K at the four-week follow-up.

Statistical analyses

Intergroup comparisons were performed using independent t-tests and Chi-squared analyses to compare the quantitative and categorical variables, respectively. The internal consistencies of the PC-PTSD-K and its subscales were assessed by estimating Cronbach's alpha coefficients for each pair, and the test-retest reliability for the PC-PTSD-K and the SIPS-K were evaluated with the Pearson correlation coefficient. The validity of scales among the PC-PTSD-K, SIPS-K, SCID, and PCL were evaluated using Pearson correlation coefficients, as well. The sensitivity, specificity, and predictive values were calculated according to standard formulae. A receiver operating characteristic (ROC) curve was used for each test to obtain the optimal cut-off score for detecting PTSD. The subjects were divided into two groups by applying the respective cut-off scores for both the positive and negative predictive values based on the SCID, PTSD, and non-PTSD diagnoses, then the percentage of patients with PTSD and without PTSD was calculated. The area under the ROC curve quantifies the degree to which a scale diagnoses the presence of a disorder accurately. An area under the ROC curve of over 0.50 means that accurate diagnosis of the disorder is more probable than chance; an ideal test has an area under ROC curve close to one.28 The statistical significance criterion was defined to be p<0.05 for the two-tailed test. IBM SPSS Statistics for Windows version 21.0. (Armonk, NY: IBM Corp, USA) was used for the statistical analysis.

RESULTS

Description of subjects

Based on the results of the SCID-PTSD module and psychiatric diagnoses, each subject was placed into one of the two groups: the PTSD group (n=41) or the non-PTSD group (n=99). Of the subjects, 102 (72.9%) were in their 60s and 68 (48.6%) were middle school graduates or lower. 74 (52.9%) had annual income of 10 million won or less, and 125 (89.3%) had been in the army (Table 1).

Table 1

Demographic characteristics of the subjects (N=140)

*unit/10,000 won

Reliability

In order to measure internal consistency, a correlation analysis of the questions was performed. Cronbach's alpha for the total score of the PC-PTSD-K was 0.76, showing high internal consistency. The correlations between each of the questions for the PC-PTSD-K and the scale were analyzed. Pearson's correlation coefficients were 0.52–0.61, showing a moderate correlation (all p<0.01) (Table 2). The evaluation of the test-retest reliability was 0.97 for the PC-PTSD-K, and 0.91 for the SIPS-K, showing a relatively high level of consistency. The retest interval was 30.3±3.3 days.

Table 2

Item-total correlation and Cronbach alpha of PC-PTSD-K

p<0.01. PTSD: post-traumatic stress disorder, PC-PTSD-K: Korean version of primary care PTSD screen

Validity

Comparison of different diagnostic groups

The mean score of the PC-PTSD-K in the PTSD group was higher than that of the non-PTSD group (t=-12.1, p=0.001). In addition, the PTSD group was found to have a significantly higher SIPS-K score than the non-PTSD group (t=-10.4, p=0.001) (Table 3).

Table 3

Comparisons of PC-PTSD and SIPS scores between PTSD and non-PTSD group

*p<0.01. PTSD: post-traumatic stress disorder, PC-PTSD-K: Korean version of primary care PTSD screen, SIPS-K: Korean version of single-item PTSD screener, SD: standard deviation

Concurrent validity

In the analysis of the relationships between the SCID, PC-PTSD-K, and SIPS-K, the sum of questions B, C, and D scores from the SCID was calculated. The PC-PTSD-K had a high correlation with the SCID (r=0.74, p=0.001), and the PCL (r=0.87, p=0.001). The SIPS-K also showed a high correlation with the SCID (r=0.69, p=0.001), and the PCL (r=0.78, p=0.001). Moreover, a significant correlation was found between the PC-PTSD-K and SIPS-K (r=0.81, p=0.001) (Table 4).

Table 4

Correlations of SCID, PCL, PC-PTSD and SIPS

**p<0.01. SCID: structured clinical interview for DSM-IV, PCL: PTSD checklist, PC-PTSD-K: Korean version of primary care PTSD screen, SIPS-K: Korean version of single-item PTSD screener

The ROC analysis

Table 5 shows different threshold scores and the corresponding sensitivity, specificity, and the positive and negative predictive values (PPV and NPV, respectively) of the PC-PTSD-K and SIPS-K. The area under curve (AUC) of the PC-PTSD-K was 0.92 (p=0.001) (Figure 1). A cut-off point of 3 for the PC-PTSD-K produced a sensitivity level of 90.2%, a specificity level of 86.9%, a PPV of 74.0%, and a NPV of 95.5%. The AUC for the SIPS-K was 0.89 (p=0.001) (Figure 2), and using the cut-off item of 'bothered a lot' yielded a sensitivity level of 85.4%, a specificity level of 89.8%, a PPV of 77.8%, and a NPV of 93.7%.

Table 5

Diagnostic utility of alternative cur off values for the PC-PTSD and SIPS

PC-PTSD-K: Korean version of primary care PTSD screen, SIPS-K: Korean version of single-item PTSD screener, Se: sensitivity, Sp: specificity, PPV: positive predictive value, NPV: negative predictive value

Figure 1

The receiver operating characteristic curve comparing the primary care PTSD screen score against PTSD module of Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders-IV. PTSD: post-traumatic stress disorder.

Figure 2

The receiver operating characteristic curve comparing the Single-item PTSD screener score against PTSD module of Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders-IV. PTSD: post-traumatic stress disorder.

DISCUSSION

The present study was performed in order to evaluate the applicability of the Korean version of the PC-PTSD and SIPS for an elderly population in a primary care setting, and to report data concerning its psychometric properties, reliability, and validity. In the present study, the internal consistency of the PC-PTSD-K was within 0.52–0.61 and the test-retest reliability coefficient was 0.97, which are consistent with those reported in previous studies from other countries.29 Prins and colleagues found a test-retest reliability coefficient of 0.83 and a Kappa coefficient of 0.58 for 188 participants with the PC-PTSD, which supports our findings.29 Thus, our PC-PTSD-K showed sufficiently high temporal consistency and reliability. The PC-PTSD-K also showed high concurrent validity and high criterion validity with the SCID, PCL, and SIPS-K. The concurrent validity of the PC-PTSD-K in comparison to the PCL showed a moderate or increased positive linear correlation. The criterion validity of the PC-PTSD-K was evaluated by comparison to the SCID's diagnosis and quantification of PTSD. Our findings demonstrate that the PC-PTSD-K is an appropriate screening instrument, with an acceptable level of validity. Gore's study of 3,234 patients using SIPS reported a test-retest reliability coefficient of 0.63.19 The present result for the test-retest reliability coefficient is 0.91, a higher value than the result of the aforementioned study, which suggests that the SIPS-K has sufficiently high temporal consistency. Moreover, the SIPS-K showed a high concurrent validity with the SCID, PCL, and PC-PTSD-K. A previous study documented a coefficient of 0.63 between the SIPS and PCL, however a higher positive linear correlation was observed in this study.19 Therefore, the SIPS-K should be a good screening tool for PTSD because of its acceptable validity.

A cut-off point was calculated for the effective discrimination of PTSD amongst the patients. In our previous study with PC-PTSD, 3 points was the suggested cut-off score which yielded a sensitivity level of 78%, a specificity level of 87%, a PPV of 65%, and a NPV of 92%.29 The PC-PTSD-K also showed the highest sum of sensitivity and specificity at a cut-off point of 3, with 90.2% in sensitivity, 86.9% in specificity, 74.0% in PPV, and 95.5% of NPV. For SIPS, previous studies reported the "bothered a little" response to be the cut-off score.19 This cut-off item generated a relatively lower sensitivity of 76%, similar that of the PC-PTSD. However, the present study showed the "bothered a lot" cut-off item to have a sensitivity level of 85.4% and specificity level of 89.8%. This discordance in results might stem from the difference in clinical settings and specific variables, including culture, prevalence, severity of illness, and subject age, gender, and education. Special considerations should be considered during the evaluation of PTSD symptoms in older adults, such as the perceived shame of reporting traumatic events or associated symptoms, inadequate labeling of an event as traumatic or misunderstanding of the terminology, and historical differences regarding gender roles. Moreover, misinterpretation of psychological symptoms as somatic concerns and aging-related sensory or cognitive impairments might affect reliability in assessment of the elderly. Thus, the SIPS-K should still be a useful screening instrument using the cut-off item of "bothered a lot", instead of "bothered a little", in the assessment of PTSD within the elderly Korean population.

The results of this study should be interpreted with caution, due to some limitations. The only subjects in this study were male Vietnamese war veterans, however, the general community is known to have twice as many women with PTSD as men. Trauma type and severity are also important determinants for the risk of developing PTSD. Our results only represent Vietnamese war veterans who are currently seeking help, rather than patients in the general community who may not be aware of their mental discomfort, thereby making it difficult to draw any general conclusions. However, because only traumatic combat events were included in this study, the sample size for the individual events was not small compared to other studies. Additionally, most of the healthy controls and patients without PTSD did not experience a traumatic event strictly according to the DSM-IV diagnostic criterion for PTSD. Instead, they recollected the most distressing traumatic event in the past year. Finally, the possibility that the patients with PTSD may have had other comorbid psychiatric disorders was not considered. Comorbidity rates are high among patients with PTSD, with approximately two-thirds having at least two other disorders. Nevertheless, the PC-PTSD-K and the SIPS-K demonstrated appropriate psychometric properties and may be used as reliable and valid tools to screen the elderly population for PTSD (although some cultural variation was apparent). Further studies will be needed to fully evaluate the PC-PTSD-K and the SIPS-K, including its application to the general population, primary care patients, patients with various non-combat trauma experiences, and those with other special psychiatric disorders.