Development of the Perceived Stress Inventory: A New Questionnaire for Korean Population Surveys.

BACKGROUND: Given emerging evidence of the association between stress and disease, practitioners need a tool for measuring stress. Several instruments exist to measure perceived stress; however, none of them are applicable for population surveys because stress conceptualization can differ by population. The aim of this study was to develop and validate the Perceived Stress Inventory (PSI) and its short version for use in population surveys and clinical practice in Korea.
METHODS: From a pool of perceived stress items collected from three widely used instruments, 20 items were selected for the new measurement tool. Nine of these items were selected for the short version. We evaluated the validity of the items using exploratory factor analysis of the preliminary data. To evaluate the convergent validity of the PSI, 387 healthy people were recruited and stratified on the basis of age and sex. Confirmatory analyses and examination of structural stability were also carried out. To evaluate discriminatory validity, the PSI score of a group with depressive symptoms was compared with that of a healthy group. A similar comparison was also done for persons with anxious mood.
RESULTS: Exploratory factor analysis supported a three-factor construct (tension, depression, and anger) for the PSI. Reliability values were satisfactory, ranging from 0.67 to 0.87. Convergent validity was confirmed through correlation with the Perceived Stress Scale, Center for Epidemiologic Studies Depression Scale, and State-Trait Anxiety Inventory. People with depressive or anxious mood had higher scores than the healthy group on the total PSI, all three dimensions, and the short version.
CONCLUSION: The long and short versions of the PSI are valid and reliable tools for measuring perceived stress. These instruments offer benefits for stress research using population-based surveys.

INTRODUCTION

There are emerging concerns about the associations between stress and related conditions, such as cardiovascular disease, depression, and cancer.1234) Early research has measured stress in different ways: external stressors, stress responses, and resilience. Contrary to the early focus on external stressors such as life events, current research commonly measures perceived stress, such as an individual's stress appraisal or emotional response. Many studies have shown an inconsistent picture of the effects of external stressors on health; therefore, recent studies have emphasized stress response, especially perceived stress.5)

Approaching this challenge, Levenstein et al.6) developed the Perceived Stress Questionnaire (PSQ) to assess perceived stress in terms of the cognitive, emotional, and symptomatic sequelae of a stress event. Factor analysis showed seven scales for the PSQ (harassment, overload, irritability, lack of joy, fatigue, worry, and tension). Follow-up studies of the PSQ revealed another psychometric construct in a large and diverse population.7) The conceptualization of stress appears to vary according to the population.

Koh et al.8) developed the Stress Response Inventory (SRI) and the Stress-induced Cognition Scale (SCS).9) The SRI is a measurement tool that comprehensively focuses on four different types of stress responses, including cognitive reactions. Koh et al.8) proposed this tool to measure individual response to a threat. The SCS, focusing on the cognitive dimension, has 21 items and three factors (extreme-negative thoughts, aggressive-hostile thoughts, and self-depreciative thoughts). The SRI and SCS might be good instruments to measure stress reactions in the Korean population; however, they were developed using a small population recruited from one hospital. In addition, a subsequent psychometric study of the SRI showed a difference from the original structure using a large population of workers, but most of those subjects were young men. Therefore, further study using a general population is necessary to generalize the SRI10) because the results of stress-measuring instruments might vary with sex and age. Validation studies of a new stress questionnaire developed using a healthy population similar in age and sex composition to census data could support a better instrument to evaluate stress in the general population.

Selecting a questionnaire for use in a large-scale survey or clinical practice study requires consideration of its length because a long questionnaire can place an unacceptable burden on survey participants or patients.11) Therefore, it is of practical importance to identify whether short versions are appropriate for discriminating factors of clinical importance. The purpose of this study was to develop a new instrument with a short version in order to measure perceived stress in a large-scale population survey.

METHODS

1. Development of a New Instrument of Perceived Stress

First, a consensus survey using the Delphi method was used to determine the questions important in measuring stress. The responses to two rounds of surveys from 17 stress experts in Korea were collected. Those experts recommended measuring perceived stress with population-based surveys in stress research and suggested using the items of the PSQ, SCS, and SRI as the pool for a new instrument. A research team of 8 psychologists and stress research experts reviewed and agreed with the results of the consensus survey. All 81 items in the item pool were thus included for the new questionnaire development before excluding several overlapping items. Responses to these items were arranged into a Likert-type format: 'not at all (1 point),' 'somewhat (2 points),' 'moderately (3 points),' 'very much (4 points),' and 'absolutely (5 points).'

Next, a preliminary study was conducted to create a questionnaire using 373 college students and healthy volunteers from several hospitals as subjects. The subjects had a mean age of 35.2±10.5 years (range, 21 to 73 years); 48.7% of them were married; 46.9% were female; and 86.9% of them were college graduates. Exploratory factor analysis (EFA) was employed to select the most important items from the pool.

The EFA of the preliminary data yielded 12 subscales with eigenvalues greater than 1. Among them, the 8 subscales of harassment, worry, tension, anger, aggression, depression, lack of joy, and loss of self-esteem were accepted as representing perceived stress by the research team. The excluded subscales were two types of somatization factors and external stressors. The items showing higher factor loading (>0.5) and higher item-total correlation were selected from the 8 subscales, with 1-4 items selected from each subscale. In total, 20 items were selected for the Perceived Stress Inventory (PSI) (Appendix 1).

2. Validation Study of Perceived Stress Inventory

Participants in the validation study were 387 individuals from several hospitals, recruited by advertisements. Most of them were patients or caregivers associated with the outpatient clinic or family members of hospital employees. We recruited as subjects only patients who were visiting the hospital for a regular health check-up or medical certificate and excluded volunteers with inappropriate conditions such as a severe disability or acute cardiac disease. The presence of many diseases that are influenced by stress reactions, such as infection, hyperthyroidism, irritable bowel syndrome, diabetes mellitus, and hypertension, along with psychiatric disease, such as depression or anxiety, and use of any medication affecting the stress response, such as nonsteroidal anti-inflammatory drugs, estrogen, steroids, or codeine, were also grounds for exclusion. We recruited to match the age and sex composition of the 2009 yearly population, in terms of expected change from the 2005 census data.12) All subjects agreed to participate in the study by signing an informed consent form and answering the questionnaire; they also received a written explanation of the study. Of the total participants, 48 randomly selected subjects agreed to repeat the test 2 weeks later. The study protocol was approved by institutional review board of Inje University Hospital (approval no. IB-1008-037).

All enrolled participants were included in the analysis. Participants completed both the 20-item and short 9-item versions of the PSI. All subjects also completed the Korean version of the Perceived Stress Scale (PSS),13) the Korean version of the Daily Stress Inventory (DSI),1415) the Korean version of the PSQ,16) and the SRI.8) Depressive mood or anxiety and quality of life (QoL) were also measured using the Center for Epidemiologic Studies Depression Scale (CES-D),17) State-Trait Anxiety Inventory (STAI),18) and Euro QoL (EQ5D) index and visual analog scale.19) The test-retest reliability of the preliminary questionnaire was calculated using the results of the first and second tests over the 2-week interval.

3. Statistical Analysis

Factor analysis was performed using varimax orthogonal rotation following a principal component analysis, and only the factors with an eigenvalue greater than 1 were retained in the final model. Depression, anxiety, and QoL were subjected to Pearson's correlation analysis in order to determine their relationships with the PSI scores. Pearson's correlation was also employed to assess the relationships between the PSI scores and the SRI, PSQ, and PSS scores. The comparison groups for the discriminant validity test were composed of subjects with depressive symptoms (CES-D score 21 or higher)17) and a group of healthy adults, defined as subjects without depressive symptoms (CES-D score less than 21). We also compared the scores between an anxious group (STAI score 57 or higher)18) and a healthy group. The comparison of PSI scores according to mood was performed using Student t-test. The test-retest reliability of the PSI was calculated using Pearson's correlation and Kappa statistics at the first and second testing sessions. Cronbach's α was calculated to measure the internal consistency of the dimensions and the total score. Confirmatory factor analysis determined the suitability of the 20 questions selected by the EFA. The IBM SPSS statistics program ver. 21.0 (IBM Co., Armonk, NY, USA) was used to analyze the data.

RESULTS

1. Participants in the Validity Study of the Perceived Stress Inventory

A total of 387 subjects aged 20 years or older were randomly selected from two hospitals, one in a metropolitan city and the other in a province. Of these subjects, 53.2% were female. The mean age was 43.10 years for men and 44.6 years for women. The proportion of college graduates was 68.2% of men and 42.9% of women; 74.2% of men and 67.7% of women were married. The health behaviors of our subjects, such as smoking and exercise, were similar to the results of the Korea National Health and Nutrition Examination Survey (Table 1).20) The proportion of depressed persons (CES-D ≥21) was 34.1% and 15% of all subjects had anxiety (STAI-S or STAI-T ≥57).

Table 1

General characteristics of study population (N=387)

Values are presented as mean±standard deviation or number (%).

STAI, State-Trait Anxiety Inventory; EQ5D, Euro quality of life.

*EQ5D_index is calculated based on result of 2007 KNHANES.

2. Factor Analysis of the Perceived Stress Inventory

The PSI is a new 20-item instrument to measure perceived stress; it is comprised of 6 items from the SRI, 8 items from the PSQ, 2 items from the SCS, and 4 overlapping items from the SRI and SCS. Explorative factor analysis was again employed to determine the psychometric constructs of the new PSI. The 9-item short version was based on those psychometric constructs. The exploratory analyses with oblique rotation yielded three factors with an eigenvalue greater than 1: tension had an eigenvalue of 12.25 and explained 61.3% of the variance; depression had an eigenvalue of 1.17 and explained 5.8% of the variance; and anger had an eigenvalue of 1.00 and explained 4.9% of the variance. The explanatory powers of the second and third factors were similar. The factor loadings of each item were greater than 0.5. The resulting three-factor model is shown in Table 2.

Table 2

Factor analysis of Perceived Stress Inventory-long form

Bold figures indicate items greater than 0.5 of factor loading.

PSQ, Perceived Stress Questionnaire; SRI, Stress Response Inventory; SCS, Stress induced Cognition Scale.

*O means items included in short version.

Most items of the first factor, tension, were drawn from the PSQ factors of tension, fatigue, and worry. Items of the second factor, depression, were drawn from all three questionnaires' factors of depression or harassment. Items of the third factor, anger, were from the anger and aggression factors of the SRI (Table 2).

Confirmatory factor analysis was performed to determine the suitability of the PSI factor structure selected by the EFA. The standardized regression weights for each question were all greater than 0.5; average variance extracted values were all ≥0.5 (tension, 0.591; depression, 0.600; anger, 0.625). Construct reliability was also greater than 0.7 (tension, 0.935; depression, 0.899; anger, 0.868). Therefore, the construct validity of the PSI is good.

3. Reliability of the Perceived Stress Inventory

All scores, including the total PSI and each of the three dimensions, were significantly correlated with each other. Each PSI item score was also highly correlated with the total score (range, 0.655 to 0.859) (Table 3). The short version also had significant correlations between the total and each item score (range, 0.625 to 0.846) (Table 4). The Cronbach's α value for the total score was 0.966, and values for the three dimensions were greater than 0.8. The test-retest reliability of the PSI scores was significant, ranging between 0.672 and 0.875. The anger dimension showed the lowest correlation between the test and retest (Table 5). The Cronbach's α value of the short version was also relatively high (0.909, shown in Table 5). The correlation coefficient of the short version's test-retest scores was lower than that for the longer version.

Table 3

Item-total and item-subtotal correlation of Perceived Stress Inventory

*All P<0.01.

Table 4

Item-total correlation of short version of PSI

PSI, Perceived Stress Inventory; PSQ, Perceived Stress Questionnaire; SRI, Stress Response Inventory; SCS, Stress induced Cognition Scale.

*All P<0.01.

Table 5

Test-retest reliability and internal consistency of the long and short version of the Perceived Stress Inventory

γ means Pearson's correlation coefficient.

*All P<0.01.

4. Validity of the Perceived Stress Inventory

As expected, the total PSI score significantly correlated with those of the other stress measurements, SRI, PSQ, and PSS. We also found a small but significant correlation coefficient between the PSI and DSI scores. The tension dimension has many items from the PSQ and showed strong correlation with other stress measurements; for example, 0.851 between tension and the SRI. The PSI total score and three dimension scores showed large, significant correlations with anxiety and depression but a weak and negative correlation with both measures of QoL (Table 6). Unlike tension, anger had a weak correlation with anxiety and depression. We observed a similar correlation between the short version of the PSI and other stress measurements, but the associations were somewhat attenuated.

Table 6

Correlation of the score of PSI with other measurement and the difference according to anxiety or depressive mood*

PSI, Perceived Stress Inventory; CES-D, Center for Epidemiologic Studies Depression Scale; STAI_S, State-Trait Anxiety Inventory_state; STAI_T, State-Trait Anxiety Inventory_trait; EQ5D, Euro QoL index.

*All P<0.01. †EQ5D_index is calculated based on result of 2007 Korea National Health and Nutrition Examination Survey. EQ-5D=1-(0.05+0.096×M2+0.418×M3+0.046×SC2+0.136×SC3+0.051×UA2+0.208×UA3+0.037×PD2+0.151×PD3+0.043×AD2+0.158×AD3+0.05×N3).

To measure the discriminant validity of the PSI, all subjects were divided into two groups: those with and without depressive mood according to CES-D scores. The anxious group was also separated from the healthy population using STAI scores. All scales differed between healthy adults and people with anxious or depressive mood. Both the depressive group and anxious group had consistently higher PSI scores than the healthy group (Table 6).

DISCUSSION

The purpose of this study was to develop a new measure of perceived stress using items drawn from three other instruments for healthy Korean people. Exploratory factor analysis showed three scales in the new measure: tension, depression, and anger. This study also demonstrates the validity and reliability of both the long and short versions of the PSI.

The item pool for the new questionnaire came from three instruments suggested by previous research. We selected 20 items covering diverse facets of perceived stress reactions. The 8 items from the PSQ were originally included in the tension, fatigue, and worries dimensions of Levenstein et al.6)'s study, which used a population with gastroenterological problems. Those same items were categorized as worry and tension in the revised model of Fliege et al.7) Tension can be regarded as an important concept in perceived stress and is a common dimension of stress measurements. As expected, the tension score in this study correlated highly with that in the PSS (γ=0.723), which was higher than the dimensions of depression and anger. The PSI tension score also showed a strong association with tension and trait anxiety, similar to Levenstein et al.6)'s study.

The dimensional structure of the PSI is meaningful for understanding perceived stress in a Korean population. We propose an anger dimension in the new questionnaire based on many items from the SRI or SCS, which were developed in Korea. Anger is frequently reported as a unique aspect of the Korean stress response, unlike in Western society. For example, the Korean culture-related anger syndrome Hwa-byung has been reported and included in the appendix of the DSM (diagnostic and statistical manual of mental disorders)-IV.21) This disease is believed to be related to the special culture of Korea, including familial collectivism and social suppression by the ruling class.2223) Thus, anger, one dimension of the PSI, is a stress response influenced by culture. The scale for anger showed good validity with depression and anxiety but relatively lesser reliability because it is temporary in nature.

Some SRI items were initially excluded from the item pool. The SRI has 39 items and a seven-factor model (tension, aggression, somatization, anger, depression, fatigue, and frustration) according to the original study. Unlike the dimensions of the original SRI, Choi et al.10) identified the somatization dimension using items such as "I suffer from indigestion," or "I feel dizzy." Those items reflect a somatic response rather than perceived stress, so we excluded them as inappropriate to the purpose of the new measurement. The PSS was not included in the item pool, even though it is widely used, because its content is nonspecific and without subscales. However, the PSS is based on the idea that stress results from experience overload, with further emphasis on the unpredictability and uncontrollability of events.2425) The PSS score showed significant correlation with the new measure.

The recently published Brief Inventory of Perceived Stress (BIPS)26) consists of 9 items drawn from the PSQ and PSS in order to compensate for the poor model fit of the PSS.1326) Lahman et al. proposed a three-factor model with 8 PSQ items and 1 PSS item, two of which are included in the tension and depression scales of the PSI. The most important advantage of the BIPS is its short questionnaire, possible for quick administration, which is similar to the short version of the PSI. Both questionnaires are useful instruments for measuring perceived stress in a population survey. However, the BIPS has a three-factor model with a different psychometric structure (lack of control, pushed, and conflict and imposition) to that of the PSI. This means each instrument reflects a different way of understanding perceived stress. The different content structure of both instruments comes from cultural influences on the stress concept. The strong correlation between the PSI short version and the PSS, CES-D, and STAI-T confirmed the good criterion validity of the short version, even though it has unique dimensions.

The purpose of the Perceived Stress Inventory (PSI) development is use for population surveys. The strength of this study is that the age and sex composition of the subjects was similar to that of the 2009 census data.12) In addition, we tried to minimize the influence of personal health status on the stress reactions being measured. The content and criterion validity of the PSI were confirmed using factor analysis and correlation analysis in a healthy population. Though this study design is good for generalizing our findings, further study using a national survey is necessary to generalize it even more broadly. Another strength of our work is the unique psychometric structure of the PSI that includes anger, which is an important factor in Korean culture. Anger is commonly associated with depression or anxiety in Korea.27) However, it is not found in instruments developed in Western cultures. Further study would thus be necessary to generalize this instrument to Western society.

The PSI, a new instrument for measuring perceived stress, has a three-factor construct and excellent reliability and validity in healthy Korean adults. It measures three dimensions with acceptable psychometric properties and can be useful for population surveys. The short version of the PSI, with similar reliability and validity, could be even more feasible for surveys or clinical practice because of its brevity.