Reliability and validity of Korean version of modified: Yale preoperative anxiety scale.

PURPOSE: The modified Yale Preoperative Anxiety Scale (mYPAS) was developed for evaluating the level of preoperative anxiety in children. The purpose of this study was to develop a Korean version of the mYPAS (K-mYPAS) and to establish its validity and reliability based on the Korean preoperative pediatric patients.
METHODS: K-mYPAS was made through stringent back-translation procedure. Total enrolled 102 patients answered questionnaires of Korean version of State-Trait Anxiety Inventory for Children (K-STAIC), and were videotaped for 2 to 5 minutes before induction of anesthesia. Three observers of experienced psychiatrist, surgeon, and nurse analyzed videotape with K-mYPAS comparing to K-STAIC. The inter- and intraobservers reliability, concurrent and construct validity, sensitivity, specificity, and predictive value were analyzed.
RESULTS: The value of Cronbach α for interobservers reliability was 0.939 and intraobserver reliability was statistically significant (P < 0.001). Concurrent and construct validity were also statistically significant (P < 0.001 and P < 0.001, respectively). Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 81.3%, 91.4%, 81.3%, 91.4%, and 88.2%, respectively.
CONCLUSION: The K-mYPAS had good psychometric properties and can be used as a reliable and valid instrument for the assessment of preoperative anxiety in children.

INTRODUCTION

The anxiety from surgical procedures in children could cause postoperative negative psychological effects such as enuresis, night terrors, separation anxiety, eating problems, temper tantrum and increased fear of doctors [12]. There have been some efforts to measure the level of preoperative anxiety in children. The standard scale for evaluation of anxiety in children is Spielberger's State-Trait Anxiety Inventory for Children (STAIC) [3]. This scale was modified as pediatric version of Spielberger's State-Trait Anxiety Inventory (STAI) which is the gold standard for anxiety evaluation [4]. However, because the STAIC is a self-reporting questionnaire that has 20 items, this scale has been using for children who can read and understand what the sentence means. Therefore, children that are too young cannot use the scale of STAIC. In general, STAIC can be used in children more than five years old [5]. It takes 5 to 10 minutes to write the answers of STAIC, thus, it is hard to have enough time to check the answers before induction of anesthesia. Moreover, because all children answered the questionnaires in the preoperative area, the resulting preoperative anxiety level written in the preoperative room might not reflect the exact level of preoperative anxiety in induction room. Actually, most of the children's anxiety reached peak level when he/she was in the induction room for anesthesia [6]. For this reason, Kain et al. [5] developed the modified Yale Preoperative Anxiety Scale (mYPAS), which was not inferior or rather superior, compared to STAIC. mYPAS consisted of five questions that were assessed by observer for only less than one minute. Observers can also check the accurate status of anxiety in children whether the patients were in the induction room for anesthesia or preoperative area [5]. Because there was no scale inventory which could evaluate the level of anxiety in Korea, physiologic data like pulse rate or visual analogue scale was used for evaluating preoperative anxiety in children.

The purpose of this study was to develop a Korean version of the mYPAS (K-mYPAS) and to establish its validity and reliability based on the Korean general children and on preoperative pediatric patients.

METHODS

This study was approved by Institute Review Board of Seoul National University Bundang Hospital. The approval number was B-1304/200-003. This study was underway with research fund supported by the authors' affiliated institution.

Patients

From April 2013 to October 2013, one hundred fourteen patients (age range, 5 to 12 years) who can read the printed letters and understand the meaning of sentences in Korean were enrolled in this study. Twelve of 114 patients were dropped from study because of refusal of study after signing on the written consent, cancellation of operation, and administration of intravenous sedative drug before entering operation room (Fig. 1). Surgical procedures included repair of inguinal hernia, appendectomy, strabismus surgeries, foreign body removal in orthopedic department, tonsillectomy and adenoidectomy under general anesthesia through the day care surgery or short-term admission by 4 surgeons and 1 pediatric anesthesiologist. The demographic and clinical characteristics were collected from patients including age, gender, history of previous surgery, education level of parents. This study was performed in a single tertiary referred hospital.

Fig. 1

Diagram of enrolled patients.

Development of K-mYPAS

The K-mYPAS was made through stringent back-translation procedure. mYPAS was translated firstly by three Korean child and adolescent psychiatric specialists. They analyzed mYPAS, searched emphasized and cored words on each sentence of questionnaire of mYPAS, and included them in K-mYPAS. The questionnaire of mYPAS translated into Korean was translated back by another 3 doctors who has lived in an English-speaking country for more than 10 years and were equally adept in English and Korean. The same procedure was repeated until all the key words were included in the back-translated questionnaire. Lastly, a doctor, proficient in English and Korean, reviewed and edited the sentences in the questionnaire to maintain the appropriate meaning for both cultures. Through these procedures, we established the final K-mYPAS (Fig. 2 and Supplementary material).

Fig. 2

Diagram of translate-back translate procedure. mYPAS, modified Yale Preoperative Anxiety Scale; K-mYPAS, Korean version of the mYPAS.

Reliability and validity analysis

When the patients visited the out-patient clinic, we received the written consent of child and the parents. The children wrote the STAIC-Trait at the out-patient clinic assisted by researchnurses who were educated for this study. This questionnaire consisted of 20-items and reflected the usual or basal anxiety condition. On the day of surgery in the preoperative room, all patients completed the STAIC-State. They should respond on 3-point scale, yes, no or intermediate. The range of total scores was 20 to 60. The higher score, children were considered as highly anxiety state.

After a nurse checked the patient in preoperative room, an anesthesiologist and a surgeon took him or her to operating room with or without parents. At operating room, anesthesia was induced using O2/N2O and sevoflurane administered via a mask. Through whole course from checking in preoperative room to anesthesia in operating room, the children's activity, vocal, emotion, arousal status and relationship with parents were recorded in smartphone for 2 to 5 minutes. The recorded videos were analyzed by three naive observers, consisting of an experienced child and adolescent psychiatrist, surgeon and nurse with K-mYPAS, separately and independently. They reviewed the video two times with two weeks interval.

Statistical analysis

To assess the reliability, two aspects were set up, one was internal consistency (interobservers) through the value of Cronbach α and the other was test-retest reliability (intraobservers) through the value of intraclass consistent coefficient (ICC). Repeated analysis of the video by three observers was done at two weeks interval, respectively. Results were also assigned the appropriate clinical significance to individual value: <0.40, poor; 0.40-0.59, fair; 0.60-0.74, good; and 0.75-1.00, excellent [7]. To assess the concurrent validity compared with a gold standard, STAIC, the correlation coefficient (r) for the K-mYPAS against the Korean version of STAIC was used. The construct validity was assessed by comparing the anxiety in preoperative room to anxiety in operating room when the child was induction of anesthesia. Statistical analysis between two-points was performed using matched pair sample t-test.

For identifying the K-mYPAS score that indicates high anxiety in children undergoing operation, we used the Korean version of STAIC as a gold standard. The cutoff score for high anxiety was derived from Korean version of Spielberger's manual which provides normative data for children aged 5 to17 years [3]. According to Spielberger's manual, the standard value of high anxiety was mean value plus 1 standard deviation value in normative group. However, the mean value and standard deviation of the Korean version of STAIC was 36.44 and 8.48, respectively [8]. The mean + 1 standard deviation (SD) was 44.92. Therefore, we could defined the value of high anxiety level in Korean children according to the Korean version of STAIC was 45 and children who scored more than 45 on the Korean version of STAIC would be classified in this study group as high-anxiety cases. Subsequently, the sensitivity, specificity, positive predictive value, and negative predictive value were examined for different cutoff points on the K-mYPAS in relation to the Korean version of STAIC as a gold standard by using a receiver operating characteristic (ROC) curve.

All statistical analyses were performed using IBM SPSS Statistics ver. 20.0 (IBM Co., Armonk, NY, USA).

RESULTS

Clinical characteristics

Total 114 of 232 eligible patients consented for this study. Twelve patients were excluded. The six of 12 patients canceled operation. One patient was transferred to the other hospital for emergent operation because of lack of beds. One guardian who has accepted the study and has signed on written consent rejected the study in preparation room at the operation day. Investigator did not meet the two patients because their operation schedules were changed at operation day. Two patients were also excluded from study because they were administrated with pentothal in preparation room. Finally, total 102 patients were enrolled and videotaped for study.

The mean age of the study population was 8.4 years old (range, 5 to 12 years). There were 56 male (54.9%) and 46 female patients (45.1%). The range of operations done included 10 appendectomies (9,8%), 19 repair of inguinal hernia (18.6%), 6 tonsillectomy/adenoidectomy (5.9%), 55 strabismus surgeries (53.9%), and 6 foreign body removal under orthopedics (5.9% ) and 6 for the others (5.9%).

The parents attained an educational level of postgraduate studies in 80.4%. There was no one who attained a level below high school education. Most of the parents/guardians were present in the preparation room. However, 57 of parents/ guardians (55.9%) were in the operating room during induction of anesthesia because the patients who were more than 7 years old would like to enter the operating room alone without their parents/guardians.

The mean age, gender, the parental education level and parental presence or not during induction of anesthesia were showed in Table 1. There were no statistically significant relationship between these factors and high anxiety.

Table 1

Patients' characteristics (n = 102)

Values are presented as mean ± standard deviation or number (%).

Reliability

In the interobservers reliability, the value of Cronbach α was 0.939, which provided evidence for the internal consistency of the measure. The intraobserver reliability (test-retest reliability) for psychiatrist, surgeon and nurse was evaluated through ICC, and the values were 0.786 (0.698-0.850, P < 0.001), 0.757 (0.660-0.829, P < 0.001) and 0.745 (0.644-0.820, P < 0.001), respectively.

The results show that the K-mYPAS is a reliable assessment tool between observers and can be used by different specialties. There is no significant difference in the ratings given between the psychiatrist, surgeon or nurse.

Validity

In terms of concurrent validity, three observers' correlation coefficient were 0.399 (P < 0.001), 0.388 (P < 0.001), and 0.393 (P < 0.001), respectively. The anxiety level might be elevated with transit from preoperative room to operation room for induction. The Table 2 showed the construct validity with statistical significance according to each observer.

Table 2

Construct validity

Values are presented as mean ± standard deviation.

K-mYPAS, Korean version of the modified Yale Preoperative Anxiety Scale.

Table 2 shows the construct validity with statistical significance according to each observer. The values in the construct validity were 33.02 ± 7.24 to 52.04 ± 12.18, 34.10 ± 7.90 to 53.15 ± 13.48, and 40.31 ± 10.30 to 51.92 ± 15.23 in psychiatrist, surgeon, and nurse, respectively (P < 0.001). These showed that the change of environments can increase the level of anxiety of patients.

ROC analysis

The reference score of high anxiety was 46 according to study of Korean version of STAIC-State [8]. Table 3 showed the cutoff point for sensitivity and specificity from the ROC curve (Fig. 3). At this score, only 12 patients (11.8%) were misclassified; 6 false positive and 6 false negative. From ROC curve, if we chose the cutoff score of K-mYPAS 50.85, the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 81.3%, 91.4%, 81.3%, 91.4%, and 88.2%, respectively. The area under the curve (AUC) was 0.887 (0.812-0.961, P < 0.001) and this represented nearly perfect performance.

Table 3

Sensitivity, specificity, positive predictive value, and negative predictive value

PPV, positive predictive value; NPV, negative predictive value.

Fig. 3

The receiver operating characteristic curve for the prediction of the diagnosis of high anxiety using Korean version of the modified Yale Preoperative Anxiety Scale. AUC, area under the curve.

DISCUSSION

mYPAS is a good and suitable scale for evaluating of preoperative anxiety in children compared with the STAIC, a gold standard in terms of anxiety through analysis of reliability and validity [5]. Proczkowska-Bjorklund et al. [9] have made Swedish version of the mYPAS and showed its reliability and validity.

STAIC was already introduced and developed in Korean version by Cho and Choi [8] in 1989. However, there has not been developed scale for evaluating of preoperative anxiety in children. Therefore, the adoption of the mYPAS and development of K-mYPAS seemed to be needed for evaluation of the reducing effect by various anxiety reducing tools such as music, video and preoperative carbohydrate-rich solution loading [101112131415].

During translation-back translation, it seemed to be important to select core keywords in every question from mYPAS, because back-translated sentences could not correspond with mYPAS with 100% precision. Three psychiatrists sorted out words or phrases from mYPAS. When two-thirds of the psychiatrists agreed with every word or phrase, that word or phrase was chosen. Lastly, 31 words or phrases were selected through this procedure. If the K-mYPAS was translated back into English and did not include the keywords or key-phrases, this translate-back translate procedure was repeated from translating into Korean version by psychiatrists till keywords or phrases were contained in the translated English version of K-mYPAS. We used the final version of K-mYPAS through this procedure for evaluating the preoperative anxiety level in children.

Kain et al. [5] used a score of 37 in the STAIC as a reference point to define high anxiety cases in the preoperative area. This cutoff score for high anxiety is explained at METHODS section. In the present study, we used the cutoff value for high anxiety derived from Korean normative data studied by Cho and Choi [8]. The normative data was from 522 Korean normative children with age 10 to 12 years old. According to this study by Cho and Choi [8], the mean ± SD was 36.44 ± 8.48 when children faced an examination. This value was quite exact because the levels of students' anxiety would be elevated most highly in Korea. From this study, the cutoff score for considerable anxiety was 45 to 48, and for high anxiety was equal or more than 49. However, cutoff score of our present study was defined for high anxiety, 45 because this value was result of the mean + 1SD as like Kain et al. [5]'s study. The fact that the children with 5 to 9 years old were not included in normative group was limitation in this study. Nonetheless, the sensitivity, specificity, positive predictive value and negative predictive value showed considerably good results. The fact that the results by the psychiatrist, surgeon and nurse as investigators were convergent and the questionnaire only consisted of five questions was affirmative. This shows that anybody can evaluate the preoperative anxiety in children easily without having to undergo further specialized studies.

In conclusion, the K-mYPAS had good psychometric properties and can be used as a reliable and valid instrument for the assessment of preoperative anxiety in children.