Validity and reliability of the Korean version of the Quality of Recovery-15 questionnaire.

BACKGROUND: The quality of recovery-40 questionnaire (QoR-40) has been widely used to assess quality of recovery after surgery, but it is too lengthy for clinical use. The short form of QoR-40, QoR-15, has been validated in many languages; however, an official Korean version of the QoR-15 (QoR-15K) has not yet been established. This study aimed to develop and validate QoR-15K.
METHODS: Based on the previously-validated Korean QoR-40, we selected 15 items; the QoR-15K was patterned on the original QoR-15. We analyzed 210 subjects who had been scheduled for elective surgery under general anesthesia. The patients completed the questionnaire before surgery and on postoperative days one and two. The validity, reliability, and responsiveness of the QoR-15K were evaluated.
RESULTS: We obtained excellent convergent validity on visual analog scale for recovery (ρ = 0.882, P < 0.001). The duration of anesthesia, post-anesthesia care unit, and overall hospital stay with the QoR-15K showed a significant negative correlation (ρ = -0.183, -0.151, and -0.185, respectively). Cronbach's α was 0.909. Cohen's effect size and standardized response mean were 0.819 and 0.721. The recruitment and completion rate were 92.9% and 100%, respectively. We based the above calculations on the results obtained on the first day following surgery.
CONCLUSIONS: The validity and reliability of the QoR-15K are comparable to those of the English version. The QoR-15K would be a good instrument to assess the quality of recovery in Korean patients after surgery.

Introduction

Postoperative recovery is a complex process affected by various factors, such as patients, surgical methods, and anesthetic characteristics. Such factors may be accompanied by many adverse sequelae. Previous studies evaluating recovery following anesthesia have primarily assessed morbidity, mortality, incidence of anesthesia-related adverse outcomes, and changes in vital signs [1-4]. These parameters are important, but most have neglected the quality of the patients’ recovery. Thus, various patient-reported outcome measurement scales and tools have been developed [1,3-8]. As anesthetic and surgical techniques have become safer, the focus has begun to shift to patients’ well-being, overall quality of life, and the quality of recovery following anesthesia [9]. One of the most widely used questionnaires is the quality of recovery-40 questionnaire (QoR-40) developed by Myles et al. [6] in 2000. The QoR-40, which is comprised of 40 items of 5 subscales, has been translated into many languages, including Japanese and Korean [10-12].

Because QoR-40 is considered to be a bit too lengthy for clinical use, a shorter form, called the QoR-15, was created by Stark et al. [4] in 2013. The QoR-15 is a self-rated 15-item questionnaire derived from the QoR-40 that is intended to evaluate the early quality of recovery and the emotional health status of patients following surgery [4,6]. This single-paged QoR-15 has been shown to be highly valid and reliable in patients who have undergone general surgery [4]. The QoR-15 has been validated and translated into many languages, including Danish, Portuguese, Chinese, Swedish, and IsiZulu [13-17]. All translated versions of QoR-15 had sufficient validity and reliability for evaluating the quality of recovery. The QoR-15 was also evaluated following a day of orthopedic surgery [18]. In a systemic review, the QoR-15 was sufficiently valid and reliable in all tested languages [19].

However, an official Korean version of the QoR-15 has not yet been developed or validated, although the Korean QoR-40 has been established. The aim of this study was to develop a Korean version of the QoR-15 (QoR-15K) and to evaluate its validity, reliability, and responsiveness for Korean patients who receive general anesthesia. The authors hypothesized that the QoR-15K would have a similar validity, reliability, and responsiveness as the original English version. This could mean that the quality of healthcare could be promoted by easily assessing the quality of recovery for Korean patients.

Materials and Methods

Patients

The Institutional Review Board of our hospital approved the clinical protocol for this study. Written informed consent was obtained from all participating patients (IRB No 2017-05-024-003). From January 2018 to August 2018, we enrolled 226 patients who had been admitted for scheduled elective surgery under general anesthesia. All procedures involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments.

The sample size was determined from the guidance of a study validating a questionnaire. This is set at 10–20 times the number of items on the questionnaire [14]. Because the QoR-15K has 15 items, we multiplied it by 15, and we obtained 225 as the sample size.

Patients were enrolled if they were able to read and write the Korean language and had been admitted to the hospital for at least one preoperative night and three days following surgery. Those with cognitive impairment, less than 18 years or greater than 80 years of age, or American Society of Anesthesiologists (ASA) physical status IV or above were excluded from the study. Patients with a history of alcohol or any other substance abuse and those who refused to participate in this study were also excluded. Those who had major postoperative complications were also eliminated. Age, gender, educational and marital status, height, and weight were recorded on the day before surgery. The duration of anesthesia and surgery, as well as the length of the post-anesthesia care unit (PACU) and hospital stay, were collected prospectively following discharge. The demographic data are presented in Table 1.

Table 1.

Demographic Data of the Patients (n = 210)

	Value
Age (yr)	47.4 ± 11.9
Gender
M	88 (41.9)
F	122 (58.1)
ASA PS (I/II/III)	139/65/6
Weight (kg)	65.9 ± 13.3
Height (cm)	162.7 ± 8.4
BMI (kg/m2)	24.8 ± 3.8
Education
High school	23 (11.0)
College	187 (89.0)
Marital status
Married	182 (86.7)
Single	28 (13.3)
Type or Surgery
General	97 (46.2)
Gynecologic	46 (21.9)
Orthopedic	37 (17.6)
Otorhinolaryngologic	25 (11.9)
Others	5 (2.4)
Duration of anesthesia (min)	124.4 ± 54.1
Duration of surgery (min)	87.4 ± 50.5
PACU stay (min)	67.1 ± 22.0
Duration of admission (day)	7.0 ± 3.5

Values are presented as mean ± SD or number of patients (%). ASA PS: American Society of Anesthesiologists physical status, BMI: body mass index, PACU: post-anesthesia care unit.

Development of QoR-15K

This study process was preauthorized by the original author of the English QoR-15. The QoR-15 is composed of 15 items in 5 subscales: physical comfort (n = 5, question number 1-4, 13), emotional status (n = 4, question number 9, 10), psychological support (n = 2, question number 6, 7), physical independence (n = 2, question number 5, 8), and pain (n = 2, question number 11, 12). In our previous study, we translated QoR-40 into the Korean language and developed the Korean version of QoR-40 (QoR-40K) via a translation procedure based on Beaton’s and Bullinger’s recommendations [11,20]. In short, two bilingual translators translated the QoR-40 into the Korean language. After reaching consensus, based on the consistency and adequacy of the meaning, backward translation into English was performed by a native English-speaking medical doctor and a linguistic specialist. An expert committee, consisting of a psychologist, a general surgeon, an anesthesiologist, and translators agreed upon the final Korean version of QoR-40. Based on translated and validated QoR-40K, the 15 items from 5 domains established by Stark et al. were selected for the development of QoR-15K [4].

The QoR-15K items are scored on an 11-point Likert scale ranging from 0 to 10. Depending on the questions, the best answers can be 10 or 0. For the positive questions, the best answers were scored 10, but they were scored 0 for the negative items. The global QoR-15K score has been derived from the summation of all items that range from 0 to 150. Higher scores indicate a better quality of recovery. The QoR-15K is presented in Appendix 1.

Investigation

Previously, the QoR-40K was evaluated in 200 selected Korean patients who had undergone surgery under general anesthesia in 2017. The results showed that all items of the QoR-40K had acceptable validity, reliability, and feasibility for assessing the quality of recovery [11]. These results were used as a pilot study, and the QoR-15K, consisting of 15 questions selected from QoR-40K, was completed.

The QoR-15K was evaluated a day before surgery and on the first and second day, following surgery by one of the authors of this study. Patients were asked to complete the QoR-15K and a 100-mm visual analog scale for recovery (VAS). VAS ranges from 0 to 100 mm, which indicates poor recovery to excellent recovery. Because there is no gold standard of measurement for postoperative recovery, the VAS was chosen for measuring criteria validity and was used as a standard control. Demographic data such as age, sex, height, weight, education level, marital status, and underlying medical condition were collected preoperatively. After the surgery, we recorded the duration of anesthesia and operation time, as well as the length of the PACU stays. We also recorded the grade classification of the procedure following surgery. All the surgeries in this study were classified as minor, intermediate, or major by the grade of surgery according to the nature of the surgical procedure, the expected duration, and the expected degree of inflammatory response using the surgical outcome risk tool (SORT) [21]. When the patient was discharged from the hospital, we noted the number of hospital days. On the first and the second postoperative days, patients were asked to complete the 100-mm VAS for recovery and the QoR-15K.

Psychometric evaluation of the QoR-15K

Validity was assessed for the accuracy of the QoR-15K. For convergent validity, we compared QoR-15K and VAS scores for recovery. Interdimensional correlations for QoR-15K were measured. We measured the associations between the QoR-15K with age, sex, duration of anesthesia, duration of stay in the PACU, and hospital stay to ascertain the construct validity of our hypothesis that they would have a negative correlation. We also measured the global QoR-15K score according to the grade to surgery. Discriminant validity showed that patients with complications and poor recovery would have a lower QoR-15K score.

Reliability indicates the consistency of QoR-15K. Reliability was assessed by internal consistency. For internal consistency, each item of QoR-15K and its own dimensions were measured by Cronbach’s α.

The clinical feasibility of QoR-15K was evaluated in terms of completion and recruitment rate. The time to complete the questionnaire was measured for some randomly selected patients. Responsiveness describes the ability to detect clinically important changes. We measured responsiveness with standardized response mean (SRM) and Cohen’s effect size. The SRM shows how the mean of the QoR-15K scores changed in terms of the change in the QoR-15K scores’ standard deviation (SD). Cohen’s effect size is the average score change divided by the SD of the pretest score. The results were calculated based on the results of the first day following surgery.

Statistical analysis

Data are presented as mean ± SD and number (%). Associations were measured using Spearman’s correlation coefficient (ρ). All statistical analyses were performed using SPSS Statistics for Windows, version 25 (IBM Corp., USA). Values were considered statistically significant when P < 0.05.

Results

Among the 226 enrolled patients, one patient canceled operation, another patient underwent regional anesthesia, and 14 patients were discharged early within two days following surgery. Finally, 210 patients were included and completed the questionnaire. Surgery and anesthesia were conducted at a single medical center. Most of the surgeries were general, gynecologic, orthopedic, and otorhinolayngologic surgeries. There was no patient who refused the questionnaire. Thus, the recruitment rate was 93%, and the completion rate among the recruited patients was 100%. The patients who fully completed the questionnaires as above were statistically analyzed for validity, reliability, and responsiveness. On the first day following surgery, we randomly selected 30 patients to measure the time it took them to complete QoR-15K questionnaire. Most patients were able to complete the questionnaire without any difficulty in less than 3 minutes.

The postoperative day-one QoR-15K score was decreased to 120.08 ± 24.74 as compared to the preoperative QoR-15K score of 136.35 ± 13.32 (P < 0.001). The QoR-15K score two days following surgery was increased to 121.80 ± 23.58 (P < 0.001). The baseline and postoperative scores showed significant differences. Thus, these results indicate excellent responsiveness. Changes in the preoperative and postoperative QoR-15K and responsiveness are presented in Table 2. Cohen’s effect size and SRM between preoperative and postoperative QoR-15K was 0.82 and 0.71, where 0.2 indicated a small, 0.5 a moderate, and 0.8 or more indicated a large effect of the intervention for both the Cohen’s effect size and the SRM [22]. The results described above were calculated based upon the results on the first day following surgery.

Table 2.

Change in QoR-15K of Patients the Day before Surgery (Preoperative), the First and Second Day Following Surgery (Postoperative)

Score	Max score	Preoperative	Postoperative day 1	Postoperative day 2	% Change from baseline	P value	Cohen’s Effect size	SRM
Global QoR-15K	150	136.4 ± 13.3	120.1 ± 24.7	121.8 ± 23.6	11.9	< 0.001	0.82	0.723
Physical comfort	50	46.3 ± 5.4	41.1 ± 8.9	41.5 ± 8.7	11.2	< 0.001	0.70	0.629
Emotional state	40	34.1 ± 6.4	32.8 ± 7.7	33.4 ± 7.2	3.8	0.012	0.19	0.170
Psychological support	20	19.0 ± 1.7	17.8 ± 3.3	17.9 ± 3.1	6.5	< 0.001	0.47	0.377
Physical independence	20	18.5 ± 2.7	14.7 ± 5.2	15.1 ± 5.1	20.3	< 0.001	0.91	0.734
Pain	20	18.5 ± 2.6	13.7 ± 4.8	14.0 ± 4.6	25.9	< 0.001	1.24	0.937

Values are presented as mean ± SD. SRM, Cohen’s effect size, and percent change from baseline were calculated, based on the postoperative QoR-15K score done on the first day following surgery. QoR-15K: Korean version of quality of recovery-15 questionnaire, SRM: standardized response mean (mean change/SD).

To assess convergent validity, we evaluated the correlation between the QoR-15K and the VAS for recovery. The spearman’s ρ was 0.882 on the first day following surgery (P < 0.001), showing excellent validity. Each domain of QoR-15K also correlated well with VAS, from 0.65 to 0.75 (P < 0.001). The correlations of VAS and each subclass of QoR-15K are described in Table 3.

Table 3.

Correlation between the VAS and the QoR-15K

Postoperative day 1			Postoperative day 2
Scores	ρ	P value	Scores	ρ	P value
QoR-15K	0.882	< 0.001	QoR-15K	0.824	< 0.001
Physical comfort	0.685	< 0.001	Physical Comfort	0.731	< 0.001
Emotional state	0.746	< 0.001	Emotional State	0.607	< 0.001
Psychological support	0.743	< 0.001	Psychological support	0.628	< 0.001
Physical independence	0.653	< 0.001	Physical Independence	0.593	0.001
Pain	0.659	< 0.001	Pain	0.482	0.007

Correlations are measured with Spearmann's correlation coefficient (ρ). QoR-15K: Korean version of quality of recovery-15 questionnaire, VAS: visual analog scale for recovery.

We evaluated the correlations of construct validity between the QoR-15K and clinical characteristics such as duration of anesthesia, duration in the PACU and hospital stays, etc. There were significant negative correlations between the global QoR-15K and the duration of anesthesia, duration of surgery, duration in the PACU, and hospital stays on the first and second postoperative days. For the first day, those spearman’s correlation coefficients were -0.183, -0.177, -0.151 and -0.185, respectively (P < 0.01). The BMI and weight were also weakly correlated with the QoR-15K on the first postoperative day (P < 0.05).

Reliability was evaluated by Cronbach α for internal consistency. The postoperative Cronbach α was high enough to reach 0.909 on the first operative day and 0.905 on the second operative day, where the recommended reliable value for these tests is more than 0.7 [23]. The median item-to-own dimension, the Cronbach α and the Spearmann’s correlation coefficient (ρ) for each dimension were physical comfort (α = 0.765, ρ = 0.878), emotional state (α = 0.816, ρ = 0.895), psychological support (α = 0.613, ρ = 0.605), physical independence (α = 0.679, ρ = 0.824) and pain (α = 0.830, ρ = 0.770). The interdimensional correlation of each dimension is presented in Tables 4 and 5.

Table 4.

Interdimensional Correlation for the QoR-15K

	Global QoR-15K	Physical comfort	Emotional state	Psychological support	Physical independence	Pain
Physical comfort	0.878	-
Emotional state	0.895	0.725	-
Psychological support	0.605	0.541	0.518	-
Physical independence	0.824	0.673	0.668	0.525	-
Pain	0.770	0.547	0.646	0.363	0.540	-

Interdimentional correlation was measured by Spearmann’s correlation coefficient (ρ). QoR-15K: Korean version of quality of recovery-15 questionnaire. Interdimensional Cronbach α = 0.909.

Table 5.

Cronbach α for Preoperative and Postoperative QoR-15K

Dimension	Preoperative	Postoperative day 1	Postoperative day 2
Global QoR-15K	0.847	0.909	0.905
Physical comfort	0.807	0.765	0.766
Emotional state	0.819	0.816	0.795
Psychological support	0.363	0.613	0.643
Physical independence	0.734	0.679	0.675
Pain	0.709	0.830	0.802

Reliability was measured by Cronbach α. QoR-15K: Korean version of quality of recovery-15 questionnaire.

When we measured the QoR-15 scores by sex, there was no difference between men’s and women’s scores (134.4 ± 13.9 vs. 136.6 ± 13.3, P = 0.457) before surgery. But a negative correlation was found that female patient scores were a bit lower than male patients’ scores postoperatively (124.1 ± 18.6 vs. 117.2 ± 28.1, P = 0.033). This may come from emotional state factors (34.28 ± 6.1 vs. 31.8 ± 8.5, P = 0.013). For the comparisons of ASA physical status, the QoR-15K of ASA III was significantly lower than that of ASA I or II (ASA I: 137.4 ± 13.3 vs. ASA II: 135.1 ± 12.6 vs. ASA III: 126.3 ± 17.8, P = 0.043). No difference was found between education levels (college 120.5 ± 25.0 vs. high school 116.3 ± 22.8, P = 0.420). Patients having major surgery had significantly lower QoR-15K scores than did patients having minor or intermediate surgery (minor 121.2 ± 22.6 vs. intermediate 121.5 ± 27.6 vs. major 114.0 ± 19.8, P = 0.013). The comparisons of the QoR-15K and the grade of surgery are presented in Table 6. Floor or Ceiling effects are generally considered to be present if more than 15% of the subjects had achieved the lowest or the highest possible score, but the effects were not observed in this study [24].

Table 6.

The Comparisons of QoR-15K and the Grade of Surgery

Grade of surgery	Numbers	Preoperative	Postoperative day 1	Postoperative day 2
Minor[*]	74	138.0 ± 11.1	121.2 ± 22.6	122.9 ± 21.0
Intermediate[†]	99	138.2 ± 13.7	121.5 ± 27.6	122.9 ± 26.7
Major[‡]	37	128.0 ± 13.6	114.1 ± 19.8	116.5 ± 19.0

Values are presented as mean ± SD or number of patients. QoR-15K: Korean version of quality of recovery-15 questionnaire.

*Minor surgery: endoscopic sinus surgery, tonsillectomy, septorhinoplasty, parotidectomy, tympanoplasty, minor mass excision, therapeutic hysteroscopic procedure, incontinence surgery, hand surgery, thyroidectomy, hardware removal, or biopsy.

†Intermediate surgery: laparoscopic abdominal surgery, laparoscopic cholecystectomy, wide mass excision, laparoscopic herniorrhaphy, pelviscopic surgery, surgery of the elbow, shoulder, or knee surgery.

‡Major surgery: open or laparoscopic gastrectomy, colorectal surgery, liver surgery, open hysterectomy, spine surgery, total hip replacement.

Discussion

The overall result of this study indicates that QoR-15K would be a valid, reliable, and easy-to-use tool for evaluating the quality of postoperative recovery following general anesthesia in the Korean population. The QoR-15K preserves the acceptability of the original English QoR-15 and is as suitable for evaluating the quality of recovery following general anesthesia for Korean patients, as are other translations [4,13-18].

Unlike the QoR-40K, the Qor-15K is a single-page form with an 11-point Likert scale. Most patients can complete the questionnaire without any problem in fewer than 3 minutes. This indicates that QoR-15K is acceptably feasible to use, although measuring time to complete the questionnaire was not recorded for all patients. This scale can be a useful tool in busy clinical circumstances. The fact that there was no floor or ceiling effect also indicates that its feasibility for use is fairly good. A high completion rate and recruitment rate also explain its suitability because nonresponse reflects poor recovery or low score QoR-15K [19].

Although there is no gold standard for quality of recovery, we compared the QoR-15K with the VAS for recovery to evaluate its convergent validity. Convergent validity showed a correlation coefficient between the QoR-15 score and VAS exceeds the published recommendation (correlation > 0.60), which was similar to the coefficient in the original study [4,23]. Without psychometric assessment, the use of a VAS as a criterion may result in a defective scale because it is an imperfect scale that overlooks the individual components of recovery and is prone to over-rating. As there is no absolute criterion for evaluating the quality of recovery, VAS was used as an alternative for assessing recovery as it had been in the original study [4,11]. The QoR-40 cannot be used as a standard since the QoR-15 is a short form of the QoR-40, which shares all the items of the QoR-15. In our previous study, the QoR-40K scores correlated with the VAS for pain, but we utilized VAS for recovery, as did the original QoR-15 study [4,11]. The relationship between the VAS for recovery and the QoR-15K has well been correlated, indicating that QoR-15K has a good convergent validity in this study.

For construct validity, the QoR-15K was negatively correlated with duration of anesthesia, surgery time, duration in PACU, and hospital stay. Spearman correlation coefficients of surgery time, duration in PACU and hospital stay are ρ = -0.177 (P = 0.007), ρ = -0.151 (P = 0.020) and ρ = -0.185 (P = 0.007). Those items were as significantly correlated as were those in our previous QoR-40K study [11].

Validity was also determined by comparing patients who had minor, intermediate, and major surgery. Although there was no difference between minor and intermediate surgery, there was a significant decreased QoR-15K score in major surgery as compared to minor or intermediate surgery [19]. Those who had a more aggressive surgery would present with a lower QoR-15K score. Moreover, patients who are a higher ASA physical status had lower QoR-15K scores. Women had lower QoR-15K scores, with a weak correlation (P = 0.033). These results had their origin from their physical comfort and emotional state. It is quite understandable that women would tend to be more susceptible to situations that evoked emotional stress [11]. There was no relation between QoR-15K with age, education, or marital status. These results correlate well with previous studies involving the original English QoR-15 [13].

The QoR-15K was found to have excellent reliability, as did all versions of QoR-15. The Cronbach α was 0.90, which exceeded the recommended criterion, above 0.7 [23]. Internal consistency was measured using a median correlation between items within each dimension, and it was established by an interdimensional correlation (ρ = 0.605–0.895, P < 0.001). These results were enough to conclude that the QoR-15K possesses adequate reliability.

The responsiveness of the QoR-15K was assessed using Cohen’s effect size and the SRM. Both of them are expressed in standardized units, assuming that 0.2 is considered to be small and 0.8 or greater is considered to be large [25]. The SRM of the overall QoR-15K was 0.72, which indicates a moderate ability to detect change. As the previous studies mentioned, East Asians may recover earlier from an emotional state than those of other cultural backgrounds, or different testing time may result in a relatively lower SRM. This result was consistent with our previous study of the QoR-40K [11]. Cohen’s effect size for global QoR-15K was 0.82, which exceeds the level for a large effect, 0.8 [25]. These results demonstrate that the QoR-15K would have acceptable responsiveness. Thus, these values can be considered to be a suitable outcome for clinical trials. Responsiveness is known to be the most important psychometric index for evaluative instruments to measure changes in health outcomes [26].

This study had several limitations. First, test-retest reliability was not conducted in this study, which can be a drawback. However, the lapse between assessments for test-retest was not clearly known. In our study, the results of the first day and the second day following surgery are similar, and the result of the second day does not reach the baseline. This means that the patients had not completely recovered during the study period. The authors mainly focused on acute recovery, which is considered to be up to two days after surgery. Moreover, we had conducted the test-retest assessment in our previous QoR-40K validation [11]. To find out what the recovery period may be, investigators should consider the timing of the postoperative QoR-15K assessment should be. Second, this study was conducted in a single hospital center. Thus, the result of our study may be restricted when applying them to all populations in various settings. Despite the limitation, this study shows validation of the Korean version of QoR-15, a shorter form of QoR-40K. Further studies would facilitate the use of the QoR-15K, which is a cultural adaptation of the translated QoR-15.

In conclusion, QoR-15K has acceptable validity, reliability, feasibility, and responsiveness for assessing the quality of recovery following general anesthesia. The QoR-15K is a single-paged, easy-to-use tool for assessing the quality of recovery, as is the original English QoR-15. The QoR-15K would be a good instrument to assess the quality of recovery in Korean patients after surgery.