Transcultural Adaptation and Validation of Quality of Dying and Death Questionnaire in Medical Intensive Care Units in South Korea.

BACKGROUND: Providing palliative care to dying patients in the intensive care unit (ICU) has recently received much attention. Evaluating the quality of dying and death (QODD) is important for appropriate comfort care in the ICU. This study aimed to validate the Korean version of the QODD questionnaire.
METHODS: This study included decedents in the ICUs of three tertiary teaching hospitals and one secondary hospital from June 2016 to May 2017. ICU staff members were asked to complete the translated QODD questionnaire and the visual analogue scale (VAS) questionnaire within 48 hours of patient death. The validation process consisted of evaluating construct validity, internal consistency, and interrater reliability.
RESULTS: We obtained 416 completed questionnaires describing 255 decedents. The QODD score was positively correlated with the 100-VAS score (Pearson correlation coefficient, 0.348; P<0.001). An evaluation of the internal consistency presented favorable results (calculated Cronbach's alpha if a given item exceeded 0.8 in all items). The interrater reliability revealed no concordance between doctors and nurses.
CONCLUSIONS: The QODD questionnaire was successfully translated and validated in Korean medical ICUs. We hope further studies that use this valuable instrument will be conducted in Korea.

Introduction

Since many patients admitted to intensive care units (ICUs) worldwide are critically ill, these units have a high mortality rate; thus, the burden of critical care is a global problem [1,2]. As a result, much attention has recently been paid to providing comfort care to dying patients in the ICU. Pain and symptom management, death preparation, achieving a sense of completion, and making decisions about treatment preferences are important factors in comfort care [3]. However, these points have not been identified as important issues in actual practice [4,5].

Providing comfort care is one of the major goals in palliative settings [6]. Maintaining an acceptable quality of life is important for dying patients in this regard. The quality of dying and death (QODD) is defined as the degree to which a person’s preferences for dying and the moment of death agree with observations of how the person actually died. Patrick et al. [7] proposed a model for evaluating the QODD that consists of 31 aspects that can be rated by clinicians after a patient dies. The QODD score, an average of the ratings of each completed item, represents the degree of a patient’s QODD as perceived by raters. Since the original QODD model was proposed, various versions of similar questionnaires intended for different raters (e.g., family members, medical staffers) and in different healthcare settings (e.g., outpatient, hospice, ICU) have been developed [8-10].

Many studies have been conducted in ICU settings using instruments like the QODD model, particularly in the United States and Europe. Currently, there is no validated questionnaire in South Korea to evaluate the quality of dying ICU patients. In this study, we translated the QODD questionnaire version 3.2 into Korean to validate the Korean version of the QODD questionnaire.

MATERIALS AND METHODS

Study Design and Data Collection

This study was conducted prospectively in the medical ICUs at three tertiary teaching hospitals and one secondary hospital from January 2015 to February 2016. Eligible patients were those who died in a medical ICU. Attending doctors (resident, fellow, or attending physician) and nurses caring for these patients at the time of death were asked to complete the translated QODD questionnaire within 48 hours of the death. The raters were permitted to complete the questionnaires only a limited number of times per month (a maximum of three times per month for a doctor and two times per month for a nurse).

All completed questionnaires were collected in a designated storage box in the ICU. The raters were also asked to complete a brief questionnaire about their own information (e.g., age, sex, religion, education, length of experience). The decedents’ demographic characteristics (e.g., sex, age, previous medical history) and clinical information (e.g., reason for ICU admission, severity, hospital day, length of ICU stay, life support equipment, resuscitations within 24 hours) were obtained from their medical records.

QODD Questionnaire

We translated version 3.2 of the QODD questionnaire, which is designed for self-administration by a healthcare professional who cared for a patient who later died in a hospital or ICU setting [11]. We obtained permission to use the questionnaire from one of the developers prior to commencement of the study. Two experts translated the original questionnaire into Korean individually and then integrated their individual translations into a single translation. This version was retranslated into English by two additional experts, and an English language expert compared that version with the original. The final Korean version of the QODD questionnaire is included in the Supplementary Material.

The QODD questionnaire has four subcategories with items to be rated: patient experiences at the end of life (15 items), medical care at the end of life (four items), experience at the moment of death (three items), and overall ratings of care (two items). For each item, all raters were asked to evaluate the quality of each experience on a scale of 0 (“terrible”) to 10 (“almost perfect”). Twenty designated items are used for calculating the QODD score, which involves summing valid ratings, dividing by the number of valid items and finally multiplying the result by 10. The QODD score ranges from 0 to 100, with a higher score indicating a better quality of death for the patient [11].

Visual Analogue Scale Questionnaire

The raters were also asked to complete a visual analogue scale (VAS) questionnaire to validate the QODD questionnaire. The VAS questionnaire consisted of two items for rating: “effect on the patient’s quality of death by medical treatment and the patient’s experience in the ICU” and “overall quality of ICU care by medical professionals.” Each item was rated on a scale from 1 (excellent) to 10 (inadequate). The mean value of valid items multiplied by 10 was designated as the “VAS score.”

Validation and Statistical Analysis

All data were presented as median (range) or median (interquartile range) for continuous variables and number (percent) for categorical variables. We used IBM SPSS version 22.0 (IBM Corp., Armonk, NY, USA) for the statistical analyses. The validation process consisted of construct validity, internal consistency, and interrater reliability. Construct validity is defined as the extent to which a test assesses the concept intended to be measured [12]. To assess construct validity, we calculated the Pearson correlation coefficient between the QODD score and the 100-VAS score. Internal consistency estimates were related to item homogeneity or the degree to which the items on a test jointly measure the same construct [13]. We assessed internal consistency by calculating the Cronbach’s alpha if a given item was deleted. A Cronbach’s alpha that exceeded 0.8 indicated good internal consistency. The interrater reliability is the extent of agreement among raters. The interrater reliability between doctors and nurses was assessed by calculating intraclass correlation coefficients (ICCs). ICCs that exceed 0.75 indicate an excellent agreement, while those below 0.40 suggest poor agreement. A P-value <0.05 was considered statistically significant.

Research Ethics

This study was approved by the Institutional Review Board of each hospital. Written informed consent was obtained from medical staff at the time of each patient’s death. The requirement for written consent from the patients was waived because their personal information was not used. We conducted this study in compliance with the Declaration of Helsinki.

RESULTS

Baseline Characteristics of Patients and Medical Staff Members

Of the 255 total patients with a mean age of 66.6 years, 39.6% had an underlying malignancy at the time of ICU admission (Table 1). Respiratory failure was the leading cause of ICU admission, and 20.0% of patients were admitted for post-resuscitation care. Forty-eight (18.8%) had received cardiopulmonary resuscitation within 24 hours of death. Documented do-not-resuscitate orders were in place for 206 patients (80.8%).

Table 1.

Baseline characteristics of patients died in ICUs

Variable	Total patients (n = 255)
Male sex	159 (62.4)
Age (yr)	69 (18–94)
Malignancy	101 (39.6)
Reason for ICU admission
Sepsis	59 (23.1)
Respiratory failure	148 (58.0)
Heart failure	22 (8.6)
Renal failure	35 (13.7)
Hemorrhagic shock	7 (2.7)
Post-resuscitation care	51 (20.0)
APACHE II score	27 (0–71)
SOFA score	12 (1–24)
Hospital day	13 (0–196)
Length of ICU stay (day)	4 (0–117)
Life support equipment
Mechanical ventilator	228 (89.4)
Continuous renal replacement treatment	124 (48.6)
Extracorporeal membrane oxygenation	17 (6.7)
Others	9 (3.5)
CPR within 24 hours prior to death	48 (18.8)
DNR documentation prior to death	206 (80.8)

Values are presented as number (%) or median (range).

ICU: intensive care unit; APACHE: Acute Physiology and Chronic Health Evaluation; SOFA: Sequential Organ Failure Assessment; CPR: cardiopulmonary resuscitation; DNR: do-not-resuscitate.

In total, 177 medical staff members completed 416 questionnaires during the 12-month study period, including 66 doctors: 52 residents (78.8%), nine fellows (13.6%), and five attending physicians (7.6%). The characteristics of the respondent ICU medical staff members are detailed in Table 2.

Table 2.

Baseline characteristics of medical staffs

Variable	Doctor (n = 66)[a]	Nurse (n = 111)
Age (yr)	31 (25–47)	27 (23–42)
Sex
Male	33 (50)	11 (9.9)
Female	33 (50)	100 (90.1)
Religion
Catholicism	15 (22.7)	14 (12.6)
Christianity	15 (22.7)	26 (23.4)
Buddhism	5 (7.6)	10 (9.0)
No religion	31 (47.0)	61 (55.0)
Experience (mo)
Medical personnel	42 (24–252)	24 (0–216)
ICU staff	12 (0–120)	24 (0–180)
Experience of EOL care education
Yes	46 (69.7)	27 (24.3)
No	20 (30.3)	84 (75.7)
No. of completed questionnaires	213	203

Values are presented as median (range) or number (%).

ICU: intensive care unit; EOL: end of life.

Sixty-six doctors were participated: resident (n = 52, 78.8%), fellow (n = 9, 13.6%), and attending physician (n = 5, 7.6%).

Validation Process

The results of the assessment of construct validity are presented in Table 3. The QODD score and subcategorical items were positively correlated with the 100-VAS score (Pearson correlation coefficient, 0.348; P<0.001).

Table 3.

Construct validity

Variable	Pearson correlation coefficient	P-value
QODD score	0.348	<0.001
Patient’s experience at the end of life	0.261	<0.001
Medical care at the end of life	0.332	<0.001
Experience at moment of death	0.234	<0.001
Overall ratings of care	0.346	<0.001

QODD: quality of dying and death.

Table 4 contains the results of the assessment of internal consistency. The calculated Cronbach’s alpha exceeded 0.8 for all items, which showed good internal consistency and item heterogeneity.

Table 4.

Internal consistency

Item	Rating	Cronbach’s α[a]
Patient’s experience at the end of life
Having pain under control	5 (3–7)	0.834
Having control over what is going on around you	2 (1–3)	0.835
Being able to feed oneself	0 (0–2)	0.837
Being able to breath comfortably	2 (1–5)	0.836
Feeling at peace with dying	2 (1–4)	0.834
Feeling unafraid of dying	3 (1–5)	0.835
Being able to laugh and smile	0 (0–2)	0.837
Keeping one’s dignity and self-respect	2 (0–3)	0.836
Spending time with family and friends	3 (1–7)	0.835
Spending time alone	2 (0–5)	0.837
Being touched or hugged by loved ones	3 (0–6)	0.835
Saying goodbye to loved ones	1 (0–3)	0.834
Clearing up bad feelings	1 (0–3)	0.837
Having one or more visits from a religious or spiritual advisor	1 (0–4)	0.836
Having a spiritual service or ceremony before death	1 (0–3)	0.836
Medical care at the end of life
Experience of receiving mechanical ventilation	3 (2–5)	0.834
Experience of receiving dialysis	4 (2–7)	0.837
Receiving right amount of sedation	6 (3–8)	0.834
Having discussed end-of-life care wishes with your doctor	2 (0–5)	0.836
Experience at moment of death
Having desired people present at the time of one’s death	7 (3–8)	0.835
State at moment of death (i.e., asleep, awake, unconscious)	4 (2–7)	0.833
Overall quality of one’s dying	3 (2–5)	0.834
Overall ratings of care
By all medical personnel	7 (5–8)	0.837
By assigned doctor	6 (5–8)	0.836

Values are presented as median (interquartile range).

Calculated Cronbach’s alpha if a given item deleted.

Three hundred six questionnaires that were completed by both doctors and nurses and described 153 patients were used for assessing interrater reliability. the ICC for the QODD score was 0.077, and there was no statistical significance (P = 0.312) (Table 5).

Table 5.

Interrater reliability

Variable	Intraclass correlation coefficient	P-value
QODD score	0.077	0.312

QODD: quality of dying and death.

DISCUSSION

We conducted this study because an instrument that can objectively evaluate the quality of life of dying patients should be available for use in Korean ICUs. An assessment of construct validity demonstrated a positive correlation between the 100-VAS score and the QODD score. The calculated scores for the subcategories also were generally correlated with the 100-VAS score. In the evaluation of internal consistency, all items presented high Cronbach’s alpha (>0.8) values. These data suggest that QODD scores calculated from the translated questionnaire could provide a useful and objective evaluation of the quality of life of dying patients, as perceived by ICU staff members in Korea.

However, in terms of construct validity, there were relatively low correlations between the QODD and 100-VAS scores despite the extremely low P-value (Table 3). This finding may be explained by the number of extreme scores. For example, 50.8% of the raters scored 0 for the item “Being able to feed oneself” (47.0% in “Having a spiritual service or ceremony before death” and 42.6% in “Having one or more visits from a religious or spiritual advisor”). Many raters might not have fully understood what was being asked or lacked clarity regarding several questions of the QODD questionnaire. A revised version of the QODD questionnaire that presents the actual status of Korean ICUs should be discussed in further studies.

In the present study, a considerable number of medical staff members had no experience with comfort care education (Table 2). Currently, comfort care education has been not incorporated into standard medical school curricula in Korea. Still, the ICU is a space where medical personnel are likely to encounter dying patients, and most ICU deaths are preceded by a decision to withhold or withdraw life-sustaining care [14,15]. Hence, comfort care education is important for medical professionals who provide comfort care. Data from a U.S. national survey showed that medical students’ sense of preparedness for comfort care increases with more coursework and bedside teaching [16]. Extended opportunities for comfort care education in formal medical curricula should therefore be provided.

Analysis of interrater reliability showed no concordance in QODD score between doctors and nurses (Table 5). We could not determine which factors were responsible for this finding. However, we suspect that actual differences between how doctors and nurses perceive the quality of life of a dying patient may be related to the amount of time spent in direct contact with the patient. Levy et al. [17] reported discordance in the QODD rating between resident physicians and nurses. A study by Breen et al. [18] also identified conflicts among staff members in 48% of cases when withdrawing life-sustaining care. In Korean medical ICUs, as is the case worldwide, nurses generally spend more time in direct contact with and closely caring for patients than do doctors. Therefore, nurses might gain a more in-depth understanding of the experiences and wishes of dying patients than doctors.

Although a good quality of death is one of the most important goals of proper comfort care, this concept is unfamiliar to Korean ICU workers. Very few studies have objectively evaluated patients’ quality of death. Many dying patients still receive meaningless treatments [19], and very limited ICU visiting policies can prohibit them from spending much time with their families. Patient autonomy is often overlooked when discussing whether life-sustaining care should be provided [20]. Thus, it can be assumed that the quality of death in Korean ICUs is relatively low. Further studies that use our validated and reliable tool can demonstrate the current status of the quality of death in Korean ICUs and offer suggestions for improving the quality of death.

This study had several limitations. We did not include patients from surgical, neurologic, or other types of ICUs where comfort care is also frequently required. Samples from just four medical ICUs of acute-care-based hospitals may not be enough to validate the assessment tool for all cases. An extended validation study in various ICU settings should be conducted in the future to confirm generalizability.

In conclusion, a translated QODD questionnaire version 3.2 was successfully validated in Korean medical ICUs. We hope that further studies conducted in Korea will include this useful instrument.