Protocol for the development and multisite validation of the Quality of Dying and Death-Revised Global Version scale.

INTRODUCTION: Evaluating the quality of dying and death is essential to ensure high-quality end-of-life care. The Quality of Dying and Death (QODD) scale is the best-validated measure of the construct, but many items are not relevant to participants, particularly in low-resource settings. The aim of this multisite cross-sectional study is to develop and validate the QODD-Revised Global Version (QODD-RGV), to enhance ease of completion and relevance in higher-resource and lower-resource settings. METHODS AND ANALYSIS: This study will be a two-arm, multisite evaluation of the cultural relevance, reliability and validity of the QODD-RGV across four participating North American hospices and a palliative care site in Malawi, Africa. Bereaved caregivers and healthcare providers of patients who died at a participating North American hospice and bereaved caregivers of patients who died of cancer at the Malawian palliative care site will complete the QODD-RGV and validation measures. Cognitive interviews with subsets of North American and Malawian caregivers will assess the perceived relevance of the scale items. Psychometric evaluations will include internal consistency and convergent and concurrent validity. ETHICS AND DISSEMINATION: The North American arm received approval from the University Health Network Research Ethics Board (21-5143) and the University of North Carolina Institutional Review Board (21-1172). Ethics approval for the Malawi arm is being obtained from the University of North Carolina Institutional Review Board and the Malawian National Health Science Research Committee. Study findings will be disseminated through publication in peer-reviewed journals and conference presentations. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A strength of the study is its aim to develop and evaluate a more culturally universal version of the widely used Quality of Dying and Death (QODD) scale across four North American hospice sites and at a Malawi palliative care clinic. This revised version, called the QODD-Revised Global Version (QODD-RGV), will allow for the assessment of cultural generalisability and relevance of content items and a comparison of QODD across diverse settings.

The North American study arm will compare QODD ratings of bereaved caregivers with those of hospice nurses and physicians who provided care for the deceased patients. Determining the concordance of these proxy ratings will inform the clinical application of the QODD-RGV.

A limitation of the North American study arm is that staff participants will complete the QODD-RGV by themselves, whereas caregiver participants will complete it in an interview with research staff. Variation in whom respondents may consult with prior to completing the measure could bias the scale responses.

Introduction

The quality of the dying and death experience is a critical outcome to ensure high quality of care near the end of life for people with serious, life-limiting diseases. The quality of dying and death is a multidimensional construct comprising physical, psychological, social and spiritual/existential experiences, the nature of healthcare, preparation for death and the circumstances of death; it is influenced by medical, individual, social and cultural factors.1 Positive experiences in these dimensions contribute to what has been termed a ‘good death’, which has been defined as a death ‘free from avoidable distress and suffering for patients, families and caregivers; in general accord with patients’ and families’ wishes; and reasonably consistent with clinical, cultural and ethical standards.”2 Evaluating the quality of the dying and death is essential to ensure the quality of end-of-life care.3

Palliative care aims to relieve serious health-related physical, psychological, social and spiritual suffering and improve quality of life in people with life-threatening diseases and their families, from diagnosis to the end of life.4 Global mortality rates related to non-communicable diseases continue to rise, particularly in lower-resource countries,5 where palliative care is not available to most individuals and families who are in need of such care.6 As global efforts to expand palliative care services continue, culturally appropriate and generalisable measures of the quality of the dying and death are needed to support advocacy for palliative care, to stimulate local progress in palliative care development and to compare the effectiveness of interventions across different settings.

Strengths and limitations of the Quality of Dying and Death scale

A number of measures have been developed to assess the quality of the dying and death3 7; of these measures, the multidimensional Quality of Dying and Death (QODD) scale is the best-validated and most widely used.3 7 The QODD includes 31 content items assessing different aspects of the dying and death experience and single-item ratings of overall quality of dying in the last 7 days of life and overall quality of the moment of death.8 9 Items are rated on the patient’s experience, from 0 (terrible experience) to 10 (almost perfect experience), in six conceptual domains: symptoms and personal care, time with family, whole person concerns, treatment preferences, preparation for death, and the moment of death. Because it is not feasible for most patients to complete the scale near the end of life,3 the QODD is completed retrospectively by proxy raters, such as bereaved caregivers or healthcare workers who provided care to the patient prior to death.

Despite its widespread use, the QODD has limitations that can affect its validity.10 11 The scale is complex, with at least two questions per each item content (a question about occurrence of the item content, followed by a quality rating), and is thus burdensome to complete. High missing item response rates that appear to be due to ambiguity or item non-relevance are commonly encountered in its use.10 The scale was developed in an American setting, and all validation studies of the QODD or adapted versions have been conducted in higher-resource settings, where palliative care has been more available and accessible.8 12–18

The development of validated outcome measures for use in lower-resource countries has been called for as part of advocacy and initiatives to stimulate palliative care development and expansion.19–21 The generalisability of the QODD outside of high-resource settings is questionable, although formal validity studies have not been conducted in low-resource settings. Our research with the QODD in Kenya, with proxy ratings provided by bereaved caregivers, revealed high missing response rates (ie, >10% omitted ratings) in QODD items that pertained to treatment preferences, preparation for death, end-of-life care discussion with doctors, medical prolongation of life and the moment of death and that suggested their lack of cultural relevance.22

A version of the QODD with cross-culturally generalisable items that are less burdensome to complete would be of value to support palliative care research and clinical care in higher-resource and lower-resource settings. The creation of such a measure is responsive to the call for the development of reliable and valid outcome measures that are generalisable to high-resource and low-resource settings. This measure would be of value to assess the quality of end-of-life care, to evaluate the impact of novel interventions near the end of life, to allow cross-cultural and cross-national comparisons, and to support advocacy for resource allocation for palliative and end-of-life care. Greater resource allocation is urgently needed in low-resource settings where palliative care is not available to the large majority of people in need.

Aim

The current protocol details the development and multisite psychometric evaluation of the QODD-Revised Global Version (QODD-RGV), a measure that addresses some of the limitations of the original QODD scale. This validation study comprises two arms: one conducted at four Northern American hospice sites, and the other at a palliative care clinic in Malawi, Africa. The primary study objectives are (1) to determine the content validity and cultural relevance of items in North American and Malawian care settings and (2) to evaluate the reliability and validity of the QODD-RGV with proxy ratings by both bereaved caregivers and primary healthcare providers of patients with advanced cancer in the North American hospice settings and by bereaved caregivers of patients with advanced cancer in the Malawian palliative care setting.

Methods and analysis

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Phase 1: development of the QODD-RGV scale

Scale revision

The QODD-RGV (see online supplemental appendix 1) was developed by the members of our study team, who have extensive clinical and research experience with the QODD in diverse cultural settings. This revision of the QODD is intended to improve the ease of completion by respondents and to enhance its universality. Team members reviewed the original QODD and revised or eliminated items with consistently high nonresponse rates in previous studies. Item revisions included rewording items so that they would be more generalisable across diverse settings. Some new items were added to reflect broadly relevant aspects of the QODD. The format of the scale items was also modified to reduce the length and complexity of the scale. The original questions that ask about occurrence of each item content were eliminated; only the quality ratings for items were retained. The content items were also modified to reflect the extent of the item content rather than the quality of the experience (eg, to what extent did X suffer from pain?), and the corresponding rating scale ranged from 1 (not at all) to 5 (extremely), with options to indicate ‘don’t know’ or ‘no response’. Only the single-item rating of overall dying and death experience retained its original format and 11-point quality rating scale: ‘How would you rate X’s overall experience of dying and death in the last 7 days of life?’ rated from 0 (terrible) to 10 (almost perfect). All proposed changes were discussed among team members until consensus was reached.

Phase 2: psychometric evaluation of the QODD-RGV

The recruitment and data collection periods for the North American and Malawi study arms are summarised in figure 1.

Figure 1

Flow chart of study procedures for the North American and Malawi study arms. CG, caregivers; DSM-5, Diagnostic and Statistical Manual of Mental Disorders-5; FAMCARE, family satisfaction with care; GDI, Good Death Inventory; HCP, staff (primary healthcare providers for patients); KCH, Kamuzu Central Hospital Oncology Clinic; KH, Kensington Hospice; MB, Margaret Bahen Hospice; PCL-5, Posttraumatic Stress Disorder Checklist for DSM-5; POS, APCA African Palliative Outcome Scale; QODD-RGV, Quality of Dying and Death Questionnaire-Revised Global Version; SECU, State Employees Credit Union Jim and Betsy Bryan Hospice; TRIG-II, Texas Revised Inventory of Grief-Part 2; YH, Yee Hong Hospice.

North American arm

Data collection for the North American arm of the multisite study is expected to require 1 year (projected time frame: June 2022–2023) and is to be conducted as part of a collaborative partnership between the University of North Carolina Palliative Care Programme and the Global Institute of Psychosocial, Palliative and End-of-Life Care (GIPPEC) of University of Toronto.

Participants and recruitment

The study includes three Canadian and one American participating hospice sites. In Ontario, Canada, the sites are Kensington Hospice (KH; Toronto, Ontario), Margaret Bahen Hospice (MB; Newmarket, Ontario) and Yee Hong Hospice (YH; Scarborough, Ontario). In North Carolina, USA, the site is the State Employees Credit Union Jim and Betsy Bryan Hospice Home (SECU; Chapel Hill, North Carolina).

Participants will be the bereaved primary caregivers and the primary staff healthcare providers of patients who died at a participating hospice site. Caregiver inclusion criteria include the following: 18 years of age or older; being the identified primary caregiver for the patient; the patient having died in the past four to six months; and being able to complete study questionnaires and interviews in English. During patient admission, clinic staff will provide caregivers with a study introduction letter and ask if they would be willing to be approached four to six months after the death of their loved ones. Caregivers will have the option to opt out of being contacted further. Four to six months after patient death, willing caregivers will be approached by a research team member by telephone, email or videoconference and provided with study details. Caregivers who agree to participate will provide written informed consent through email or mail.

Staff inclusion criteria include the following: 18 years of age or older, being the primary healthcare provider for a patient who died in the past week, and able to complete study questionnaires and interviews in English. A site lead at each participating hospice will provide staff members with a fact sheet detailing the study; interested staff will be asked to let the site lead know, who will then provide staff contact information to the research team. Interested staff will then be contacted by a member of the research team and provided with study details. Those who agree to participate will provide written informed consent through email.

Sample size

The study will aim to recruit the following number of participants at each North American site: at the Ontario, Canada, sites, 50 participants at KH, 50 participants at MB and 30 participants at YH; and at the North Carolina, USA, site, 100 participants at SECU (N=230). These are the maximum subsample sizes achievable per site within the 1-year data collection period, given the number of patients treated annually at each location. Using G*Power V.3.1.9.2, a power analysis to detect a convergent validity correlation (bivariate normal model) of 0.70 indicates that the total sample size of 230 will provide more than sufficient (power=1.00). Based on recruitment rates of bereaved caregivers in previous studies conducted by our team, we anticipate that approximately 50% of eligible caregivers approached will consent to participate.

Measures and procedures

In addition to the QODD-RGV, measures will assess caregiver, staff, and patient demographic information and patient medical information, palliative care services available, and standardised, validated measures of the end-of-life experience (Good Death Inventory (GDI)23), family satisfaction with care (FAMCARE)24 and caregiver bereavement (Texas Revised Inventory of Grief-Part 2)25 and distress (Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders-5).26 Details of these measures are summarised in table 1.

Table 1

Additional study measures to be administered with QODD-RGV

Measures	Variables measured	Psychometric information	Administered to
			North American caregivers	North American staff	Malawi caregivers	Malawi staff
Measures developed for study
Caregiver demographic form	Caregiver sociodemographic characteristics	--	X		X
Patient demographic form	Patient sociodemographic characteristics, diagnosis	--	X		X
Staff demographic form	Staff sociodemographic, professional characteristics	--		X
Patient chart review form	Patient hospice referral process, care services provided, date of death	--		X		X
Palliative care services	Palliative, psychosocial, supportive care services	--		X(site leads)		X
Standardised scales
Good Death Inventory (GDI),23 shortened version	18-item measure of 10 domains, 8 optional domains of end-of-life experience.	Bereaved family members of patients with cancer: internal consistency (α=0.74–0.95) and test–retest reliability (ICC=0.38–0.72); 10 domains, 8 optional domains identified; correlations with end-of-life care evaluation, care satisfaction.23	X
Texas Revised Inventory of Grief–Part 2 (TRIG-II)25	13 statements about present grief symptoms.	Internal consistency across cultures (mean α=0.90).32 Bereaved older adults: internal consistency (α=0.75–0.87), test–retest reliability (r=0.70–0.84); three factors (emotional response, thoughts, non-acceptance of loss) correlated with demographic, validation measures.33 Bereaved family members: single factor; internal consistency (α=0.90–0.95); correlations with past grief (r=0.80), anxiety (r=−0.59), depression (r=−0.51).34	X
Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5)26	20-item measure of posttraumatic stress symptoms.	College students: internal consistency (α=0.94), test–retest reliability (r=0.82); correlations with original PCL (r=0.56–0.84), other PTSD measures (r=0.84–0.85); correlations with depression (r=0.60) antisocial personality (r=0.39), mania (r=0.31).26	X
FAMCARE24	20-item measure of family satisfaction with advanced cancer care.	Family members of advanced cancer patients: internal consistency (α: scale=0.93, subscale=0.61–0.88), test–retest reliability (r=0.92); correlations with patient, family care satisfaction (r=0.77–0.80), overall care satisfaction (r=0.60–0.58), caregiver education, ethnicity, patient age.24	X		X
APCA African Palliative Outcome Scale (POS)30 31	10-item measure of physical/psychological, spiritual, practical, emotional concerns.	South African, Ugandan palliative care patients: internal consistency (α=0.60), test–retest reliability (ICC=0.78–0.89); face validity; correlations with quality of life (r=0.12–0.57).30			X

DSM-5, Diagnostic and Statistical Manual of Mental Disorders-5; FAMCARE, family satisfaction with care; ICC, intraclass correlation; PTSD, post-traumatic stress disorder; QODD-RGV, Quality of Dying and Death-Revised Global Version.

Caregiver participant procedures

At four to six months after patient death, consenting bereaved caregivers will be emailed a unique link to the study questionnaires or mailed a hard copy of the questionnaire package, along with a preaddressed, stamped return envelope. They will be asked to complete the demographic and other questionnaires (except the QODD-RGV) on their own or over the telephone with study staff. Caregiver participants will also be asked to schedule a time to complete the QODD-RGV with a member of the research team by telephone or videoconference. The entire set of questionnaires will take a total of about 30–40 min to complete, including 20 min to complete the QODD-RGV. Participants who request psychosocial support or who are identified by study staff as being significantly distressed will be referred to bereavement support services at each study site.

Cognitive interview

Across sites, the first 15–20 caregiver participants recruited for the study will be asked to participate in a cognitive interview when completing the QODD-RGV. After rating each item, interview participants will be asked to verbalise their thought processes about the item and their response to it to ascertain the extent of the item’s importance/relevance to the person’s experience of dying and death and the nature of the response. Since verbalisation of thought processes occurs after ratings are provided, quantitative data from this subgroup will be merged with the study database if no changes to the QODD-RGV are made following the interviews. The interviews will be audio-recorded and transcribed for qualitative analysis.

Staff participant procedures

One to two weeks after patient death, the hospice site lead will identify the two consenting staff participants (one physician and one nurse) who spent the most time caring for the patient and inform the research team. Each site lead will also report on the palliative care services available within their site. A team member will contact the staff participant and confirm their consent to complete a self-administered version of the QODD-RGV. The staff participant will be emailed a unique link to the self-administered staff demographic form and QODD-RGV or mailed a hard copy of the questionnaires that can be left after completion in a drop box. Staff completion of the questionnaires will take about 15–20 min.

Analysis strategy

All quantitative analyses will be conducted using SPSS V.25. Alpha will be set at 0.05 unless indicated. Each analysis will be conducted separately for each site, unless indicated.

Qualitative analysis

Cognitive interview data analysis will be managed using QSR NVivo software. Verbatim interview transcripts will be analysed using content analysis methodology.27 To ensure validity, comprehensiveness and appropriateness of analysis, data will be analysed separately by two or more members of the research team, who will code transcripts, compare codes and review category descriptions. Qualitative responses will additionally be used to refine items that demonstrate a high level of confusion or misunderstanding.

Descriptive analyses

Descriptive statistics will be calculated to summarise the QODD-RGV item ratings and total score for each site. Floor or ceiling effects (>15% indicating lowest or highest rating, respectively)28 and amount of non-responding for each QODD-RGV item will be inspected to identify problematic items that may require further refinement; these data will also be taken as indices of item relevance (eg, high frequency of nonresponses to an item may suggest that it is a poor item or that the item content is not relevant to many individuals).

Reliability

To assess internal consistency, we will calculate Cronbach’s α for the 25 QODD-RGV content items. Cronbach’s α of 0.70 is considered the minimum level for acceptable internal consistency.28 Inter-rater reliability of the caregiver and clinical staff ordinal item ratings will be evaluated using Fleiss’s kappa (>2 raters), with a kappa of 0.70 being considered the minimum standard for inter-rater reliability.28

Validity

Caregiver and clinical staff item ratings/total scores will be compared using independent-samples t-tests. Given the possibly large number of comparisons in these analyses, a more conservative alpha of 0.001 will be set to indicate significance.

To assess convergent and concurrent validity of the QODD-RGV, we will calculate Pearson’s correlation coefficients (r) between the QODD-RGV total score and total scores of the GDI and FAMCARE, respectively. While no standardised criteria for sizes of validity coefficients to support convergent and concurrent validity have been established, we will consider that convergent validity with the GDI is met if r is large (ie, r>0.70) and concurrent validity with FAMCARE is met if r is moderate (ie, r>0.40). We expect this pattern of validity coefficients to be consistent across hospice sites, which would enhance confidence in validity.

Malawi arm

Participants

Participating hospice site

Data collection for the Malawi arm of the multisite study will be conducted over a 2-year period (projected time frame: November 2022–2024) as part of a collaborative partnership between the University of North Carolina Project-Malawi (UNCPM) Cancer Programme and GIPPEC of University of Toronto. UNCPM is a leader in oncologic research in sub-Saharan Africa and collaborates with Kamuzu Central Hospital (KCH), one of two national teaching hospitals in Malawi that provide cancer treatment. The participating Malawi palliative care site is the KCH Oncology Clinic in Lilongwe, Malawi, which is staffed by the Ministry of Health and UNCPM nurses.

Caregiver participants and recruitment procedures

Participants will be bereaved primary caregivers of patients who died of cancer after receiving care at KCH. Inclusion criteria include being 18 years of age or older, being the primary caregiver of a loved one who died of cancer after receiving care at KCH in the last four to six months, and being able to complete study questionnaires and interviews in Chichewa.

Participants will be recruited from the KCH Oncology Clinic four to six months after the death of their loved ones. A research assistant will contact eligible caregivers by telephone or by a short message service to ask about interest in participation and to provide study details. Caregivers who agree to participate will provide verbal consent that will be documented. Two sets of participants will be recruited: 25 bereaved caregivers for the cognitive interview study, and 200 bereaved caregivers for the validation study. The latter sample size is the maximum achievable within the 2-year data collection period given the number of patients treated annually at the site. Using G*Power V.3.1.9.2, a power analysis to detect a convergent validity correlation (bivariate normal model) of 0.70 indicates that the total sample size of 200 will provide more than sufficient (power=1.00). Based on the recruitment rates for similar studies, we anticipate that approximately 50% of eligible caregivers will consent to participate.

All measures will be translated to Chichewa using a rigorous stepwise forward-translation and back-translation approach.29 A panel of bilingual translators will conduct the forward translation, expert panel review and back-translation of measures.

As indicated in table 1, caregiver and patient demographic and patient medical information will be gathered through caregiver or staff self-report and medical chart review. Caregiver participants will complete the QODD-RGV, another measure of quality of dying (APCA African Palliative Outcome Scale (POS)30 31) and a measure of family satisfaction with advanced cancer care (FAMCARE).24

Four to six months after patient death, the research assistant will contact caregiver participants by telephone to complete the study questionnaires in a 30 min interview. All interview data will be collected using an encrypted tablet-based software, REDCap, which is password protected and accessible only to study staff.

After rating each item, interview participants will be asked to verbalise their thought processes about the item and their response to it. Interviewers will also probe to seek greater understanding of responses chosen for each item and of how important or relevant each item is to the patient’s experience of dying and death. If caregivers have difficulty providing a quality rating, they will be asked, ‘Why is this question difficult for you to answer?’ The nature of this difficulty and whether modification of the wording would enhance its understanding or relevance will be explored. Interviews will be audiorecorded, transcribed and translated into English for qualitative analysis.

All quantitative analyses will be conducted using SPSS V.25. Alpha will be set at 0.05 unless indicated. The same qualitative, descriptive and internal consistency analyses as described for the North American arm will be conducted with the Malawi QODD-RGV data as well.

Factor analysis

To identify conceptually meaningful item clusters, an exploratory factor analysis will be conducted of the QODD-RGV response data, using principal axis factoring with oblique promax rotation; factors will be identified (eigenvalues>1.0) and retained based on inspection of the scree plot and on their conceptual coherence.

To assess convergent and concurrent validity of the QODD-RGV, we will calculate Pearson’s correlation coefficients (r) between the QODD-RGV total score and total scores of the POS and FAMCARE. While no standardised criteria for sizes of validity coefficients to support convergent and concurrent validity have been established, we will consider that convergent validity is met if r with the POS, as a similar measure of quality of life near the end of life, is large (ie, r>0.70) and concurrent validity with the FAMCARE is met if r is moderate (ie, r>0.40).

To compare the quality of dying between North American and Malawi sites, we will first compare characteristics between the two arms (with pooled North American site data) using univariate statistics. We will conduct analysis of covariance (ANCOVA) to compare the QODD-RGV total scores from the Malawian caregivers and North American caregivers, adjusting for characteristics that significantly differ between sites. We will also conduct ANCOVA to compare the Malawi and North American groups on each of the item ratings; given the large number of comparisons, a more conservative alpha of 0.001 will be set.

Ethics and dissemination

Ethics approvals for the multisite North American study were obtained from the University Health Network Research Ethics Board (#21-5143) and the University of North Carolina Institutional Review Board (#21-1172). The North American arm is currently recruiting caregiver and staff participants. Ethics approval for the Malawi study is currently being obtained from the University of North Carolina Institutional Review Board and the Malawian National Health Science Research Committee. The aim of the study is to validate a revised version of the QODD that addresses some of its main limitations and increases its cultural generalisability to palliative and end-of-life care settings across resource-level countries. We hope that a culturally generalisable, caregiver-reported measure of QODD will stimulate much-needed palliative care research in lower-resource settings and thereby inform the essential expansion of palliative care policies and services. Findings from the studies will be disseminated to palliative care clinicians and scientists through publication in peer-reviewed journals and conference presentations.