Psychometric properties of carer-reported outcome measures in palliative care: A systematic review.

BACKGROUND: Informal carers face many challenges in caring for patients with palliative care needs. Selecting suitable valid and reliable outcome measures to determine the impact of caring and carers' outcomes is a common problem. AIM: To identify outcome measures used for informal carers looking after patients with palliative care needs, and to evaluate the measures' psychometric properties.
DESIGN: A systematic review was conducted. The studies identified were evaluated by independent reviewers (C.T.J.M., M.B., M.P.). Data regarding study characteristics and psychometric properties of the measures were extracted and evaluated. Good psychometric properties indicate a high-quality measure. DATA SOURCES: The search was conducted, unrestricted to publication year, in the following electronic databases: Applied Social Sciences Index and Abstracts, Cumulative Index to Nursing and Allied Health Literature, The Cochrane Library, EMBASE, PubMed, PsycINFO, Social Sciences Citation Index and Sociological Abstracts.
RESULTS: Our systematic search revealed 4505 potential relevant studies, of which 112 studies met the inclusion criteria using 38 carer measures for informal carers of patients with palliative care needs. Psychometric properties were reported in only 46% (n = 52) of the studies, in relation to 24 measures. Where psychometric data were reported, the focus was mainly on internal consistency (n = 45, 87%), construct validity (n = 27, 52%) and/or reliability (n = 14, 27%). Of these, 24 measures, only four (17%) had been formally validated in informal carers in palliative care.
CONCLUSION: A broad range of outcome measures have been used for informal carers of patients with palliative care needs. Little formal psychometric testing has been undertaken. Furthermore, development and refinement of measures in this field is required.

What is already known about the topic?

The involvement of informal carers is essential for the provision of palliative care, but informal caregiving can have a major impact on carers’ outcomes.

Studies of informal carer outcomes use a wide range of endpoints.

Selecting suitable and appropriate carer outcome measures seems problematic.

What this paper adds?

An increasing number of studies are conducted in informal carers looking after patients with palliative care needs.

Only four outcome measures have been formally developed and validated within this population, and limited psychometric information is available on most measures.

While there has been an increasing trend since 2008 in the use of outcome measures for informal carers in palliative care research, most measures used in these studies were developed more than 20 years earlier and may not adhere to current standards for measure development.

Implications for practice, theory or policy

Existing carer outcome measures need to be validated for the palliative care setting and new measures need to be developed in accordance with current guidelines in order to meet the requirements of the growing number of studies, including intervention studies, of informal carers looking after patients with palliative care needs.

When using an existing outcome measure, the authors should report their rationale for selecting it and should refer to the publications that report the original development of the measure.

Interventions for supporting informal carers should be evaluated using outcome measures for which appropriate psychometric properties have been reported before they are implemented as policy.

Introduction

The World Health Organization (WHO)[1] defines palliative care as an approach that focuses on the quality of life of patients and their relatives facing problems associated with life-threatening illness, through prevention and relief of suffering. Annually, around 20 million people worldwide need palliative care,[2] and an ageing population and increases in long-term conditions mean that need is likely to continue to rise.[3,4]

Informal carers make an important contribution in the provision of palliative care and are regarded as integral to its delivery.[5,6] Informal carers are defined as carers who are not financially compensated for their services typically spouses, children, siblings or friends.[7] In 2011, the contribution of approximately six million informal carers in the United Kingdom was estimated at the equivalent of £119 billion a year.[8] About half a million people are caring for patients during the end-of-life phase and this number is expected to increase to 3.4 million in the coming 30 years.[9]

Palliative care has become an important component of health care, and policy makers are putting more emphasis on informal carers.[10] Informal caregiving may provide emotional benefits and togetherness for carers,[11,12] but it also involves considerable challenges including adverse psychological, physical, social and financial consequences.[13,14] Studies indicate that informal caregiving affects carers’ wellbeing and their own health resulting in isolation, fatigue, sleeping problems, exhaustion, weight loss, depression and anxiety.[15-19] It is therefore important that carer outcomes are assessed in order to be able to provide effective support and to reduce negative consequences of caregiving. Carer outcomes refer to a range of concepts including quality of life, burden and strain. While these terms are not well defined and frequently get used interchangeably, it is generally accepted that they comprise multiple dimensions such as physical impact, mental strain and social functioning. It has been argued that quality of life is a broader concept as it assesses a wider spectrum of wellbeing, whereas burden and strain suggest a more direct measure of duty of care.[20]

Evidence on effective strategies to reduce the burden of caring and improve their quality of life of informal carers is limited.[21,22] Although interventions have been developed that aim to improve outcomes for informal carers, their results are difficult to compare as studies focus on a wide range of endpoints.[23] One systematic review identified 62 questionnaires used among informal family carers in various palliative care settings.[24] These questionnaires included instruments on carer satisfaction, experience (of health services and support), needs bereavement and outcomes. Previous reviews on interventions for informal carers concluded that it was unclear what kind of support was beneficial, partly due to the lack of appropriate outcome measures.[21,25]

In order to assess the impact of the caring role on carers, an appropriate choice of outcome measures is required; however, selecting suitable and appropriate measures seems a common problem.[25-27] This requires reliable and valid measures with robust psychometric properties, which are appropriate for a palliative care context, as this forms the foundation for evaluating caregiver interventions.

This systematic review aimed to identify and evaluate outcome measures that have been used for informal carers in palliative care studies. The measures used in palliative care are described and their psychometric properties (e.g. reliability, validity, feasibility and precision), when available, are evaluated.

Methods

Search strategy

We conducted a systematic review of carer outcome measures used in palliative care, according to Cochrane guidelines.[28] The databases, Applied Social Sciences Index and Abstracts (ASSIA), the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), EMBASE, PubMed, PsycINFO, Social Sciences Citation Index and Sociological Abstracts, were searched using four main terms: palliative care, informal carers, outcomes and measures. The search strategy is presented in Table 1 and further detailed search histories are available from the corresponding author on request. All identified citations were imported into the bibliographic database of EndNote, version X5 (Thomas Reuters, New York, NY). Reference lists of the retrieved articles were screened for additional studies.

Table 1.

Search strategy employed in systematic review of studies on psychometric properties of carer-reported outcome measures in palliative care.

Main search terms	Search terms (PubMed database)
Palliative care	palliative care[Mesh Terms] OR hospice care[Mesh Terms] OR hospices[Mesh Terms] OR palliative*[title/abstract] OR terminal care[title/abstract] OR terminal ill[title/abstract] OR hospice*[title/abstract] OR end-of-life care[title/abstract] OR end-of-life care[title/abstract] OR end-stage[title/abstract]
	AND
Caregivers	caregivers[Mesh Terms] OR family[Mesh Terms] OR spouses[Mesh Terms] OR volunteers[Mesh Terms] OR (family[title/abstract] AND (caregiver*[title/abstract] OR care giver*[title/abstract] OR caregiving[title/abstract] OR care giving[title/abstract] OR carer*[title/abstract])) OR (informal[title/abstract] AND (caregiver*[title/abstract] OR care giver*[title/abstract] OR caregiving[title/abstract] OR care giving[title/abstract] OR carer*[title/abstract])) OR (volunteer*[title/abstract] AND (caregiver*[title/abstract] OR care giver*[title/abstract] OR caregiving[title/abstract] OR care giving[title/abstract] OR carer*[title/abstract])) OR (unpaid[title/abstract] AND (caregiver*[title/abstract] OR care giver*[title/abstract] OR caregiving[title/abstract] OR care giving[title/abstract] OR carer*[title/abstract])) OR spouse*[title/abstract] OR husband*[title/abstract] OR wife*[title/abstract] OR family[title/abstract] OR volunteer*[title/abstract] OR unpaid[title/abstract] OR informal[title/abstract]
	AND
Outcomes	quality of life[Mesh Terms] OR quality of life[title/abstract] OR QOL[title/abstract] OR anxiety[title/abstract] OR benefit*[title/abstract] OR burden[title/abstract] OR competence*[title/abstract] OR coping[title/abstract] OR confidence[title/abstract] OR impact[title/abstract] OR need*[title/abstract] OR preparedness[title/abstract] OR satisfaction[title/abstract] OR self-assurance[title/abstract] OR strain*[title/abstract] OR stress[title/abstract] OR support[title/abstract] OR wellbeing[title/abstract]
	AND
Questionnaires	questionnaires[Mesh Terms] OR self-report[Mesh Terms] OR outcome assessment (health care)[Mesh Terms] OR psychometrics[Mesh Terms] OR assessment*[title/abstract] OR instrument*[title/abstract] OR measure*[title/abstract] OR outcome*[title/abstract] OR psychometric*[title/abstract] OR psychometry[title/abstract] OR tool*[title/abstract] OR questionnaire*[title/abstract] OR reliability[title/abstract] OR reliable [title/abstract] OR reproducibility[title/abstract] OR scale*[title/abstract] OR self-report[title/abstract] OR survey [title/abstract] OR validated[title/abstract] OR validation[title/abstract] OR validity[title/abstract]

Study selection

All types of multidimensional measures (generic, carer-specific for any condition and carer-specific for patients with a specific condition) were eligible for inclusion. The study focused on multidimensional measures as we were interested in measures that assess the overall impact of caring in palliative care rather than measures that assess one specific dimension of outcome or impact. A study was included if all of the following were fulfilled: (1) the study used a self-reported multidimensional measure that assessed caregiver outcomes (i.e. burden, strain or quality of life), (2) measures were directed at unpaid informal carers (e.g. spouse, relatives, siblings, friends or neighbours), (3) the patients they supported were diagnosed with an advanced progressive illness or were receiving palliative care (end-of-life care, terminal care or hospice care), (4) both carers and patients were ⩾18 years old and (5) the study was reported in English.

A study was excluded if any of the following were fulfilled: (1) only unidimensional measures were used; (2) only subscales or individual items and not the full measure were included; (3) only clinician-assessed measures or patient-reported measures were used; (4) all measures completed by carers were on behalf of the patient or (5) it was a qualitative study, comment, editorial, protocol, conference article or grey literature. There were no restrictions regarding publication date and research methods.

Data extraction and analysis

After retrieving all records, the duplicates were removed. All studies were initially screened on the basis of title and abstract, and then on the basis of full-text. Three authors (C.T.J.M., M.B. and M.P.) independently assessed the eligibility of studies: C.T.J.M. assessed all articles, M.B. and M.P. each assessed half of the articles. Any uncertainties were discussed with the other two authors (A.A. and B.W.) and resolved by consensus. C.T.J.M. extracted the data on study characteristics (publication year, country, sample size, research setting, type of disease, intended outcome measure and information on measure) and psychometric charac-teristics. The following information on psychometrics was collected: content validity, internal consistency, construct validity, reproducibility (agreement and reliability), res-ponsiveness, floor or ceiling effects, acceptability and feasibility. As guidance, we used the definitions given by Terwee et al.[29] and Fitzpatrick et al.[30] Additionally, when an included study did not report any psychometric information but referred to other articles regarding a measure or its psychometric values, we assessed these additional articles in order to evaluate the evidence they provided.

Results

Our electronic search, performed on 4 September 2014, identified 8569 studies. Figure 1 provides an overview of the number of studies identified at each stage of the search. After duplicates were removed, 4505 studies were screened on the basis of titles and abstracts, and 231 studies were screened on the basis of full text. This identified 112 studies using 38 different measures for informal carers in palliative care.

Figure 1.

PRISMA flow diagram of study selection.

Study and measure characteristics

A total of 112 studies (18 randomized controlled trials (RCTs), 78 observational studies and 16 methodological studies) were included. The methodological studies included translation, development and validation studies about an outcome measure for informal carers in palliative care. The patient population mainly consisted of cancer patients (n = 67, 60%) or a mixture of conditions (n = 29, 26%). Of the studies, 37% were conducted in the United States. Most studies included a mix of spouses, children, parents or friends (n = 99, 88%) and a small number of studies included only spouse carers (n = 4, 3%).

Most studies used only one outcome measure that fit our selection criteria (n = 91, 81%) and 19% of the studies administered two outcome measures to carers. Studies mainly used carer-specific measures only (n = 69, 62%), a quarter used a generic measure (n = 29, 26%), and 14 studies used both types (i.e. generic and carer-specific). In total, 38 measures were identified, including 25 carer-specific measures and 13 generic measures. The main study characteristics are presented in Table 2 and in detail in Supplement 1.

Table 2.

Study characteristics of the included studies (n = 112).

Study characteristic		Number of studies (%)	References
Type of study	Methodological	16 (15%)	31–46
	Observational	78 (70%)	17,47–123
	RCT	18 (16%)	124–141
Country	United States	37 (33%)	46,47,51,59–61,65,66,68,70,84,86–88,90,94–96,98,102,105–108,117,120,121,124,125,128,129,131,132,134–136,141
	Australia	12 (11%)	17,35,45,50,56,57,82,83,89,123,133,138
	Canada	11 (10%)	34,36,39,52,53,55,62,63,77,78,139
	Norway	8 (7%)	72–76,103,118,119
	United Kingdom	7 (6%)	37,64,80,100,126,137,140
	Other (e.g. Brazil, China, Germany, Spain, The Netherlands, Japan, Korea, Sweden and Taiwan)	37 (33%)	31–33,38,40–44,48,49,54,58,67,69,71,79,81,85,91–93,97,99,101,104,109–116,122,127,130
Study population	Mixture of informal carers (e.g. spouse, child and parent)	99 (88%)	17,31–37,39–47,49,51–63,65–74,76–81,83–92,94,96–120,122–125,127,129,130,132,134–141
	Not reported	9 (8%)	38,64,75,82,95,121,126,128,133
	Spouse	4 (3%)	48,50,93,131
Patient population (disease)	Cancer	67 (60%)	17,31,34,35,38,41–48,51,54,55,59,62,63,65,68–70,72,73,75–78,80,82,84,86–88,91,92,94,97–104,109–116,118,119,122,124,129,131–135,138–140
	Mixture of various diseases	29 (26%)	32,37,39,40,52,53,56–58,66,67,71,74,81,83,85,89,90,95,96,105,106,108,120,123,127,128,130,136
	Other (e.g. ALS, ESRD, dementia, heart failure, MND and MS)	10 (9%)	33,49,50,64,79,93,107,121,126,137
	Not reported	6 (5%)	36,60,61,117,125,141
Sample size study population	N < 50	22 (20%)	47,48,50,51,55–57,59,60,63,65,79,90,95,96,100,107–109,121,125,137
	N = 50–100	30 (27%)	17,31,38,41,45,52–54,62,71–73,75,77,80,84,86–88,92–94,97,106,117,120,122,123,129,138
	N=101–200	31 (28%)	33,35,39,42,46,58,61,64,66,69,70,76,78,82,85,89,98,99,101,104,110,111,115,118,119,124,127,130,133,135,139
	N > 200	29 (25%)	32,34,36,37,40,43,44,49,67,68,74,81,83,91,102,103,105,112–114,116,126,128,131,132,134,136,140,141
	Not reported	1 (1%)	126
Type of measure	Generic measure only	29 (26%)	37,40,42,55,63,64,67–69,72,73,76,89,90,93,97,101,103,107,109,110,119–121,127,130,132,137,141
	Carer-specific measure only	69 (62%)	17,31,32,34–36,38,39,41,43–48,50–54,56–62,66,70,74,75,78–88,91,92,94–96,98,100,104–106,111–117,122, 124–126,129,131,133,134,138,140
	Both generic and carer-specific measure	14 (12%)	33,49,65,71,77,99,102,108,118,123,128,135,136,139
Number of measures asked in study	One outcome measure	91 (81%)	17,31,34–40,42,43,45–48,50–64,66–69,72–74,76,78–95,97,98,100,101,103–107,109–113,115–117,119–122,124–127,129–134,137,138,141
	Two outcomes measures	21 (19%)	32,33,41,44,49,65,70,71,75,77,96,99,102,108,114,118,123,128,135,136,139,140

RCT: randomized controlled trial; ALS: amyotrophic lateral sclerosis; ESRD: end-stage renal disease; MND: motor neurone disease; MS: multiple sclerosis.

The most frequently used generic measure was the SF-36 (n = 16, 14%). The most frequently used carer-specific measures were the Caregiver Reaction Assessment (n = 21, 19%), Caregiver Quality of Life Index–Cancer (n = 14, 13%) and the Zarit Burden Inventory (n = 10, 9%). The primary focus of studies using a carer-specific measure was burden (n = 14, 13%), followed by quality of life (n = 8, 7%) and strain (n = 3, 2.6%). An overview of the identified measures and their frequency of use are presented in Table 3.

Table 3.

Identified outcome measures and frequency of use in the included studies.

Measures			Number of studies	References
Generic measures	SF-36		16	33,49,64,68,73,76,77,89,90,99,102,103,119,123,128,139
	SF-12		3	63,65,136
	SF-8*		3	69,97,121
	EORTC QLQ-C30*		3	37,72,118
	EQ-5D*		3	40,110,137
	QOLS		3	67,72,108
	Other (i.e. MS*, MQOL, SWED-QOL, OQOLI*, QOLI, WHOQOL and WHOQOL-BREF*)		9	42,93,101,107 –109,127,128,130
Carer-specific measures	Burden	CRA	21	41,47,62,74,75,82,83,99,106,111–116,118,122–124,133,138
		ZBI (including 4 item, 6 item, 8 item, 12 item, 22 item 29 item version)	10	50,53,58,66,77,80,81,91,102,126
		CBS	4	33,49,70,135
		Other (i.e. BASC*, BCOS, BSFC, CBS-EOLC, CBI, CIS, FACS*, HP*, MBCBS, RCAS* and BIC)	15	17,31,32,36,38,41,52,54,55,65,71,79,95,105,135
	Quality of life	CQOLI–Cancer	14	44,46,56,57,59,78,92,96,114,117,125,131,139,140
		CQOLI–Revised	5	60,61,120,136,141
		QOLLTI-F	4	34,39,43,71
		Other (i.e. AQOL-EOL*, CH-QOL-F*, FACT, HQOLI* and QOL–Family*)	8	48,51,70,86,94,98,132,134
	Strain	CSI	7	32,84,87,96,100,129,140
		FACQ-PC	4	35,45,85,88
		FSQ*	1	104

SF: short form; EORTC QLQ-C30: European Organization for Research and Treatment of Cancer quality-of-life–30-item questionnaire; EQ-5D: EuroQol–5 dimensions; QOLS: Quality of Life Scale; MS: Montgomery Scale; MQOL: McGill Quality of Life Questionnaire; SWED-QOL: Swedish Health-Related QOL Survey; OQOLI: Overall Quality of Life Index; QOLI: Quality of Life Index; WHOQOL: World Health Organization Quality of Life; WHOQOL-BREF: World Health Organization Quality of Life–brief form; CRA: Caregiver Reaction Assessment; ZBI: Zarit Burden Inventory; CBS: Caregiver Burden Scale; BASC: brief assessment scale for caregivers; BCOS: Bakas Caregiving Outcomes Scale; BSFC: Burden Scale for Family Caregivers; CBS-EOLC: Caregiver’s Burden Scale in end-of-life care; CBI: Caregiver Burden Inventory; CIS: Caregiver Impact Scale; FACS: Feelings about Caregiving Scale; HP: Hausliche Pflegeskala; MBCBS: Montgomery Borgatta Caregiver Burden Scale; RCAS: Revised Caregiving Appraisal Scale; BIC: burden index of caregivers; CQOLI: Caregiver Quality of Life Index; QOLLTI-F: Quality of Life in Life-Threatening Illness–Family Carer Version; AQOL-EOL: Assessment Quality of life–End of life–Spouses; CH-QOL-F: City of Hope–QOL Scale–Family Version; FACT: Functional Assessment of Cancer Therapy; HQOLI: Hospice Quality of Life Index; QOL: quality of life; CSI: Caregiver Strain index; FACQ-PC: Family Appraisal of Caregiving Questionnaire for Palliative Care; FSQ: Family Strain Questionnaire.

No information reported and available on psychometric properties.

Psychometrics of measures

More than half of the 112 (n = 60, 54%) studies reported no information on psychometric properties. The 52 (46%) studies that did included 33 observational studies, 15 methodological studies and 4 RCTs. Psychometric data were available for only 23 of the 38 measures including 7 generic measures (i.e. McGill Quality of Life Questionnaire,[142] World Health Organization Quality of Life,[143] Quality of Life Scale,[144] Quality of Life Index,[94] SF-36,[145] SF-12[145] and Swedish Health-Related Quality of life[146]) and 17 carer-specific measures. These measures consisted of 4–64 items, with a median of 16 items. Table 4 presents an overview of the 24 measures with the available psychometric information. This consisted mainly of information on the Cronbach’s alpha (n = 45, 40%), construct validity (n = 27, 24%), reliability (n = 14, 12%), content validity (n = 8, 7%), responsiveness (n = 8, 7%) and acceptability and feasibility (n = 8, 7%).

Table 4

Identified psychometric information in studies identified from the search (n=52).

Measure	Study	Type of study	No. of items	Psychometric information						Measure references cited by study	Original validation studies of measure (study population)
				Content Validity	Internal consistency	Construct validity	Reliability	Responsiveness	Acceptability and feasibility
BCOS	Buscemi et al [54]	OBS	15 items	NR	α=0.75	NR	NR	NR	NR	Bakas 1999[147], [+]Bakas 2000[148], [+]Bakas 2005[149]	Bakas 2006[150] (15 items, caregivers of stroke survivors)
	Govina et al[38] (Greek version)	MES	15 items	NR	α=0.83	BCOS−LASA (r=0.7), BCOS−G-HADS (r=−0.52), Criterion validity r=0.57	ICC=0.985ITC=0.47–0.76	91% sensitivity 86% specificity	10–15 minutes	NR
BSFC	Brogaard et al [31] (Danish version)	MES	28 items	NR	α=0.91	Social isolation (ρ =0.33, p=0.01), Dyspnoea (ρ=0.32, p=0.01),Self-reported health (ρ=0.03, p=0.80)	ITC=0.02–0.72	Tendency towards floor effect.	NR	Graesel 1995[151], *Graesel 1998[152], *[$]Graesel 2001[153], Hecht 2003[79], [$]Holz 1999 [154], [#]Zarit 1980[155]	Graesel 1995[151] (Caregivers of patients with various illnesses: no end-of-life diseases)
BIC	Misawa et al[41]	MES	11 items	NR	NR	“CRA subscales strongly correlated with supposed subscales of BIC”	NR	NR	NR	Miyashita 2006[156]	Miyashita 2006[156] (Informal caregivers of patients with neurological conditions or stroke)
CBS	Akinci and Pinar[33] (Turkish version)	MES	22 items	NR	α=0.91	CFA=0.43-0.81 (All CBS factors correlated with each other in positive direction. All sub dimensions scores were negatively correlated with SF-36 (–0.58; –0.65)	ICC=0.985ITC=0.37–0.70	NR	30 minutes	Elmstahl 1996[157]	Elmstahl 1996[157] (Caregivers to stroke patients 3 year after a primary stroke)
CBI	Merluzzi et al [95]	OBS	24 items	NR	α=0.88	Correlations CBI-CGI factors 2, 3 and 4 (p<0.05). Strongest correlations for both PSS-CBI were with factors 3+4 of the CGI.	NR	NR	10–15 minutes	Novak 1989[158]	Novak 1989[158] (Caregivers of confused/disoriented elderly)
CBS-EOLC	Dumont et al[36]	MES	16 items	Focus group + qualitative interviews	α=0.95	Construct validity= Most inter-item associations were consistent with conceptual framework qualitative study. Divergent validity=Interscale-correlations ZBI=0.72 (p<0.01), POMS (fatigue)=0.69 (p<0.01), POMS (vigour)=0.-0.27(p<0.05). Explaining overall variance = 64.8%	NR	Sensitivity showed consistent associations with EGOG and unmet needs.	NR	NR	Dumont 2008[36] (Family caregivers of terminal cancer patients)
CIS	Cameron et al[55]	OBS	14 items	NR	α=0.87	NR	NR	NR	NR	NR	Cameron 2002[55] (Caregivers of advanced cancer patients), Devins 1983[159] (ESRD patients)
CQOLI	McMillan and Mahon[94]	MES	4 items	Expert panel	α=0.76–0.88	Comparison to control group.Correlation between 4 items (r=0.43-0.64)	No test–retest differences	No significantdifferences found	Might be too short for reliability	NR	Weitzner 1999[160](Caregivers of cancer)
CQOLI-C	Connell et al[57]	OBS	35 items	NR	α=0.90	NR	Test–retest =0.95	NR	NR	Weitzner 1999[160]	Weitzner 1999[160](Caregivers of cancer)
	Delgado-Guay et al[59]	OBS	35 items	NR	α=0.91	NR	Test–retest =0.95	NR	NR	Weitzner 1999[160]
	Leow et al[92]	OBS	35 items	Expert panel (89%)	α=0.87–0.90	NR	NR	NR	NR	[#]Axelsson 1998[48], Weitzner 1999[160]
	Meyers and Gray[96]	OBS	35 items	NR	α=0.91	NR	Test–retest =0.95	NR	NR	Weitzner 1999[160]
	Tang et al [114]	OBS	35 items	NR	α=0.91	NR	NR	NR	NR	[^]Edwards 2002[161] (Refers to study Weitzner 1999[160]), Weitzner 1999[46]
	Tang et al[44] (Mandarin version)	MES	35 items	Translated	α=0.87	EFA showed 7 underlying factors explaining 48.15% of the variance. Caregivers’ QOL was inversely related to both patients’ (F=0.90, p=0.008) and caregivers’ pain (t=−4.22, p<0.001). Correlation CQOLC-M– MOS-SS scores (r=0.26, P<0.01), CQOLC-M–SWBS scores (r=0.30, P<0.01).	NR	NR	NR	NR
	Tang 2009 [117]	OBS	35 items	NR	α=0.90	NR	NR	NR	NR	Rhee 2005[162] [Korean version], Weitzner 1997[163]
	Weitzner 1999[46]	MES	35 items	Expert panel	α=0.91	Correlations with mental health (r=0.64), emotional distress (r=−0.52), burden (r=−0.65) and patient’s performance (r=−0.47), physical health (r=0.13), social support (r=0.22) and social desirability (r=0.08). SF-36–CQOLC correlations social support and social desirability were low range (range=0.08–0.20) Correlated with SF-36, BDI, ECOG, STAI, CBS, MSPSS, and MCSDS.	Test–retest =0.95	Correlation CQOLC–Performance status r=−0.46, p<0.0001	10 minutes	NR
CQOLI-R	Wittenberg-Lyles et al[141]	RCT	4 items	NR	α=0.77	NR	Reliability=0.94	NR	NR	Courtney 2005[164]	McMillan 1994[94] (Caregivers of hospice cancer patients)
CRA	Andrews [47]	OBS	24 items	NR	NR	Correlated with CES-D scale.	NR	NR	NR	Given 1992[165], Given 1993[166], [#]Radloff 1997[167]	Given 1992[165] (Caregivers of persons with physical impairments or dementia)
	Grov et al[72]	OBS	24 items	NR	α=0.57–0.85	NR	NR	NR	NR	Given 1992[165], Grov 2006[72]
	Hudson and Hayman-White[82]	OBS	24 items	NR	α=0.76–0.83	The initial rotation indicated that a number of items (3, 5, 7, 10, 16, 18) had high loadings. Items with high loading on more than one component were removed. A third and final Principal Component Analysis (PCA) was performed on the remaining items indicating extraction of 5 components, which accounted for 23.9%, 16.7%, 10.1%, 7.9%, and 7.2% of the total variance.	NR	NR	NR	Given 1992[165], [^]Kinsella 1998[168], Nijboer 1999[169]
	Misawa et al[41] (Japanese version)	MES	24 items	“Face and content validity checked.”	α=0.73–0.89	EFA= 5 factors were extracted, and thecumulative proportion of variance explained was 76.6%. Correlations Impact on schedule–time dependent burden (r=0.75), Caregiver’s self-esteem–emotional burden (r=0.66), Caregiver’s self-esteem–Existential burden (r=0.54), Impact on health–physical burden (r=0.75)	NR	No ceiling effect or floor effect.	NR	NR
	Morishita and Kamibeppu[99]	OBS	24 items	NR	α=0.83–0.91	NR	NR	NR	NR	Given 1992[165], Misawa 2009[41]
	Stein et al[138]	RCT	NR	NR	α=0.82	NR	NR	NR	NR	Given 1992[165]
	Tang [111]	OBS	NR	NR	α=0.90	NR	NR	NR	NR	Given 1992[165], Nijboer 1999[169]
	Tang et al[115]	OBS	NR	NR	α=0.88	NR	NR	NR	NR	Given 1992[165]
	Tang and Li[113] (Chinese version)	OBS	24 items	NR	α=0.88	NR	NR	NR	NR	Given 1992[165], Tang 2007[115]
	Tang et al[114]	OBS	NR	NR	α=0.68–0.85	NR	NR	NR	NR	Given 1992[165]
	Tang et al[116]	OBS	24 items	NR	α=0.88	NR	NR	NR	NR	Given 1992[165]
	Utne et al[118]	OBS	24 items	NR	NR	Variance in each domain of CRA was explained by different factors, with total explained variance=[5.5 % (lack of family support)-31.8% (impact on daily schedule)]	NR	NR	NR	Fletcher 2009[170], Mazanec 2011[171], Nijboer 1999[169], Park 2012[172]
	Yoon et al [122] (Korean version)	OBS	24 items	NR	α=0.63–0.85	NR	NR	NR	NR	*Rhee 2009[173], Grov 2006[174]
CSI	Hwang et al[84]	OBS	NR	NR	α=0.84	Rotated factor patterns showed that items from PPUN, FIN unmet need subscale, CSI, FAMCARE clearly loaded on different factors and suggested that these scales were measuring separate conceptual constructs. CSI correlated with PPUN (r=0.61, p<0.01), FIN (r=0.39,p=0.01)	NR	NR	NR	Robinson, 1983[175]	Robinson, 1983[175] (Caregivers for people who have arteriosclerotic heart disease or hip operation/ replacement)
	Meyers and Gray[96]	OBS	13 items	NR	α=0.86	NR	NR	NR	NR	Robinson, 1983[175]
	Chan and Suen[32] (Chinese version)	MES	13 items	NR	α=0.91	Factor analysis yielded a single factor as the original M-CSI, which explained 49% of variance. Factor of scale was constructed from item loading of at least 0.59 and was not subjected to any rotation. Higher scores in total score of C-M-CSI were substantially associated with high scores of C-CBI and its subscales.	NR	NR	NR Good range of responses to items (Mean= 0.73–1.26; SD=0.72–0.80)	Robinson, 1983[175], Thornton & Travis 2003[176] (Modified CSI)
FACQ-PC	Cooper et al[35]	MES	26 items	Experts	α=0.73–0.86	FA=Presence of 6 initial factors with eigenvalues exceeding 1, explaining 25, 14, 8, 7, 5, and 4% of the variance, respectively. Caregiver strain subscale the strongest correlation with subjective burden.	ITC= Strain [0.41-0.69], Distress [0.28–0.65], Family wellbeing [0.37–0.78]	NR	NR	NR	Cooper 2006[35] (Caregivers of palliative care patients)
	Keir et al[88]	OBS	25 items	NR	NR	Correlation with distress r=−0.245, p=0.04	NR	NR	<15 minutes	Cooper 2006[35]
FACT	Northouse et al[134]	RCT	NR	NR	α=0.81–0.84	NR	NR	NR	NR	Northouse 2002[177]	Cella 1993[178] (cancer patients)
MQOL	Sherman et al[108]	OBS	17 items	NR	α=0.76–0.73	Correlations between subscales MQOL r<0.39, though psychological symptoms and support are significantly correlated to existential wellbeing (p<0.0015, p<0.005).	NR	NR	NR	Cohen Hassan 1996[179], Cohen Mount 1996[142]	Cohen Mount 1996[142] (cancer patients)
MBCBS	O’Hara et al[135]	RCT	14 items	NR	α=0.76–0.90	NR	NR	NR	NR	[$]Montgomery 2000[180]	Montgomery 1989[181] (Caregivers of elderly persons)
WHO-QOL	Paiva et al[42]	MES	26 items	NR	α=0.80	Correlation HCQ-c –WHOQOL-Bref = Overall quality of life (r=0.688, p<0.01), Physical domain (r=0.415, p<0.01), psychological domain (r=0.570, p<0.01), social domain (r=0.561, p<0.01), environmental domain (r=0.619, p<0.01), and global spirituality (r=0.639, p<0.01).	NR	NR	NR	Berlim 2005[182] (Brazilian version), Chachamovich 2007[183] (WHOQOL-BREF)	WHOQOL Group 1998[184] (general population)
QOLLTI-F	Cohen et al[34]	MES	16 items	Expert panel	α=0.86	Factor loadings, only one regarding amount of control the carer has over his/her life remained problematic. 7 domain scores were created with items that loaded most heavily on each factor. Correlations 7 components r⩽0.36. The 16 items predicted 55% of variance in global QOL and 53% in 7 domain scores. QOLLTI-F Total score predicted 43%.	Test-retest r=0.77–0.8	All scores significantly different between days, with exception of financial. 2 sub measures limited by ceiling effect.	NR	NR	Cohen 2006[34] (Caregivers of palliative cancer patients)
	Schur et al[43] (German version)	MES	16 items	Group of various stakeholders (relatives, clinicians)	α=0.85	Correlations: QOLLTI-F–HIS (integrative hope scale) r=0.40 (p=0.000), explained variance 16.2%	Test-retest r=0.92	NR	All items showed a rate of missing responses below 5%	NR
						QOLLTI-F –HADS Stress r=−0.41 (p=0.000), explained variance 16.26%. QOLLTI-F –HADS Anxiety r=−0.51 (p=0.000), explained variances 26%. QOLLTI-F –HADS Depression r=−0.52 (p=0.000), explained variances 26%. QOLLTI-F – Subjective burden r=−0.55 (p=0.000), explained variances 29.1%. FA= problematic cross-loading above 0.32 for items 1, 2, 3, 8, 11, 14 and 16. Scale showed a stable 4-factor structure, instead of initial 7.
QOLS	Grov et al[72]	OBS	16 items	NR	α=0.88	NR	NR	NR	NR	[^]Burckhardt 2003[185], Wahl 1998[186]	Flanagan 1978[144] (General population)
QOLI	Scott[107]	OBS	36 items	NR	α=0.70	NR	Total reliability r=0.92–0.96	NR	NR	Ferrans 1992[187]	McMillan 1994[94] (Caregivers of hospice cancer patients)
SF-36	Ringdal et al[103]	OBS	36 items	NR	α=0.70–0.94	Correlations from subscales ranges r=0.50–0.89	NR	NR	NR	Ware 1992[145], Loge 1998[188], Jenkinson 1997[189], McHorney 1993[190], Jenkinson 1993[191]	Ware 1992[145] (General population)
	Weitzner et al[46]	MES	35 items	NR	NR	r= 0.52 to 0.78 for subscales with other QoL measures.	Test–Retest r=0.68–0.93	NR	NR	Ware 1992[145], McHorney 1993[190], McHorney 1994[192]
SF-12	Duggleby et al[63] (SF12v2)	OBS	12 items	NR	NR	√ SF-36 r=0.95 and r=0.96	NR	NR	NR	[$]Ware 1998[193]
SWED-QOL	Persson et al[101]	OBS	64 items	NR	α=0.68–0.93	NR	NR	NR	NR	[#]Brown, 2005[194], Eriksson 2005[195], Lindqvist 2000[93], *Jönsson 1999 [196]	Brorsson 1993[146] (General population)
ZBI	Bentley et al[50]	OBS	NR	NR	α=0.87	NR	NR	NR	NR	Bedard 2001[197]	Zarit 1980[155] (Caregivers of dementia patients)
	Brink et al[53]	OBS	12 items	NR	α=0.78–0.90	CFA examined factor structure of 12 abridged items. Correlation 12 items–22 items r=0.92-0.97. Mean scores were high 19.89, compared to Bedard 2001[197] =11.20, O’Rourke 2003[198]=8.29, Higginson 2010[81]=12.0	NR	NR	NR	NR
	Higginson et al[81]	OBS	22 items	NR	α=0.69–0.93	ZBI-12 ρ=0.95–0.97, ZBI-8 ρ=0.86-0.93, ZBI-7 ρ=0.90–0.95, ZBI-6 ρ=0.89–0.95, ZBI-4 ρ=0.88–0.92, ZBI-1 ρ=0.63– 0.78	NR	NR	NR	Bedard 2001[197] (ZBI-4), *Gort 2003[199] (ZBI-7), *Arai 2003[200] (ZBI-8), Bedard 2001[197] (ZBI-12)
	Prigerson et al[102]	OBS	9 items	NR	NR	Mean caregiver burden score was 8.17 for with MDD versus 5.71 for without MDD (p<0.002).	NR	NR	NR	Zarit 1980[155]

Measure not used in study, +Conference article, $Grey literature, *Non English, ^Review, NR=Not Reported, MES=Methodological study, OBS=Observational study, RCT=Randomized Controlled Trial.

BCOS=Bakas Caregiving Outcomes Scale, BDI=Beck Depression Inventory, BIC=The Burden Index of Caregivers, BSFC=Burden Scale for Family Caregivers, CBI= Caregiver Burden Inventory, CBS=Caregiver Burden Scale, CBS-EOLC=Caregiver’s Burden Scale in End-of-Life Care, CES-D=Center for Epidemiologic Studies Depression Scale, CFA=Confirmatory factor analysis, CGI= Clinical Global Impression, CIS= Caregiver Impact Scale, CQOLI=Caregiver Quality of Life Index, CQOLI-C=Caregiver Quality of Life Index-Cancer, CQOLI-R=Caregiver Quality of Life Index–Revised, CRA=Caregiver Reaction Assessment, CSI=Caregiver Strain Index, ECOG=Eastern Cooperative Oncology Group, EFA=Exploratory factor analysis, ESRD=End stage renal disease, FA=Factor Analysis, FACQPC=Family Appraisal of Caregiving Questionnaire for Palliative Care, FACT=Functional Assessment of Cancer Therapy, HADS= Hospital Anxiety and Depression Scale, IHS =Integrative Hope Scale, HCQ-C= Holistic Comfort Questionnaire – Caregiver, ICC= Intra-class Correlation, ITC=Corrected Item-Total Correlation, LASA= Linear Analogue Scale Assessment, MBCBS=Montgomery Borgatta Caregiver Burden Scale, MCSDS=Marlowe-Crowne Social Desirability Scale, MDD=Major depressive disorder, MOS-SS= Medical Outcomes Study- Social Support, MQOL=McGill Quality of Life Questionnaire, MSPSS=Multidimensional Scale of Perceived Social Support, PSR=Performance Status Rating, PSS=Perceived Stress Scale, QOL=Quality of life, QOLI=Quality of Life Index, QOLLTI-F=Quality of Life in Life Threatening Illness-Family Carer Version, QOLS=Quality of Life Scale, SF=Short Form, STAI= State-Trait Anxiety Inventory, SWED-QOL=Swedish Health-Related QOL Survey, WHO-QOL=World Health Organization Quality of Life, ZBI=Zarit Burden Inventory.

Of the 24 measures, four were originally developed in a palliative care context, that is, the Quality of Life in Life-Threatening Illness–Family Carer Version (QOLLTI-F),[34] the Family Appraisal of Caregiving Questionnaire for Palliative Care (FACQ-PC),[35] the Caregiver Burden Scale in end-of-life-care (CBS-EOLC)[36] and the Caregiver Quality of Life Index (CQOLI).[94] The content validity (which examines the extent to which the concepts of interest are represented by the items[201]), internal consistency (which measures the extent to which items in a scale are inter correlated[29]) and construct validity (the extent to which scores relate to other similar measured concepts[29]) were adequate in all four measures. Interpretability (the degree to which one can assign qualitative meaning to quantitative scores) was not reported in all four studies. The reliability (which concerns the degree to which repeated measurements in stable persons provide similar answers[29]) was positive in two measures[34,94] and negative for FACQ-PC.[35] Floor and ceiling effects (considered to be present if more than 15% of respondents achieved the lowest or highest possible score, indicating that it is likely that extreme items are missing in the lower or upper ends of the scale[202]) was negative for QOLLTI-F[34] and not reported for the other three measures.[35,36,94]

For studies (n = 60) that did not report psychometric properties but referred to previous publications about the measure, C.T.J.M. additionally extracted psychometric information from the referenced articles (see Supplement 2). An additional 139 references were assessed for study type, study population and psychometric properties. Although this provided information on how the measures were originally developed, it did not result in additional psychometric information for the measures in the context of carers in a palliative care setting.

Discussion

The aim of this systematic review was to identify and evaluate the psychometric properties of self-reported measures used in informal carers in palliative care studies. A total of 112 studies were found, which used 38 different outcome measures for informal carers. The most commonly used generic measure was the SF-36 (n = 27) and the most commonly used carer-specific measure was the Caregiver Reaction Assessment (n = 21). Psychometric information was available for only 24 carer outcome measures (52 studies). We identified only four measures that were formally tested in a palliative care context.

Measures were mainly used in descriptive studies (n = 78) and the overall study sample sizes tended to be quite small. This could be due to methodological and structural challenges in palliative care research.[203] For example, uncertainties in patients’ prognosis, heterogeneity of the palliative care population, relatively small palliative care centres, ethical concerns or attrition of patients during the study could inhibit research in palliative care.

We noted an increasing trend in the use of measures in informal carers in palliative care. The majority of the included studies were published relatively recently, with more than 70% published since 2008. However, the majority of measures were developed much longer ago, including the most frequently used such as the Caregiver Reaction Assessment[165] or the Zarit Burden Interview.[155] It is therefore unclear whether measures adhere to the current development guidelines, such as those set by the Food and Drug Administration for patient-reported outcome measures.[204] Evaluating publications on the development of these outcome measures was beyond the scope of our review, and the information would have been of limited value as the measures were mainly developed in other carer populations.

Due to the wide range of identified carer outcome measures and the variety of versions of the measures (e.g. Zarit Burden Interview; Table 3), it is difficult to draw overall conclusions about psychometric properties. The most commonly reported psychometric information was Cronbach’s alpha (n = 45, 40%), which is a psychometric property that is commonly used, relatively easy to calculate and easy to interpret. In all, 60 did not report any psychometric information. It was not expected that all studies would contain psychometric information, as the lack of psychometrics was not an exclusion criterion. For studies that did not report psychometric properties but referred to previous publications about the measure, we screened an additional 139 references for information on psychometrics. However, these resulted in limited extra psychometric data, and none of the studies met the inclusion criteria of this systematic review.

Although psychometric information was generally limited, it was even more limited in relation to some psychometric properties such as responsiveness. Responsiveness (or sensitivity to change) is particularly important to highlight as carer-reported outcome measures may be used to assess the effectiveness of interventions. Interventions to support carers in palliative care settings are likely to be complex and require measures that are able to detect change following the intervention.

We identified only four carer-specific measures that were formally developed and tested in this population: QOLLTI-F,[34] FACQ-PC,[35] CBS-EOLC[36] and CQLI.[94] These four measures were used less frequently than either the Caregiver Reaction Assessment or the Zarit Burden Interview that have not been validated in this population.

Regarding the generic measures, none have been formally validated in this carer population but we found psychometric information on seven[94,142-146] measures. As these have been widely validated in a large number of different populations, it seems reasonable to assume that they are applicable for carers in a palliative context as well.

It is interesting that limited psychometric information was reported for the most widely used carer-specific measure, the Caregiver Reaction Assessment.[165] This suggests that psychometric properties of the measures may not be the key factor in researchers’ choice of outcome measures. It would be worthwhile exploring in further studies what considerations researchers take into account when selecting their measures and why some carer-specific measures are used more frequently than others, particularly those developed specifically for carers in a palliative context.

Choosing the right measure for a particular study can be challenging because there may be a number of relevant measures from which to choose.[205] A systematic review would be appropriate valuable method to identify the most suitable measure, but it may not always be feasible to conduct a systematic review. Alternatively, as our systematic review highlights, no measure may seem entirely appropriate due to a lack of psychometric information. Additionally, measures may include items irrelevant to the study population, but developing new measures is costly and time consuming. Measure listings such as the Mapi research trust[206] and published systematic reviews can assist in selecting an appropriate measure.[205] Studies in this review did not always reference the measures used or when a reference was provided, it was frequently not the reference of the development of the measure. We encourage authors to reference the original development paper(s) of the measure(s) used and to justify their choice of instrument.

The findings of this systematic review are in line with previously published reviews. Hudson et al.[24] identified 62 tools covering a range of topics including satisfaction, experience, bereavement, needs, preparedness, family functioning and outcomes. Hudson et al.[24] identified a larger number of tools than we did as they included instruments, which we specifically excluded. The review concluded that appropriate tools were lacking but the authors only gave a broad critical appraisal across substantially different types of instruments. In 2009, Whalen and Buchholz[207] identified 74 caregiver burden screening tools for children or adults providing informal care, not specific to a palliative care context. Whalen and Buchholz[207] reported that burden measures might seem appropriate for informal carers but many are lacking psychometric information. Deeken et al.[208] searched MEDLINE and PubMed from 1966 to 2002 and identified 28 tools on burden (n = 17), needs (n = 8) and quality of life (n = 3). Neither Whalen and Buchholz nor the Deeken et al. reviews focused on palliative care. In contrast, our systematic review was conducted in a broader range of databases, specifically focused on self-reported multidimensional carer outcome measures in a palliative care context.

A strength of this systematic review is the comprehensive search of eight databases using four main search terms and no date restrictions, which meant we could collate and examine the variety of outcome measures that have been used with informal carers in a palliative care context. This review shows that although there is an increasing number of studies of informal carers in palliative care, most of the outcome measures used have not been formally validated within this carer population.

Another strength of the review is the care that was taken with regard to the inclusion criterion of palliative care. Palliative care is a complex process and involves a broad spectrum of health care services and treatments. Not all palliative care studies are labelled as such but refer to ‘hospice care’ or ‘end-of-life care’. These search terms were included but provided some challenges. For example, end-stage renal failure is for some patients a chronic disease but when dialysis or treatment is no longer effective, patients need a palliative approach. Two palliative care experts (A.A. and B.W.) independently assessed each study where there was uncertainty to determine whether or not it was in a palliative care population.

A limitation of the review is the exclusion of the grey literature and literature in languages other than English. It is likely that this meant we missed measures published outside the standard academic field or validation studies of translated measures, which might have provided further psychometric information.

A second limitation is rooted in the limitations of literature itself. Limited psychometric information was available, as more than half of the studies (n = 60) did not report any psychometric data. We included all studies that used multidimensional outcome measures in informal carers in palliative care, rather than only development or validation studies, as this corresponded to our study aims. We did not intend to include only development or validation studies, but this may be more appropriate for assessing psychometrics. However, if our inclusion criteria had been limited to development or validation studies alone, only four studies[34-36,94] would have been identified. Trends regarding the increasing number of publications on carer outcomes in palliative care would have been missed. As most of the studies did not include psychometric information, we could not critically assess the quality of most of the measures.

Conclusion

Support for patients receiving care is likely to continue to be devolved to informal carers. The WHO has called for health care provision to be extended to families, ensuring their needs, coping and outcomes are addressed alongside those of patients receiving health care services at the end of life.[209] As more interventions are developed to support carers, carers’ outcomes will increasingly be assessed in palliative care context. Although a wide range of measures have already been used in this context, very limited formal psychometric testing has been undertaken. The frequently used measures contain limited psychometric information, while the outcome measures developed or validated in this context are not frequently used in research. Hence, further development and refinement of measures for informal carers in palliative care is required in order to be able to sufficiently support informal carers.