Patient-reported outcome measures for chronic obstructive pulmonary disease : the exclusion of people with low literacy skills and learning disabilities.

BACKGROUND: Patient-reported outcome measures (PROMs) are intended to reflect outcomes relevant to patients. They are increasingly used for healthcare quality improvement. To produce valid measures, patients should be involved in the development process but it is unclear whether this usually includes people with low literacy skills or learning disabilities. This potential exclusion raises concerns about whether these groups will be able to use these measures and participate in quality improvement practices.
METHODS: Taking PROMs for chronic obstructive pulmonary disease (COPD) as an exemplar condition, our review determined the inclusion of people with low literacy skills and learning disabilities in research developing, validating, and using 12 PROMs for COPD patients. The studies included in our review were based on those identified in two existing systematic reviews and our update of this search.
RESULTS: People with low literacy skills and/or learning disabilities were excluded from the development of PROMs in two ways: explicitly through the participant eligibility criteria and, more commonly, implicitly through recruitment or administration methods that would require high-level reading and cognitive abilities. None of the studies mentioned efforts to include people with low literacy skills or learning disabilities.
CONCLUSION: Our findings suggest that people with low literacy skills or learning disabilities are left out of the development of PROMs. Given that implicit exclusion was most common, researchers and those who administer PROMs may not even be aware of this problem. Without effort to improve inclusion, unequal quality improvement practices may become embedded in the health system.

Key Points for Decision Makers

• Patient-reported outcome measures (PROMs) are health questionnaires asking patients to report on their symptoms and quality of life

• Normally patients are involved in the rigorous process to develop PROMs to ensure the questionnaires make sense and are relevant to them. However, this study demonstrates that people with low literacy skills and learning disabilities are generally left out of the development of PROMs so they may not be able to use these measures

• Increased involvement of these groups in research and specific efforts to ensure PROMs are accessible for them could prevent unequal access to further uses of PROMs, for example, in healthcare quality improvement

Background

Patient-reported outcome measures (PROMs) are quickly becoming an important part of healthcare quality improvement processes internationally [1]. Health professionals and authorities can use these questionnaires to understand the burden of disease from the patients’ perspective and gather information to improve the quality of care they receive. Although PROMs are mainly used for acute conditions, expanding their use into long-term conditions is imminent [2].

Multiple stages involving patients and subject experts are required to develop PROMs so that they will produce meaningful data. Typically, developers first consult with the concerned groups about what to include in the PROM. Several forms of validation and reliability testing follow before the PROM is finalized. The development of a PROM does not stop at its conception; the process of development is ongoing, encompassing further validation and reliability testing of PROMs through use with different patient groups. The initial and ongoing development and validation are meant to ensure the items and tool are meaningful to clinicians and patients and that it is easy to administer [3]. Since patients are consulted, PROM development appears patient centered. But not all groups of patients may have an opportunity to offer their views on PROMs.

People with low literacy skills or learning disabilities may be excluded from PROM development and ultimately from the uses of PROMs, countering national and international policies. The causes of learning disability and low literacy vary greatly, and people with low literacy and those with learning disabilities are not usually grouped together. Nevertheless, the functional difficulties (e.g., reading ability and comprehension) they face can be similar and both groups may find help in processing written material and managing health interventions useful [4]. Together, these groups form a large proportion of people in the UK, where one in five adults have low literacy skills [5] and there are approximately 1.5 million people with learning disabilities [6]. Literacy difficulties are associated with general exclusion from trials and research [7]. If people with low literacy skills or learning disabilities are left out of the development of PROMs specifically, they may struggle to complete them since patient involvement is crucial to ensure appropriate wording, format, and other needs are met. When PROMs are then used in further healthcare initiatives, they may face exclusion again. This exclusion would counter the UK Government’s policy for the NHS to achieve full inclusion [8]. It also violates the principles outlined in the United Nations Convention on the Rights of Persons with Disabilities [9].

To prevent people with low literacy skills and/or learning disabilities from possible exclusion from PROMs initiatives, knowledge of the extent and nature of exclusion in the development of PROMs is required. There is little previous research on this issue in the PROMs literature. Others have, however, acknowledged the potential for the exclusion of people with low literacy skills and learning disabilities from PROMs initiatives. Devlin and Appleby [10] write that gathering PROMs data from people with “cognitive deficits” (p. 12) is difficult. Mencap, a leading charity in the UK for people with learning disabilities, say their clients are at risk of being left out of efforts to gather health information from patients [11]. But neither describes the current extent of exclusion from the development of PROMs in the first place.

Chronic obstructive pulmonary disease (COPD) is a good exemplar case to investigate potential exclusion. This long-term condition causes significant burden to over 3 million people in the UK [12]. It is also more prevalent in lower socio-economic groups where lower levels of literacy are also more common. This study examines the most commonly used PROMs for COPD to investigate whether there is systematic exclusion of people with learning disabilities and/or low literacy from PROM use and, if so, aims to suggest how PROMs can be developed more inclusively.

Method

Our methods had two phases.

In Phase I, we identified the existing PROMs that are considered suitable for use with people with COPD using systematic reviews by Fitzpatrick et al. [13] and Davies et al. [14]. The most promising PROMs they identified included the respiratory condition-specific Clinical Respiratory Questionnaire (CRQ) and St George’s Respiratory Questionnaire (SGRQ), and the generic EQ-5D and SF-36. We went through Fitzpatrick et al. [13] and Davies et al.’s [14] reference lists to select only the publications involving the above-listed PROMs. The inclusion and exclusion criteria are summarized in Table 1. Given the quality of their review, these were the basis for our study for articles published prior to or in 2008.

Table 1

Inclusion and exclusion criteria for the papers identified in Phases I and II

Inclusion criteria	Exclusion criteria
Phase I a
Study with primary research involving patients or members of the public	Included PROM only to validate another PROM that was not relevant to our study
Inclusion of one or more of the following PROMs:  Condition specific:   • Chronic Respiratory Disease Questionnaire (CRQ)b   • Short-Form Chronic Respiratory Disease Questionnaire (SF-CRQ)b   • St George’s Respiratory Questionnaire (SGRQ)c   • COPD-specific St George’s Respiratory Questionnaire (SGRQ-C)c  Generic:   • SF-36   • EQ-5D	Systematic reviews
Phase II d
Study with primary research involving patients or members of the public	Included PROM only to validate another PROM that was not relevant to our study
Inclusion of one or more of the following PROMs:  • McGill COPD Quality of Life Questionnaire  • Functional Assessment of Chronic Illness Therapy (FACIT)-Dyspnea Scale  • Disability Related to COPD Tool (DIRECT)  • Capacity of Daily Living during the Morning Questionnaire  • Visual Simplified Respiratory Questionnaire (VSRQ)  • Living with COPD Questionnaire  • COPD Assessment Test (CAT)  • EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT)	Systematic reviews
Study was published between 2009 and 2012

COPD chronic obstructive pulmonary disease, PROM patient-reported outcome measure

aThese criteria were applied to the papers previously identified in Fitzpatrick et al. [13] and Davies et al. [14] until 2008

bThese were not differentiated; both are included as ‘CRQ’

cThese were not differentiated; both are included as ‘SGRQ’

dThese criteria were applied to papers describing and using PROMs developed from 2009 to 2012

In Phase II, we updated the search to identify papers describing PROMs developed from 2009 to 2012. To identify these papers, we used a pragmatic approach based on the search strategy provided to us by one of the authors of Davies et al. [14], searching in MEDLINE. We were not in a position to recommend which of the newly identified PROMs were suitable for population use, as was done in Fitzpatrick et al. [13] and Davies et al. [14], so we included all PROMs that were used with COPD patients, were in English, and were not too narrowly focused (for example, focusing only on one or two specific symptoms such as breathlessness). We also included two PROMs that were identified, but had only just been developed, at the end of Davies et al.’s [14] systematic review in 2009: the COPD Assessment Test (CAT) and the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT). The inclusion and exclusion criteria for this phase are also summarized in Table 1.

The abstracts from Phase I and Phase II were independently reviewed by two researchers for inclusion and exclusion. If there was a discrepancy, the abstract went to a third reviewer to decide. We retrieved the full text of articles that met the inclusion criteria and extracted the relevant data from each paper including:

The aim of the study

The PROM(s) used

The participant recruitment and data collection processes (if stated)

The inclusion and exclusion criteria for research participants (if stated)

Any mention of people with low literacy skills or learning disabilities

We distinguished between ‘explicit’ and ‘implicit’ modes of exclusion. Studies were classed as having ‘explicit exclusion’ if they excluded people with low literacy skills or learning disabilities ‘explicitly’ or immediately as a result of the study’s eligibility criteria. While we did not consider a specific category of disability severity, the terms considered to imply the potential exclusion of groups with low literacy skills or learning disabilities included those who were ‘unable to read,’ or ‘unable to complete or comprehend the questionnaire,’ or those with ‘cognitive deficits,’ ‘cognitive impairments,’ or ‘cognitive limitations.’ Studies that used literacy-dependent methods for recruitment or data collection (e.g., unassisted self-completion of the PROM) were classed as leading to ‘implicit exclusion.’

A second researcher in the team checked the data extraction for 20 % of the articles. Finally anything else in the study set-up that could impact the inclusion of these groups was recorded.

Results

We reviewed 108 abstracts and included 53 papers in Phase I. We identified six new PROMs in Phase II and reviewed 67 abstracts, eventually including 17 papers. The processes for Phase I and Phase II are summarized in Figs. 1 and 2, respectively. In addition to the CRQ, SGRQ, EQ-5D, SF-36, CAT, and EXACT tools, the new PROMs identified and included were:

Fig. 1

Phase I review flow chart

Fig. 2

Phase II review flow chart

The McGill COPD Quality of Life Questionnaire

The Functional Assessment of Chronic Illness Therapy (FACIT)-Dyspnea Scale

The Disability Related to COPD Tool (DIRECT)

The Capacity of Daily Living during the Morning Questionnaire

The Visual Simplified Respiratory Questionnaire (VSRQ)

The Living with COPD Questionnaire

We extracted data from 70 papers in total. None of the studies mentioned specific efforts to include people with low literacy skills or learning disabilities. Three studies excluded these groups explicitly, or based on eligibility criteria alone. For example Aaron et al. [15] excluded those with cognitive impairments. Thirty-two studies had implicit exclusion only, relating to non-inclusive recruitment and questionnaire administration processes. For instance Schofield and Mishra [16] recruited study participants through postal packages. People with low literacy skills or learning disabilities may not be able to read or comprehend such packages excluding them from the study at the outset; other examples included recruitment through newspaper advertisements [17] and the requirement for self-administration [18], which was the most common reason for classifying studies as having implicit exclusion. Sixteen studies had both explicit and implicit exclusion. Finally, 19 studies were unclear about recruitment or administration processes so we could not verify whether they excluded people with low literacy skills or learning disabilities (Table 2).

Table 2

Data extraction results

Study	PROMs involved	Explicit exclusion	Implicit exclusion	Unclear or no evidence of exclusion	Notes
Aaron et al. [15]	CRQ	X			• Excluded those with cognitive impairments
Aguilaniu et al. [37]	DIRECT		X		• Self-administration
Aiken et al. [19]	SF-36		X		• Proxy responses not accepted
Ashmore. et al. [38]	SF-36		X		• Self-administration
Berry et al. [39]	CRQ		X		• Recruited through media advertising • Self-administration
Bauldoff et al. [40]	SGRQ	X			• Excluded those with inability to read or cognitive impairments
Benzo et al. [41]	SF-36		X		• Self-administration
Brightling et al. [42]	CRQ			X
Brown et al. [43]	SF-36, SGRQ		X		• Self-administration
Carr et al. [44]	CRQ	X	X		• 19/234 were excluded due to cognitive impairments • Self-administration • Excluded those who could not communicate clearly
Carrieri-Kohlman et al. [45]	CRQ, SF-36			X
Coultas et al. [46]	SGRQ, SF-36		X		• Recruited through postal mail-outs
Crockett et al. [47]	SF-36	X	X		• Excluded those who were not mentally able to complete questionnaires • Postal questionnaires
de Torres et al. [48]	SF-36, SGRQ, CRQ	X	X		• Excluded those who could not complete questionnaires • Self-administration
Desikan et al. [49]	SF-36, SGRQ and CRQ		X		• Recruited through postal mail-outs
Dodd et al. [50]	CAT		X		• Self-administration
Eaton et al. [51]	CRQ	X	X		• Excluded those who could not complete questionnaires • Self-administration
Fan et al. [52]	SGRQ			X
Gross et al. [53]	SGRQ			X
Gupta et al. [54]	CRQ		X		• Self-administration
Guyatt et al. [55]	CRQ			X
Guyatt et al. [56]	CRQ			X
Harper et al. [57]	CRQ, SGRQ, SF-36, EQ-5D		X		• Self-administration
Hazell et al. [58]	EQ-5D		X		• Self-administration • Recruitment and questionnaire completion through the post
Hinchcliff et al. [59]	FACIT-Dyspnea Scale		X		• Self-administration
Johnson et al. [60]	SGRQ			X
Jones et al. [61]	SGRQ			X
Jones and Bosh [62]	SGRQ, SF-36		X		• Self-administration
Jones et al. [63]	CAT			X
Jones et al. [64]	CAT		X		• Self-administration
Jones et al. [65]	CAT			X
Leidy et al. [66]	EXACT	X	X		• Excluded those with an inability to read or cognitive limitations • Self-administration
Mahler and Mackowiak [67]	SF-36		X		• Self-administration
Maleki-Yazdi et al. [17]	SGRQ		X		• Self-administration • Recruitment through media advertising
Martin [68]	CRQ		X		• Self-administration
McGlone et al. [69]	SGRQ	X			• 4 were excluded because they were unable to read
McKenna et al. [70]	Living with COPD Questionnaire	X	X		• Excluded people who could not read or respond to questionnaires • Self-administration
Miller et al. [71]	CRQ, SF-36		X		• Self-administration
Nagata et al. [72]	CAT	X	X		• Excluded those who could not read or understand the questionnaire • Self-administration
Okubadejo et al. [73]	SGRQ	X	X		• Excluded those with an inability to read • Self-administration
Pakhale et al. [74]	McGill COPD Quality of Life Questionnaire			X
Partridge et al. [25]	Capacity of Daily Living during the Morning Questionnaire		X		• Self-administration
Perez et al. [75]	VSRQ			X
Petrillo and Cairns [76]	EXACT	X	X		• Self-administration • 2 removed for lack of comprehension
Puhan et al. [77]	CRQ, SGRQ, SF-36	X	X		• 2 were excluded because they were unable to read • Self-administration
Punekar et al. [78]	EQ-5D		X		• Self-administration
Ries et al. [79]	SGRQ, SF-36		X		• Self-administration
Ringbaek et al. [23]	CAT	X	X		• Excluded those who could not read • Self-administration
Roth et al. [80]	SGRQ, SF-36			X
Schofield and Mishra [16]	SF-36		X		• Self-administration • Recruitment through postal mail-outs
Schunemann et al. [81]	CRQ, SGRQ	X	X		• Excluded those with cognitive limitations • Self-administration
Schunemann et al. [82]	SGRQ, CRQ	X	X		• Excluded those with cognitive limitations • Self-administration
Schunemann et al. [83]	CRQ, SGRQ, SF-36	X	X		• Excluded those with cognitive limitations • Self-administration
Sewell et al. [84]	CRQ		X		• Self-administration
Sewell et al. [85]	CRQ	X	X		• Excluded patients with cognitive problems • Self-administration
Singh et al. [18]	CRQ, SGRQ		X		• Self-administration
Sprenkle et al. [21]	SF-36		X		• Self-administration • Done by postal recruitment/survey
Stapleton et al. [86]	SGRQ	X	X		• Excluded people with “mental status that precluded participation” • Self-administration
Tashkin et al. [87]	SGRQ			X
Tonnel et al. [88]	VSRQ			X
Tsai et al. [89]	CRQ			X
Viramontes and O’Brien [90]	SF-36			X
Williams et al. [91]	CRQ		X		• Self-administration
Williams et al. [20]	CRQ		X		• Self-administration
Wilson et al. [92]	SGRQ, SF-36		X		• Self-administration
Wyrwich et al. [22]	SF-36			X
Xue et al. [93]	CAT		X		• Recruitment through media advertising
Yeo et al. [94]	SGRQ		X		• Self-administration
Yohannes et al. [95]	CRQ			X
Yount et al. [24]	FACIT-Dyspnea Scale		X		• Self-administration

CAT COPD Assessment Test, COPD chronic obstructive pulmonary disease, CRQ Clinical Respiratory Questionnaire, DIRECT Disability Related to COPD Tool, EXACT EXAcerbations of Chronic Obstructive Pulmonary Disease Tool, FACIT Functional Assessment of Chronic Illness Therapy, PROM patient-reported outcome measure, SQRQ St George’s Respiratory Questionnaire, VSRQ Visual Simplified Respiratory Questionnaire

We also recorded other notable issues in the studies concerning people with low literacy skills and learning disabilities. Some of the studies’ protocols may have given consideration for these groups. For example, Aiken et al. [19] used telephone interviews to administer the PROM, although they did not accept proxy responses. Williams et al. [20] provided an opportunity for participants to clarify the self-administered questionnaire at the end. Some studies also described the sample composition, which could give an indication of the prevalence of low literacy. In Sprenkle et al. [21], the majority of participants had minimum high school education, while 26 % of Wyrwich et al.’s [22] sample had a grade school education only. However, the latter study did use telephone interviews. Ringbaek et al. [23] found patients needed a lot of assistance to complete PROMs, particularly those with less education, and especially while attempting the SGRQ. Yount et al. [24] were explicit about their sample’s high education level and used a touch screen to administer the FACIT-Dyspnea Scale. Technology was also used to send automatic alerts through an e-diary to support patients to complete the Capacity of Daily Living during the Morning Questionnaire [25].

Discussion

We have found that people with low literacy skills or learning disabilities are excluded from the development of PROMs for COPD in two distinct ways: explicitly through the study eligibility criteria and, of potentially greater significance, implicitly through recruitment or administration processes. These results indicate that these groups may struggle to complete PROMs. Healthcare initiatives like quality improvement for COPD that depend on PROMs may exclude people with low literacy skills or learning disabilities as a result.

Implicit exclusion is where researchers and readers may not realize exclusion takes place; we found implicit exclusion was most common largely due to the majority of studies’ requirement for self-administration of the PROM. Almost all PROMs are only validated to be self-administered but their readability and administration may be more or less inclusive. For example, technology support or easy read formatting can enhance accessibility. While some studies did use technology [24, 25], none of the papers we reviewed included discussion of the groups they potentially excluded in the recruitment processes they described. This could partly reflect the word limitations imposed on authors, but at least to some extent indicates a lack of critical awareness of the potential for exclusion. Where the authors did highlight heterogeneity in their sample demographic [24], we could not find specific mention of approaches to include people who may have low literacy skills or learning disabilities. There was also a lack of description of efforts to accommodate specific support to complete PROMs, for example through assisted completion. While other authors have found substantial disagreements between self- and proxy reporting especially to collect subjective information [26-28], the valid provision of assistance in completing PROMs has proved useful in population survey research, for example in Kaye [29].

Identifying and distinguishing forms of exclusion in research to develop PROMs is a prerequisite for future research aiming to make PROMs practice more inclusive. We consider implicit exclusion to be more concerning than explicit exclusion because it implies a lack of awareness of the potential for inequality in resulting healthcare and research practices. Studies with explicitly stated criteria excluding people with low literacy skills or learning disabilities set a clear barrier preventing them from participating. This suggests awareness on the part of the researchers and gives clear notification to readers that the findings apply only to the included population groups, provided this caveat is not overlooked when discussing generalizability [30]. Implicit exclusion practices have a higher likelihood than explicit exclusion to result in barriers to inclusive participation without any realization. The more recent increased focus on equality and diversity issues in research may improve future development practices compared with those used to develop the PROMs we identified. Inclusion might be given a higher priority and researchers given more support to overcome inevitable time, skill, and funding constraints that can limit feasibility to include people with low literacy skills or learning disabilities in their work. But if we do not realize there are barriers, we will not recognize the need for efforts to be more inclusive. In a current policy environment with increasing interest in PROMs [10], the potential result is unequal access to quality improvement processes.

Specific efforts can support the inclusion of people with low literacy skills or learning disabilities in research to develop PROMs. Research funders can confront issues of inequality by attaching equality criteria as a condition for funding the development of a PROM (similar to the requirement from the National Institutes of Health in the USA [31] to justify not making efforts to include groups normally left out of research). Responsibility among researchers to think through the implications of their methods and practices for people’s ability to participate in their study may also be helpful. This may involve consulting service users in the commissioning, design, and conduct of research to determine best practice options for PROM development including when (timeliness), where (setting), and how (visual assistance) they can be administered, and any required assistance [32]. Other literature outlines issues and provides guidance to conduct research with people with learning disabilities [33], and tools to aid communication also exist [34]. Guidance on increasing the accessibility of information including the need for ample white space and large font sizes [35] are also easily applicable to typical research documents like information leaflets, consent forms, and PROM design. Our findings also suggested some promising results to build on. The e-diary with automatic alerts used to administer the Capacity of Daily Living during the Morning Questionnaire [25] and the touch screen used to administer the FACIT-Dyspnea Scale [24] may be promising as technology-based methods to support PROMs administration for people with low literacy skills or learning disabilities. These are similar to previous efforts to administer PROMs to people with low literacy skills in a cancer setting using an audiovisual touch screen [36].

Our work is a first step in highlighting the presence of exclusive practices in the development of PROMs and it has limitations. The findings have to be interpreted in the light of word limits imposed on article length. We did not infer sample demographics, recruitment, or administration practices where they were not explicitly stated in the studies and authors’ descriptions of these may have in turn been limited by journal word counts. The data extraction was partly subjective as we sometimes assumed terms used by the authors to describe people who are not eligible for their study (like excluding those with ‘cognitive impairments’) would exclude someone with low literacy skills or learning disabilities. However, the extraction was checked by a second researcher to ensure consistency. Finally, we recognize that although people with low literacy skills and learning disabilities should not be ‘lumped together’ as one group, both groups face similar practical and functional challenges with reading and understanding written material. Other groups may also face similar exclusion from PROM development and validation processes, and substantial difficulties with PROMs, for example people with dementia or people with acquired brain injuries resulting in cognitive impairments.

Conclusion

While the use of PROMs is growing as a major part of healthcare quality improvement internationally, we have demonstrated that, at least in relation to COPD, people with low literacy skills or learning disabilities might be left out because they are not included in the research to develop PROMs. Most exclusion is implicit, suggesting the potential for lack of awareness of exclusionary practices. To prevent this form of exclusion in the future, efforts on the part of funders and researchers and more research into validating PROMs to be used with support through technology-based accessible completion or assisted completion are important. Ultimately PROMs are a promising opportunity to involve patients in quality improvement for COPD healthcare but the exclusive nature of PROM development may further alienate people with low literacy skills and learning disabilities from healthcare practices.