emPHasis-10: development of a health-related quality of life measure in pulmonary hypertension.

The aim of this study was to develop a measure of the impact of pulmonary hypertension (PH) on health-related quality of life (HRQoL) as there is a need for a short, validated instrument that can be used in routine clinical practice. Interviews were conducted with 30 PH patients to derive 32 statements, which were presented as a semantic differential six-point scale (0-5), with contrasting adjectives at each end. This item list was completed by patients attending PH clinics across the UK and Ireland. Rasch analysis was applied to identify items fitting a uni-dimensional model. 226 patients (mean age 55.6±14 years; 70% female) with PH (82% had pulmonary arterial hypertension) completed the study questionnaires. 10 of the 32 items demonstrated fit to the Rasch model (Chi-squared 16; p>0.05) and generated the emPHasis-10 questionnaire. Test-retest (intraclass correlation coefficient 0.95, n=33) and internal consistency (Chronbach's α=0.9) were strong. emPHasis-10 scores correlated consistently with other relevant measures and discriminated subgroups of patients stratified by World Health Organization functional class (ANOVA F=1.73; p<0.001). The emPHasis-10 is a short questionnaire for assessing HRQoL in pulmonary arterial hypertension. It has excellent measurement properties and is sensitive to differences in relevant clinical parameters. It is freely available for clinical and academic use.

Introduction

Pulmonary hypertension (PH) has many causes, and is a debilitating and progressive condition that can affect people of any age and shortens life expectancy [1, 2]. It ranges from severe elevations of pressure, seen in the context of pulmonary arterial hypertension (PAH), for which drug therapies are available, and chronic thromboembolic PH (CTEPH), which may be cured by surgery, to relatively mild elevations of pressure seen in the setting of respiratory and cardiac disease, where treatment is principally aimed at the underlying condition [3].

PH is characterised by breathlessness, decreased exercise tolerance, psychological distress and lack of energy and fatigue, resulting in a reduced health-related quality of life (HRQoL) [4-7]. As there is no cure, except in selected patients with CTEPH, current management strategies aim to prolong survival, alleviate symptoms and improve HRQoL. Given the central importance of HRQoL as a goal of therapy, a tool capable of accurately quantifying this in both the clinical and trial environments would be highly desirable.

The routine collection of valid and reliable HRQoL data necessitates the availability of a short, simple, patient-reported outcome to measure the impact that PH has on patients’ lives. Available patient-reported outcomes frequently used in PH are often multi-dimensional and include the Cambridge Pulmonary Hypertension Outcome Survey (CAMPHOR) [8], medical outcome study 36-Item Short Form [9], the Nottingham Health Profile [10] and the Minnesota Living with Heart Failure Questionnaire (MLHFQ) [11]. The CAMPHOR is the only currently available PH-specific questionnaire and has been validated in many countries to account for differences in language and culture [12-14]. Whilst these tools have demonstrated validity and reliability for use in a research context [15, 16], they have limited clinical utility. In addition, the multi-dimensional nature of a number of these tools makes the weighting process more complex in routine clinical practice.

The aim of this multicentre study was to develop and assess the validity of an easily and rapidly administered, simple-to-score and interpret, disease-specific questionnaire to assess HRQoL in a large population of patients with PH, with the aim of developing a tool to use in the clinic to assess patients with PAH and CTEPH.

Methods

The study was approved by the NRES Committee East of England (Hertfordshire, UK; 11/EE/0345), Mater Misericordiae University Hospital (Dublin, Ireland; 1/378/1493), and the University of Manchester (Manchester, UK). Patients provided written informed consent prior to study entry. Patients were recruited from nationally designated specialist PH centres in the UK and Ireland, and were approached at the time of their routine clinic visit or identified from the database of the Pulmonary Hypertension Association UK (PHA-UK).

Inclusion criteria required a confirmed diagnosis of PH with a mean pulmonary artery pressure (PAP) of ≥25 mmHg at cardiac catheterisation and required detailed assessment to allow accurate classification of PH as per international guidelines [17]. Exclusion criteria included age <18 years and a history of previous successful pulmonary endarterectomy with normalisation of PAP.

Time from diagnosis was taken as time from right heart catheterisation unless the patient had congenital heart disease, in which case time of diagnosis was taken as when the patient was diagnosed at the PH referral centre as having PH. Targeted pulmonary vascular therapies were recorded at the time of questionnaire completion. The 6-min walk distance (6MWD) or incremental shuttle walking test (ISWT) and World Health Organization (WHO) functional class at the time of administration of the questionnaires were recorded.

Stage 1: item generation for inclusion in questionnaire

A qualitative study was conducted with 30 patients to explore the concept of HRQoL and their experience of living with PH. Five of these patients also conducted self-videoing for a minimum of 5–10 min every day for a total of 7 days. Participants were free to record any aspects of living with PH that was important to them. Following the extraction of a draft item list, cognitive debriefing interviewing was employed [18] with 14 patients and two family members to ensure that all items were clear and easy to understand. Items were presented to patients as a semantic differential six-point scale (0–5), anchored at each end by contrasting adjectives. The draft item list and scaling range was reviewed by clinical experts for clinical relevance.

Stage 2: item reduction

Stage 2 involved the administration of the draft item list to patients with PH. We also collected participants’ WHO functional class, 6MWD (or ISWT) and demographic details at the time of enrolment in the study. All patients were also requested to complete the 32-item list and the following questionnaires. 1) The MLHFQ modified for PH is a modified version [19] of the original MLHFQ [11]. The MLHFQ-modified for PH has been found to be highly reproducible and demonstrated internal consistency and reliability, and moderately good validity comparable with the values found with the CAMPHOR [20]. 2) Dyspnoea-12 (D-12) consists of 12 items that reflect the physical perception and emotional effects of dyspnoea and has been validated for use in PH [21, 22]. The instrument uses simple summation scoring to yield scores from 0–36, with higher scores corresponding to greater impairment. 3) The Hospital Anxiety and Depression scale (HAD) assesses psychological distress (seven items for anxiety with a score of 0–21, and seven items for depression with a score of 0 –21; higher scores indicate greater emotional distress) [23].

Statistical analyses

Item reduction process

A hierarchical approach to item reduction was used followed by the application of Rasch analysis [24, 25]. The aim was to develop a uni-dimensional instrument that captured a wide range of PH experiences using the least number of items required. The process of identifying items for potential deletion was iterative and included items with >25% missing responses and items that demonstrated significant age or sex bias (p<0.05). To remove the least discriminative descriptors, items demonstrating floor (>25% endorsement) or ceiling effects (>25% endorsement) were removed. Items were also marked for potential deletion if they demonstrated high item–item correlations (r>0.7) and low item–total correlations (r<0.3) [26].

Items that survived hierarchical reduction were scrutinised with a Rasch uni-dimensional model (RUMM2030; www.rummlab.com). Rasch models provide a template for testing how well each item contributes to the concept being measured [24, 27].

Individual item fit was assessed by examining the residual and Chi-squared fit statistic for each item (item residual ±2.5 and Chi-squared p-value >0.05) [27]. Items with the worst model fit were removed whilst ensuring that the balance of retained items and content validity for the total item-set was retained. The overall fit of the final item set was determined by examining the person item trait interaction Chi-squared statistic, where a nonsignificant p-value (>0.05) indicates fit to the model [27]. See the online supplementary material for details.

Stage 3: preliminary reliability and validity of the final item list

We tested the preliminary reliability and validity of the final item set using the same data set as in stage 2. Internal consistency was tested using Cronbach’s α, for which values from 0.7–0.9 are acceptable [28].

Concurrent validity was assessed by evaluating correlations between the newly developed questionnaire and the MLHFQ-modified for PH, D-12, HAD and 6MWD. 66 patients attending the Sheffield Pulmonary Vascular Disease Unit (Sheffield, UK) who performed the ISWT did not perform a 6MWD and were excluded from this part of the analysis. Associations with WHO functional class were calculated using ANOVA.

Test–retest reliability of the finalised item list was tested in a different group of patients, using a postal survey sent to members of PHA-UK. Participants were sent two copies of the final item list to complete 7 days apart. Each copy of the questionnaire was placed in a separate sealed envelope clearly marked with the date for completion (day 0 and day 7, respectively), and participants received written and telephone directions for completion. Test–retest reliability was tested using intraclass correlation coefficient for which values >0.7 indicated good reliability [29].

Results

Stage 1: item generation for questionnaire

30 patients (table 1) participated in the qualitative interview work that yielded a preliminary list of 32 items. Items covered breathlessness, activity and social limitations, lack of energy, emotional issues, and treatment-related issues. Cognitive interviewing with 14 patients and two family carers resulted in only minor adjustments, and participants indicated that the items and response scale used were relevant and easy to understand. PH clinicians confirmed the clinical relevance of the draft instrument.

Table 1–

Participant characteristics: qualitative phase

Subjects	30
Females/males	18/12
Age years	56.3±38 (26–80)
Aetiology of PH
Group 1: PAH
Idiopathic	12
Connective tissue disease	7
Congenital heart disease	5
Heritable	1
Portal hypertension	1
Drugs/toxins	1
Group 3: owing to respiratory disease in PH lung	1
Group 4: CTEPH	2
Time since PAH diagnosis years
<1	2
1 to <3	2
3 to <5	7
5 to <10	12
>10	7
WHO functional class
I	0
II	9
III	19
IV	2

Data are presented as n or mean±sd (range). PH: pulmonary hypertension; PAH: pulmonary arterial hypertension; CTEPH: chronic thromboembolic PH; WHO: World Health Organization.

A total of 226 patients with PH (mean age 55±14 years; 69% female) participated in stage 2 (table 2).

Table 2–

Participant characteristics: item reduction phase

Patients n	226
Females	157 (70)
Age years	55.6±14
Aetiology of PH
Group 1: PAH
Idiopathic	89 (40)
Congenital heart disease	50 (22)
Connective tissue disease	43 (19)
Heritable	3 (1)
Portal hypertension	1 (0.5)
Group 2: PH left heart disease	1 (0.5)
Group 3: PH lung	1 (0.5)
Group 4: CTEPH	36 (16)
Group 5: neurofibromatosis and sarcoidosis	2 (1)
PH therapy
Monotherapy	125 (55)
Dual therapy	78 (35)
Triple therapy	11 (5)
Not recorded	12 (5)
Route
Inhaled iloprost	20 (9)
i.v. iloprost	8 (4)
i.v. treprostinil	5 (2)
i.v. epoprostenol	4 (2)
Continuous or as needed oxygen use	87 (38)
6MWD m#	336±130
Employment status
Full-time work	25 (11)
Part-time work	31 (14)
Student	2 (1)
Unemployed/retired	168 (74)
WHO functional class
I	3 (1)
II	73 (32)
III	115 (51)
IV	34 (15)
MLHFQ modified for PH	47.8±25
Hospital Anxiety and Depression scale
Total	13.3±7.5
Anxiety	6.9±4.4
Depression	6.3±3.9
Dyspnoea-12 questionnaire
Total	12.8±9.8
Physical	8.5±5.8
Affective	4.4±4.6

Data are presented as n (%) or mean±sd, unless otherwise stated. PH: pulmonary hypertension; PAH: pulmonary arterial hypertension; CTEPH: chronic thromboembolic PH; 6MWD: 6-min walk distance; WHO: World Health Organization; MLHFQ: Minnesota Living with Heart Failure Questionnaire. #: n=98, patients attending the Sheffield Pulmonary Vascular Disease Unit, Sheffield, UK (n=66) underwent exercise testing with the incremental shuttle walking test and did not perform the 6MWD.

A complete summary of the item reduction can be found in the online supplementary material. The full scaling range (0–5) for all 32 items was used (mean item responses ranged from 2.04±1.5 for “being breathlessness never interrupts my conversations” to 3.88±1.3 for “when I walk up one flight of stairs I am not breathless”). All items had <5% missing responses. Sex bias was significant in two items (one relating to feeling “anxious” and the other to feeling “unattractive” because of PH), and age bias was significant in three items; all five of these biased items were removed. Three items demonstrated a high degree of floor effects (25–42%) and four items demonstrated a high degree of ceiling effects (26–30%); these seven items were removed. The item “walking up a flight of stairs makes me breathless” demonstrated a ceiling effect (38%); however, this item had strong face validity during the item generation stage and was retained to enable further scrutiny with Rasch analysis.

Item–item correlations and item–total correlations were assessed with the remaining 20 items. Eight items were marked for possible removal due to high item-to-item correlations. Each of these items correlated with more than one other item. Based on the frequency of an item’s high correlation with other items and our previously conducted qualitative work, four of these items were removed at this stage, thus ensuring that a balance between statistical significance and face validity was maintained. One item (“I feel dizzy or lightheaded”) was removed due to a low item–total correlation.

Following the above tests, 17 items were deleted with the remaining 15 items entered into RUMM2030 for Rasch analysis. During the iterative process of analysing the 15 items, five items were removed to achieve an overall fit to the uni-dimensional Rasch model. This resulted in a 10-item solution demonstrating a good Rasch model fit for individual items (see online supplementary material) and the 10 items in aggregate (Chi-squared 16.2, p=0.7). There was a good distribution of the item scores across the severity scale (fig. 1). The 10 items form the emPHasis-10 with a total score range from 0 to 50 (fig. 2).

Figure 1–

Distribution of a) patients and b) items based on Rasch logit locations showing the distributions of patient health-related quality of life severity and item severity (locations) along the same linear scale measured in logits. Most items are located between -2 and +2 logits. Total n=226, mean±sd 0.200±1.334.

Figure 2–

The final emPHasis-10 questionnaire. The Pulmonary Hypertension Association UK (Rotherham, UK) intends to make this questionnaire free to use by both the clinical and academic communities. It can be downloaded free of charge from the Pulmonary Hypertension Association UK website (www.phassociation.uk.com) and reproduced without cost for clinical and academic use. Different language versions will be made available in due course. Reproduced with permission from the Pulmonary Hypertension Association UK.

Stage 3: preliminary reliability and validity

emPHasis-10 demonstrated excellent internal reliability (Chronbach’s α=0.9). 33 patients completed the test–retest study and emPHasis-10 demonstrated good stability over time (intraclass correlation coefficient 0.95).

The emPHasis-10 demonstrated excellent concurrent validity with related patient reported outcome measures including the MLHFQ-modified for PH (r=0.61), HAD total (r=0.77) and D-12 (r=0.74) (all p<0.001). There was a moderate correlation between the emPHasis-10 and 6MWD (r= -0.40, p<0.001). emPHasis-10 scores discriminated subgroups of patients stratified on WHO functional class II and III (mean difference 10.9, 95% CI 7.3–14.5; p<0.001) (fig. 3). There were insufficient numbers of patients in WHO functional class I and IV to draw any conclusive results.

Figure 3–

Mean emPHasis-10 scores for each World Health Organization (WHO) functional class. Error bars represent 95% confidence intervals.

Discussion

We have developed emPHasis-10, a rapidly administered, simple-to-score, disease-specific questionnaire to assess HRQoL in PH. The final 10 items are formatted as a semantic six-point differential scale, a format that has previously proven to be easy to administer and easy for patients to complete in the clinical setting [30]. We have demonstrated that emPHasis-10 scores correlate strongly with measures of HRQoL, dyspnoea and psychological distress. Although the emPHasis-10 has primarily been developed for evaluative purposes, this study shows that the emPHasis-10 also has strong discriminative measurement properties according to WHO functional class; emPHasis-10 scores increased significantly as functional class declined. The performance characteristics of emPHasis-10 support the validity of this tool for assessing the impact of PH on the lives of patients with PAH.

Important components of the impact of PH are covered in emPHasis-10 including breathlessness, fatigue and lack of energy, social restrictions, and concerns regarding effects on patient’s significant others, such as family and friends [4-7]. Despite this, the emPHasis-10 fits a uni-dimensional model where each question carries a similar weighting. Many existing HRQoL instruments are multidimensional, comprising separately scored sub-domains that are often lengthy with complex scoring algorithms. A concern with many multidimensional instruments is the reporting of a total HRQoL score when the totality of its parts might not necessarily provide a valid overall score of HRQoL [26]. Our primary objective was to develop an overall measure of PH HRQoL with a simple scoring system that does not require complex analysis and interpretation.

In developing this instrument we used Rasch modelling to facilitate the development of a questionnaire that provides measurement of HRQoL using a parsimonious collection of items that form a uni-dimensional measure. HRQoL is generally acknowledged to be a multidimensional construct. The emPHasis-10 is compatible with this view since it contains items that cover a range of experiences (e.g. energy, breathlessness, social confidence and independence). The 10 items contribute reliably to the measurement of HRQoL in PH. To do this, we required all items to fit a uni-dimensional model whereby the underlying trait (i.e. HRQoL) has a single dimension on which all of the test items rely. This process may have excluded items that may have behaved differently with different measurement properties. Our approach created a pragmatic instrument with clinical utility.

There are a number of limitations in this study. The population studied had various forms of PH and are representative of the typical patients under chronic follow-up in a UK PH centre. The vast majority had PAH, where we feel this questionnaire is likely to have most value. Further studies need to be performed in other important forms of PH such as CTEPH, given the relatively small numbers in this group. The questionnaire is not currently recommended for use in other forms of PH such as PH owing to left heart disease or respiratory disease, given the small number of patients included in this study. The patients who were interviewed were at different points on the illness trajectory with many having relatively long standing disease and it is likely that the duration of disease will impact on HRQoL. The current study presents cross-sectional data; a prospective longitudinal study would be required to test the sensitivity of the instrument to change overtime. We used data from the item reduction stage (stage 2) to test concurrent validity of the emPHasis-10 with other relevant outcomes. These preliminary results require subsequent validation in a separate study. We have compared emPHasis-10 with a number of other questionnaires but not to CAMPHOR; unfortunately, at the time this study was conducted permission to use CAMPHOR was declined. Currently, emPHasis-10 is only available in English but the PHA-UK is in the process of making it available in a number of different languages.

In conclusion, the emPHasis-10 is a short, simple questionnaire for assessing HRQoL in PH. It has excellent measurement properties and construct validity, and is sensitive to differences in clinical parameters. Further work is required to evaluate its value in routine clinical practice and to assess the impact of treatment interventions in various form of PH. A number of HRQoL questionnaires are not freely available and various copyright laws preclude their use by both clinicians and academics. The PHA-UK has funded this study with the specific intention of making emPHasis-10 freely available for clinical and academic use.