Pulmonary Arterial Hypertension-Symptoms and Impact Questionnaire: feasibility of utilizing one-day versus seven-day symptom reporting.

Patient-reported outcomes are important measures to include in pulmonary arterial hypertension clinical trials but are not widely utilized in clinical practice. Pulmonary Arterial Hypertension-Symptoms and Impact Questionnaire (PAH-SYMPACT) is the only pulmonary arterial hypertension-specific patient-reported outcomes instrument developed and validated in accordance with the US Food and Drug Administration guidance on patient-reported outcomes development. The PAH-SYMPACT tool measures pulmonary arterial hypertension-related symptoms and impact of pulmonary arterial hypertension on daily life. Symptoms are reported each day for seven consecutive days, and the impact of pulmonary arterial hypertension over one week is recalled and reported on day 7; however, daily symptom reporting may overburden patients and healthcare resources, limiting the practicality of PAH-SYMPACT outside of clinical trials. To determine the practicability of an abridged version of PAH-SYMPACT for which all reporting is completed on one day, symptom data from the SYMPHONY trial (NCT01841762; PAH-SYMPACT validation study) were retrospectively analyzed to assess whether symptoms reported on each day correlated with the weekly average and whether one-day symptom scores were sensitive to disease severity. Correlation coefficients comparing the weekly average and individual day symptom scores were mostly high or very high regardless of the day they were measured. Findings were similar when using either Spearman's rank correlation or weighted kappa method. One-day symptom scores differentiated well between World Health Organization functional classes II and III/IV pulmonary arterial hypertension and were sensitive to change in disease severity as measured by the Patient Global Assessment of Disease Severity. These data suggest that the one-day PAH-SYMPACT is feasible and appropriate for routine implementation in clinical practice.

Introduction

Pulmonary arterial hypertension (PAH) is a debilitating condition that can severely impact patients' abilities to carry out daily activities, with symptoms becoming more severe—and even fatal—as the disease progresses.[1,2] A key goal of PAH therapy is to improve symptoms and thereby patient quality of life.[3] The utilization of validated PAH-specific instruments for clinical use is increasingly considered essential to evaluate the effect of PAH and treatments on symptoms and quality of life and for the holistic management of patients with PAH.[4-7] There is evidence to suggest that quality of life may be associated with outcomes in patients with PAH[8] and that PAH treatment can improve health-related quality of life.[9] PAH-specific patient-reported outcomes (PROs) instruments, including Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire (PAH-SYMPACT)[10] and emPHasis-10,[11] have been developed and are being validated to provide a comprehensive evaluation of the health status and symptoms of patients with PAH.

PROs are recommended as end points in clinical trials, providing subjective information on the impact of disease symptoms and therapeutic interventions on patient lives that are not captured by objective clinical end points.[12-16] Instruments that measure PROs are used to inform treatment decisions in many disease settings, such as urologic diseases[17] and left-sided heart failure[16,18]; however, PROs are not used routinely in the PAH patient population. PAH-specific PRO instruments have been developed to provide a comprehensive evaluation of the health status and symptoms of patients with PAH that cannot be captured by generic quality-of-life instruments.[10,11,19,20]

PAH-SYMPACT is a new PAH-specific instrument developed and validated in accordance with guidance on PRO development from the US Food and Drug Administration (FDA).[10] In its original form, the 11 symptom items are reported on seven consecutive days and averaged, while the 11 impact items are reported on day 7 with a one-week recall period.[10] Although daily reporting of symptoms may be both beneficial and practical in the clinical trial setting, it may be impractical to incorporate into routine clinical practice. The objective of this analysis was to determine the practicability of a one-day version of the PAH-SYMPACT that enables patients to report their symptoms on one day instead of daily for seven days. We conducted a retrospective analysis of the PAH-SYMPACT symptom data from the SYMPHONY trial to assess (1) whether symptom scores reported on a single day correlated with the weekly average of symptom scores collected daily over seven days and (2) the sensitivity of one-day symptom scores to differentiate between patients with different disease severity and their ability to detect improvement or worsening in the severity of disease.

Methods

This is a retrospective analysis of the previously described SYMPHONY trial, a prospective, open-label, single-arm, multicenter, phase 3b study in patients with PAH who received the endothelin receptor antagonist macitentan 10 mg once daily (NCT01841762).[21] Eligible patients were 18–80 years of age with PAH as classified by the fourth World Symposium on Pulmonary Hypertension (idiopathic; heritable; drug- or toxin-induced; or associated with connective tissue disease, congenital heart disease with simple systemic-to-pulmonary shunt at least one year after surgical repair, or HIV infection),[22] had World Health Organization (WHO) functional class (FC) II to IV PAH, a six-minute walk distance of 150 m or more, and could be receiving stable doses of inhaled prostacyclin analogs or phosphodiesterase type 5 inhibitors. SYMPHONY comprised a 20-week study period: two weeks for screening, a two-week baseline period, and a 16-week treatment period. An electronic version of the PAH-SYMPACT instrument (ePRO)[10] was completed by screened patients twice during the baseline period (week –2 to week –1 and from week –1 to week 0) and twice in the treatment period at weeks 8 and 16 using a tablet (ePRO 1–4; Fig. 1). The draft PAH-SYMPACT completed by patients in SYMPHONY included a yes/no question about oxygen use assessed daily, 16 symptom items with a 24-h recall period assessed daily, and 25 impact items with a seven-day recall period assessed at the end of the week. Item reduction, content validation, and psychometric validation were performed to produce a final validated PAH-SYMPACT that comprises 11 impact questions (reported on day 7 with a one-week recall period) and 11 symptom questions (reported daily for seven days with a 24-h recall period) (Figs 1 and 2). The question regarding oxygen use was recorded but not used as part of the domain score.[21] Each PAH-SYMPACT symptom item was scored using a five-point Likert scale, with a score of 0 for no effect, 1 for mild, 2 for moderate, 3 for severe, and 4 for very severe.

Fig. 1.

PAH-SYMPACT symptom assessment in SYMPHONY (adapted from).

Source: reproduced with permission from Chin et al., 2018.[21]

PAH-SYMPACT: Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire; ePRO, electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.

Fig. 2.

PAH-SYMPACT domains.

Source: reproduced with permission from Chin et al., 2018.[21]

PAH-SYMPACT: Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire; PAH: pulmonary arterial hypertension.

SYMPHONY was conducted in accordance with the amended Declaration of Helsinki. Local institutional review boards or independent ethics committees at each participating institution approved the protocol, and all patients provided written informed consent.

Retrospective analysis

Using data for the 11 symptom questions over the four study periods (ePRO 1–4), the symptom domain scores collected each day (days 1–7) were compared with the weekly average using descriptive statistics. Any correlation between the weekly average and the one-day scores was assessed using Spearman's rank correlation (which allows determination of the correlation between two variables/assessments that have a non-normal distribution[23]), using the mean score for each symptom domain per day. A correlation coefficient of 0.9 to 1.0 (−0.9 to −1.0) indicates a very high positive (negative) correlation, 0.7 to 0.9 (−0.7 to −0.9) a high positive (negative) correlation, 0.5 to 0.7 (−0.5 to −0.7) a moderate positive (negative) correlation, 0.3 to 0.5 (−0.3 to −0.5) a low positive (negative) correlation, and 0.0 to 0.3 (0.0 to −0.3) a negligible correlation.[23] The sensitivity of the Spearman's rank correlation was assessed using the weighted kappa method.[24] The correlation of scores between two days of the weekly average collection was also evaluated using Spearman's rank correlation.

The sensitivity of one-day scores to changes in disease severity was assessed using a t-test to compare scores for patients with FC II vs FC III/IV symptoms. Patient Global Assessment of Disease Severity (PGA-S, with response none = 1, very mild = 2, mild = 3, moderate = 4, severe = 5, and very severe = 6) was used to assess change in severity of disease (based on changes from baseline to week 8 and week 16); the sensitivity of one-day scores to this change was evaluated using an analysis of variance (ANOVA) model. PGA-S groups were included as the independent variable, and PAH-SYMPACT domain scores were included as the dependent variable. Pairwise comparisons between least squares means were performed using the Scheffe test and adjusted for multiple comparisons.

Results

As reported previously, 284 patients were enrolled in SYMPHONY between April 2013 and October 2015 at 71 centers in the United States.[21] Final analysis included 278 patients; six were excluded due to protocol deviations. Most patients had idiopathic (48.9%) or connective tissue disease-associated (39.9%) PAH. The majority of patients were female (78.8%), the median age was 59.7 years, most had WHO FC II (39.6%) or FC III (59.0%) symptoms, and 46.4% were receiving background PAH therapy.[21]

Correlation between the weekly average and one-day symptom scores

Correlations between daily symptom reports and the weekly average were computed for each day and grouped by symptom domain. Correlations between the weekly average and the mean one-day scores were mostly high (i.e. 0.70–0.9023) for both cardiopulmonary and cardiovascular domain symptoms across all four study periods and all seven daily reports (Table 1). The exceptions were chest pain on day 1 during ePRO period 1 (0.68) and 3 (0.67), chest pain on day 2 during ePRO period 3 (0.66), and chest pain on day 4 during ePRO period 3 (0.68). Correlations increased modestly as the week progressed from day 1 through day 7.

Table 1.

Spearman correlation coefficient ranges by symptom domains on days 1 through 7 for each period.

	ePRO period
Day and symptom domain	1	2	3	4
Day 1
Cardiopulmonary	0.778–0.862	0.857–0.894	0.777–0.886	0.831–0.910
Cardiovascular	0.677–0.775	0.775–0.843	0.667–0.834	0.721–0.835
Day 2
Cardiopulmonary	0.801–0.886	0.846–0.899	0.828–0.888	0.868–0.926
Cardiovascular	0.747–0.819	0.775–0.843	0.664–0.824	0.732–0.825
Day 3
Cardiopulmonary	0.824–0.890	0.836–0.904	0.838–0.910	0.837–0.922
Cardiovascula	0.765–0.804	0.776–0.835	0.719–0.812	0.761–0.839
Day 4
Cardiopulmonary	0.829–0.895	0.847–0.924	0.824–0.915	0.849–0.921
Cardiovascular	0.776–0.823	0.786–0.850	0.682–0.826	0.721–0.810
Day 5
Cardiopulmonary	0.800–0.896	0.855–0.912	0.831–0.898	0.877–0.926
Cardiovascular	0.815–0.837	0.784–0.873	0.752–0.829	0.745–0.846
Day 6
Cardiopulmonary	0.811–0.893	0.846–0.912	0.838–0.920	0.805–0.925
Cardiovascular	0.804–0.847	0.767–0.848	0.729–0.809	0.699–0.797
Day 7
Cardiopulmonary	0.832–0.881	0.825–0.914	0.809–0.906	0.832–0.925
Cardiovascular	0.756–0.840	0.794–0.856	0.771–0.836	0.769–0.840

ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.

Spearman correlation coefficients were also calculated between days of the week for each symptom during each period. In general, a high correlation was observed between any of the days within the seven-day period, with higher correlations between days that were closer together (data not shown). For example, for stomach area swelling during period 1, the correlation coefficients between days 1 and 2 (0.78) and days 6 and 7 (0.81) were higher than the correlation coefficient between days 1 and 7 (0.67).

In PAH-SYMPACT, the impact of symptoms is reported on a single day (day 7) with a weekly recall period. Single-day reporting of symptoms on the same day would provide the highest correlation with impact and offer the greatest clinical utility. Therefore, day 7 symptom scores were further analyzed. For the cardiopulmonary symptom domain, Spearman's correlation coefficients between day 7 scores and weekly average scores were all in the very high range (i.e. ≥0.90[23]). For the cardiovascular symptom domain, correlation coefficients between day 7 scores and weekly average scores ranged from 0.87 to 0.90, all in the high range (Table 2).[23]

Table 2.

Mean score in symptom domains. Spearman's correlation coefficients between weekly average and day 7.

	Cardiopulmonary symptoms		Cardiovascular symptoms
	Weekly	Day 7	Weekly	Day 7
ePRO period 1, n = 256
Mean (SD)	1.0 (0.5)	1.0 (0.6)	0.5 (0.5)	0.5 (0.6)
Correlation coefficient	0.919		0.872
ePRO period 2, n = 248
Mean (SD)	1.0 (0.6)	1.0 (0.6)	0.5 (0.5)	0.5 (0.6)
Correlation coefficient	0.922		0.894
ePRO period 3, n = 199
Mean (SD)	0.9 (0.5)	0.9 (0.6)	0.4 (0.4)	0.4 (0.5)
Correlation coefficient	0.926		0.883
ePRO period 4, n = 192
Mean (SD)	0.8 (0.6)	0.8 (0.6)	0.4 (0.5)	0.4 (0.5)
Correlation coefficient	0.936		0.896

ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire; SD: standard deviation.

Spearman correlation coefficients demonstrated a favorable correlation between the weekly average and day 7 data for each individual symptom (Table 3a). Correlation coefficients of 0.76–0.80 (high positive correlation[23]) or ≥0.9 (very high positive correlation[23]) were demonstrated for all symptoms during all four study periods. The correlation coefficients varied modestly across the periods. During period 4, the strongest correlations were for swelling in the stomach area (0.92) and swelling in the ankles (0.91). The weakest correlation occurred for chest pain (0.77). Sensitivity analysis of the correlation using the weighted kappa method also confirmed these findings (Table 3b), with the strongest correlation in period 4 being for swelling in the stomach area (0.83) and cough (0.83), though the weakest was for shortness of breath (0.62).

Table 3a.

Correlation between day 7 and weekly average, by symptom, using Spearman's correlation coefficients.

ePRO period	Cardiopulmonary symptoms
	Shortness of breath	Fatigue	Lack of energy	Swelling in ankles	Swelling in stomach area	Cough
1	0.83	0.88	0.88	0.88	0.86	0.85
2	0.83	0.87	0.87	0.91	0.90	0.89
3	0.81	0.84	0.87	0.91	0.91	0.85
4	0.83	0.87	0.87	0.91	0.92	0.89
	Cardiovascular symptoms
	Heart palpitations	Rapid heartbeat	Chest pain	Chest tightness	Light-headedness
1	0.79	0.76	0.80	0.83	0.79
2	0.82	0.86	0.80	0.83	0.84
3	0.81	0.84	0.76	0.79	0.82
4	0.84	0.80	0.77	0.83	0.80

ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.

Table 3b.

Correlation between day 7 and weekly average, by symptom, using the weighted kappa method.[a]

ePRO period	Cardiopulmonary symptoms
	Shortness of breath	Fatigue	Lack of energy	Swelling in ankles	Swelling in stomach area	Cough
1	0.65	0.74	0.73	0.72[b]	0.79	0.73
2	0.67	0.71	0.72	0.80	0.79	0.80
3	0.69	0.71	0.72	0.82	0.85	0.77
4	0.62	0.78	0.77	0.78	0.83	0.83
	Cardiovascular symptoms
	Heart palpitations	Rapid heartbeat	Chest pain	Chest tightness	Light-headedness
1	0.64[b]	0.65[b]	0.63[b]	0.67[b]	0.64[b]
2	0.73	0.73[b]	0.71[b]	0.72[b]	0.74[b]
3	0.75	0.80	0.61	0.68	0.79
4	0.79	0.72	0.81	0.81	0.76

ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.

Source: reproduced with permission from Landis and Koch, 1977.[24]

Kappa value of 0.61–0.80 indicates substantial correlation and 0.81–1.00 indicates almost perfect correlation.

In order to calculate the weighted kappa value, weekly average and daily scores that were in the same range were included.

Sensitivity of one-day symptom scores to severity of disease and ability to detect change in disease severity

One-day symptom scores differentiated well between WHO FC II and FC III/IV PAH, particularly for cardiopulmonary domain symptoms (Table 4).

Table 4.

Mean score in symptom domains on day 7 by ePRO period (WHO FC II versus WHO FC III/IV).[a]

	Cardiopulmonary symptoms		Cardiovascular symptoms
WHO FC II	WHO FC III/IV	WHO FC II	WHO FC III/IV
ePRO period 1	n = 104	n = 157	n = 104	n = 157
Mean (SD) day 7 score	0.8 (0.5)	1.1 (0.6)	0.4 (0.5)	0.6 (0.7)
Mean difference between FC groups	−0.4 (P < 0.0001)		−0.2 (P = 0.0250)
ePRO period 2	n = 102	n = 150	n = 102	n = 150
Mean (SD) day 7 score	0.7 (0.6)	1.1 (0.6)	0.4 (0.5)	0.6 (0.6)
Mean difference between FC groups	−0.4 (P < 0.0001)		−0.1 (P = 0.0404)
ePRO period 3	n = 115	n = 95	n = 115	n = 95
Mean (SD) day 7 score	0.7 (0.5)	1.1 (0.6)	0.3 (0.5)	0.4 (0.5)
Mean difference between FC groups	−0.4 (P < 0.0001)		−0.1 (P = 0.0903)
ePRO period 4	n = 123	n = 78	n = 123	n = 78
Mean (SD) day 7 score	0.6 (0.4)	1.2 (0.7)	0.2 (0.4)	0.5 (0.6)
Mean difference between FC groups	−0.6 (P < 0.0001)		−0.3 (P < 0.0001)

ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire; SD: standard deviation; WHO FC: World Health Organization functional class.

WHO FC and symptoms are collected at baseline and during the same period.

One-day scores were also sensitive to change in disease severity as measured by the PGA-S, using an ANOVA model that included PGA-S change category (i.e. decline in PGA-S [where change from baseline was ≥1], no change [where change from baseline = 0], small improvement [where change from baseline = −1], greater improvement [where change from baseline was < −1]) as the independent variable and change score in PAH-SYMPACT domains (collected on day 7) as the dependent variable. Based on the F-test, the model demonstrated statistically significant associations between improvements in symptom domain scores and improvements in PGA-S scores from baseline to week 16. These associations were also statistically significant in three of the four patient subgroups: WHO FC II and cardiovascular symptom domain (P = 0.0194); WHO FC III/IV and cardiopulmonary symptom domain (P = 0.0001), and WHO FC III/IV and cardiovascular symptom domain (P = 0.0001). The relationship between WHO FC II and the cardiopulmonary symptom domain was not statistically significant (P = 0.151).

Discussion

In this study, correlation coefficients comparing the weekly average and individual day symptom scores were mostly high (i.e. 0.70–0.90) or very high (i.e. > 0.90). One-day symptom scores also differentiated well between WHO FC II and III/IV PAH, and were sensitive to change in disease severity, as measured by the PGA-S. These results suggest that the PRO SYMPACT tool, when taken on a single day, may be a clinically useful and simpler alternative compared with completion on a daily basis for seven days.

There is widespread and increasing focus on the use of PROs in routine clinical practice to guide, individualize, and improve patient management in diverse disease settings.[25-27] In addition, there is a move away from generic PRO tools to those that are disease-specific to capture the most relevant information for the patient population.[28] However, there are challenges to implementation of PRO instruments in clinical practice, which include the need to devise user-friendly PRO tools that can be introduced seamlessly into daily workflow without presenting an additional burden to routine patient management; training healthcare teams on the use of PRO tools and the interpretation of PRO data; and overcoming potential skepticism on the value or benefit that incorporation of PROs may achieve in clinical practice.[26,29-31]

We describe the development of a one-day version of the PAH-SYMPACT instrument that aims to simplify the reporting of PROs to facilitate its implementation in routine clinical practice. The original PAH-SYMPACT tool is the first PAH-specific PRO developed and validated per FDA guidance; however, it requires daily reporting of symptoms over a seven-day period, with impact reported only on one day, with a weekly recall period. Similarities and differences between the one-day and seven-day versions are summarized in Table 5. The one-day version of PAH-SYMPACT streamlines reporting for convenience so that it can be completed on a single day. Importantly, evaluation of the PAH-SYMPACT tool demonstrated that symptom scores reported on a single day correlated well with the weekly average symptom score, irrespective of the day of reporting. An impact assessment analysis showed that the day 7 symptom score demonstrated mostly high correlation with the weekly average symptom score, and the symptom impact was typically recorded on day 7. This suggests that reporting of both symptom score and impact score on the same day is feasible, thereby improving the convenience and applicability of the one-day PAH-SYMPACT tool in standard clinical practice. Furthermore, the modest increase in correlations observed as the week progressed from day 1 through day 7 tends to suggest that collection of quality-of-life data closer to the clinic visit may provide a better representation of current PAH symptoms.

Table 5.

Similarities and differences between the 1-day and 7-day PAH-SYMPACT versions.

	PAH-SYMPACT version
	Seven day (full version)[21]	One day
PAH symptoms
Domains	Cardiopulmonary (six items) Cardiovascular (five items)
Reporting (recall period)	Daily (24-h)	Day 7[a] (24-h)
PAH impacts
Domains	Physical impacts (seven items) Cognitive/emotional (four items)
Reporting (recall)	Day 7 (one week)	Day 7 (one week)
Potential setting for use	Clinical trials/research Clinical practice	Clinical practice

PAH: pulmonary arterial hypertension; PAH-SYMPACT: Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.

Study results show that this could be any day (1 through 7); however, using day 7 means the symptom and impact data will all be collected on the same day.

Source: reproduced with permission from Chin et al., 2018.[21]

The validated PAH-SYMPACT tool can differentiate between patients according to disease severity (measured by WHO FC II vs FC III/IV and by PGA-S), and evaluation of the one-day version demonstrated that one-day symptom scores were also able to differentiate between these patient subgroups, particularly within cardiopulmonary domain symptoms.

PRO instruments require robust validation, and the original PAH-SYMPACT tool was developed according to the guidance of the FDA and validated in the phase 3b SYMPHONY study (NCT01841762). While this study demonstrates strong correlation between the one-day version of the instrument and the original PAH-SYMPACT tool, the former has not been formally and independently validated. However, since it is based on the validated PAH-SYMPACT instrument and shows good correlation with that instrument, the one-day PAH-SYMPACT may be more practical for use in clinical practice. Taken together, PAH-SYMPACT and the modified one-day version provide PAH-specific instruments applicable to the research and clinical practice environments, respectively.

Implementation of the one-day version of PAH-SYMPACT in clinical practice, including serial use following a change in therapy, holds the promise of providing quantifiable information about patient quality of life and the impact of changing therapy. A pilot project of administering the one-day PAH-SYMPACT tool when patients check in for their appointments at the Mayo Pulmonary Hypertension Clinic was recently initiated. The goal of this project is to assess in clinical practice how the instrument performs relative to other measures of disease severity, the impact of change in therapy, and relationship to outcome. In addition, flagging of particularly problematic symptoms and impacts may facilitate directed intervention. The initial reaction of patients has been positive, including ease of completing the instrument and sense of gratitude that quality of life is being taken seriously.

In conclusion, we have confirmed the feasibility of a one-day version of the PAH-SYMPACT instrument for reporting and monitoring patient outcomes. We encourage further evaluation of the one-day PAH-SYMPACT in both clinical and research settings to confirm the suitability and usefulness of the modified instrument in both scenarios.