Do We Need to Assess Quality-of-Life with Both the St George's Respiratory Questionnaire and the EuroQol 5-Dimension Questionnaire in a Clinical Study with an Economic Component: Insights from the REVOLENS Study in Severe Emphysema.

Introduction: The use of two quality-of-life questionnaires in a single clinical trial with an economic component can be challenging due to the associated workload in terms of data collection and an increased risk of missing data. The aim of our study was to determine whether the questionnaire chosen to measure health status, the St. George's Respiratory Questionnaire (SGRQ), could be administered on its own without adding the EuroQol five dimensions questionnaire (EQ-5D) for economic evaluation in emphysema studies. Materials and
Methods: Data were prospectively collected during the REVOLENS trial assessing endobronchial coil treatment in severe emphysema. To quantify the association between the two questionnaires, correlations between the EQ-5D and the SGRQ were first tested and the concordance was then studied in order to know whether the two questionnaires were interchangeable. Finally, the Starkie et al algorithm predicting EQ-5D utility values from the SGRQ was used on REVOLENS's individual patient data. The Student's t-test, correlation and concordance between EQ-5D individual value (from the REVOLENS study) and predicted value (from the Starkie et al algorithm) were studied to test this algorithm.
Results: Results showed a strong correlation but no concordance between the EQ-5D and the SGRQ, demonstrating that the two questionnaires are not interchangeable. Moreover, the algorithm predicting EQ-5D utilities from the SGRQ did not provide utility values comparable to those observed in the REVOLENS study. Indeed, our study demonstrated a strong correlation between predicted and individual EQ-5D values but no concordance.
Conclusion: The use of both the EQ-5D and the SGRQ in a clinical study with an economic component is justified. Based on our results, the SGRQ should not be used to obtain a utility score to calculate the incremental cost-effectiveness ratio and conclude on the efficiency of an intervention in emphysema patients.

Introduction

The EuroQol five dimensions questionnaire (EQ-5D) is a standardized instrument used to measure health-related quality of life (HRQoL). The 5-level EQ-5D version (EQ-5D-5L) was introduced by the EuroQol group in 2009 to improve the instrument’s sensitivity and to reduce ceiling effects, as compared to the previous version, the EQ-5D-3L.1,2 The EQ-5D is used in economic evaluation, and many countries generated a population-specific utilities value set. This utility value is a component of the quality-adjusted life-year (QALY) which has a number of useful properties and is the current standard for conducting cost-effectiveness analyses.3

The validity of the EQ-5D compared to disease-specific HRQoL questionnaires including the St. George’s Respiratory Questionnaire (SGRQ) has been demonstrated in outpatients with chronic obstructive pulmonary disease (COPD).4 The use of two quality-of-life questionnaires in a clinical trial can be challenged due to the overload of work and the risk of increasing the number of missing data related to the multiplication of data to be collected from patients. Starkie et al had developed an algorithm predicting EQ-5D utility from the SGRQ in subjects with COPD, with the following resulting model: EQ-5D = 0.9617–0.0013*SGRQ total - 0.0001*SGRQ total2 + 0.0231*male.5

The aim of our study was to determine whether the questionnaire chosen to measure health status, the SGRQ, could be administered on its own without adding the EQ-5D for economic evaluation in emphysema studies.

Materials and Methods

The REVOLENS Study

The design and results of the REVOLENS study (NCT01822795) had previously been published.6,7 This study was a prospective randomized open-blinded end-point (PROBE) trial conducted in ten French sites throughout France. Patients with severe emphysema were randomly assigned in a 1:1 ratio to lung volume reduction coil treatment or usual care and were followed during 5 years. This study was approved by the Ethics Committee of Dijon Est I (N°2012-A01477-36), and by the French Agency for Medicines and Health Products (ANSM). All participants provided written informed consent to participate in the study. The primary endpoint of the REVOLENS study was the improvement of at least 54 meters in the 6 mins walk test at 6 months. The aim of the economic evaluation was to estimate the 1-year cost-effectiveness of lung volume reduction coil treatment compared to usual care in patients with severe emphysema included in the REVOLENS trial.6,8

Quality-of-Life Questionnaires Used in the REVOLENS Study

Two quality-of-life questionnaires were used in the REVOLENS study, the generic EQ-5D and the disease-specific SGRQ.

The EQ-5D-5L comprises a descriptive system and a visual analogue scale (VAS). The descriptive system is composed of five health dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) with 5 levels of health state (no problems, slight problems, moderate problems, severe problems and extreme problems). For each of the 5 dimensions, the participant’s answer is converted to a number between 1 and 5, expressing the health state reported. The responses are combined to produce a five-digit number describing the participant’s health status which is converted to a utility value from the country-specific value set. The French EQ-5D-5L value set has utility between −0.530 (health condition worse than death) and 1 (best possible health). The VAS records the respondent’s self-rated health status on a graduated scale from 0 to 100, with higher scores for higher HRQoL.1,4

The SGRQ is a 50-item questionnaire developed to measure health status in patients with chronic airflow limitation. The responses to these items can be aggregated into a total score and three sub-scores for symptoms, activity and impact domains. There may be between 2 and 5 response options for each question. The participant’s answers are weighted, and scores are calculated by dividing the summed weights by the maximum possible weight. The result is expressed as a percentage ranging from 0%, the best possible score, to 100%, the worst possible score.9,10

Statistical Analysis

Data were prospectively collected during the REVOLENS trial and statistical analyses were performed on the REVOLENS intention-to-treat population. The sample size has been previously calculated based upon the primary endpoint of the clinical study (improvement of at least 54 meters in the 6 mins walk test at 6 months) and 100 patients were included in the study accordingly.6 Questionnaires responses were analyzed at baseline for the 2 groups together due to randomization and at 1 year for the 2 separated groups (coil treatment group versus usual care group). The remaining statistical analyses were done at baseline and at 1 year for the 2 groups together in order to obtain more robust results. 1-year results for the 2 separated groups were presented in secondary analyses. Data were presented using means with 95% confidence interval (CI) as appropriate and missing data were imputed by the average in each group.

To quantify the association between the 2 questionnaires, correlations between the EQ-5D and the SGRQ and between the EQ-5D and each domain of the SGRQ were first tested at baseline and 1 year. The correlation coefficient on the study population was estimated using the Pearson correlation test if data follow a normal distribution and otherwise the Spearman correlation test. The sign of the correlation coefficient indicates the direction of the association and the magnitude of the correlation coefficient indicates the strength of the association. The correlation between the 2 questionnaires can be positive (higher scores of one questionnaire were associated with higher scores of the other) or negative (higher scores of one questionnaire are associated with lower scores of the other). Strength of correlation was classified as follows: absent (r < ±0.20), weak (r = ±0.20 to ±0.34), moderate (r = ±0.35 to ±0.50), and strong (r > ±0.50).11–13 Correlation diagrams were then plotted.

In order to know if the 2 questionnaires are interchangeable, the concordance between the EQ-5D and the SGRQ was studied at baseline and 1 year using the Lin’s concordance correlation coefficient (CCC)14 and the Bland-Altman diagram.15 Values for Lin’s coefficient were interpreted according to these levels of concordance: poor (CCC < 0.40), fair (CCC = 0.40 to 0.59), good (CCC = 0.60 to 0.74) and excellent (CCC > 0.75). The principle of the Bland-Altman diagram is to appreciate the difference between the 2 health utility scores obtained with each of the two questionnaires and to deduce, for the study population, the bias (which is the mean difference between the two questionnaires), the precision (which describes how close the values are) and the 95% agreement limits. It is expected that the 95% agreement limits include 95% of differences between the two questionnaires. In case of perfect match, the Lin’s coefficient will be 1, with a bias of zero and agreement limits close to the bias on the Bland Altman plot. In order to appreciate the interchangeability SGRQ data were adjusted to the same scale as the EQ-5D with the following equation: 1 – (SGRQ score/100).

Finally, the Starkie et al algorithm predicting EQ-5D utility value from the SGRQ utility value in subjects with COPD was used with the REVOLENS individual data.5 The Student’s t-test, correlation and concordance between EQ-5D individual value (from the REVOLENS study) and EQ-5D predicted value (from the Starkie et al algorithm) were studied to test this algorithm at baseline and 1 year.

A p-value less than 0.05 was considered significant. SAS (Version 9.4, SAS Institute, Cary, NC, USA) and R (Version 3.4, R Foundation for Statistical Computing, Vienna, Austria) were used for analysis.

Results

A hundred patients were included in the REVOLENS study. Baseline characteristics in each group were presented in .

Questionnaires Responses

Table 1 shows the distribution of patient answers to the EQ-5D descriptive system at baseline and 1 year. The mobility and usual activities dimensions were those where we found the greatest number of severe or extreme problems at baseline and 1 year with no significant difference between the 2 groups at 1 year (p= 0.16 and 0.55, respectively). Figure 1 showed SGRQ scores at baseline and 1 year for each domain. The activity domain was the one where the average score was the highest at baseline and 1 year with significant difference between the 2 groups at 1 year (p= 0.03). Total EQ-5D and SGRQ scores at baseline and 1 year are presented in Figure 2. There was no significant difference between the 2 groups at 1 year for the 2 scores (p= 0.06 and 0.10, respectively).

Table 1

Distribution of Responses to the EQ-5D Descriptive System at Baseline and 1 Year

Number of Patients	Baseline	1 Year Coil Treatment	1 Year Usual Treatment
Mobility	N=100	N=97	N=100
No problem	7	5	2
Slight problem	11	9	1
Moderate problem	28	12	19
Severe problem	53	16	24
Extreme problem	1	5	4
Self-care	N=100	N=97	N=100
No problem	27	14	12
Slight problem	17	9	9
Moderate problem	39	12	18
Severe problem	15	8	6
Extreme problem	2	4	5
Usual activities	N=100	N=97	N=100
No problem	5	4	1
Slight problem	10	11	8
Moderate problem	41	9	15
Severe problem	39	18	20
Extreme problem	5	5	6
Pain/discomfort	N=99	N=96	N=100
No problem	29	21	9
Slight problem	19	11	12
Moderate problem	30	7	21
Severe problem	20	3	4
Extreme problem	1	4	4
Anxiety/depression	N=100	N=97	N=100
No problem	22	15	9
Slight problem	19	10	16
Moderate problem	32	10	10
Severe problem	20	4	10
Extreme problem	7	8	5

Figure 1

Average values of the SGRQ at baseline and 1 year.

Figure 2

Total EQ-5D and SGRQ scores at baseline and 1 year.

Correlation

A strong negative correlation between the EQ-5D and the SGRQ total score was observed at baseline and 1 year (p<0.0001 for both). The EQ-5D was strongly correlated with each domain of the SGRQ at baseline and at 1 year (p<0.0001 for all) with the exception of the Symptoms domain at baseline for which the correlation was moderate (p=0.0004) (Table 2). The correlation diagram is presented in . The strong negative correlation between the EQ-5D and the SGRQ total score was also observed when the correlation was studied separately in the coil treatment group and in the usual care group at 1 year (p<0.0001 for both) ( and ).

Table 2

Correlation Between the EQ-5D and the SGRQ at Baseline and 1 Year

		Mean (95% CI)	Correlation with the EQ-5D
		Correlation Coefficient (r)	P-value
Baseline	SGRQ (%)	58.9 (56.2; 61.6)	−0.66	<0.0001
	Symptoms domain	57.2 (53.6; 60.8)	−0.35	0.0004
	Activity domain	80.2 (77.7; 82.7)	−0.55	<0.0001
	Impact domain	47.2 (48.3; 50.5)	−0.63	<0.0001
EQ-5D	0.30 (0.25; 0.35)
1 year	SGRQ (%)	54.8 (51.5; 58.2)	−0.71	<0.0001
	Symptoms domain	52.3 (48.4; 56.2)	−0.48	<0.0001
	Activity domain	76.6 (73.4; 79.9)	−0.700	<0.0001
	Impact domain	42.4 (38.4; 46.3)	−0.65	<0.0001
EQ-5D	0.32 (0.27; 0.37)

Concordance

The Lin’s coefficient was 0.48 (0.37; 0.58) at baseline and 0.54 (0.43; 0.64) at 1 year. The Bland Altman plots presented in Figure 3 showed significant points dispersion at baseline and 1 year. The agreement limits were between −0.47 (−0.54; −0.41) and 0.26 (0.20; 0.33) at baseline and between −0.49 (−0.56; −0.43) and 0.23 (0.17; 0.29) at 1 year. In addition, there was a negative bias with a 95% confidence interval excluding zero of −0.11 (−0.07; −0.14) at baseline and −0.13 (−0.17; −0.10) at 1 year, indicating that the EQ-5D score was smaller than the SGRQ score. The significant point’s dispersion and the Lin’s coefficient showed fair concordance between the 2 questionnaires. The fair concordance between the 2 questionnaires was also observed when the concordance was studied separately in the coil treatment group and in the usual care group at 1 year ().

Figure 3

Bland Altman Plot of the EQ-5D and the SGRQ score at baseline and 1 year.

EQ-5D Predicted Value from the SGRQ with Starkie et al Algorithm

A statistically significant difference between the EQ-5D value predicted by the SGRQ and the EQ-5D individual value was observed at baseline and 1 year with a strong positive correlation (p<0.0001) (Table 3). The correlation diagram is presented in . The Lin’s coefficient was 0.39 (0.28; 0.49) at baseline and 0.41 (0.30; 0.50) at 1 year. The Bland Altman plots presented in Figure 4 showed significant points dispersions at baseline and 1 year. Indeed, the agreement limits were between −0.13 (−0.19; −0.06) and 0.59 (0.53; 0.66) at baseline and between −0.12 (−0.18; −0.05) and 0.64 (0.57; 0.70) at 1 year. In addition, there was a positive bias with a 95% confidence interval excluding zero of 0.23 (0.20; 0.27) at baseline and 0.26 (0.22; 0.30) at 1 year, meaning that the EQ-5D predicted value was higher than the EQ-5D individual value. The significant point’s dispersion and the Lin’s coefficient showed poor to fair concordance between the 2 questionnaires.

Table 3

EQ-5D Predicted Value and EQ-5D Individual Value at Baseline and 1 Year

Mean (95% CI)	EQ-5D Predicted Value	EQ-5D Individual Value	P-value	Correlation Coefficient (r)	r p-value
Baseline	0.54 (0.50; 0.57)	0.30 (0.25; 0.35)	<0.0001	0.67	<0.0001
1 year	0.58 (0.54; 0.62)	0.32 (0.27; 0.37)	<0.0001	0.68	<0.0001

Figure 4

Bland Altman plot of the EQ-5D predicted value and EQ-5D individual value at baseline and 1 year.

Discussion

To our knowledge, this is the first study specifically assessing utility data obtained from a population of patients with severe emphysema who responded to both EQ-5D, a generic quality-of-life questionnaire and SGRQ, a specific quality-of-life questionnaire. The agreement between these 2 questionnaires was studied in order to know if they are both required for a clinical study with an economic component. Results showed a strong correlation but no concordance between the EQ-5D and the SGRQ at baseline and 1 year, demonstrating that these 2 questionnaires are not interchangeable. The correlation analysis showed a moderate rather than strong correlation of the EQ-5D total score with the SGRQ symptom domain at baseline, unlike with other domains. This may be due to the fact that the EQ-5D is a generic questionnaire while the SGRQ is specific to COPD patients and therefore better captures the symptoms of this specific population. Comparable results were found when the analysis was performed at 1 year on the 2 separate groups. Moreover, the algorithm predicting EQ-5D utility from the SGRQ in subjects with COPD did not provide utility values comparable to those observed in the REVOLENS study. Indeed, this study demonstrated a strong correlation between the EQ-5D predicted value and the EQ-5D individual value but no concordance.

Comparability

A 2017 Indian study investigated the relationship between health-related quality of life, measured using the EQ-5D, and respiratory health status, measured using the SGRQ, among coal-based sponge iron plant workers in India.16 This study showed no-to-weak correlation between each component of the 2 questionnaires whereas our study found a strong correlation. This difference can be explained by the difference between patient characteristics in the 2 studies, with severe COPD emphysematous patients in the REVOLENS study. In a COPD population, Wilke et al showed a moderate to strong correlation between the EQ-5D and the SGRQ depending on the time points (r = −0.42 to −0.54, p<0.01)11 and Nolan et al also showed a strong correlation between the 2 questionnaires (r = −0.62, p<0.001). However, it has to be pointed out that the concordance between the EQ-5D and the SGRQ has never been investigated.

One of the limitations of this study is that in the Bland Altman plots, we did not establish the a priori agreement limits due to the lack of data in the literature. So our conclusion on the mismatch was only based on the significant point’s dispersion on the Bland Altman plots and on the Lin’s concordance correlation coefficients. Another limitation is due to the fact that the Starkie et al algorithm was developed for the English population and even if utility weights are close in France, they are not identic. Moreover, we suppose that the EQ-5D-3L was used and not the EQ-5D-5L like in the REVOLENS study which may explain the non-concordance between the EQ-5D predicted value and the EQ-5D individual value. Finally, the sample size has been calculated on the primary endpoint of the clinical study and 100 patients were thus included [6]. The sample size was also calculated using the concordance parameter. With a power of 90%, the sample size required would be 111 which is very close to the initial sample size calculation.

To conclude, due to the lack of concordance between the SGRQ score and the EQ-5D score and to the poor performance of the Starkie et al algorithm, the use of both questionnaires in a clinical study with an economic component is justified. Based on our results, the SGRQ should not be used to obtain a utility score to calculate the incremental cost-effectiveness ratio (ICER) in euro per QALY gained for an economic study.