Variation in Assignment of the COPD Patients into a GOLD Group According to Symptoms Severity.

Introduction: The Global Organization of Lung Disease (GOLD) classifies patients with chronic obstructive pulmonary disease (COPD) taking into account the symptoms. The modified Medical Research Council's dyspnea scale (mMRC) and the COPD assessment test (CAT) are used to assess these symptoms. In this study, we analyze the concordance of GOLD classification using mMRC and CAT. Patients and
Methods: This is an observational study of a cohort of 169 patients with COPD, who were classified following the GOLD 2017 recommendations, using both mMRC and CAT. A concordance analysis was applied, and a ROC curve was generated to identify the CAT score that best concorded with the mMRC scale.
Results: The concordance for the GOLD groups classified by CAT and mMRC was moderate (kappa 0.492). For mMRC score of 1 and 2, a CAT score of ≥9 and ≥16 showed the maximum value of the Youden index, respectively. By reclassifying the patients with the new cut-off points obtained, the best concordance was obtained between the cut-off point for CAT of 16 and for mMRC of 2, followed by CAT of 9 and mMRC of 1.
Conclusion: Because of the deficient concordance between CAT and mMRC, we propose the use of new cut-off points in future updates of the GOLD strategy.

Introduction

The World Health Organization (WHO) describes chronic obstructive pulmonary disease (COPD) as a progressive lung disease that can cause life-threatening dyspnea and predisposes to suffer exacerbations and serious illnesses. It is estimated that about 3 million people worldwide died of this cause in 2015, representing 5% of all deaths recorded that year. The main cause of COPD is exposure to tobacco smoke.1

The GOLD strategy (Global initiative for the Diagnosis, Management, and Prevention of Chronic Obstructive Lung Disease) is an international reference document for the management of COPD.2 In recent years it has undergone significant changes in light of new scientific evidence, especially on issues related to the diagnosis and classification of disease. All of this has contributed to improving the global approach to COPD.

The first GOLD report was published in 2001 and proposed a classification of severity of COPD based on spirometry and arterial blood gas abnormalities.3 Spirometry remained the gold standard for diagnosis, classification and follow-up of the COPD patient until the document was updated in 2011.4 In this edition, spirometry was maintained as a method for diagnosing subjects with suspected COPD and a new classification system was proposed that combined symptoms, comorbidities, risk of exacerbations and lung function (ABCD). Despite the ABCD proposal was relatively well received, it was criticized for some aspects that were controversial. Thus, this classification was more complex and did not offer any advances in determining the prognosis of patients compared to the previous version.5 In this sense, some studies suggested that there would probably be an improvement in the prediction of exacerbations, but these results were not reproduced in predicting mortality or decreased lung function.6

Based on the above, one of the main changes introduced in the 2017 update was the introduction of a new diagnostic algorithm in which lung function was removed from all four quadrants of the ABCD classification. According to this classification, patients were stratified with either mMRC score (0–1 vs ≥2) or CAT score (<10 or ≥10) resulting in two low-symptom categories (A and C) and two high-symptom categories (B and D). The patient’s risk was then established based on the exacerbations and hospital admissions suffered by the patient in the previous year. Patients who had suffered 0–1 exacerbation or no hospital admission in the previous year were stratified into groups A or B (low risk) and patients with ≥2 exacerbations or income prior year were stratified to groups C or D (high risk). Despite these changes, criticism of this model has also emerged. One is based on the choice of the tool used to measure the symptoms, either the Medical Research Council’s modified dyspnea scale (mMRC) or the COPD Assessment Test questionnaire (CAT). The GOLD strategy recommends using either mMRC or CAT, choosing one but not indicating the performance of both. In this regard, the assignment of a patient to a group may change depending on the symptom scale used.7

The COPD assessment test (CAT) is a short questionnaire developed to help patients and clinicians to assess the impact of symptoms in routine clinical practice. It consists of eight items and each item has a score from 0 to 5. An overall score is calculated by adding the score from each item; a total score ranging from 0 to 40. The higher the score, the worse the subject’s health status.8 The CAT questionnaire presents excellent internal consistency and a very good correlation with the Saint George Respiratory Questionnaire (SGRQ) specific for COPD.9 The correlation of the CAT with the SGRQ has made it possible to establish that a value of 10 points (equivalent to 25 points on the SGRQ questionnaire) should be used as a cut-off point below which the disease has a low impact on the subject’s health.10

The mMRC dyspnoea scale is an assessment tool that establishes the severity of this symptom in relation to various physical tasks. This scale was developed by the American Thoracic Society as a modification of the originally proposed British Medical Research Council dyspnea index (scale 1–5). It consists of five items and its value is set in a range from 0 (No dyspnea or dyspnea only with strenuous exercise) to 4 (dyspnea at rest).11,12 As the scale is one-dimensional and focuses only on respiratory distress, it may lose accuracy in older patients or major co-morbidities such as obesity, osteoarticular disease, or anxiety and depression disorders.13

The GOLD strategy considered an mMRC dyspnea scale score of 2 and a CAT score of 10 to be equivalent, despite the fact that there were no studies to support these data.

The purpose of our study is to find out if the classification of COPD patients into a group (ABCD) could change depending on the symptom scale used. Another objective is to determine whether the cut-off points set at 10 for the CAT questionnaire and 2 for the mMRC dyspnea scale, which separate mildly symptomatic and highly symptomatic patients, are the ideal cut-off points where there is the least degree of discrepancy between the two assessment tools.

Patients and Methods

The observational study included COPD patients followed-up in pneumology outpatient department. Demographic, clinical and functional variables were analyzed in patients with stable COPD. Sequential sampling and inclusion of all subjects presenting with a diagnosis of COPD (newly diagnosed patients and follow-up patients) were performed.

Diagnosis of COPD was established by spirometry according to the GOLD criteria: post-bronchodilator FEV1/FVC<0.7, history of smoking more than 10 pack-year and presence of respiratory symptoms. We did not consider COPD patients due to exposures other than tobacco because this is the main cause at present and the difficulty of documenting the wide range of exposures to other substances. Patients who had suffered an exacerbation of COPD in the last month were excluded to avoid the score on the CAT and mMRC dyspnea scale being influenced by the exacerbation.

Those with active respiratory disorders such as pneumonia, diffuse bronchiectasis and interstitial lung disease seen on chest radiograph or chest computed tomography were also excluded, so that the respiratory symptoms produced by these diseases do not interfere with the scores of the CAT and mMRC scales related to COPD. Patients must have had the ability to undertake spirometry and answer the CAT questionnaire and the mMRC dyspnea scale. The period of inclusion of patients was from September 2017 to December 2018.

Subjects were classified according to GOLD 2017 recommendations in ABCD. This classification was performed on two occasions using the two recommended tools for measuring symptoms, the CAT questionnaire and the mMRC dyspnea scale. Both were applied on the same day and by the same interviewer.

Statistic Analysis

A descriptive analysis of the characteristics of the patients included was carried out. Quantitative variables are presented as mean and standard deviation (SD). The ANOVA test was used to compare the means of clinical and functional variables of the four GOLD categories.

The degree of agreement when applying both evaluation instruments to our cohort was calculated with the Kappa coefficient (k).The interpretation of agreement in this study is based on kappa value as described previously in the literature in which k=0 is “poor”, 0.01

In addition, the Spearman correlation (non-parametric rho estimator) was used to assess the consistency between the two evaluation methods collected in the qualitative ordinal scale.

A receiver operating characteristics (ROC) curve was generated to identify the CAT questionnaire score that best correlated with the mMRC scale. The Youden index was used to find the cut-off point for the best combination of sensitivity and specificity. The analyses were performed with the SPSS statistical program. Values of p<0.05 (2-tailed) were taken as significant in all comparisons.

Ethics Committee and Informed Consent

The study was approved by the Ethics Committee of the Hospital Universitario Puerta de Hierro Majadahonda de Madrid (Acta nº 02.18). Patients gave their signed informed consent before participating in the study.

This study was conducted in accordance with the Declaration of Helsinki.

Results

A total of 169 patients with COPD completed the questionnaire: CAT and mMRC dyspnea scale. The data on anthropomorphic, clinical and functional characteristics are shown in Table 1.

Table 1

Baseline Patient Characteristics

Characteristics	Value
Total	169
Age(years)	67.64 ± 9.77
Sex	116 males (68.6%)53 women (31.4%)
Active smoking	68 (40.2%)
Pack-years index	50 ± 24.58
BMI (kg/m2)	27.49 ± 5.76
Comorbidity
Hypertension	77 (45.56%)
Dyslipidemia	50 (29.58%)
Diabetes	27 (16%)
Cardiovascular events	37 (22%)
Charlson Index	4 ± 1.5
CAT questionnaire	15.7 ± 7.32
Dyspnea Scale (mMRC)	1.09 ± 0.927
Dyspnoea (mMRC)
0	40 (23.7%)
1	93 (55.1%)
2	21 (12.4%)
3	10 (5.9%)
4	5 (3%)
FEV1 (%)	68.34 ± 20.96
FEV1(l)	1.76 ± 0.677
Bronchodilator test positive	30 (17.75%)
GOLD
Stage I	55 (32.5%)
Stage II	79 (46.9%)
Stage III	25 (14.8%)
Stage IV	10 (5.9%)

Abbreviations: BMI, body mass index; CAT, COPD assessment test; mMRC, modified Medical Research Council; FEV1, forced expiratory volume in one second; FVC, forced vital capacity; GOLD, Global Initiative for Chronic Obstructive Lung Disease.

The cohort was classified according to the symptoms. The78.7% of patients were considered to be slightly symptomatic and 21.3% were considered to be very symptomatic when using the mMRC dyspnea scale. Patients were also classified using the CAT questionnaire and 43.2% were not highly symptomatic and 56.8% were highly symptomatic (Figure 1).

Figure 1

Classification of subjects according to symptoms.

Using the mMRC dyspnea scale to classify patients into ABCD groups, 108 (63.9%) patients were assigned to group A, 18 (10.7%) to group B, 25 (14.8%) to group C and 18 (10.7%) to group D. When the patients were classified according to the CAT questionnaire, 61 (36.1%) were assigned to group A, 65 (38.5%) to group B, 12 (7.1%) to group C and 31 (18.3%) to group D (Table 2).

Table 2

Cohort Classification According to the 2017 GOLD Update for the mMRC Dyspnea Scale and the CAT Questionnaire

GOLD mMRC
GOLD CAT	A	B	C	D	Total
A	61	0	0	0	61
B	47	18	0	0	65
C	0	0	12	0	12
D	0	0	13	18	31
Total	108	18	25	18	169

Abbreviations: GOLD, Global Initiative for Chronic Obstructive Lung Disease; CAT, COPD assessment test; mMRC, modified Medical Research Council.

The degree of agreement between the two GOLD groups was obtained with a kappa value of 0.492, which corresponds to a moderate agreement. The degree of concordance between low-risk (A-B) and high-risk (C-D) patients was evaluated, obtaining a kappa value of 0.271 and 0.436, respectively, showing that in low-risk patients the concordance is weaker than in high-risk patients.

The correlation between the mMRC dyspnea scale and the CAT questionnaire score was statistically significant (p<0.001), with a value of the Spearman correlation coefficient of 0.661 (moderate-high correlation) (Figure 2). Despite this, we observed that there was a wide variation in the CAT score among patients with the same mMRC scale score (Figure 3).

Figure 2

Correlation between the CAT questionnaire and the mMRC dyspnea scale.

Figure 3

Histogram of CAT scores according to mMRC score.

In Table 3, taking as reference a score ≥10 in the CAT questionnaire, the variation of the degree of concordance was analyzed when modifying the cut-off point of dyspnea according to the mMRC scale. By varying the cut-off point of the mMRC scale (≥1, ≥2, ≥3, ≥4) new groups (A, B, C and D) and new degrees of agreement were obtained. By analyzing the cut-off point of 2 in dyspnea according to the mMRC scale proposed by the guidelines, a group distribution of ABCD of 108, 18, 25 and 18, respectively, was obtained, with a kappa value of 0.492. There was substantial agreement between the MRC dyspnea scale ≥1 with a CAT score ≥10 (kappa=0.623). The mMRC ≥1 value was the group with the highest degree of agreement. On the other hand, the mMRC value ≥4 was the group with the lowest degree of agreement, with a kappa value of 0.242.

Table 3

Distribution of Subjects Classified in Groups a, B, C and D According to the Cut-off Point Used for the mMRC Dyspnea Scale and Variation in Agreement (Kappa) with CAT 10 (Reference)

	A	B	C	D	Kappa
CAT ≥10	61	65	12	31	Reference
mMRC ≥ 1	34	92	6	37	0.623
mMRC ≥2	108	18	25	18	0.492
mMRC≥ 3	120	6	34	9	0.321
mMRC ≥ 4	124	2	40	3	0.242

Abbreviations: CAT, COPD assessment test; mMRC, modified Medical Research Council.

A ROC curve was used to identify the CAT score that was best correlated with the mMRC scale. For an mMRC scale of 1, the best value was CAT ≥9, with a sensitivity and a specificity of 0.78 and 0.80, respectively (Figure 4). For an mMRC score of 2, a CAT score of 16 showed the maximum value of Youden’s index with a sensitivity and specificity of 0.80 and 0.85, respectively. (Area under curve -AUC- =0.878; 95% Confidence Interval -CI-: 0.820 to 0.924).

Figure 4

ROC curves for dyspnea mMRC 1 and 2.

By reclassifying the patients with the new cut-off points obtained, the highest value of the kappa was obtained between the cut-off point for CAT of 16 and for mMRC of 2 (kappa=0.71), followed by a kappa of 0.676 for the cut-off point of the CAT questionnaire of 9 and of mMRC of 1 (Figure 5).

Figure 5

Classification of COPD patients according to the new cut-off points. *p <0.001.

Discussion

The GOLD strategy recommends measuring the presence and intensity of symptoms by the CAT questionnaire or the mMRC dyspnea scale, choosing one of them but not indicating the performance of both. The cut-off points of mMRC ≥2 and CAT≥10 are considered the recommended cut-off points that separate symptomatic and slightly symptomatic patients, despite the fact that there is no scientific evidence that these cut-off points are similar.

The main conclusion of our study is that a cut-off point of 10 on the CAT questionnaire is not equivalent to a score on the mMRC dyspnea scale of 2, so the classification of COPD patients according to the GOLD recommendations is modified depending on the symptom scale used. This phenomenon of change or migration of patients from one group to another according to the symptom scale used was already described by Rieger-Reyes et al15, who identified that the use of one or another assessment method led to the reclassification of more than 25% of patients.

Discrepancy between the mMRC and CAT score is well described in previous studies. In our study, 35.51% of the patients were discordant, in other words, 35.51% of the subjects when classified by CAT and mMRC did not agree on their assignment within the groups A, B, C and D of the GOLD recommendations. This difference is important, since the recommended initial treatment depends on it. No differences were observed between these discordant and concordant patients when analyzing their age, BMI, FEV1, year package index and Charlson’s index.

In our cohort, the agreement obtained between the CAT questionnaire and the mMRC dyspnea scale was moderate (κ = 0.492) and similar to that described in other studies.16 In the subgroup analysis, concordance in the low-risk categories (A-B) was weak and increased to moderate in the high-risk categories (C-D). Furthermore, for each of the dyspnea scores, the CAT score presented a wide range of values. The degree of dispersion of the CAT score was greater in patients with dyspnea 0 and 1.

The highest concordance of the CAT questionnaire and the mMRC dyspnea scale in the high-risk groups has already been recorded in other studies, in which the group with more symptoms and high risk achieved better correlations than the group that combined fewer symptoms and low risk.17 In addition, a score on the questionnaire CAT≥10 occurs in up to 50% of individuals who smoke and ex-smoke without lower airway obstruction has been described.18

The mMRC breathlessness scale is a simple and easy scale. It correlates better with the quality of life of COPD patients than functional parameters.19 It also evaluates the efficacy of treatment and correlates closely with the 6-minute walk test and with 5-year survival in COPD patients.20,21

However, in its simplicity, there is its greatest limitation. The mMRC scale is one-dimensional and only focuses on breathing difficulty when measuring the magnitude of symptoms. COPD affects patients beyond dyspnea; it is a complex and heterogeneous syndrome with pulmonary and extrapulmonary characteristics.

In this sense, the CAT questionnaire has the advantage of providing a multidimensional assessment of the impact of COPD on the patient.8 It is probably this dimensionality that explains the discrepancy between the two methods. Furthermore, the CAT cut-off points and their impact on health are well validated. Thus, scores in the questionnaire CAT<10 reflect a low impact in terms of the effect of the disease on the health of the patient; CAT values of ≥10 and <20 reflect a moderate impact; scores of CAT≥20 and <30 indicate a high impact; and a value of CAT≥30 suggests a very high impact of COPD on the health of the patient.9

By taking a score on the CAT questionnaire ≥10 (recommended) and varying the cut-off point of the mMRC scale, the degree of agreement (kappa) is modified according to the cut-off point of the mMRC dyspnea. If we compare the ≥2 score on the dyspnea scale, which is the recommended cut-off point according to the latest GOLD recommendations, and the mMRC score ≥1, we see how the concordance increases from moderate to substantial in relation to the recommended cut-off point. Other authors support these results, suggesting that an mMRC dyspnea scale of 1 would be more consistent with a value of CAT 10 than a score of 2.7,22

The new CAT cut-off points proposed in our study for the mMRC scores of ≥1 and ≥2 are ≥9 and ≥16, respectively. By performing a new reclassification with the new cut-off points and applying a concordance analysis, better degrees of agreement were obtained between the CAT cut-off point and the mMRC dyspnea scale with respect to the reference ones. The highest kappa coefficient value is between the cut-off point of CAT≥16 and dyspnea mMRC≥2 (kappa=0.71), followed by a kappa coefficient of 0.676 for the cut-off point ≥9 and dyspnea mMRC≥1. These data would support the idea of redefining new cut points and making the appropriate modifications in the next GOLD review. By increasing the CAT score for dyspnea 1 and 2, the concordance increases, so the choice of the new cut-off point for CAT should be increased with respect to the mMRC dyspnea scale scores. By increasing the CAT score for dyspnea 1 and 2, the concordance increases, so the choice of the new cut-off point for CAT should be increased from the mMRC dyspnea scale scores.

Rhee et al16 described a new CAT cut-off point for an mMRC of 2. They showed that a CAT cut-off point of 15 had a higher concordance for mMRC of 2, increasing from a CAT score of 10. These authors also noted that the concordance between an mMRC cut-off point of 1 and a CAT cut-off point of 10 was higher than for any of the other combinations and indicated that an mMRC cut-off point of 1 was more appropriate than a cut-off point of 2 for defining the highly symptomatic patients. Other studies have established other cut-off points, proposing that an mMRC dyspnea of 2 would correspond with a CAT value ≥18, concluding that these changes would reclassify up to a third of patients.23

Our study has limitations. First, the sample size is relatively small due to the capacity of the center where the patients came from and the data have been collected in a single center; therefore, the results should be interpreted with caution. Another limitation is determined by the collection of patients. Subjects were referred from an outpatient pneumology service, excluding patients who recently had an exacerbation (avoiding that the score on the CAT and mMRC dyspnea scale was influenced by the exacerbation). In this way, exacerbation patients or those who had recently been admitted and who attended our consultations were excluded. Despite this, this is a descriptive observational study in which the number of exacerbations and hospital admissions in the previous months has been recorded. Furthermore, it is based on actual data from daily clinical practice, representing the actual distribution of COPD groups. On the other hand, the study was conducted by respiratory specialists, with a rigorous design and with spirometry performed in a respiratory function laboratory, following the criteria of the American Thoracic Society (ATS).

In conclusion, the classification of COPD patients into GOLD revision varies according to the assessment method (CAT or mMRC) used to measure the intensity and presence of symptoms. A CAT score of 10 and the mMRC dyspnea scale of 2 are not equivalent cut-off points. In this sense, a CAT score of 16 was the optimal cut-off point for an mMRC dyspnea score of 2. On the other hand, a CAT score of 10 was the optimal cut-off point for a score of 1 on the mMRC dyspnea scale. Our results are consistent with those obtained in other studies where both assessment methods have been compared, also concluding that the currently established cut-off points are not similar.7,24–26 Thus, we propose that in future updates of the GOLD strategy, a readjustment should be made and new cut-off points planned to separate highly symptomatic patients from those who are mildly symptomatic.