Brazilian version of the Clinical COPD Questionnaire, administered by interview: reliability and validity measurement properties.

OBJECTIVE: To test the reliability, validity, and interpretability of the Brazilian version of the Clinical COPD Questionnaire (CCQ) in patients with COPD.
METHODS: Fifty patients with COPD completed the CCQ by interview on two occasions. At the first visit, the CCQ was administered twice, by two different raters, approximately 10 min apart; the patients also underwent spirometry and were administered the COPD Assessment Test, the modified Medical Research Council scale, and Saint George's Respiratory Questionnaire (SGRQ). At the second visit (1-2 weeks later), the CCQ was readministered. We tested the hypothesis that the CCQ total score would correlate positively with the total and domain SGRQ scores (r ≥ 0.5).
RESULTS: Of the 50 patients, 30 (60%) were male. The mean age was 66 ± 8 years, and the mean FEV1 was 44.7 ± 17.9% of the predicted value. For all CCQ items, Cronbach's alpha coefficient (95% CI) was 0.93 (0.91-0.96). To analyze the interrater reliability and test-retest reliability of the CCQ, we calculated the two-way mixed effects model/single measure type intraclass correlation coefficient (0.97 [95% CI: 0.95-0.98] and 0.92 [95% CI: 0.86-0.95], respectively); the agreement standard error of measurement (0.65 for both); the smallest detectable change at the individual level (1.81 and 1.80, respectively) and group level (0.26 and 0.25, respectively); and the limits of agreement (-0.58 to 0.82 and -1.14 to 1.33, respectively). The CCQ total score correlated positively with all SGRQ scores (r ≥ 0.70 for all).
CONCLUSIONS: The Brazilian version of the CCQ showed an indeterminate measurement error, as well as satisfactory interrater/test-retest reliability and construct validity.

INTRODUCTION

Assessments of disease impact and clinical stability in patients with COPD should help physicians to make therapeutic decisions. Patient-reported outcome measures (PROMs), such as the Clinical COPD Questionnaire (CCQ) and the COPD Assessment Test (CAT), are useful to assess disease impact cross-sectionally and clinical stability longitudinally. In accordance with GOLD recommendations, the CCQ and the CAT are comprehensive and suitable PROMs for the assessment of symptoms in patients with COPD.

Most COPD patients prefer CCQ to CAT, because the CCQ incorporates more details about daily respiratory problems than does the CAT and, therefore, reflects their health status better. Some patients also point out that the CCQ presents a system of response options which is easier to understand when compared to that of CAT. Furthermore, the International Primary Care Respiratory Group elected the CCQ as the best PROM to evaluate COPD patients in primary care. The CCQ was the only PROM that received top marks in the survey.

The selection of a suitable PROM for health status assessment should be based on the quality of its measurement properties-a PROM should be reliable and valid. There are numerous PROMs that can be used in order to measure health status. However, careful selection is of utmost importance to avoid the risk of imprecise or biased results, which could lead to wrong conclusions. Measurement properties may differ between populations and therefore should be tested and considered appropriate for the specific population to be assessed.

The CCQ was developed by van der Molen et al. in 2003 with the purpose of promoting the evaluation of clinical control in patients with COPD. The domains selected as the most important for clinical control were functional state, symptoms, and mental state. A Portuguese version of the CCQ for use in Brazil is available on the CCQ website, but its measurement properties have yet to be investigated. The present study aimed to analyze the internal consistency, reliability, measurement error, and construct validity, as well as the floor and ceiling effects, of the Brazilian version of the CCQ, when administered by interview, in patients with COPD.

METHODS

Patient selection

Patients referred to a public outpatient clinic specializing in COPD were invited to participate in the study. Inclusion criteria were as follows: having a confirmed diagnosis of COPD, being ≥ 40 years of age, being a smoker or a former smoker, having no other comorbidity (such as cardiovascular, neurological, orthopedic, rheumatic, or respiratory diseases other than COPD) that negatively impacted on the activities of daily life, and having a Mini-Mental State Examination score ≥ 25 (literates) or ≥ 19 (illiterates). Exclusion criteria were having changes in clinical stability or disease impact in the month prior to the study or during data collection, assessed by closed questions, and not participating in the evaluations of the study. All patients who agreed to participate signed an informed consent form. This study was approved by the Human Research Ethics Committee of the Federal University of Santa Catarina (CAAE no. 33299214.8.0000.0121).

Study design

The study was conducted in a public outpatient clinic specializing in COPD in two visits in the morning period between 2017 and 2019. The selected PROMs-CAT, CCQ, modified Medical Research Council (mMRC) scale, and Saint George’s Respiratory Questionnaire (SGRQ)-were administered by interviews due to the low level of education of part of the sample. The PROMs were administered in a dedicated room where only the patient and the rater were present. The raters only read the instructions and items of the PROM and wrote down the choices of the patients. The raters are physiotherapists with experience in assessing health status in patients with COPD. At the first visit, spirometry was performed, and the CAT, the mMRC scale, and the SGRQ were administered. In addition, CCQ was first administered by rater 1 and then, approximately 10 min later, by rater 2, for interrater reliability analysis. At the second visit, between one and two weeks later, the CCQ was readministered by rater 1 for test-retest reliability analysis.

Assessments

Lung function was assessed following the standards recommended by the American Thoracic Society/European Respiratory Society using a spirometer (KoKo Sx 1000; nSpire Health Inc., Longmont, CO, USA). The reference values for post-bronchodilator spirometric variables were those established by Pereira et al. The severity of airflow limitation was based on FEV1 and classified as GOLD I, II, III ou IV.

The CAT and the mMRC scale scores, as well as the number of exacerbations within the last 12 months, regardless of hospital admissions, were used in order to classify the impact of COPD on health status and the risk of future events as GOLD A, B, C, or D.

The SGRQ was used in order to assess health-related quality of life. The questionnaire consists of 76 items distributed into three domains (symptoms, activity, and psychosocial impact). The total score ranges from 0 to 100, higher scores meaning poorer quality of life.

The CCQ was used in order to evaluate clinical control. It consists of 10 items distributed into three domains (symptoms, mental state, and functional state). Total and domain scores range from 0 to 6, higher scores representing poorer control.

Statistical analysis

Data normality was analyzed by the Shapiro-Wilk test. The level of statistical significance was set at p < 0.05. The internal consistency of the CCQ items was analyzed by the Cronbach’s alpha coefficient (α) and the corresponding 95% CI. To compare the scores between raters and between test and retest, the Student’s t-test or the Wilcoxon test was used according to data normality. Interrater and test-retest reliability of the CCQ scores were analyzed by the two-way mixed effects model/single measurement type intraclass correlation coefficient (ICC3,1) and the corresponding 95% CI. For interrater reliability, test-retest reliability, and measurement error analyses, we calculated the agreement standard error of measurement (SEMagreement), the smallest detectable change at the individual level (SDCindividual) and group level (SDCgroup), and the limits of agreement (LoA). To visualize the total score and the agreement between the CCQ measurements, Bland-Altman plots were used. To analyze construct validity, the following hypothesis was used: the total CCQ score would positively correlate with the total and domain SGRQ scores, and the correlation coefficient (r) would be ≥ 0.5. The percentage of occurrence of minimum and maximum CCQ scores was used in order to analyze the floor and ceiling effects, respectively, which were classified as absent or present.

RESULTS

Fifty patients with COPD were included in the study. The general characteristics of the sample are described in Table 1.

Table 1

General characteristics of the sample.a

Variable	(N = 50)
Male gender	30 (60)
Age, years	66 ± 8
BMI, kg/m2	24.7 ± 4.7
Smoking history, pack-years	50 [23-73]
Pulmonary function
FEV1/FVC	0.53 [0.44-0.61]
FEV1, L	1.15 [0.80-1.69]
FEV1, % of predicted	44.7 ± 17.9
FVC, L	2.14 [1.69-2.80]
FVC, % of predicted	66.2 [54.0-75.5]
GOLD, severity
I	1 (2)
II	17 (34)
III	21 (42)
IV	11 (22)
GOLD, classification
A	12 (24)
B	19 (38)
C	0 (0)
D	19 (38)
CAT score	18 ± 10
mMRC scale score	1 [1-4]
SGRQ score
Total	38.3 [18.0-67.8]
Symptoms	44.6 ± 22.9
Activity	51.5 [25.1-85.1]
Psychosocial impact	28.9 [14.4-55.8]

CAT: COPD Assessment Test; mMRC: modified Medical Research Council; and SGRQ: Saint George’s Respiratory Questionnaire. aValues expressed as n (%), mean ± SD, or median [IQR].

The median administration time of the CCQ by interview was 2.76 min (2.38-3.38 min). In the internal consistency analysis, the values of α (95% CI) for all of the CCQ items (CCQ total score) and for the symptoms (items 1, 2, 5, and 6), mental state (items 3 and 4), and functional state domains (items 7, 8, 9, and 10) were, respectively, 0.93 (0.91-0.96); 0.77 (0.66-0.85); 0.79 (0.64-0.87); and 0.94 (0.91-0.96).

Table 2 shows the CCQ total and domain scores in each administration. Differences between raters were observed for the total, the mental state domain, and the functional state domain scores, as well as for the mental state domain score between test and retest (p ≤ 0.05 for all). Table 2 also shows the analysis of interrater reliability, test-retest reliability, and measurement error. All ICC3,1 were ≥ 0.80, 95% CI ranges being broader between test and retest than between raters. The results of SEMagreement, SDCindividual, and SDCgroup were similar between raters and between test and retest for the CCQ total score, but they were lower between raters for the symptoms and mental state domains, as well as between test and retest for the functional state domain. Figure 1 also presents the measurement error by the LoA ranges of the CCQ total score, which were broader between test and retest than between raters.

Table 2

Scores, reliability, and measurement error of the Clinical COPD Questionnaire between measurements.a

CCQ	Rater 1 (Test)	Rater 2					Rater 1 (Retest)
	Score	Score	ICC3,1 (95% CI)	SEMa	SDCi	SDCg	Score	ICC3,1 (95% CI)	SEMa	SDCi	SDCg
Total	1.85 [0.77-3.52]	1.65 [0.60-3.07]	0.97 (0.95-0.98)	0.65	1.81	0.26	1.50 [0.80-3.10]*	0.92 (0.86-0.95)	0.65	1.80	0.25
Symptoms	2.21 ± 1.44	2.25 ± 1.60*	0.92 (0.86-0.95)	0.47	1.30	0.18	2.30 ± 1.42	0.81 (0.69-0.89)	0.77	2.14	0.30
Mental state	1.5 [0.0-4.0]	1.00 [0.00-3.50]*	0.96 (0.92-0.97)	0.70	1.91	0.27	0.75 [0.00-2.50]	0.80 (0.67-0.88)	2.04	5.66	0.80
Functional state	1.75 [0.68-3.5]	1.50 [0.44-3.00]	0.93 (0.85-0.96)	1.46	4.06	0.57	1.37 [0.25-3.06]*	0.90 (0.83-0.94)	0.90	2.49	0.35

CCQ: Clinical COPD Questionnaire; ICC3,1: two-way mixed effects model/single measurement type intraclass correlation coefficient; SEMa: agreement standard error of measurement; SDCi: smallest detectable change at the individual level; and SDCg: smallest detectable change at the group level. aValues expressed as median. [IQR] or mean ± SD. *p > 0.05 vs. rater 1 (test).

Figure 1

Bland-Altman plots of the total Clinical COPD Questionnaire score, showing interrater reliability (in A) and test-retest reliability (in B). LoA: limit of agreement; and MD: mean difference.

Table 3 and Figure 2 show the correlations between the CCQ scores and SGRQ scores. All correlations were strong (r > 0.70), except for the correlation between the CCQ mental state domain and the SGRQ symptoms and activity domains, which were good (0.50 < r < 0.70).

Table 3

Correlations of the Clinical COPD Questionnaire (CCQ) domain scores with the Saint George’s Respiratory Questionnaire (SGRQ) total and domain scores.

SGRQ	CCQ domain
	Symptoms	Mental state	Functional state
Total	0.83*	0.78*	0.88*
Symptoms	0.75**	0.64*	0.73*
Activity	0.75*	0.64*	0.82*
Psychosocial impact	0.78*	0.81*	0.85*

*p < 0.01; Spearman’s rank correlation. **p < 0.01; Pearson’s rank correlation.

Figure 2

Correlations of the Clinical COPD Questionnaire (CCQ) total score with the Saint George’s Respiratory Questionnaire (SGRQ) total score (in A) and domain scores (in B, C and D).

Only 4 patients had a minimum score (8%), and 1 had a maximum score (2%), indicating the absence of the floor and ceiling effects.

DISCUSSION

In the present study, the measurement properties of the Brazilian version of the CCQ were tested in a sample of patients with COPD in Brazil. The results suggest that this PROM is reliable and valid when administered by interview. To our knowledge, this was the first study to analyze the measurement properties of the Brazilian version of the CCQ.

The internal consistency analysis of the Brazilian version of the CCQ revealed values between 0.77 and 0.94. In the original development study of the CCQ, these values ranged from 0.78 to 0.91. Among the domains, the highest value was in the functional state domain, which was similar in the present study. In other validation studies, values above 0.70 - were also found. By definition, internal consistency determines the degree of interrelationship between items. Values below 0.70 indicate a lack of correlation between the PROM items. However, values above 0.95 may indicate that the PROM contains many items that are evaluating the same construct, suggesting redundancy. Therefore, the internal consistency of the Brazilian version of the CCQ and its domains was positive (0.70 ≤ α ≤ 0.95) and sufficient (α ≥ 0.70).

This study presented the reliability analysis of the Brazilian version of the CCQ between two raters and over a time interval. The minimum ICC was 0.80. In the study that presented the measurement properties of the original CCQ version, the ICC was 0.94 for the total score between test and retest. Our finding is similar to those reported in the validation studies for the Italian (ICC = 0.99) and Persian (ICC = 0.98) versions of the CCQ. Reliability is defined as the proportion of the total variance in the measurements that is due to true differences among patients. Statistical analysis should preferably be done by calculating the ICC, because it considers systematic errors between repeated measures. In the present study, we chose to use the ICC3,1, in which each individual is evaluated by each rater, these being the only raters of interest, and reliability is calculated from a single measure. ICCs range from 0 to 1; values close to 1 indicate small error variation when compared with patient variation. This means that such values also depend on the heterogeneity of the population, that is, when the population is more homogeneous, it is easier to find an ICC closer to 0. Considering an ICC of at least 0.70 as a quality criterion, it can be said that the reliability of the Brazilian version of the CCQ between raters and between test and retest was positive and sufficient in our sample.

There were differences in the scores between the administrations of the CCQ. However, comparison tests are not recommended by the Consensus-based Standards for the Selection of Health Measurement Instruments for reliability analysis. This is due to the fact that such tests show only the agreement between administrations for the central values but do not provide information about agreement between administrations for individual values. In the present study, the CCQ scores in the three administrations were higher in the symptoms and the functional state domains, corroborating the results in the original validation of the CCQ.

Measurement error was analyzed by means of SEMagreement, SDCindividual, SDCgroup, and LoA to determine interrater and test-retest reliability for the Brazilian version of the CCQ. The SEMagreement of the total CCQ score was the same between raters and between test and retest (0.65). Tsiligianni et al. , studying a sample of clinically stable COPD patients, reported a lower value of SEM for the CCQ regarding test-retest reliability. However, those authors calculated SEM using a different equation, which does not consider the variance due to systematic differences between raters. This measurement property represents the systematic and random error of a patient’s score which is not attributed to real changes in the construct to be measured. In the present study, we calculated SEM, which represents the standard deviation of repeated measures of an individual; and then SDC, which consists of the minimum change that must be overcome to guarantee a real change in the individual. This means that an observed change must be greater than the limit of SDC to be considered true. LoA were demonstrated in Bland-Altman plots to support the interpretation of measurement error size. It is possible to visualize the magnitude of the measurement error when we relate LoA with the score range. By definition, 95% of the differences between repeated measures must be within the LoA range. A value outside that range can indicate a real change. In order to know whether the measurement error is acceptable or not, one must also analyze the minimal important change (MIC). SDC and MIC can be used to decide whether a real and clinically relevant change has occurred with a patient. Other studies about CCQ described a MIC value close to 0.4. - In the present study, however, MIC was not calculated. Therefore, the interrater and test-retest measurement errors of the Brazilian version of the CCQ was classified as undetermined. ,

According to the results, the hypothesis related to the construct analysis was confirmed. There was a positive correlation of r values of at least 0.5 between the total score of the Brazilian version of the CCQ and the SGRQ total and domain scores. For the elaboration of the hypothesis, the minimum r value (0.53) found between the CCQ total score and the SGRQ scores in the validity study of the original CCQ version was considered. Other studies also reported similar r values between the CCQ and the SGRQ. , Similar results were also found between the CCQ and other PROM scores that assess overall and specific health-related quality of life in patients with COPD , . The CCQ symptoms and functional state domain scores showed a strong correlation (r > 0.70) with all SGRQ domains. The CCQ mental state domain score strongly correlated only with the total and the impact domain scores of SGRQ, probably because this is the only SGRQ domain that has questions about psychosocial changes. The construct validity estimates the degree to which PROM scores are consistent with assumptions based on the hypothesis that the PROM validly measures the construct that is intended to be measured. , In the sample studied, the construct validity of the CCQ reached the quality criteria, being rated as positive and sufficient, because the hypothesis of the construct validity was met.

In the interpretability analysis, the floor and ceiling effects were not observed. In similar studies, the floor and ceiling effects were not detected in the CCQ total score either. , The presence of a floor or ceiling effect may indicate that extreme items are missing in the lower or upper end of the PROM and can limit its ability to discriminate patients and to measure changes.

Due to the low level of education of part of the sample studied, the CCQ was administered by interview. As expected, the completion of the CCQ by interview in the present study was slightly longer than that of the original self-administered CCQ reported in the original study (approximately 2 min). Agreement between the self-administered CCQ scores and clinician-administered CCQ scores obtained during a medical visit, as well as between self-administered CCQ scores and clinician-administered CCQ scores obtained through semi-structured, in-depth interviews, have been reported. However, this was the first study to analyze measurement properties of the CCQ completed by interview, with no interference from the raters. A meta-analysis reported that, in general, the self-completion and assisted completion of a PROM produce equivalent scores, supporting that the interview format is a valid mode of administration. Moreover, in the present study, assisted completion of the CCQ allowed the unprecedented analysis of interrater reliability and measurement error.

The time interval between the administrations of the CCQ by the raters might have been too short to avoid recall bias and, therefore, might have compromised interrater reliability. However, as far as we know, there is not a recommendation regarding an appropriate time interval for the application of a PROM by raters in the literature. In addition, although the 95% CI range between raters was shorter than it was between test and retest, interrater and test-retest ICC3,1 were similar and higher than 0.70. , Another possible limitation of the study was the cross-sectional design, which prevented the sufficiency of the measurement error from being tested. However, this was the first study that reported the values of SEMagreement, SDCindividual, SDCgroup, and LoA of the Brazilian version of the CCQ. These values reveal that the changes are real, and not due to measurement error, by showing how much the score needs to change before ensuring that a real change has occurred, providing conditions to interpret longitudinal measurements. In addition, the values of SEMagreement, SDCindividual, SDCgroup, and LoA are presented in the same measurement unit of the PROM being studied, which facilitates the interpretation of the scores by health professionals in clinical practice.

In conclusion, the Brazilian version of CCQ has sufficient internal consistency and reliability, that is, the PROM items are interrelated, and their scores are stable and capable of reproducing consistent results in repeated measures between different raters and over time. In addition, the Portuguese version of the CCQ for use in Brazil demonstrates sufficient construct validity, and the correlations between the CCQ total scores and the SGRQ scores are consistent. In the present study, SEMagreement, SDCindividual, SDCgroup, and LoA parameters of the measurement error were shown. However, it is recommended that further studies be conducted to test the sufficiency of measurement error by calculating the MIC. No floor or ceiling effects in the total score of the Brazilian version of the CCQ were found. To our knowledge, this was the first study to evaluate the measurement properties of the CCQ in a sample of patients with COPD in Brazil, which contributes to disseminating this PROM to and promoting its use by health professionals and researchers in order to assess the health status of their patients.