Patient-reported outcomes in primary care patients with COPD: psychometric properties and usefulness of the Clinical COPD Questionnaire (CCQ). A cross-sectional study.

BACKGROUND: Chronic obstructive pulmonary disease (COPD) is a common disease with considerable consequences for patients' daily lives. The Clinical COPD Questionnaire (CCQ) was designed to measure these consequences in daily practice. Although the CCQ is widely used, its original structure has never been tested. AIMS: This study examines the psychometric properties of the CCQ with regard to its latent structure in a sample of primary care patients with COPD.
METHODS: Two cross-sectional studies were conducted; in study 1 (N=243) exploratory analyses, including exploratory factor analysis (EFA) and Mokken scale analysis, were performed to explore the latent structure of the CCQ. In study 2 (N=244), confirmatory factor analysis (CFA) was conducted to evaluate the model fit of the structure found in study 1.
RESULTS: Both EFA and Mokken scale analysis revealed a structure of two dimensions ('general impact' α=0.91 and 'cough' α=0.84). This structure, however, was not confirmed in study 2, nor was the original structure. However, subsequently removing items that violated the assumption of a normal response distribution did result in an excellent model fit with two dimensions measuring 'dyspnoea' and 'cough' (CFA: comparative fit index (CFI) 0.98; normed fit index (NFI) 0.97; root mean squared error of approximation (RMSEA) 0.08 (0.04)).
CONCLUSIONS: In primary care, factor analyses on the CCQ revealed a two-component structure measuring 'general impact', and 'cough'. A shortened and more specific version of the CCQ could be regarded as a useful instrument to screen for exacerbations by measuring dyspnoea, coughing and producing phlegm.

Introduction

Chronic obstructive pulmonary disease (COPD) places a considerable burden on patients’ daily lives[1] and is the fourth leading cause of death worldwide.[2] Although the forced expiratory volume in 1 s (FEV1) has long been the gold standard to determine COPD severity, it is now well known that COPD is a multi-component disease and that ‘FEV1 is an unreliable marker of the severity of breathlessness, exercise limitation, and health status impairment’.[1] Therefore, in their guidelines of 2011, the Global Initiative for Chronic Obstructive Lung Disease (GOLD) suggests a combined COPD assessment in which the patients’ spirometric classification should be combined with a more patient-centred assessment of symptoms.[1]

In 2003, Van der Molen et al.[3] developed the Clinical COPD Questionnaire (CCQ) with the intention to incorporate both clinicians’ and patients’ treatment goals by assessing symptoms and functional state. The CCQ is a short, 10-item scale that measures ‘clinical control’, which was defined as ‘the full range of clinical impairment that patients with COPD may experience as a result of their disease’.[4] When developing a new instrument that will serve as a measure of patient-reported outcomes, it is strongly recommended to incorporate the patient’s opinion.[5] Therefore, 34 patients diagnosed with COPD were involved in the initial development of the CCQ through focus group discussions.[3] However, the constructs and final content of the current CCQ were determined by clinicians and ‘experts’ in the field of COPD. These clinicians and experts received 16 potential items that derived from the focus group discussions, and were invited to rank them in the order of importance for the assessment of clinical COPD control; 10 items remained and were divided into three subscales: ‘symptoms’ 4 items, ‘functional state’ 4 items, and ‘mental state’ 2 items. Subsequently, the reliability and validity of the CCQ and its subscales were determined in a cross-sectional study involving 119 (mainly primary care) patients who were either ‘healthy’ ex-smokers, at risk of COPD or diagnosed with COPD (GOLD I–III).

Because the CCQ was among the first short and user friendly, disease-specific questionnaires, it has been translated in over 20 languages and become a widely used instrument in both clinical practice and clinical trials. Previous research has extensively studied the concurrent validity of the CCQ. However, its original latent structure of three dimensions has never been tested in replication studies. In addition, although studies claim that both the CCQ and its subscales are valid and reliable,[3,6-11] the methodological approach of these studies has several limitations (e.g., 40% of these studies had a sample size <100 or included a rather heterogeneous COPD population). Moreover, when compared with other COPD-specific health status instruments, the CCQ often appears less reliable in measuring constructs like quality of life,[12] functional status[13] and breathlessness.[14]

Therefore, the aim of the present study was to examine the latent structure of the CCQ to provide better knowledge of its underlying dimensions and to further enhance the potential usefulness of this patient-reported outcome measure in clinical practice.

Materials and Methods

Setting

This study was conducted in collaboration with PoZoB, a large primary care organisation in the south of the Netherlands. PoZoB supports ~250 general practitioners in organising their care for patients with a chronic disease through disease management programmes targeted at specific conditions such as diabetes, asthma and COPD.[15] In 2008, PoZoB started a disease management programme for patients with asthma and mild to moderate COPD (GOLD I and GOLD II), the ‘ASCOZOB’ programme.[16] All data in the present study were collected within this setting in patients diagnosed according to the GOLD guidelines.

Participants and procedure

During a 12-month period (2008–2009) all COPD patients within the ASCOZOB programme were invited to complete a set of questionnaires, including the CCQ, as part of a baseline assessment; questionnaires were sent by postal mail. Four hundred and two patients (20%) completed and returned the CCQ. The data of this first sample were combined with the data of another sample of primary care patients with COPD from the same population. Data collection for this second sample started in 2011 and was part of a randomised controlled trial testing a disease management approach for co-morbid depression and anxiety in patients with COPD participating in the ASCOZOB programme. In accordance with the first sample, data came from a baseline assessment; an extensive description of the trial has been published elsewhere.[17] Eighty-five patients (34%) completed and returned the questionnaire resulting in an overall study sample of 487 primary care patients with COPD; missing values were replaced using expectation maximization imputation (missings per item varied from 0% (item 1) to 2% (item 7)). Subsequently, this overall study sample was randomly divided into two separate subsamples for studies 1 and 2. The data of study 1 were used to explore the latent structure of the CCQ; the data of study 2 were used to confirm the scale structure found in study 1.[18] Patient characteristics of studies 1 and 2 are presented in Table 1; all patients were diagnosed with COPD GOLD I or GOLD II.

Table 1

Characteristics of patients with COPD in both study 1 (N=243) and study 2 (N=244)

	Study 1: exploratory factor analysis					Study 2: confirmatory factor analysis
Characteristics	N	%	Mean	s.d.	Range	N	%	Mean	s.d.	Range	P value
Demographics
Sex											0.26
Male	147	62				162	66
Female	92	38				82	34
Age			67	10.7	35–88			67	9.8	33–87	0.95
Marital status											0.75
With partner	187	77				183	75
Widowed	27	11				28	12
Single	29	12				32	13
Education level											0.71
Low	152	63				150	62
Middle	60	25				66	28
High	27	11				25	10
Life style
Any smoking	117	49				135	57				0.09

Abbreviation: COPD, chronic obstructive pulmonary disease.

Measurements and statistical methods

All patients completed a questionnaire including questions regarding demographics (sex, age, marital status, education level and smoking status) and the CCQ. Statistical analyses were performed using the Statistical Package for Social Sciences (SPSS version 18.0, IBM, Chicago, IL, USA). Confirmatory factor analysis (CFA) was done using AMOS (version 18, IBM, Chicago, IL, USA).

Descriptive analyses

To test for differences between the two study samples, chi-square analyses were used for all categorical data (sex, marital status, education and smoking), continuous data (age) were analysed with an independent samples t-test.

Study 1

Exploratory analyses were conducted to discover the latent structure of the CCQ. First, skewness, kurtosis and response distributions were reviewed to explore the relevance of each item to the present patient population (Table 2). Subsequently, although factor analyses are relatively robust against violations of normality,[19] both exploratory factor analysis (EFA) using principal axis factoring and oblimin rotation (Table 3) and Mokken scale analysis were performed to explore the structure of the CCQ. Mokken scale analysis is the non-parametric equivalent of EFA, based on the principles from item response theory,[20] and is less sensitive to violations of normality. In EFA, Cattell’s scree test and eigenvalues (>1)[21] were explored to determine the number of eligible factors; in the Mokken scale analysis the lower bound of the H-coefficient was set to 0.4 (comparable to a factor loading of 0.4) to determine the latent structure. Finally, internal consistency was determined by calculating Cronbach’s alphas.

Table 2

Study 1: skewness, kurtosis and response distribution in % of all individual items

	Skewness (s.d.=0.16)	Kurtosis (s.d.=0.31)	Never	Hardly ever	A few times	Several times	Many times	A great many times	Almost all the time
On average, during the past week, how often did you feel
CCQ 1. Short of breath at rest?	1.06	1.50	33	28	27	8	3	1	1
CCQ 2. Short of breath doing physical activities?	0.56	0.19	7	16	30	30	7	5	6
CCQ 3. Concerned about getting a cold or your breathing getting worse?	2.40	4.83	62	22	10	3	2	1	<1
CCQ 4. Depressed (down) because of your breathing problems?	1.73	2.86	55	21	13	6	2	3	1
In general, during the past week, how much of the time
CCQ 5. Did you chough?	0.58	0.09	13	22	26	22	10	3	4
CCQ 6. Did you produce phlegm?	0.59	−0.36	23	17	21	20	9	3	7
	Skewness (s.d.=0.16)	Kurtosis (s.d.=0.31)	Not limited at all	Very slightly limited	Slightly limited	Moderately limited	Very limited	Extremely limited	Totally limited/or unable to do
On average, during the past week, how limited were you because of your breathing problems
CCQ 7. Strenuous physical activities?	0.40	−0.44	12	19	25	21	12	7	4
CCQ 8. Moderate physical activities?	0.74	−0.10	31	22	21	17	5	4	1
CCQ 9. Daily activities at home?	1.83	3.54	56	23	10	6	3	<1	1
CCQ 10. Social activities?	1.72	2.69	57	21	9	9	1	3	<1

Abbreviation: CCQ, Clinical COPD Questionnaire.

Table 3

Study 1: two-component structure from EFA using principal axis factoring with oblimin rotation; the structure matrix

	Impact	Coughing
Eigen values	5.3	1.4
Percentage of variance	53.0	14.0
Factor loadings
CCQ 1. Short of breath	0.69	0.41
CCQ 2. Short of breath doing physical activities	0.70	0.47
CCQ 3. Concerned about getting a cold or your breathing getting worse	0.70
CCQ 4. Depressed (down) because of your breathing problems	0.75
CCQ 5. Did you cough?		0.84
CCQ 6. Did you produce phlegm?	0.42	0.84
CCQ 7. Limited in strenuous activities	0.75	0.49
CCQ 8. Limited in moderate physical activities	0.84	0.42
CCQ 9. Limited in daily activities at home	0.75
CCQ 10. Limited in social activities	0.76

All factor loadings >0.40 are displayed.

Abbreviations: CCQ, Clinical COPD Questionnaire; EFA, exploratory factor analysis.

Study 2

The structure found in study 1 was evaluated with CFA. Adequate fit can be assumed with comparative fit index (CFI) ⩾0.80, in combination with normed fit index (NFI) ⩾0.80 and (the lower limit of) root mean squared error of approximation (RMSEA) ⩽0.05 for good and (lower limit) RMSEA ⩽0.08 for adequate fit.[22-24]

Ethical principles

This study was conducted in accordance with the principles described in the Declaration of Helsinki and in accordance with ‘The Medical Research Involving Human Subjects Act’ (Wet Maatschappelijke Ondersteuning) and approved by the medical ethics committee of the Elizabeth Hospital Tilburg, the Netherlands, NL33363.008.10.

Results

The study samples of studies 1 and 2 were similar on all baseline characteristics as displayed in Table 1.

Initial analyses in study 1

Skewness and kurtosis of each individual item are displayed in Table 2. ‘Floor effects’ were found in four items as over 79% of patients indicated that the content of these items was not applicable to them: item 3, ‘On average during the past week, how often did you feel concerned about getting a cold or your breathing getting worse?’; item 4, ‘On average during the past week, how often did you feel depressed (down) because of your breathing problems?’; item 9, ‘On average, during the past week, how limited were you in daily activities at home because of your breathing problems?’; item 10, ‘On average, during the past week, how limited were you in social activities because of your breathing problems?’

All assumptions to perform EFA were met, the Kaiser–Meyer–Oklin value was >0.60 (0.86) and Bartlett’s test of sphericity was significant (P<0.01). The scree test (Figure 1) revealed a clear pattern of two dimensions that together explained 67% of all variance (‘general impact’ 53% and ‘cough’ 14%). In addition, all corresponding factor loadings were >0.65 (Table 3). With regard to the Mokken scale analysis, a lower bound of 0.4–0.5 (comparable with factor loadings >0.4) resulted in the same two dimensions with sufficient differential quality (H); the differential quality of ‘general impact’ varied between H 0.60 (lower bound 0.4) and H 0.66 (lower bound 0.5), with H 1.00 suggesting perfect differential quality. The differential quality of ‘cough’ was found to be H 0.76. ‘General impact’ was covered by eight items with α=0.91, ‘cough’ was covered by two items with α=0.84.

Figure 1

The scree plot from the exploratory factor analysis in study 1.

Replication of findings in study 2

Neither the structure found in the first study (CFI 0.87, NFI 0.85, RMSEA 0.14, lower limit 0.12) nor the original model (CFI 0.80, NFI 0.79, RMSEA 0.18, lower limit 0.16) revealed an adequate fit of the data. An inadequate model fit can be the result of severe violations of normality. Therefore, skewness, kurtosis and response distributions were reviewed again, and the results were consistent with those of study 1 (Table 4). Subsequently removing items 3, 4, 9 and 10 from the model because of floor effects and non-normal distributions led to an excellent model fit with two, more homogeneous dimensions measuring ‘dyspnoea’ (α=0.86) and ‘cough’ (α=0.86) (Figure 2): CFI 0.98; NFI 0.97; RMSEA 0.08 and lower limit 0.04. Moreover, the remaining items cover the frequency and impact of the key symptoms that characterise an exacerbation (dyspnoea, cough and sputum production).[25]

Table 4

Study 2: skewness, kurtosis and response distribution in % of all individual items

	Skewness (s.d.=0.16)	Kurtosis (s.d.=0.31)	Never	Hardly ever	A few times	Several times	Many times	A great many times	Almost all the time
On average, during the past week, how often did you feel
CCQ 1. Short of breath at rest?	0.77	−0.10	41	22	24	10	2	1	0
CCQ 2. Short of breath doing physical activities?	0.54	−0.15	12	13	35	18	9	6	6
CCQ 3. Concerned about getting a cold or your breathing getting worse?	2.14	5.60	64	21	10	3	1	<1	<1
CCQ 4. Depressed (down) because of your breathing problems?	2.12	6.05	62	24	11	2	2	<1	0
In general, during the past week, how much of the time
CCQ 5. Did you chough?	0.67	0.05	14	20	28	21	8	3	7
CCQ 6. Did you produce phlegm?	0.78	−0.31	26	21	22	12	8	5	7
	Skewness (s.d.=0.16)	Kurtosis (s.d.=0.31)	Not limited at all	Very slightly limited	Slightly limited	Moderately limited	Very limited	Extremely limited	Totally limited/or unable to do
On average, during the past week, how limited were you because of your breathing problems
CCQ 7. Strenuous physical activities?	0.43	−0.54	14	21	23	22	10	7	4
CCQ 8. Moderate physical activities?	0.97	0.62	30	28	21	13	5	3	2
CCQ 9. Daily activities at home?	2.11	5.02	57	24	9	6	<1	2	1
CCQ 10. Social activities?	2.50	7.58	62	23	8	3	1	1	1

Abbreviation: CCQ, Clinical COPD Questionnaire.

Figure 2

Best-fitting model from the confirmatory factor analysis in study 2.

Discussion

Main findings

The aim of the present study was to examine the psychometric properties of the CCQ. Both exploratory (study 1) and confirmatory (study 2) factor analyses were conducted to discover its latent structure. EFA and Mokken scale analysis revealed a structure with two dimensions measuring the general impact of COPD, cough and phlegm production. The stability of this structure was, however, not confirmed in the second study, nor was the stability of the original structure with three dimensions. An inadequate fit can be caused by many different things, among which severe violations of normality as found in items 3, 4, 9 and 10.[18]

Interpretation of findings in relation to previously published work

Nonetheless, the guidelines on developing instruments that measure patient-reported outcomes clearly state that statistical analyses should guide, not dictate, the process of scale development,[26] the relevance of specific items and constructs to the patient and their clinical importance should also always be considered.[26] The content of items 3, 4, 9 and 10 covered issues such as being concerned/depressed because of one’s breathing problems and feeling limited in daily and social activities. However, in the Netherlands, primary care patients with COPD most likely suffer from a mild form of COPD (GOLD I and GOLD II) that does not necessarily lead to functional limitations in their daily and social activities or cause anxiety or depression because of their breathing problems. Therefore, one could choose not to include these items when using the CCQ in a primary care COPD population. Removing items 3, 4, 9 and 10 from the model resulted in an excellent fit with two homogenous dimensions, measuring both the frequency and impact of dyspnoea and the frequency of coughing and producing phlegm. Subsequently, the remaining six items may be a useful tool to screen for exacerbations. An exacerbation is characterised by ‘worsening dyspnea, cough, sputum production and sputum purulence, as well as worsening airflow obstruction’,[27] which are symptoms covered by the remaining items. As can be seen in Figure 1, the ‘cough’ items were correlated with each other and the dyspnoea subscale. We chose not to include a ‘cough’ subdimension because there is debate on whether two items are sufficient to measure an overall underlying construct.[28,29] However, when the CCQ is used to compare patients from primary care with those from, for example, secondary or tertiary care, the items on being concerned, depressed and/or limited in daily and social functioning could be relevant.

Implications for clinical practice and future research

Taken together, when using the CCQ in primary care patients with COPD one should be aware of the considerable differences between the originally described scale structure,[3] and the structure that was discovered in the present study. On the basis of the present results, we recommend the use of an alternative, shortened version of the CCQ containing six items on dyspnoea and cough to screen for exacerbations.

To develop a questionnaire that measures the full range of clinical impairment related to COPD, the use of both qualitative and quantitative methods is essential. For this purpose, the relevant aspects of perceived functional impairment should be obtained from focus group interviews with patients and health-care workers (including COPD experts, nurses and pulmonologists). Subsequent appropriate statistical methodology, i.e., factor analyses followed by reliability (internal consistency, test–retest) and concurrent and convergent validity analyses, should guide the process of item reduction and structure development. The guidelines even state that ‘a questionnaire is not considered valid until the statistical properties have been tested’.[26] The methodology that was used to develop the CCQ does not meet all of these criteria. For example, no statistical procedures or techniques were used to guide and validate the process of item selection and determining scale structure; instead this process was based on experts’ opinions. As a consequence, the original scale structure was not confirmed by factor analysis and some items appeared not to be relevant.

Strengths and limitations

The key strengths of the present study design are the use of both confirmatory and exploratory factor analyses to determine the latent structure of the CCQ. Furthermore, both studies were conducted in relatively large samples of patients with COPD who were diagnosed according to the GOLD guidelines.

Limitations of the study include the relatively low response rate and the inclusion of primary care patients only, which may bias the results and limit generalisation to more severe patients; however, the CCQ was developed primarily for use in primary care as reflected in the patient population included by van der Molen et al. [3] In addition, there might be some overlap between the two samples that were combined. However, the second study sample was relatively small (N=85) and their data were collected 2 years after the data collection of the first sample had finished.

Conclusions

This study revealed a clear two-component structure of the CCQ that, in a shortened version, could be an excellent instrument to screen for patient-reported exacerbations of COPD. Future research should further validate the stability and test–retest validity of the CCQ dimensions in primary care patients and in patients with advanced COPD. Our findings clearly support the potential usefulness of the CCQ to screen for exacerbations of COPD as a major patient-reported outcome measure.