Comparison of the COPD Population Screener and International Primary Care Airway Group questionnaires in a general Japanese population: the Hisayama study.

BACKGROUND: The incidence of chronic obstructive pulmonary disease (COPD) is increasing worldwide. In Japan and other countries, epidemiological studies have found that many patients with COPD are underdiagnosed and untreated, and thus, early detection and treatment of COPD has been emphasized. Screening questionnaires may have utility in the initial detection of COPD.
OBJECTIVE: This study aimed to validate and compare the COPD Population Screener (COPD-PS) and the International Primary Care Airway Group (IPAG) questionnaires in a general Japanese population. PATIENTS AND METHODS: Eligible subjects 40 years of age and older living in the town of Hisayama were solicited to participate in a health checkup in 2012. All subjects 40-79 years of age without physician-diagnosed asthma or lung resection were recruited, and 2,336 subjects who fully completed both questionnaires and who had valid spirometry measurements were analyzed. Persistent airflow obstruction (AO) was defined by a postbronchodilator forced expiratory volume in 1 second/forced vital capacity <0.70. Receiver operating characteristic curves, net reclassification improvement, and integrated discrimination improvement were used to examine the ability of the COPD-PS and IPAG questionnaires to discriminate between subjects with and without AO.
RESULTS: The overall area under the receiver operating characteristic curve for the COPD-PS questionnaire was 0.747 (95% confidence interval [CI], 0.707-0.788) and for the IPAG was 0.775 (95% CI, 0.735-0.816), with no significant difference (P=0.09). The net reclassification improvement and integrated discrimination improvement were -0.107 (95% CI, -0.273-0.058; P=0.203) and -0.014 (95% CI, -0.033-0.006; P=0.182), respectively.
CONCLUSION: The five-item COPD-PS questionnaire was comparable to the eight-item IPAG for discriminating between subjects with and without AO. The COPD-PS is a simple and useful screening questionnaire for persistent AO.

Introduction

COPD is a common, usually progressive disease characterized by chronic inflammation of the airways and persistent airflow limitation. Its prevalence is increasing worldwide, and it is expected to be the third largest global cause of mortality by the year 2030.1 In Japan, a large epidemiological study (the Nippon COPD Epidemiological Study) reported that the prevalence of airflow limitation was 10.6%, and at least 8.6% of subjects were sought to have COPD.2 Several studies have shown that the prevalence of potentially undiagnosed airflow limitation in both Western3–5 and Asian6 countries is ~ 3%–15%. Many patients with COPD continue to be underdiagnosed and untreated.2,7 Since COPD is a preventable and treatable disease, the importance of early detection has been emphasized.8

The use of simple COPD screening questionnaires to detect persistent airflow limitation may help in the early diagnosis of COPD. These tools reliably detect airflow limitation in the general population and may facilitate the early, accurate diagnosis of COPD in general practice settings.9,10 Some COPD diagnostic questionnaires have already been reported.11–14

The COPD Population Screener (COPD-PS), which was developed by a clinician working group in the United States, is a five-item, self-administered questionnaire that was validated for screening individuals in the general population who are at high risk of COPD. It is composed of three COPD-related items (breathlessness, productive cough, and activity limitation) and one question, each regarding smoking history and age.15 In a previous population-based study, we verified the validity of the Japanese version of the COPD-PS questionnaire for the identification of individuals at increased risk of airflow limitation.16 This instrument is easy to score and would be suitable for large-scale screening of possible airflow obstruction (AO). Another COPD screening tool, the International Primary Care Airway Group (IPAG) questionnaire,11 is in use worldwide. It consists of eight items and takes more time to administer than the COPD-PS questionnaire.

The purpose of this study was to compare these two questionnaires in the general Japanese population, as no previous research has done so, as well as to assess the ability of both instruments to discriminate between subjects with and without persistent AO.

Methods

Study population

This study was based on data from the Hisayama study, which is an ongoing population-based epidemiologic study designed to investigate the morbidity, mortality, and risk factors of cardiovascular and smoking-related diseases in the town of Hisayama, Japan. The town is located in a suburban area adjacent to Fukuoka city, a large urban center on Kyushu island in the southern part of Japan. The population of the town is ~8,000 and has been stable for over 50 years. According to national census data, the distributions of age and occupations in Hisayama have been almost identical to those across Japan since the 1960s.17

Study design

This cross-sectional study compared the screening efficacy of the COPD-PS and IPAG questionnaires in COPD patients. In 2012, registered subjects 40 years of age and older were solicited to participate in a town-wide health checkup that included spirometry. Of the 2,643 subjects who were enrolled between June 2012 and October 2012, 307 were excluded for the following reasons: 105 had physician-diagnosed asthma, 22 had a previous lung resection, 159 had poor studied data, and 21 had records with missing data. The final analysis included data for 2,336 subjects with fully completed COPD-PS and IPAG questionnaires and valid spirometry measurements.

The subjects who provided informed consent to participate in the health checkup independently completed the Japanese versions of the COPD-PS and IPAG questionnaires, and then, in addition to their usual clinical tests, underwent spirometry using a CHESTGRAPH HI-105 spirometer (Chest MI, Tokyo, Japan). Each subject performed at least three forced vital capacity (FVC) maneuvers according to the recommended method. The results were assessed by two pulmonary physicians, who visually inspected the flow–volume curve and excluded subjects without at least two satisfactory tests. The highest forced expiratory volume in 1 second (FEV1) and FVC values were used for analysis. Reference values for percent predicted FEV1 were derived from Japanese criteria. The subjects who had pre-bronchodilator (BD) FEV1/FVC <0.70 were eligible for post-BD testing, in which spirometry was performed 15 minutes after inhalation of salbutamol (GlaxoSmithKline, Tokyo, Japan) via a metered-dose inhaler with a spacer, according to the procedure recommended.18 Persistent AO was defined as having a post-BD FEV1/FVC <0.70. The subjects with persistent AO were categorized according to the Global Initiative for Chronic Obstructive Lung Disease criteria (mild, FEV1 ≥80% predicted; moderate, 50%≤ FEV1 <80% predicted; severe, 30%≤ FEV1 <50% predicted; very severe, FEV1 <30% predicted).19

The study protocol was approved by the institutional review board for clinical research of Kyushu University (numbers 21–37, 24–82, and 24–123) and of Kagoshima University (numbers 156 and 279), and all subjects provided their written informed consent prior to participation in the study.

COPD-PS

The COPD-PS is a brief, reliable, self-scored questionnaire to identify individuals likely to have COPD. It consists of five items, three assessing COPD-related symptoms on a 5-point scale, one on cigarette use (3-point scale), and one on the subject’s age (four categories). These five items are scored 0, 1, or 2 with a summed total score ranging from 0 to 10. In the Japanese version of the COPD-PS questionnaire, a cutoff point of 4 has been found to be useful for COPD screening.16

IPAG questionnaire

The IPAG questionnaire is also a self-scored questionnaire and has been validated in smokers as a screening tool for COPD diagnosis.11 The IPAG questionnaire is composed of eight items: one on the subject’s age (four categories), one on body mass index (3-point scale), one on cigarette use (4-point scale), and five on symptoms/history (2- or 3-point scales). Each question is scored individually, with a summed total score ranging from 0 to 38. A suggested cutoff score of 17 is used for smokers in general health checkup settings and general practices in Japan. However, there is no cutoff score for persistent AO in the general Japanese population, including never-smokers, and we therefore investigated this issue in this study.

Statistical analysis

Baseline data for the demographic characteristics of the study population and the informant questionnaires were evaluated in descriptive analyses. The Kruskal–Wallis test was used to compare nonparametric data between groups. Spearman correlations were used to examine the strength of associations between informant and performance measures. For the subjects who used a BD, the values of post-BD FEV1/FVC were utilized as FEV1/FVC. Sensitivity, specificity, and positive and negative predictive values were calculated. Another way to evaluate the utility of screening tests is with the likelihood ratio;20 likelihood ratios range from 0 to infinity; larger numbers provide more convincing evidence of a disease, smaller numbers indicate that the disease is less likely, and ratios close to one lack diagnostic value. Likelihood ratios (positive and negative) were calculated for both the COPD-PS and the IPAG questionnaires. Receiver operating characteristic (ROC) curves and areas under the ROC curves (AUCs) were generated to reflect graphically and quantitatively the ability of the COPD-PS and the IPAG questionnaires to discriminate between subjects with and without persistent AO using the DeLong method.21 Another method proposed by Pencina et al22 was also utilized for this purpose; this approach assesses the ability of a model to reclassify case and control subjects, respectively, on the basis of the individual-estimated probability of an event. The ability of the model to reclassify is summarized by the net reclassification improvement (NRI) and integrated discrimination improvement (IDI). The categorical NRI requires the definition of strata risk. We defined four strata dividing the risk of persistent AO into four quartiles, namely, Q1, Q2, Q3, and Q4. The NRI considers only the changes in the estimated prediction probabilities that imply a change from one category to another. In contrast, the IDI does not require a prior definition of strata risk and considers the change in the estimation prediction probabilities as a continuous variable. We computed the NRI and IDI and examined the accuracy of the two screening questionnaires in diagnosing AO. All statistical analyses were performed using R Version 3.1.0.23 The results were considered statistically significant when P<0.05.

Results

Participant characteristics

The baseline characteristics of the 2,336 subjects stratified by airflow limitation category following post-BD spirometry are presented in Table 1. The majority of subjects (88.9%) showed an initial FEV1/FVC ≥0.70. Following post-BD spirometry, 6.4% of subjects were found to have AO. Almost all the AO subjects (94.0%) were classified as having mild or moderate COPD; only 6.0% of the AO subjects had severe or very severe COPD. The AO subjects were older, were more likely to be men, had lower body mass index, had a higher number of pack-years smoked, and were more likely to be former or current smokers (Table 1).

Table 1

Baseline characteristics

Characteristics	No AOa		AOa	Kruskal–Wallis test (P-value)
	Pre-BD FEV1≥70%	Pre-BD FEV1 <70%
Subjects, n	2,076	110	150
Age (years), mean (SD)	60.3 (10.5)	65.0 (9.4)	66.5 (8.9)	<0.0001
Male (%)	41.7	45.5	68.7
BMI (kg/m2), mean (SD)	23.3 (3.4)	22.8 (2.9)	22.2 (3.0)	<0.001
Pack-year				<0.0001
Mean (SD)	11.6 (19.7)	15.6 (21.5)	31.0 (30.7)
Median (IQR)	0 (0–19.5)	0 (0–30.8)	26.5 (0–47.5)
Smoking status (%)
Never	59.3	52.7	28.0
Former	25.4	24.5	35.3
Current	15.3	22.7	36.7
COPD-PS score				<0.001
Mean (SD)	2.4 (1.4)	2.9 (1.4)	3.9 (1.7)
Median (IQR)	2 (2–4)	2 (2–4)	4 (2–5)
IPAG questionnaire score, mean (SD)	16.7 (4.9)	18.8 (4.0)	22.0 (5.0)	<0.0001
%FVC, mean (SD)	100.7 (13.0)	101.4 (13.6)	96.4 (16.6)	<0.01
Pre-BD FEV1% predicted, mean (SD)	95.4 (13.2)	84.8 (12.2)	74.6 (16.7)	<0.0001
Post-BD FEV1% predicted, mean (SD)	NA	91.2 (13.0)	77.9 (16.7)	<0.0001
Pre-BD FEV1%, mean (SD)	77.7 (4.6)	67.6 (2.6)	62.2 (6.4)	<0.0001
Post-BD FEV1%, mean (SD)	NA	73.6 (3.6)	63.2 (6.8)	<0.0001
Reversibility (%)	NA	29.1	18.0
COPD stage (%)
Mild	NA	NA	46.0
Moderate	NA	NA	48.0
Severe	NA	NA	5.3
Very severe	NA	NA	0.7

Notes:

No AO, post-BD FEV1/FVC ≥0.7; AO, post-BD FEV1/FVC <0.7.

Abbreviations: AO, airflow obstruction; BD, bronchodilator; FEV1, forced expiratory volume in 1 second; BMI, body mass index; IQR, interquartile range; COPD-PS, COPD Population Screener; IPAG, International Primary Care Airway Group; FVC, Forced Vital Capacity; NA, not applicable; SD, standard deviation.

The mean and median scores on the COPD-PS questionnaire were higher for the AO subjects. For the AO and non-AO subjects, respectively, the mean scores were 3.9 and 2.4, while the median scores were 4 and 2. The mean scores on the IPAG questionnaire were also higher for the AO subjects. The IPAG and the COPD-PS questionnaires were correlated with FEV1/FVC (r=−0.356 and r=−0.301, respectively, P<0.001), and the two questionnaires correlated with each other (r=0.622, P<0.001; Table 2).

Table 2

Strength of the association between informant and performance measures

Variable	COPD-PS	IPAG questionnaire
IPAG questionnaire	0.622*	–
%FVC	−0.117*	−0.033**
FEV1%	−0.301*	−0.356*

Notes:

P-values <0.001 for Pearson’s correlation coefficients.

P-values <0.01 for Pearson’s correlation coefficients.

Abbreviations: COPD-PS, COPD Population Screener; IPAG, International Primary Care Airway Group; FVC, forced vital capacity; FEV1, forced expiratory volume in 1 second.

Discriminating subjects with and without AO

The previously identified cutoff point of 4 was used for the COPD-PS questionnaire. The crude odds ratio (OR) of the COPD-PS questionnaire for AO was 5.52, and the sensitivity of COPD-PS questionnaire was 66.7% and the specificity was 73.4%. In this population-based study that included never-smokers, a cutoff point of 20 on the IPAG questionnaire would be adequate for screening for AO (Table 3). The crude OR of the IPAG questionnaire for AO, using a cutoff point of 20, was 6.56. If a cutoff point of 17 was used on the IPAG questionnaire, the sensitivity was higher (86.0%) but both the specificity and AUC were much lower, 46.2% and 0.66%, respectively. If a cutoff point of 20 was used instead of 17, the sensitivity was slightly lower (71.3%) but both the specificity and AUC were higher, 75.2% and 0.72%, respectively.

Table 3

Cutpoint evaluation of COPD-PS and IPAG questionnaires to discriminate between AO and no AO states

Screening questionnaire	Cutpoint	OR	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)	Percent correctly classified	AUC
COPD-PS	4	5.52	66.7	73.4	14.7	97.0	73.0	0.70
IPAG	16	5.20	88.7	39.9	9.2	98.1	43.1	0.64
	17	5.28	86.0	46.2	9.9	98.0	48.8	0.66
	18	5.51	84.0	51.2	10.6	97.9	53.3	0.68
	19	5.48	78.7	59.8	11.8	97.6	61.0	0.69
	20	6.56	71.3	72.5	15.1	97.4	72.4	0.72
	21	6.62	70.0	73.9	15.6	97.3	73.7	0.72

Abbreviations: COPD-PS, COPD Population Screener; IPAG, International Primary Care Airway Group; AO, airflow obstruction; OR, odds ratio; AUC, area under the receiver operating characteristic curve.

ROC curves were generated to measure the properties of the COPD-PS and the IPAG questionnaires in discriminating subjects without AO from those with AO. The AUC values obtained from the ROC curve by discriminating AO from no AO were 0.747 (95% confidence interval [CI], 0.707–0.788) for the COPD-PS questionnaire and 0.775 (95% CI, 0.735–0.816) for the IPAG questionnaire (Figure 1). There was no significant difference in the AUC values with the two questionnaires (P=0.09).

Figure 1

Receiver operating characteristic curve of the COPD-PS and the IPAG questionnaires to discriminate between the subjects with and without AO.

Abbreviations: COPD-PS, COPD Population Screener; IPAG, International Primary Care Airway Group; AO, airflow obstruction.

Between the COPD-PS and IPAG questionnaires, no statistically significant differences were founded in terms of sensitivity, specificity, or positive and negative predictive values. However, compared with the COPD-PS questionnaire, the IPAG questionnaire had superior likelihood ratios for both a positive (2.51 vs 2.59) and negative test (0.45 vs 0.40), with nonoverlapping CIs (Table 4).

Table 4

Discriminative ability of the COPD-PS and IPAG questionnaires

Variable	AO vs no AO
	COPD-PS	IPAG questionnaire
AUC	0.747 (0.707–0.788)	0.775 (0.735–0.816)
Sensitivity	66.7 (59.1–74.2)	71.3 (64.1–78.6)
Specificity	73.4 (71.6–75.3)	72.5 (70.6–74.4)
PPV	14.7 (9.0–20.3)	15.1 (9.4–20.8)
NPV	97.0 (96.2–97.8)	97.4 (96.6–98.1)
LR+	2.51 (2.49–2.53)	2.59 (2.57–2.61)
LR−	0.45 (0.44–0.47)	0.40 (0.38–0.41)

Note: Values are represented as AUC (95% CI).

Abbreviations: COPD-PS, COPD Population Screener; IPAG, International Primary Care Airway Group; AO, airflow obstruction; AUC, area under the receiver operating characteristic curve; PPV, positive predictive value; NPV, negative predictive value; LR, likelihood ratio.

Reclassifications of subjects with and without AO are summarized in Table 5. For 672 (30.7%) subjects without AO, classification improved using the model with the COPD-PS questionnaire, and for 549 (25.1%) subjects, it became worse, with the net gain in reclassification proportion of 0.056. For subjects with AO, classification was improved in 21 subjects (14.0%) and less accurate in 28 subjects (18.7%), with the net gain in reclassification proportion of −0.047. Thus, the categorical NRI was 0.0096 (95% CI, −0.0868–0.106; P=0.845). The continuous NRI and IDI for predicting the presence of AO using the model with the COPD-PS questionnaire against the IPAG questionnaire were −0.107 (95% CI, −0.273–0.058; P=0.203) and −0.014 (95% CI, −0.033–0.006; P=0.182), respectively, and these were not statistically significant (Table 5).

Table 5

Reclassification among subjects with and without AO

Model with IPAG questionnaire	Model with COPD-PS
	Q1	Q2	Q3	Q4	Total
No AO
Q1	302	135	46	21	504
Q2	165	251	110	89	615
Q3	11	249	89	148	497
Q4	3	197	47	323	570
Total	481	832	292	581	2,186
AO
Q1	4	5	0	0	9
Q2	1	7	3	4	15
Q3	0	9	3	9	21
Q4	0	13	5	87	105
Total	5	34	11	100	150

Notes: NRI =−0.1073; IDI =−0.0135. Frequency across the four strata risks. Model with COPD-PS vs model with IPAG questionnaire.

Abbreviations: AO, airflow obstruction; COPD-PS, COPD Population Screener; IPAG, International Primary Care Airway Group; NRI, net reclassification improvement; IDI, integrated discrimination improvement; Q1, first quartile; Q2, second quartile; Q3, third quartile; Q4, fourth quartile.

The COPD-PS and the IPAG questionnaires had only a marginal difference in their ability to discriminate between the subjects with and without AO.

Discussion

In the present population-based study, the Japanese version of the COPD-PS questionnaire was compared with the IPAG questionnaire in a general Japanese population at the age of 40 years or older. The IPAG questionnaire had superior likelihood ratios of both positive and negative tests compared with those of the COPD-PS questionnaire. However, in comparison with two questionnaires, no significant differences were founded in the AUC values obtained from the ROC curves discriminating between subjects with and without AO and in the sensitivity, specificity, or positive and negative predictive values.

The two questionnaires were correlated with each other, FEV1/FVC, and for both of them, the ORs were significantly greater than 1.0. These findings suggest that the COPD-PS and the IPAG questionnaires are useful screening tools for detecting persistent AO. In addition, the subjects with a ≥4 point on the COPD-PS questionnaire and those with a ≥20 point on the IPAG questionnaire are at increased risk for AO.

Overall, the categorical and continuous NRI and IDI for predicting the presence of AO using the model with the COPD-PS questionnaire were 0.0096 (95% CI, −0.0868–0.106; P=0.845), −0.107 (95% CI, −0.273–0.058; P=0.203), and −0.014 (95% CI, −0.033–0.006; P=0.182), respectively. Reclassifications showed the IPAG questionnaire to be superior to the COPD-PS questionnaire; however, there was no significant difference between the two questionnaires.

A cutoff point of 17 on the IPAG questionnaire is used in general health checkup settings and general practices in Japan.24 In this study, we found that a cutoff point of 20 was superior. The reason for this discrepancy is not clear, although it may be due at least in part to the backgrounds of the study subjects. The present study was population based and included never-smokers. In contrast, in a previous study that enrolled subjects in general health checkup settings, a cutoff point of 17 resulted in a higher sensitivity of 86.0% and a smaller AUC of 0.66 (lower than 0.70) than a cutoff point of 20.24 The present study was implemented in the town of Hisayama, in which the age and occupational distributions of the population have been almost identical to those of Japan as a whole, and thus, we recommend a cutoff point of 20 for the general Japanese population.

Conclusion

There were only marginal differences between the two questionnaires in terms of ability to discriminate between subjects with and without AO. Moreover, the COPD-PS questionnaire consists of fewer items than the IPAG questionnaire and requires less time to complete. It takes ~5 minutes to fill out COPD-PS questionnaire, however, the completion of the IPAG questionnaire takes ~5–10 minutes. In this regard, the COPD-PS is a simple and useful screening questionnaire for persistent AO. The Japanese version of the COPD-PS questionnaire should be an adequate measure for large-scale screening for possible AO.