Diagnostic accuracy of pre-bronchodilator FEV1/FEV6 from microspirometry to detect airflow obstruction in primary care: a randomised cross-sectional study.

BACKGROUND: Forced expiratory volume in 1s/forced expiratory volume in 6 s ( FEV1/FEV6) assessment with a microspirometer may be useful in the diagnostic work up of subjects who are suspected of having COPD in primary care. AIM: To determine the diagnostic accuracy of a negative pre-bronchodilator (BD) microspirometry test relative to a full diagnostic spirometry test in subjects in whom general practitioners (GPs) suspect airflow obstruction.
METHODS: Cross-sectional study in which the order of microspirometry and diagnostic spirometry tests was randomised. Study subjects were (ex-)smokers aged ≥50 years referred for diagnostic spirometry to a primary care diagnostic centre by their GPs. A pre-BD FEV1/FEV6 value <0.73 as measured with the PiKo-6 microspirometer was compared with a post-BD FEV1/FVC (forced vital capacity) <0.70 and FEV1/FVC<lower limit of normal (LLN) from diagnostic spirometry.
RESULTS: One hundred and four subjects were analysed (59.6% males, 42.3% current smokers). Negative predictive values from microspirometry for airflow obstruction based on the fixed and LLN cut-off points were 94.4% (95% confidence interval (CI), 86.4-98.5) and 96.3% (95% CI, 88.2-99.3), respectively. In all, 18% of positive microspirometry results were not confirmed by a post-BD FEV1/FVC <0.70 and 44% of tests were false positive compared with the LLN criterion for airflow obstruction.
CONCLUSIONS: Pre-bronchodilator microspirometry seems to be able to reliably preselect patients for further assessment of airflow obstruction by means of regular diagnostic spirometry. However, use of microspirometry alone would result in overestimation of airflow obstruction and should not replace regular spirometry when diagnosing COPD in primary care.

RCT Entities:   Population Interventions Outcomes

Introduction

Chronic obstructive pulmonary disease (COPD) is widely underdiagnosed in primary care.[1-4] The hallmark of COPD is chronic airflow obstruction objectified by spirometry after the administration of a bronchodilator (BD).[5] High-quality spirometry requires extensive training of staff, reliable equipment and well-standardised measurement procedures.[6,7] Although the majority of general practitioners (GPs) recognise the importance of confirmatory spirometry testing when diagnosing COPD, it is still underutilised.[8] According to GPs, inability to apply spirometry during consultation is an important barrier.[8]

Diagnostic spirometry requires measurement of the ratio of forced expiratory volume in 1s (FEV1) and forced vital capcity (FVC) in order to calculate the FEV1/FVC ratio, which is the main diagnostic criterion for COPD. Forced expiratory volume in 6 s (FEV6) can be used as a valid surrogate for FVC[9] and is less prone to measurement error.[10]

Simple hand-held microspirometers such as the PiKo-6 (nSpire Health Inc., Longmont, CO, USA) and COPD6 (Vitalograph Ltd, Ennis, Ireland) can be used to measure the FEV1/FEV6 ratio. Results of previous studies indicate that these devices are effective and reliable screening tools that can reduce underdiagnosis of COPD in primary care.[11,12] A pre-BD FEV1/FEV6 assessment takes little time and has the potential to be integrated into a GP's office consultations, which is not the case with a full (pre- and post-BD) spirometry test. On the basis of microspirometry test results, candidates for further diagnostic spirometric assessment can be selected, which could reduce underdiagnosis of COPD and at the same time increase the efficiency of full diagnostic spirometry use in primary care. However, this will only be the case when a negative (i.e., normal) pre-BD FEV1/FEV6 value from a microspirometry test rules out the presence of airflow obstruction with sufficient certainty.

The aim of this study was to determine the diagnostic accuracy of a pre-BD FEV1/FEV6 ratio from microspirometry relative to a post-BD FEV1/FVC ratio from a diagnostic spirometry test in patients referred for spirometry by GPs because of respiratory symptoms that may suggest underlying COPD. Because we focus on the potential role of microspirometry to select patients for further diagnostic spirometry testing, we were especially interested in the negative predicted value of a normal microspirometry result.

Materials and Methods

Study design and subjects

A randomised cross-sectional diagnostic study was set up at the ‘Stichting Huisartsen Laboratorium’ (SHL), a regional primary care diagnostic centre that performs diagnostic spirometry tests at several sites for general practitioners (GPs) in the South-Western part of the Netherlands. Participants were recruited from among individuals who visited the diagnostic centre for a diagnostic spirometric test based on a referral by their GP for respiratory symptoms that may suggest underlying COPD. We included subjects who were 50 years or older and who were current or former smokers (⩾1 pack year). Exclusion criteria were: (1) refusal or inability to give informed consent; (2) having undergone a spirometry test in the previous 5 years; (3) having already been diagnosed with COPD; and (4) anticipated inability to perform 12 forced blows as presumed by the lung function technician.

The study was conducted between October 2010 and January 2012. According to the medical ethics review board of the Radboud University Medical Centre, the study was exempted from ethics review (file number 2010/286). All participants gave written informed consent before any study procedure took place.

Study procedures

All participants underwent diagnostic spirometry and a microspirometry test before and after administration of 400 μg of aerosolised salbutamol by means of a Volumatic spacer, all during the same visit to the diagnostic centre. The order of diagnostic spirometry and microspirometry testing was randomised. Participating sites of the diagnostic centre received sealed envelopes with the randomisation code. Subjects had to start with either microspirometry or the diagnostic spirometry test before as well as after the administration of the BD, which resulted in the following two possible test sequences:

pre-BD microspirometry–pre-BD spirometry–post-BD microspirometry–post-BD spirometry; or

pre-BD spirometry–pre-BD microspirometry–post-BD spirometry–post-BD microspirometry

Following the diagnostic centre’s protocol, all tests were performed at least 12 h after the last inhalation of a short-acting BD or a long-acting beta-2-agonist, and at least 72 h after inhalation of tiotropium.

Microspirometry

Microspirometry testing was performed by trained lung function staff who were given a uniform and brief training on how to use the PiKo-6 according to the manufacturer’s instructions. PiKo-6 devices were checked for calibration errors before the start of the study by the investigators.

Subjects were required to inhale maximally, and exhale as hard and as fast as possible into the mouthpiece of the PiKo-6 until an end-of-test beep was heard after 6 s. At least three valid attempts were taken before and after the administration of the BD. The PiKo-6 has an automatic test quality alert and indicates attempts that were invalid because of coughing or abnormal blow. The highest FEV1 and FEV6 value of the three pre-BD measurements were used (which were not necessarily from the same blow) and the FEV1/FEV6 ratio was calculated. We used a fixed FEV1/FEV6 cut-off point of <0.73 as an indicator for airflow obstruction, which was shown to be a valid alternative to FEV1/FVC <0.70 in previous studies.[13,14]

Diagnostic spirometry

Diagnostic spirometry testing was performed by the same lung function technicians, using the PC-based SpiroPerfect spirometer (Welch Allyn, New York, NY, USA). The spirometer was calibrated each day before testing started. The spirometric tests had to meet the recommendations of the ATS/ERS guidelines.[15] At least three reproducible blows of good quality were taken for pre-BD and post-BD measurements. The post-BD measurement with the highest FEV1/FVC ratio was recorded and used for analysis. The lower limit of normal (LLN) values used for analysis were calculated using the 2012 Global Lungs Initiative (GLI-2012) spirometric prediction equations.[16]

Questionnaire

Participants filled out a questionnaire about possible previous diagnoses of chronic respiratory conditions, cigarette smoking, respiratory medication use, previous spirometry tests and reason for referral by the GP. During the waiting time between the pre- and post-BD tests, demographic and disease-specific information (e.g., respiratory symptoms and exacerbations) were inquired in a standardised way by the lung function technician.

Outcomes, sample size and analysis

The main outcome of interest for the study was the negative predictive value (NPV) of a pre-BD FEV1/FEV6 value <0.73 as measured with the PiKo-6 microspirometer compared with a post-BD FEV1/FVC value <0.70 from a diagnostic spirometry test, the latter serving as the gold standard. Positive predictive value, sensitivity and specificity of a pre-BD FEV1/FEV6 value <0.73 were also analysed. The same calculations were made with a post-BD FEV1/FVC 17]

Sample size was chosen to be able to demonstrate an NPV of 95%, for microspirometry with a lower confidence limit of 90%. Earlier research showed a prevalence of 12–30% of undiagnosed COPD in male smokers aged 40 years and over.[18] However, because of the inclusion criteria the subjects in our study would be older and all would have respiratory symptoms. Therefore, we assumed a 35% prevalence of undetected airflow obstruction in subjects referred for spirometry by their GP. With the aforementioned assumptions, we calculated a sample size of n=112. Given the cross-sectional design of the study, no drop-outs were expected.

Descriptive statistics (numbers (%)) were used to describe the study population’s characteristics. The diagnostic accuracy measures (i.e., NPV, positive predictive value, specificity and sensitivity) were calculated using crosstabs, and 95% confidence interval (CI) were determined. Kappa statistics for agreement between pre-BD FEV1/FEV6 and post-BD FEV1/FVC cut-off points were also calculated. Moreover, a receiver operating characteristic curve and its area under the curve were calculated, with post-BD FEV1/FVC<0.7 as the gold standard for chronic airflow obstruction.

Not all subjects were referred specifically for suspected COPD. In some cases, the GP’s referral indication for the diagnostic spirometry test was asthma or reasons less clear (e.g., ‘dyspnoea’ or ‘chronic cough’). Therefore, a sub-analysis was conducted with the subjects who had been referred for spirometry by their GP specifically for evaluation of possible underlying COPD. Statistical analyses were performed using IBM SPSS software (Chicago, IL, USA, version 20).

Results

Study population

A total of 121 subjects were recruited (see Figure 1), of whom 111 subjects were eligible for the study. Of them, six failed to complete the microspirometry test, and diagnostic spirometry data were incomplete for one patient. Valid diagnostic spirometry and microspirometry data were available for 104 study participants. Table 1 shows baseline and clinical characteristics of the subjects. Airflow obstruction (i.e., post-BD FEV1/FVC <0.7) was observed in 44 subjects (42.3%), with most of them (88.6%) being classified as having mild to moderate airflow obstruction (see Table 1). Forty-three per cent of subjects with no airflow obstruction used prescribed respiratory medication. Twelve subjects (11.5%) met the criteria for a reversible airflow obstruction.

Figure 1

Flow chart of subject recruitment and selection.

Table 1

Characteristics of the study sample (n=104)

Characteristic	Number	%
Male	62	59.6
Age group
50–59 years	42	40.4
60–69 years	51	49.0
⩾70 years	11	10.6
Current smokersa	44	42.3
Packyears b
1–14	32	30.8
15–30	32	30.8
⩾30	39	37.5
Reason to refer for spirometry
COPD	55	52.9
Asthma	16	15.4
Asthma/COPD	5	4.8
Otherc	11	10.6
None mentioned	17	16.3
Post-BD FEV1/FVC<0.70	44	42.3
Severity of airflow obstruction
Mild (FEV1 ⩾80% predicted)	22	50
Moderate (50% ⩽FEV1<80% predicted)	17	38.6
Severe (30% ⩽FEV1<50% predicted)	5	11.4
Reversible airflow obstructiond	12	11.5
Use of respiratory medication a
Short-acting bronchodilators	25	24.5
Long-acting bronchodilators	11	10.8
Inhaled corticosteroids	15	14.7

Abbreviations: COPD, chronic obstructive pulmonary disease; FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity.

Two missing.

One missing.

Other: three referred on account of dyspnoea, three for coughing and sputum production and five for an unspecified diagnostic referral for spirometry.

Post-BD FEV1 12% and 200 ml higher compared with pre-BD FEV1.

Diagnostic accuracy of pre-BD FEV1/FEV6 ratio <0.73

Of the 54 subjects with a negative microspirometry test (i.e., pre-BD FEV1/FEV6 ⩾0.73), absence of airflow obstruction was confirmed by a negative diagnostic spirometric test (i.e., post-BD FEV1/FVC ⩾0.70) in 51 subjects (see Figure 2). Thus, the NPV was 94.4% (95% CI, 86.4–98.5). Table 2 shows the diagnostic test characteristics both (i.e., ⩾0.73 and ⩾LLN) for the diagnostic spirometry cut-off points and for the subgroup of subjects with a specific referral for suspected COPD. The NPV of pre-BD FEV1/FEV6 ⩾0.73 was high for both definitions of airflow obstruction but with 96.3% (95% CI, 88.2–99.3) slightly better for the LLN cut-off point. The NPV for subjects referred specifically for suspected COPD was 96.3% for both diagnostic spirometry cut-off points. Positive microspirometry tests were confirmed by positive diagnostic spirometry tests in only 82.0% (95% CI, 73.3–86.3) and 56.0% (95% CI, 47.3–59.3), respectively (see Table 2 for these positive predictive values and the results for specificity, sensitivity and Kappa). Figure 3 shows the receiver operating characteristic curve for different FEV1/FEV6 cut-off points. The area under the curve was 0.937.

Figure 2

Scatter plot of pre-bronchodilator FEV1/FEV6 from microspirometry against post-bronchodilator FEV1/FVC from diagnostic spirometry.

Table 2

Diagnostic test characteristics of pre-BD FEV1/FEV6 <0.73 values from microspirometry versus post-BD FEV1/FVC from diagnostic spirometry for different cut-off points

Cut-off point	All subjects (n=104)		Subjects referred for COPD (n=55)
	<0.70	<LLN	<0.70	<LLN
NPV (95% CI)	94.4 (86.4–98.5)	96.3 (88.2–99.3)	96.3 (88.3–99.8)	96.3 (83.1–99.8)
PPV (95% CI)	82.0 (73.3–86.3)	56.0 (47.3–59.3)	78.6 (66.1–82.0)	57.1 (44.4–60.5)
Sensitivity (95% CI)	93.2 (83.3–98.1)	93.3 (78.8–98.8)	95.7 (80.4–99.8)	94.4 (73.1–99.7)
Specificity (95% CI)	85.0 (77.8–88.6)	70.3 (64.4–72.5)	81.3 (70.3–84.2)	68.4 (59.0–70.9)
Kappa	0.77 (0.60–0.85)	0.53 (0.36–0.60)	0.75 (0.49–0.82)	0.53 (0.27–0.60)

Abbreviations: FEV1, forced expiratory volume in 1 s; FEV6, forced expiratory volume in 6 s; FVC, forced vital capacity; LLN, lower limit of normal; NPV, negative predictive value; PPV, positive predictive value.

Estimates are shown with 95% confidence intervals in parentheses.

Figure 3

Receiver operating characteristic (ROC) curve and its coordinates for the forced expiratory volume in 1 s /forced expiratory volume in 6 s (FEV1/FEV6) ratio using post-BD FEV1/FVC<0.7 as criterion for chronic airflow obstruction (AUC: 0.937).

Discussion

Main findings

This study shows that the use of microspirometry seems to be a method for preselecting subjects for further diagnostic spirometry testing in the diagnostic work up of patients who, according to their GP, may have chronic airflow obstruction. Taking into account that the NPV of microspirometry was 94.4%, a subject with a negative microspirometry test (pre-BD FEV1/FEV6 ⩾0.73) is rather unlikely to show airflow obstruction in a subsequent diagnostic spirometry test and thus to have COPD. On the other hand, 18% of subjects with a positive microspirometry test (i.e., pre-BD FEV1/FEV6 <0.73) had no airflow obstruction (post-BD FEV1/FCV >0.70) according to diagnostic spirometric assessment and 44% of these subjects did not fulfil the LLN criterion for airflow obstruction. This illustrates that (pre-BD) microspirometry should not be used to diagnose COPD.

Strengths and limitations of this study

In our study the test sequence was randomised and therefore, the effect of possible fatigue and learning effect of previous blows was minimised. However, in three subjects the randomisation sequence was violated. Data of these subjects were used for analysis as the effect of this protocol violation was considered negligible.

The fixed post-BD <0.70 FEV1/FVC cut-off is a widely accepted criterion to define airflow obstruction in COPD.[5] As FEV1 decreases more quickly with age than does FVC, this criterion tends to overdiagnose COPD in the elderly.[19,20] Therefore, some studies have suggested the use of LLN to define airflow obstruction when diagnosing COPD.[21] The LLN is based on patient characteristics (age, height and race) and an FEV1/FVC value below the lower fifth percentile of an appropriate healthy reference group is considered abnormal. Given the changing perception of how to detect airflow obstruction, we also used the LLN as a gold standard diagnostic spirometry outcome to validate the FEV1/FEV6 ratio from microspirometry. The analyses based on LLN showed slightly higher NPVs (96.3%) compared with the fixed cut-off value (94.4%).

Subjects were referred for diagnostic spirometry by their GPs. As shown, there were several reasons for referral, of which the largest part (n=55/104) consisted of ‘COPD’. ‘Asthma’ or ‘asthma/COPD’ were other recurring indications for referral (n=21). For GPs it is difficult to differentiate between COPD and asthma, given the significant overlap in the clinical presentation of these two conditions.[22,23] Therefore, all subjects were included in the main analysis regardless of the indication as written on the referral form. The NPVs of the subgroup analyses with patients referred for COPD only were very similar to the NPVs when all subjects were included, indicating that this was a valid approach.

A limitation of a PiKo-6 test is that it displays the highest FEV1 and FEV6 of a set of attempts that are not necessarily reproducible. Therefore, any outliers in these values may influence the results for an individual substantially. In fact, 19% of PiKo-6 tests (i.e., three valid attempts) failed to fulfil the criteria for reproducibility for the pre-BD FEV1 value and 34% failed to fulfil the criteria for the reproducibility of the pre-BD FEV6 value.[15] This is another good reason why a full spirometric exam remains essential to confirm a diagnosis of COPD.

In this study we did not look at the validity (i.e., accuracy and precision) of the PiKo-6 device itself, and to our knowledge there are no published reports about the validity of this particular device. Moreover, different types or brands of microspirometers may differ in terms of their accuracy and precision for the FEV1 and FEV6 values they measure. Therefore our observations in the current study cannot be extrapolated to other devices than the PiKo-6.

A final limitation is the fact that we did not have information on the quality and reproducibility of the full spirometric test as well. The diagnostic centre was used to save only the best FEV1 and FVC value in their database, and therefore we have to rely on the professional judgement of the lung function staff of the primary care diagnostic centre.

Interpretation of findings in relation to previously published work

Previous studies have suggested different cut-off points for the FEV1/FEV6 ratio (i.e., <0.75,[9] <0.80[8] and <0.70[24,25] to detect airflow obstruction in primary care settings. In our study, we used the cut-off point of <0.73, which has been recommended as the preferred alternative for FEV1/FVC <0.70.[9,13,26] It is to be expected that a higher FEV1/FEV6 cut-off point decreases the number of false-negative results and increases the number of false-positive results. This was also the case in our study: at a cut-off FEV1/FEV6 ratio <0.75 the NPV would have been 97.8% and the positive predictive value 72.9% (instead of 94.4% and 82.0%, respectively).

In previous studies, the study population consisted of current and former smokers aged ⩾40, ⩾45 or ⩾50 years.[11,12,19,26] In our current study, subjects were referred by their GP for spirometry testing because of respiratory symptoms that could suggest underlying COPD in addition to being at risk on the basis of age and smoking history. This might explain the high prevalence of spirometry-confirmed airflow obstruction in our study (44.4%) compared with previous studies.[11,12,19,26]

Moreover, previous studies focused on the sensitivity and specificity of microspirometry. We specifically looked at the NPV as the primary outcome. In our view NPV best reflects the purpose that microspirometry can have in primary care: to preselect candidates for full diagnostic spirometry in order to avoid unnecessary testing. In our study, the NPV of the pre-BD FEV1/FEV6 ratio from microspirometry was slightly different from the NPVs reported by Sichletidis et al. [24] and Frith et al.,[12] who reported NPV values of 98% and 91%, respectively. Both values are within the 95% CI (86.4–98.5) of the NPV that we found in our study. The main difference of the study by Sichletidis and co-workers compared with our study is their use of post-BD FEV1/FEV6 microspirometry values, which may be expected to correlate better with the post-BD FEV1/FVC value, but is less practical than using a pre-BD test. Dissimilarities with the study by Frith et al. are the aforementioned differences in subject recruitment.

Implications for future research and practice

This is the first validation study to determine the diagnostic accuracy of a hand-held microspirometer in subjects with clinically suspected airflow obstruction that point to underlying COPD in a primary care setting. Given the findings from this study, a microspirometer could provide GPs with a simple and reliable method to preselect patients for full diagnostic spirometry. The simplicity of a microspirometer fits into the tight work schedule of GPs during office consultations. However, one question that remains is whether the same diagnostic test characteristics are obtained when microspirometry is performed in primary care practices by GPs themselves. Thorn et al. found that FEV1/FEV6 assessment (with a COPD6) by asthma/COPD nurses in primary care practices had acceptable sensitivity and specificity, although the test characteristics were not as good as in our study.[26] Furthermore, microspirometers have the potential to optimise early referral for spirometry and may facilitate early, targeted interventions aimed at reducing the burden of COPD. The effectiveness of such interventions needs to be evaluated as well, as robust evidence on this issue is currently lacking. However, based on the high percentage of patients that will be misdiagnosed with COPD when such a device is used only (with a cut-off <0.73), confirmative diagnostic spirometry remains essential. A false-positive microspirometry test could be the result of reversible airflow obstruction (i.e., the hallmark of asthma). Therefore, there might be a positive ‘side-effect’ of microspirometry that warrants further study as well, although pre-BD microspirometry cannot rule out asthma and should not be used to preselect patients at risk for asthma.

Conclusions

Pre-BD microspirometry seems to be a valid method to preselect subjects for full diagnostic spirometry in the diagnostic work up of subjects who are suspected of having COPD in primary care. However, microspirometry should not replace regular diagnostic spirometry.