Influence of the practice setting on diagnostic prediction rules using FENO measurement in combination with clinical signs and symptoms of asthma.

OBJECTIVES: To evaluate the influence of the practice setting on diagnostic accuracy of fractional exhaled nitric oxide (FENO) for diagnosing asthma; and to develop prediction rules for diagnostic decision-making including clinical signs and symptoms (CSS).
SETTING: Patients from 10 general practices and 1 private practice of 5 pneumologists in ambulatory care. PARTICIPANTS: 553 patients, 57.9% female. Consecutive inclusion of diagnostic-naive patients suspected of suffering from obstructive airway disease. Exclusion criteria were respiratory tract infections within the last 6 weeks.
INTERVENTIONS: The index test was FENO measurement. Reference standard was the Tiffeneau ratio (forced expiratory volume in 1 s/vital capacity) or airway resistance as assessed by whole body plethysmography, with additional bronchoprovocation or bronchodilator testing. PRIMARY AND SECONDARY OUTCOME MEASURES: Asthma as determined by pneumologists, who were blind to FENO measurement results. Prediction rules were derived from multiple logistic regression analysis. A freely available calculator that allows computing all combinations was developed.
RESULTS: The practice setting only had minor influence on sensitivities of FENO cut-off points. In the final model (n=472), allergic rhinitis, wheezing and previous medication were positively associated with asthma. Increasing age and recurrent respiratory tract infections were negatively associated. The area under the curve (AUC) of FENO (AUC=0.650; 95% CI 0.599 to 0.701) increased significantly (p<0.0001) when combined with CSS (AUC=0.753; 95% CI 0.707 to 0.798). Presence of wheezing and allergic rhinitis allowed ruling in asthma with FENO >30 ppb. Ruling out with FENO <16 ppb in patients <43 years was only possible without allergic symptoms when recurrent respiratory tract infections were present.
CONCLUSIONS: FENO results should be interpreted in the context of CSS to enhance their diagnostic value in primary care. The final diagnostic model appears as a sound algorithm fitting well to the established diagnostic rules related to CSS of asthma. FENO appears more effective for ruling in asthma than for ruling it out. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

We used data from 553 patients to develop prediction rules for diagnostic decision-making with fractional exhaled nitric oxide (FENO) measurement including clinical signs and symptoms.

The general practice patients seemed to be selected more than those of the pneumologists’ practice, which might be explained by the study design. Therefore, it appeared adequate to extrapolate our FENO findings more cautiously to allow generalisation of the diagnostic algorithm.

The final model fitted well with the established clinical decision rules used by many physicians and led to a more conservative interpretation of the FENO measurements. However, a validation study would be desirable to confirm our findings.

We used the maximum concentration of methacholine for bronchial provocation as a reference standard to rule in and rule out asthma. Therefore, the potential of FENO for ruling out moderate and severe asthma might be underestimated.

A freely available calculator that allows computation of the probability of asthma based on the combination of clinical signs and symptoms, and FENO results, was developed.

Introduction

Asthma is a common chronic disease with a prevalence of up to 5% in industrialised countries. It is characterised by chronic inflammation, bronchial hyper-responsiveness (BHR) and usually reversible airway obstruction. Many efforts continue to be undertaken to improve the diagnostic process to allow an early diagnosis, as early treatment is important for the management of the disease. Investigation of the diagnostic accuracy of clinical signs and symptoms (CSS) showed that these were not very effective in ruling in or ruling out the disease.1 2 Spirometry is considered a reference standard for diagnosing airway obstruction,3 but it is not possible to rule out milder forms of asthma, as obstruction is not present in these cases.4 Guidelines also suggest the use of peak flow variability to diagnose BHR,5 but its diagnostic accuracy is low.6 Therefore, bronchoprovocation for determining BHR still remains as a reference standard, particularly in cases with inconclusive spirometric results.7 It is considered valuable in confirming or excluding asthma, despite being a time-consuming and costly, and not always available, procedure, and carrying a small risk of severe bronchospasm.8

Compared to bronchoprovocation, fractional exhaled nitric oxide (FENO) is an easily available, truly non-invasive marker. Increased FENO has been consistently demonstrated in asthma, including milder stages of the disease.9 10 The major pathophysiological basis seems to be that nitric oxide has a modulatory role in airway hyper-responsiveness11 and eosinophilic airway inflammation.12 Therefore, FENO has a potential in identifying specific asthma phenotypes, which might also allow the prediction of steroid responsiveness due to eosinophilic inflammation.13 This might be especially helpful for establishing or confirming the diagnosis safely and quickly in the primary care setting. Its diagnostic accuracy has been investigated in a large number of studies. In general, the results were promising, but different cut-off points were suggested to rule in or rule out asthma. As an example, to rule in the diagnosis of asthma with FENO, >50 parts per billion (ppb),14 15 or FENO >35 ppb,16 or FENO >46 ppb, has been suggested.17 To rule out the disease with FENO, <15–25 ppb has been suggested.14 FENO <16 ppb17 or even lower18 might be more useful in the primary care setting.

An important reason for the variation in cut-off points might be the selection of patients who participated in the diagnostic studies. The influence of the patient spectrum on the variation of diagnostic accuracy was already demonstrated by Ransohoff and Feinstein.19 Knottnerus20 explained the increase of sensitivity and decrease of specificity by referral processes in a methodological framework. The understanding of this process is important as patients present to the general practitioner (GP) with early symptoms and thus often with lower severity of disease.21 Beyond that, the interpretation of a test result is often hampered by low positive predictive values of tests, because the pretest probability of the target disease is often low in general practice. This phenomenon is described by Bayes’ Theorem.22 Especially in the primary care setting, in which few objective methods are available, it seems reasonable to combine information from a diagnostic test with the CSS presented by the individual patient, to enhance the diagnostic accuracy. This approach has been followed previously, for example, for pneumonia and C reactive protein.23 24 The aim of the present study was to evaluate the influence of patient selection on the diagnostic accuracy of FENO measurement on the basis of two diagnostic studies from different clinical settings,17 18 and to develop prediction rules including CSS in order to enhance the diagnostic value.

Methods

Design and sample

The first part of this prospective diagnostic study was performed in 10 German general practices in the area of Heidelberg in Baden-Württemberg, between February 2006 and June 2007.17 Two hundred and ten patients visiting their GP for the first time, with symptoms suggestive of OAD (obstructive airway disease) or the respective differential diagnoses (such as restrictive airway disease), were included consecutively. The patients had to present with symptoms such as dyspnoea, cough or expectoration of more than two months, thus leading to the clinical suspicion of obstructive or restrictive airway disease as the most important differential diagnoses. The presence of at least one of these symptoms was used as inclusion criterion (indicated population design).25 GPs were advised to exclude patients who had suffered from respiratory tract infections within 6 weeks preceding the evaluation. After the initial judgement by the GP, patients were sent to the lung function laboratory of the University Medical Hospital for diagnostic assessment including FENO measurement. Patients with a previously established diagnosis of OAD were excluded. Other exclusion criteria related to known contraindications for bronchodilator reversibility testing or bronchial provocation, namely untreated hyperthyreosis, unstable coronary artery disease and cardiac arrhythmia. Pregnancy also led to exclusion. Medical history was recorded using a structured questionnaire (table 1).

Table 1

Characteristics of the study population

	Practices of five pneumologists			General practices (n=13)				p Value
	Asthman (%)	COPDn (%)	No OADn (%)	Asthman (%)	COPDn (%)	ACOSn (%)	No OADn (%)
n	154 (39.2)	5 (1.3)	234 (59.5)	75 (46.9)	25 (15.6)	8 (5.0)	52 (32.5)
Female	91 (59.1)	2 (40.0)	142 (60.7)	44 (58.7)	15 (60.0)	4 (50.0)	22 (42.3)	0.341
FENO (mean in parts per billion [SD]))	42.4 [46.4]	16.6 [6.8]	22.0 [16.5]	42.6 [47.9]	16.2 [11.1]	20.4 [18.6]	24.7 [16.0]	0.710
Age (mean in years [SD])	40.5 [15.4]	60.8 [17.0]	44.6 [16.5]	38.7 [15.1]	55.7 [11.9]	63.5 [10.5]	42.8 [15.8]	0.684
FEV1 (mean of absolute values in L [SD])	3.32 [0.90]	2.85 [1.72]	3.97 [1.16]	3.43 [0.76]	2.12 [0.73]	1.93 [0.55]	3.52 [0.92]	0.034
FEV1 (mean of % of predicted [SD])	101.3 [17.0]	74.1 [12.3]	107.7 [16.3]	100 [12.2]	67.8 [18.5]	68.8 [18.4]	107.4 [12.8]	<0.001
FEV1/VC (mean of % [SD])	81.8 [8.4]	66.8 [9.8]	85.6 [7.3]	78.45 [7.02]1	59.7 [8.4]	58.2 [7.6]	82.1 [5.8]	<0.001
Do you suffer from dyspnoea attacks (yes)	33 (21.4)	1 (20.0)	32 (13.7)	27 (36.0)2	4 (16.0)	2 (25.0)	13 (25.0)	<0.001
Do you suffer from dyspnoea on exertion (yes)	72 (46.8)	4 (80.0)	95 (40.6)	21 (28.0)3	16 (64.0)	4 (50.0)	14 (26.9)	0.033
Have you ever suffered from wheezing in your chest? (yes)	97 (62.9)	5 (100.0)	79 (33.8)	39 (52.0)	15 (60.0)	3 (37.5)	19 (36.5)	1.000
Do you often suffer from a cough? (yes)	65 (42.2)	3 (60.0)	112 (47.9)	32 (42.7)	15 (60.0)	4 (50.0)	40 (76.9)5	0.047
Do you often suffer from respiratory tract infections? (yes)	54 (35.1)	1 (20.0)	79 (33.8)	17 (22.7)	6 (24.0)	4 (50.0)	29 (55.8)6	0.350
Do you often suffer from expectoration? (yes)	44 (28.6)	3 (40.0)	58 (24.8)	19 (25.3)	10 (40.0)	3 (37.5)	19 (36.5)	0.532
Have you ever woken up with a feeling of tightness in your chest? (yes)	54 (35.1)	1 (20.0)	59 (25.2)	19 (25.3)	4 (16.0)	3 (37.5)	8 (15.4)	0.070
Have you ever been woken up by an attack of shortness of breath? (yes)	35 (22.7)	1 (20.0)	28 (12.0)	21 (28.0)	3 (12.0)	2 (25.0)	9 (17.3)	0.180
Do you suffer from any nasal allergies? (yes)	76 (49.4)	0 (0)	47 (20.1)	40 (53.3)	7 (28.0)	1 (12.5)	23 (44.2)	0.008
Do you already take medication against asthma? (yes)	17 (11.0)	1 (20.0)	8 (3.4)	19 (25.3)4	5 (20.0)	0 (0)	2 (3.8)	<0.001
Do you smoke? (yes)	19 (12.3)	0 (0)	20 (8.5)	16 (21.3)	15 (60.0)	5 (62.5)	13 (25.0)	<0.001
Did you ever smoke? (yes)	56 (36.4)	4 (80.0)	79 (33.8)	30 (40.0)	24 (96.0)	8 (100)	24 (46.2)	0.001
How much do/did you smoke? (mean in pack-year [SD])	10.1 [10.7]	42.5 [3.5]	12.9 [15.8]	6.7 [13.3]	35.6 [20.6]	26.5 [17.4]	5.0 [11.1]	<0.001

Values indicate the number (proportion) or mean (SD).

Subgroup differences of asthma patients from general practice versus private practice of pneumologists: 1, p<0.001; 2, p=0.002; 3, p=0.009; 4, p=0.001; Subgroup differences between patients without OAD from general practice versus private practice of pneumologists: 5, p<0.001; 6, p=0.003.

ACOS, asthma-COPD overlap syndrome; COPD, chronic obstructive pulmonary disease; FENO, fractional exhaled nitric oxide; FEV1, forced expiratory volume in 1 s; OAD, obstructive airway disease; VC, vital capacity.

The second part of the study was performed in a private practice of five pneumologists in Bavaria, between June 2010 and October 2011.18 In Germany, specialists also work in primary care in their private practices, and ambulatory care comprises almost all specialists. There is no formal gatekeeping role for a GP in the German healthcare system. However, referrals from a GP to a specialist are requested in most cases. Only patients presenting for the first time for diagnostic work up to include or exclude an OAD or the respective differential diagnoses, were included. Patients with respiratory tract infections within the last 6 weeks were excluded.

Reference test—whole body plethysmography (WBP) and bronchial provocation: the spirometric manoeuvre performed during investigation with WBP was used as reference test in every setting. The procedures were performed according to standard protocols.26 Lung function reference values corrected for sex, age and height were used.27 Patients with forced expiratory volume in 1 s (FEV1) <80% predicted received salbutamol with an additional WBP investigation 20 min later. An OAD was diagnosed if FEV1/vital capacity (VC) was ≤0.70. It was classified as asthma if clinical symptoms and history fitted, and if the change in FEV1 compared to baseline was both ≥12% and ≥200 mL, and lung function returned to the predicted normal range. An incomplete bronchodilator response was stated if the response was ≥12% and ≥200 mL, but where lung volumes remained below predicted. We labelled this group as having asthma-COPD (chronic obstructive pulmonary disease) overlap syndrome (ACOS), because it shows spirometric properties of both, asthma and COPD.5 It was classified as COPD if clinical symptoms and history fitted and the bronchodilator response of FEV1 after salbutamol was both <12% compared to baseline and <200 mL.3 If there was no bronchial obstruction, bronchial provocation was performed to determine BHR. Trained lung function technicians measured BHR to methacholine according to the American Thoracic Society (ATS) guideline8 in the GP study.17 A modified bronchial provocation procedure was used in the practices of the pneumologists,18 according to the 1-concentration-4-step dosimeter protocol.28 This yields similar results as the ATS multiconcentration protocol but is less time consuming. An ‘asthma’ diagnosis required a 20% fall in FEV1 from baseline after inhaling methacholine stepwise until the maximum concentration (16 mg/mL), and, alternatively, a doubling of airway resistance and its increase to ≥2.0 kPa s.29 The diagnostic superiority of WPB compared with spirometry for ruling out asthma was demonstrated previously.30 The final diagnosis was made under consideration of medical history and clinical examination by a pneumologist.

Index test—FENO measurement: all patients underwent standard measurement of FENO (NioxMino, Aerocrine, Solna, Sweden) at a flow rate of 50 mL/s, according to the ATS/European Respiratory Society guideline,31 using feedback signals for control. This was performed prior to WBP and bronchial provocation, as the breathing manoeuvres involved could distort FENO results. The responsible pneumologist was blinded to the FENO results and made the diagnostic decision only on basis of medical history, physical examination, WBP investigation and bronchial provocation results.

Patients gave written informed consent.

Statistical methods

Power calculation was based on previous studies related to the prevalence of asthma in the respective setting and the diagnostic accuracy of FENO. We wanted to include at least 149 patients in the first part of the study17 and at least 302 patients in the second part.18 Differences between lung function values (not normally distributed) were statistically evaluated with the Mann-Whitney U test. Differences between clinical symptoms were evaluated with the χ2 test. The data were analysed with IBM Statistics SPSS V.22.0 for Windows.

Independent clinical and diagnostic contributions of symptoms and signs to the prediction of asthma were assessed using multiple logistic regression analysis. As the number of available variables was too large to meet the rule of thumb in 10 cases, per independent variable,32 we checked univariate associations with asthma and included only significant variables (p<0.05) in the model. Multiple logistic regression analysis using backward elimination with p>0.1 for exclusion was performed with the selected variables, resulting in the final covariate model. Several potentially relevant interaction terms between covariates were first included and then removed from the model if they did not contribute to the diagnostic accuracy. Considering the resulting covariate effects estimated from the data, a rule could be derived from the multiple logistic regression approach, predicting the probability of asthma in each individual case. Respective 95% CIs for predicted probabilities are given in parentheses and were calculated using the δ-method.33 A calculator that allows computing all combinations is provided as an internet supplement. If the δ-method is not applicable, in particular at the border of the domain of predicted probabilities, the CI is not calculated.

In accordance with everyday practice, where an additional FENO measurement is performed after medical history information has been acquired, multiple logistic regression analysis was repeated, adding FENO at different cut-off values and as exact numerical variable. Receiver operating characteristic (ROC) curves display the diagnostic performance of the final models. The area under the curves (AUC) were used to quantify the added value of the CCS+FENO model beyond the FENO model. Comparison of AUC is performed with the empirical test implemented in NCSS V.9.0.534 using a non-parametric approach described in DeLong et al35 and Zhou et al.36

The results of the diagnostic models were interpreted with respect to clinical significance. A satisfactorily high posterior probability of asthma is assumed, when the positive predictive value is ≥70%. This corresponds with the positive predictive value of bronchial provocation, which was estimated around 70% for a pretest probability of asthma of 30%,8 37 and was demonstrated recently.30 A satisfactorily low posterior probability is assumed at 20%, corresponding to the probability of 80% of having ‘no asthma’. This corresponds to the negative predictive value of a 20% fall in FEV1 from baseline during bronchial provocation.30

Results

Study population

A total of 553 patients participated (320 female (57.9%)). The recruitment rate in general practice was 76%. Nearly every patient from the practices of the pneumologists participated; the data of seven patients could not be used due to incompleteness (figure 1). The diagnosis of asthma was based mainly on bronchial provocation (n=206; 90%); positive bronchodilator response of pre-existing airway obstruction was recorded in only 23 (10%) cases. The prevalence of asthma was highest in the general practice group (table 1). Patients suffered mainly from shortness of breath, wheezing and cough. The patient sample from general practice suffered significantly more from dyspnoea attacks, cough and nasal allergy, and less from dyspnoea on exertion. They used more antiasthmatic medication than patients from the practices of the pneumologists. We found more smokers in the general practice sample, with higher nicotine use. Correspondingly, there were more patients with COPD and ACOS in the general practice sample, accompanied by a significantly lower FEV1, VC and FEV1/VC ratio. Patients with asthma in general practice had significantly more dyspnoea attacks and less dyspnoea on exertion than patients from the practices of pneumologists (p values of subgroups are depicted at the bottom of the table). They also used more antiasthmatic medication. The asthma patients from the general practice showed a significantly lower FEV1/VC ratio compared to the patients with asthma from the pneumologists practices; FEV1 and VC showed no significant difference. Patients in general practice without OAD suffered from cough and recurrent respiratory tract infections significantly more than the patients from the practices of the pneumologists. There were no further significant differences between the patient groups with respect to the other CSS.

Figure 1

Diagnostic work up of patients (ACOS, asthma-COPD overlap syndrome; COPD, chronic obstructive pulmonary disease; FEV1, forced expiratory volume in 1 s; OAD, obstructive airway disease; VC, vital capacity).

Diagnostic accuracy of FENO of the different patient collectives

A comparison of patients from general practice and pneumologists’ practice showed a trend towards slightly higher sensitivities around the cut-off point >40 ppb in the general practices; there were no remarkable differences related to specificity (table 2). Multiple logistic regression analyses were performed with either 3, 4 or 5 selected covariates from clinical history or physical examination, respectively. This resulted in three groups of models and the respective equations displayed in table 3.

Table 2

Diagnostic accuracy of FENO in different patient collectives

	Pneumologists practice (n=393)p(asthma)=39.2%		General practice (n=160)p(asthma)=46.9%
NO [ppb]	Sensitivity	Specificity	Sensitivity	Specificity
≥4	100 (97.6 to 100)	0 (0 to 1.6)	100 (95.1 to 100)	0 (0 to 4.3)
≥6	99.4 (96.5 to 99.9)	2.1 (0.9 to 4.8)	96.0 (88.9 to 98.6)	7.1 (3.3 to 14.6)
≥9	97.4 (93.5 to 99.0)	7.5 (4.8 to 11.6)	92.0 (83.6 to 96.3)	15.3 (9.2 to 24.4)
≥10	95.5 (91.0 to 97.8)	13.0 (9.3 to 17.9)	89.3 (80.3 to 94.5)	16.5 (10.1 to 25.8)
≥11	90.9 (85.3 to 94.5)	18.8 (14.4 to 24.2)	88.0 (78.7 to 93.6)	20.0 (12.9 to 29.7)
≥12	85.1 (78.6 to 89.9)	28.5 (23.2 to 34.5)	85.3 (75.6 to 91.6)	23.5 (15.8 to 33.5)
≥15	69.3 (76.6 to 82.6)	36.4 (30.6 to 42.7)	76.0 (65.2 to 84.3)	37.6 (28.1 to 48.2)
≥17	69.5 (61.8 to 76.2)	47.3 (41.1 to 53.6)	69.3 (58.1 to 78.6)	52.9 (42.4 to 63.2)
≥19	65.6 (57.8 to 72.6)	55.2 (48.9 to 61.4)	64.0 (52.7 to 73.9)	57.6 (47.0 to 67.6)
≥21	59.7 (51.8 to 67.1)	63.2 (56.9 to 69.1)	60.0 (48.7 to 70.3)	61.2 (50.6 to 70.9)
≥26	48.7 (40.9 to 56.5)	75.3 (69.5 to 80.3)	49.3 (38.3 to 60.4)	72.9 (62.6 to 81.2)
≥31	38.3 (31.0 to 46.2)	83.3 (78.1 to 87.5)	37.3 (27.2 to 48.6)	76.5 (66.5 to 84.3)
≥36	32.5 (25.6 to 40.2)	87.9 (83.2 to 91.4)	32.0 (22.5 to 43.2)	83.5 (74.2 to 89.9)
≥41	28.6 (22.1 to 36.2)	90.8 (86.5 to 93.8)	32.0 (22.5 to 43.2)	88.2 (79.6 to 93.5)
≥48	23.4 (17.4 to 30.7)	92.5 (88.4 to 95.2)	30.7 (21.4 to 41.9)	92.9 (85.4 to 96.7)
≥51	22.7 (16.8 to 29.9)	93.7 (89.9 to 96.1)	28.0 (19.1 to 39.0)	92.9 (85.4 to 96.7)
≥57	20.8 (15.1 to 27.9)	95.4 (92.0 to 97.4)	25.3 (16.8 to 36.2)	94.1 (86.9 to 97.5)
≥65	18.8 (13.4 to 25.7)	95.8 (92.5 to 97.7)	20.0 (12.5 to 30.4)	98.8 (93.6 to 99.8)
≥72	17.5 (12.3 to 24.3)	97.1 (94.1 to 98.6)	14.7 (8.4 to 24.4)	98.8 (93.6 to 99.8)
≥99	12.3 (8.0 to 18.4)	99.2 (97.1 to 99.8)	10.7 (5.5 to 19.7)	100 (95.7 to 100)

FENO, fractional exhaled nitric oxide; ppb, parts per billion.

Table 3

Regression models of FENO (NO) and clinical signs and symptoms in different patient collectives, where k is the number of covariates in the final model

General Practice: k=5;
Variable	i	βi NO exact	βi NO ≤10	βi NO ≤16	βi NO ≥40	βi NO ≥50	βi NO ≥60	βi NO ≥70	βi NO ≥80
Constant	0	1.51	2.45	2.74	2.09	1.95	2.06	2.09	2.19
NO	1	0.03	−0.96	−1.05	1.03	1.53	1.91	1.89	20.82
Age	2	−0.04	−0.04	−0.04	−0.04	−0.04	−0.04	−0.04	−0.04
Medication	3	1.58	1.87	1.74	1.62	1.68	1.70	1.64	1.59
Infection	4	−1.31	−1.25	−1.23	−1.24	−1.27	−1.40	−1.29	−1.21
Cough	5	−1.04	−0.98	−1.14	−1.05	−1.02	−0.90	−0.99	−1.05
Pneumologist: k=3;
Variable	iI	βi NO exact	βi NO ≤10	βi NO ≤16	βi NO ≥40	βi NO ≥50	βi NO ≥60	βi NO ≥70	βi NO ≥80
Constant	0	−1.85	−1.39	−1.27	−1.55	−1.56	−1.51	−1.50	−1.50
NO	1	0.02	−0.57	−0.45	0.86	1.02	1.19	1.09	1.84
Wheezing	2	0.82	0.97	0.96	0.88	0.90	0.85	0.87	0.86
Allergic Rhinitis	3	1.28	1.43	1.40	1.33	1.38	1.37	1.37	1.34
Combined: k=6;
Variable	i	βi NO exact	βi NO ≤10	βi NO ≤16	βi NO ≥40	βi NO ≥50	βi NO ≥60	βi NO ≥70	βi NO ≥80
Constant	0	−0.79	−0.10	0.00	−0.43	−0.49	−0.45	−0.42	−0.47
NO	1	0.02	−0.91	−0.64	0.90	1.07	1.40	1.32	2.09
Age	2	−0.02	−0.02	−0.02	−0.02	−0.02	−0.02	−0.02	−0.01
Wheezing	3	0.66	0.83	0.82	0.74	0.75	0.70	0.72	0.71
Allergic rhinitis and medication	4	2.34	2.40	2.40	2.32	2.29	2.28	2.36	2.36
Allergic rhinitis	5	0.73	0.85	0.81	0.76	0.83	0.84	0.83	0.82
Infection		−0.44	−0.51	−0.51	−0.46	−0.43	−0.46	−0.47	−0.44

Further subgroup models were defined dependent on the treatment of FENO measures as either exact numerical or dichotomised at cut-offs 10, 16, 40, 50, 60, 70 or 80 ppb. The resulting covariate effects estimated from the data are given in table 3 as βi, i=0, 1, …, k, where k is the number of covariates in the respective model. This allowed the predicted probability of asthma for individual patients to be calculated. Figure 2 illustrates that the diagnostic accuracy of FENO increases remarkably when the results are combined with CSS. The AUC differences were significant in general practice (p=0.001), pneumologists’ practice (p=0.0002) and in the combined sample (p<0.0001). Beyond that, the AUCs of the general practice sample were higher than in the pneumologists’ practice sample. Box 1 gives examples of using estimate covariate effects and equations from table 3 in order to calculate posterior predicted probabilities of asthma dependent on selected combinations of symptoms and FENO measurements. In principle, diagnostic trees with all possible posterior predicted probabilities of asthma can be derived from table 3. The results can be computed with the calculator that is added as a supplement.

Figure 2

Receiver operating characteristic curves of fractional exhaled nitric oxide and clinical signs and symptoms.

The predicted probability (P) of asthma for each individual patient can be calculated from the equation:

in which for the pneumologist model, where β0 is the estimated coefficient of the grand mean in the model and β1, β2 and β3 are the regression coefficients of the variables in the model.

Examples of calculations for the final models are given below:

Pneumologist model using exact numerical values for fractional exhaled nitric oxide (FENO):

Examples

A patient with wheezing, allergic rhinitis and a FENO value of 80 ppb has a prediction score of −0.25+2.1=1.85, resulting in a probability of 84.1% (95% CI 75.5% to 92.7%) of having asthma. Similarly, a patient with wheezing and FENO=80 ppb, but no allergic rhinitis, has a probability of 59.5% (95% CI 44.1% to 74.8%) of having asthma. A patient without any of these two items, however, with FENO=80 ppb, has a predicted probability of 39.3% (95% CI 22.8% to 55.9%) of having asthma.

Pneumologist model using a cut-off value <16 ppb for FENO:

Examples

A patient with FENO measurement less or equal to 16 ppb, wheezing and allergic rhinitis has a probability of 65.3% (95% CI 52.7% to 78.0%) of having asthma. Similarly, a patient with the same symptoms and same FENO measurement, but without allergic rhinitis, has a probability of 31.8% (95% CI 21.6% to 42.0%) of having asthma. A patient without any of these items, but FENO below 16 ppb, has a probability of 15.1% (95% CI 9.2% to 21.1%) of having asthma. Conversely, this means a probability of 84.9% (95% CI 78.9% to 90.8%), for this group of patients, of not having asthma.

According to these calculations, predicted probabilities for the sputum and general practice patients can be calculated.

Reviewing the equations (table 3), the patients’ age turned out to be an important predictor in general practice. If the patient was 20 years old, the resulting posterior probability of FENO ≥30 ppb was 87.0% (calculating 95% CI was not possible). However, it was only 66.5% (95% CI 44.2% to 88.7%) when the patient was 50 years old. Ruling out was only effective when a patient was suffering from cough and recurrent respiratory tract infections; for example, the posterior probability of asthma was 18.8% (95% CI 2.1% to 35.6%) when FENO was ≤16ppb in a 20-year-old patient. Previously taken medication was strongly associated with asthma. For example, in a 40-year-old patient, the posterior probability was 86.6% (calculating 95% CI was not possible), even when FENO was ≤16 ppb.

The patients from the pneumologists’ practices showed different characteristics. When a patient, independent of age, reported wheezing and nasal allergy, the posterior probability of asthma was 77.3% (95% CI 68.1% to 86.4%) when FENO was ≥30 ppb. Without these symptoms, the posterior probability was only 26.2% (95% CI 14.9% to 37.5%) when FENO was ≥30 ppb. Ruling out was possible when the patient had no allergic symptoms; with FENO ≤16 ppb, the resulting posterior probability for asthma was 15.1% (95% CI 9.2% to 21.1%).

Wheezing, allergic rhinitis, medication, infection and age remained as significant covariates when data of all patients were pooled. Previously taken medication remained as a strong predictor for asthma and was interrelated with allergic rhinitis. Within this model, wheezing and allergic rhinitis helped to rule in, and recurrent infections helped to rule out, asthma. The positive predictive value of FENO increased considerably with decreasing age. As an example, the final prediction rule allowed ruling in asthma in a 20-year-old patient with wheezing and allergic rhinitis; probability of asthma was 78.4% (95% CI 68.8% to 88.1%) when FENO was ≥30 ppb. Without wheezing but with allergic rhinitis, p was 75.0% (95% CI 61.3% to 88.7%) when FENO was ≥50 ppb. In patients who were at least 43 years of age, the probability of asthma was lower than 20% when FENO was ≤16 ppb. However, ruling out in younger patients was only effective with recurrent respiratory tract infections when allergic signs were absent; then, as an example, the probability of asthma was 18.1% (95% CI 9.58% to 26.7%) when FENO was ≤16 ppb in a 20-year-old patient.

Discussion

To the best of our knowledge, this is the first study to evaluate FENO in different clinical settings in combination with CSS. We found that the selection of patients only had a slight influence on the sensitivities of the various FENO cut-off points. However, there was a meaningful influence on diagnostic patterns. The ROC analyses illustrated that the diagnostic accuracy of FENO increased remarkably when the test results were combined with CSS.

The variation of the diagnostic accuracies of CSS related to respiratory diseases was shown in a few studies,1 38 illustrating that sensitivity increases and specificity decreases during the selection process of the patients. The explanation for this phenomenon previously was worked out theoretically and methodologically.20 Whiting et al39 found, in their systematic review about sources of variation and biases on diagnostic accuracy of diagnostic instruments, that sensitivity increased with disease prevalence and severity, whereas the effects on specificity were inconsistent. This might fit with our findings around the critical cut-off point of 40 ppb, as the sensitivities of the various cut-off points >40 ppb were slightly higher in the general practice setting. The higher pretest probability in general practice might be surprising at first sight. This could be explained by the study design, which required that participating patients had to travel to the lung function laboratory of the University Medical Hospital Heidelberg, which might have unintentionally caused a selection of patients with a higher probability and/or severity of disease. It might explain why ruling in the diagnosis of asthma appeared more straightforward in the general practice sample than in the pneumologists’ sample. In the latter, ruling in of asthma was only reasonable with FENO ≥30 ppb, when the patient suffered from wheezing and allergic rhinitis.

The strength of both settings was that only diagnostically naive patients presenting for the first time for diagnostic investigation were included. As we observed no strong influence of the setting on the sensitivities and specificities of FENO cut-points, we pooled the data of all patients. The AUC increased remarkably from 0.650 to 0.753 when CSS were included in the diagnostic model. The final prediction rule conclusively illustrates that the potential of FENO to rule in or rule out asthma depends on the age of patients and the presentation of CSS. This might explain why varying cut-off points were found in the different studies when various patient collectives were evaluated. The prediction rule revealed that, especially, allergic rhinitis and wheezing are helpful to identify patients who will benefit from FENO measurement in terms of a high positive predictive value. This fits in with previous studies illustrating the relationship between asthma, increased FENO values, wheezing40 and allergic rhinitis,41 which is explained by the common type of eosinophilic inflammation.12 Thus it seems possible to diagnose asthma with FENO ≥30 ppb in patients with a compatible medical history, which is 20 ppb lower, as suggested by the ATS guideline.15 Another important point is the strong impact of previously given medication on the diagnostic model. Medications are prescribed occasionally ‘ex juvantibus’ in case of clinical uncertainty in general practice when asthma is suspected.42 This is crucial to avoid deterioration of asthma until the definite diagnosis is established by bronchial provocation in the practices of pneumologists or in a hospital. Thus, there seems to be a high probability of asthma when the patient continues inhaler therapy.

It was difficult to exclude the diagnosis of asthma in younger patients solely on the basis of FENO measurement. In general practice, it was only possible when there were no specific allergic signs, FENO measurement showed low values and the patient was suffering from recurrent respiratory tract infections and cough. The latter appears contradictory to guidelines.5 However, the negative association with cough was already shown previously1 43 and seems reasonable from a clinical point of view, as many patients in general practice are coughing and/or have respiratory tract infections, but only few are really suffering from asthma. The low performance of ruling out asthma might be explained by the blind spot of FENO regarding neutrophilic inflammation.18 44 Patients with this type of inflammatory pattern are less responsive to inhaled corticosteroids, but absence of eosinophilia does not indicate an absence of steroid response.45 Therefore, patients with negative test results have to be referred for bronchial provocation in case of persistent symptoms, to definitely rule in or rule out the diagnosis of asthma.

The strength of the study is that the diagnostic accuracy of FENO was evaluated in two different settings. This was accompanied by the use of two slightly different reference standards with respect to bronchial provocation, which could have influenced the evaluation. However, the 1-concentration-4-step dosimeter protocol shows results similar to the ATS multiconcentration protocol.28 Thus, a major distortion seems unlikely. We used the maximum concentration of methacholine for bronchial provocation as a reference standard to rule in and rule out asthma. Consequently, borderline bronchial hyper-reactivity also led to the diagnosis of asthma.8 Therefore, the potential of FENO for ruling out moderate and severe asthma might be underestimated.17 Further diagnostic studies would be necessary for differentiation of such subgroups, in particular with respect to the necessity for therapy with inhaled corticosteroids. Another inherent limitation is that two different patient collectives were used for analysis. However, recruitment in different settings was intended to analyse potentially different diagnostic patterns. It might be speculated whether two different diagnostic algorithms related to each practice setting need to be used. The general practice patients seemed to be comparatively selected, which might be explained by the study design. Therefore it appeared adequate to extrapolate our FENO findings more cautiously to allow generalisation of the diagnostic algorithm. Thus we decided to pool the data of both patient samples, because the clinical setting had only a minor influence on the sensitivities of the various cut-off points of FENO. As a result, the final model fitted well with the established clinical decision rules used by many physicians and led to a more conservative interpretation of the FENO measurements. However, a validation study would be desirable to confirm our findings.

Another crucial issue is to decide the ideal cut-off point with respect to clinical significance. FENO ≥30 ppb resulting in a probability of asthma of 78.4% might be regarded as too low. However, this is considerably better than the predictive value of bronchial provocation with methacholine.8 30 37 Ruling out asthma with FENO ≤16 ppb is equal to a 20% fall of FEV1 during bronchial provocation, which can be detected with spirometry manoeuvres. However, the negative predictive value of specific airway resistance response on methacholine as determined with WBP would be much lower with a negative predictive value of 97.8%.30 Finally, eight patients with ACOS were labelled as non-asthmatics because of the uncertainty of their diagnostic entity. However, we expect that this did not distort the results, due to the low number of cases.

Conclusion

The ROC analysis revealed that FENO results should be interpreted in the context of CSS to enhance their diagnostic value in primary care. The final diagnostic model appears as a sound primary care algorithm fitting to the established diagnostic rules related to CSS of asthma. Importantly, FENO appears more promising for ruling in asthma than for ruling it out. Ruling in asthma with FENO ≥30 ppb is reasonable when allergic symptoms such as wheezing and allergic rhinitis are present. Previously taken medication is a strong predictor for asthma. Ruling out younger patients only seems possible in case of recurrent respiratory tract infections when no allergic symptoms are present.