Diagnostic accuracy of FeNO in asthma and predictive value for inhaled corticosteroid responsiveness: A prospective, multicentre study.

Background: Fractional exhaled nitric oxide (FeNO) is promising for diagnosing asthma and could replace bronchial provocation (BP). To date, cut-off values have been derived by post hoc analysis only. The aim was to validate the diagnostic accuracy for predefined FeNO cut-off values and the predictive value for responsiveness to inhaled corticosteroids (ICS).
Methods: We conducted a prospective, diagnostic, multicentre study with patients attending three private practices of pneumologists in Upper Bavaria, Germany, from July 3, 2020 to Jan 21, 2022. Index test was FENO measurement. Reference standard was Tiffeneau ratio (FEV1/VC) or airway resistance as assessed by whole body plethysmography, with additional BP or bronchodilation test. Follow-up was performed after 12 weeks. Analyses of Receiver Operating Characteristics curves were conducted to determine the diagnostic accuracy and predictive value of FeNO. Findings: 308 patients with complete follow-up were recruited, 186 (60·4%) were female, average age was 44·7 years, 161 (52·3%) had asthma. Regarding diagnostic accuracy, the area under the curve (AUC) was 0·718 (95% CI 0·661-0·775; p < 0·001). Sensitivity at FeNO >50 ppb was 0·24 (95% CI 0·18-0·32), specificity 0·99 (0·95-1·0), positive predictive value (PPV) 0·95 (0·84-0·99), negative predictive value (NPV) 0·54 (0·48-0·60). In 66 patients with ´wheezing´ and ´allergic rhinitis´, the sensitivity at FeNO >33 ppb was 0·49 (0·34-0·64), specificity 0·88 (0·64-0·99), PPV 0·92 (0·75-0·99), NPV 0·38 (0·23-0·54). In 68 patients with ICS medication, responsiveness was predicted at the cut-off >43 ppb, with a sensitivity of 0·55 (95%CI 0·36-0·74), specificity 0·82 (0·66-0·92), PPV 0·70 (0·47-0·87), NPV 0·71 (0·56-0·84). Interpretation: FeNO measurement allows a valid ruling-in of an asthma diagnosis, whereas ruling-out of asthma is not possible. Enhanced probability of ICS responsiveness is also given with increased FeNO values. Funding: Circassia Germany gave 25% discount on the purchase of three NIOX VERO devices.

Evidence before this study

Measurement of fractional exhaled nitric oxide (FeNO) has a high potential for diagnosing asthma, and might also be suitable for predicting responsiveness to inhaled corticosteroids (ICS). However, a systematic review including diagnostic studies until 2015 has shown that cut-off values were derived by post hoc analysis only. We updated the literature search using PubMed on May 20, 2022 using the search terms ((“FeNO” OR “nitric exhaled oxide”) AND (“sensitivity” OR “specificity” OR “diagnostic accuracy” OR diagnostic study” OR “ROC curve”) AND asthma) to identify 255 additional publications; and examined national and international guidelines. We found no confirmatory diagnostic study using pre-defined cut-off values. The weaknesses of post hoc findings are apparent as they led to contradictory recommendations in the guidelines.

Added value of this study

To determine its appropriate place in asthma diagnosis and routine care, we conducted a prospective confirmatory multi-center study to validate its diagnostic accuracy for predefined FeNO cut-off values and predictive value for responsiveness to inhaled corticosteroids (ICS). We found with our large study population comprising 308 patients with 3-month follow up, that FeNO values >50 ppb allow a valid ruling-in of an asthma diagnosis with a positive predictive value (PPV) of 0·95; sensitivity was 0·24, and specificity was 0·99. A FeNO cut-off >33 ppb showed a PPV of 0·92, when patients are also experiencing wheezing and allergic rhinitis. A FeNO cut-off >43 ppb showed a PPV of 0·70 for ICS responsiveness.

Implications of all the available evidence

In patients with FeNO values >50 ppb or >33ppb if wheezing and allergic rhinitis are present, bronchial challenge is no longer required to diagnose asthma. In addition, the probability of ICS responsiveness is enhanced with increased FeNO cut-off values, which alleviates therapeutic decision making.

Alt-text: Unlabelled box

Introduction

Asthma is a respiratory disease with a prevalence of approximately 10% in industrialised countries. It is characterised by chronic airway inflammation leading to recurrent respiratory symptoms. Regarding diagnostic decision making, airway obstruction is often absent during investigation by spirometry or whole body plethysmography (WBP) at least in periods of mild symptoms, thus leading to diagnostic uncertainty. For these cases, diagnostic guidelines recommend bronchial provocation (BP) tests in order to diagnose or exclude asthma.2, 3, 4, 5 Peak-flow variability may be assessed, but is considered as a second-choice method due to its low diagnostic value. Unfortunately, BP is time-consuming and often only available in specialised lung function laboratories.

Numerous studies have demonstrated that the measurement of fractional exhaled nitric oxide (FeNO) has a high potential for diagnosing asthma and could possibly replace BP. NO is regarded as a biomarker of type-2 airway inflammation, and it could be shown that patients with asthma, even in mild stages of the disease, exhale NO in higher concentrations than healthy individuals. In contrast to BP, FeNO is a non-invasive measurement that can be performed without risk to the patient in a much shorter time. The available studies indicate that a cut-off value of 50 ppb (parts per billion) is well suited for diagnosing asthma. However, the cut-off values were identified only by post hoc analyses involving multiple and exploratory testing.

Beyond that, it was shown in a secondary analysis that even lower FeNO values than 50 ppb could be useful for diagnosis when considering appropriate anamnestic information. If, for example, the patient suffers from allergic rhinitis and wheezing, an asthma diagnosis can be established with a high degree of certainty when FeNO is >33 ppb. However, this post hoc derived algorithm needs to be validated in a multi-centre study. Another important aspect is that FeNO could be suitable for predicting responsiveness to inhaled corticosteroids (ICS) in asthma. The study by Martin et al. showed that FeNO >33 ppb could be used with a high degree of certainty but these values were also identified by post hoc analysis.

The weaknesses of post hoc findings are apparent as they led to contradictory recommendations. The Global Initiative for Asthma (GINA) guideline and the German National Guideline stated FeNO as currently not proven for ruling in or ruling out a diagnosis of asthma., In contrast, the British Thoracic Society/Scotish Intercollegiate Guidelines Network (BTS/SIGN) guideline suggested to use FeNO ≥40 ppb for ruling in asthma, and the National Institute for Health and Care Excellence (NICE) recommended to offer a FeNO test to adults if a diagnosis of asthma is considered. The FeNO guideline of the American Thoracic Society (ATS) from 2011 recommends FeNO greater than 50 ppb be used to indicate that eosinophilic inflammation and, in symptomatic patients, responsiveness to corticosteroids are likely; whereas the ATS guideline from 2021 suggests no definite cut-off point. The recently published asthma guideline of the European Respiratory Society (ERS) stated, that a cut-off value of 40 ppb offers the best compromise between sensitivity and specificity while a cut-off of 50 ppb has a high specificity >90% and is supportive of a diagnosis of asthma.

To determine its appropriate place in asthma diagnosis and routine care, we conducted a prospective confirmatory multi-centre study to validate its diagnostic accuracy for predefined FeNO cut-off values and predictive value for responsiveness to inhaled corticosteroids (ICS).

Methods

Study design and setting

The multicentre, prospective, diagnostic study was performed in three practices of pneumologists in Upper Bavaria, Germany, from July 3, 2020 to January 21, 2022. The study desing was described in the study protocol. Patients coming for the first time for diagnostic work-up with complaints suggestive of asthma were consecutively included. Patients completed a questionnaire with structured questions about medical history and symptoms (Table 1) and the ‘Asthma Control Questionnaire (ACQ)’. The ACQ is used to determine the extent of asthma control (controlled, partially controlled, uncontrolled) and the responsiveness to ICS. ICS responsiveness is assumed if the ACQ score improves by at least 0.5 in the sense of a ‘minimal important difference’.

Table 1

Characteristics of the study population.

	Asthma	COPD	No OAD
N	161 (52·3)	6 (1·9)	141 (45·8)
Female	90 (55·9)	4 (66·7)	92 (65·2)
Age in years	44·3±17·2	60·8±19·8	44·6±15·8
Height in cm	172·8±9·8	167·0±13·0	171·6±9·3
Weight in kg	78·9±16·7	73·7±18·6	73·6±16·9
BMI in m/kg²	26·5±5·3	26·3±5·3	24·9±4·9
FeNO in parts per billion	27·0 (15·5-49·0)	15·0 (11·5-25·8)	16·0 (11·0-23·5)
FEV1 absolute values in litre	3·34±0·97	2·07±1·07	3·46±0·82
FEV1 in % of predicted (missing data: 1)	94·1±14·6	70·2±11·9	101·3±12·3
FEV1/VC in % (missing data: 1)	80·3±8·8	70·2±11·5	84·2±7·3
Baseline ACQ-Score (missing data: 9)	1·17 (0·50-1·83)	0·92 (0·00-2·54)	0·83 (0·17-1·33)
Do you often suffer from allergic rhinitis (e.g., hay fever)? (yes) (missing data: 1)	85 (52·8)	0 (0·0)	39 (27·7)
Have you ever suffered from wheezing in your chest during the last 12 months? (yes)	86 (53·4)	2 (33·3)	37 (26·2)
Do you suffer from dyspnoea attacks (yes) (missing data: 7)	41 (25·5)	1 (16·7)	27 (19·1)
Do you suffer from dyspnoea on exertion (yes) (missing data: 7)	101 (62·7)	4 (66·7)	70 (49·6)
Have you ever been woken up by a feeling of tightness in your chest during the last 12 months? (yes) (missing data: 3)	48 (29·8)	1 (16·7)	30 (21·3)
Have you ever been woken up by an attack of shortness of breath during the last 12 months? (yes) (missing data: 3)	29 (18·0)	6 (100·0)	13 (9·2)
Have you ever been woken up by an attack of coughing during the last 12 months? (yes) (missing data: 1)	54 (33·5)	6 (100·0)	42 (29·8)
Have you ever had an asthma attack during the last 12 months? (yes) (missing data: 21)	30 (18·6)	6 (100·0)	6 (4·3)
Do you already take medication against asthma? (yes) (missing data: 2)	39 (24·2)	3 (50·0)	10 (7·1)
Do you often suffer from respiratory tract infections? (yes) (missing data 4)	36 (22·4)	0 (0·0)	31 (22·0)
Do you often cough? (yes) (missing data: 7)	58 (36·0)	6 (100·0)	65 (46·1)
Do you suffer from coughing for more than 3 months per year? (yes) (missing data: 3)	45 (28·0)	6 (100·0)	51 (36·2)
Do you often suffer from expectoration? (yes) (missing data: 4)	36 (22·4)	2 (33·3)	28 (19·9)
Do you smoke? (yes)	9 (5·6)	1 (16·7)	9 (6·4)
Did you smoke in the past? (yes) (missing data: 10)	64 (39·8)	5 (83·3)	48 (34·0)
How much do/did you smoke? in pack years (missing data: 45)	6·9(2·5-15·3)	35·3 (30·0-35·3)	4·5(1·7-12·5)

Values indicate the number n (%), mean ±SD, median (1. quartile – 3. quartile); OAD = obstructive airway disease; COPD = chronic obstructive pulmonary disease; BMI = body mass index; FeNO = fractional exhaled nitric oxide; ACQ = Asthma Control Questionnaire; FEV1 = Forced Expiratory Volume in the first second; VC = Vital Capacity.

Patients with a previously established diagnosis of obstructive airway disease were excluded. Patients who smoked at the day of examination (affecting FeNO and BP testing), with nitrate-rich meal before examination for less than three hours before the examination (false high FeNO values), and patients with respiratory infection <6 weeks prior to the visit (affecting FeNO and BP testing) were also excluded. Other exclusion criteria referred to known contra-indications for bronchodilator reversibility testing or bronchial provocation, namely untreated hyperthyreosis, unstable coronary artery disease, and cardiac arrhythmia. Pregnancy also led to exclusion. The study was approved by the Ethics Committee of the Medical Faculty of the Technical University Munich, and all patients gave their written informed consent.

FeNO measurement was performed first (T1). Afterwards, patients were routinely examined with WBP and BP when necessary. This diagnostic procedure is routinely performed in German practices of pneumologists in ambulatory care if asthma is suspected.

All patients with a positive BP test or the diagnosis of asthma, respectively, were invited by telephone for a follow-up investigation after 3 months (T2). Patients were re-examined with FeNO measurement and WBP, and BP when appropriate (if the result of WPB examination were inconclusive).

Patients with a negative BP test were also interviewed by telephone after 3 months in order to rule out a false negative test result. Patients with a positive BP test or the diagnosis of asthma who refused or were not able to attend the follow-up examination were also interviewed. Patients were asked about their symptoms and use of inhaled medication within this structured telephone interview. If patients report persistent respiratory symptoms although the BP test was negative, a follow-up examination at the practice of the respective pneumologist was offered.

Ethics and dissemination

The study was approved by the Ethical Committee of the Technical University of Munich (Reference number 122/20 S). Written, informed consent to participate was obtained from all participants.

Index test

The FeNO measurement was performed with the electrochemically-based NO-measuring device NIOX VERO. This device is CE-certified and available in national and international markets. The FeNO measurements were performed once for each patient, following ATS (American Thoracic Society) and ERS (European Respiratory Society) recommendations. The pneumologists were blinded to the results of the FeNO measurement.

Reference test

Whole body plethysmography

WBP including spirometry was considered as the reference standard for the diagnosis of obstructive airway disease. An obstructive airway disease was diagnosed when Forced Expiratory Volume in the first second / Vital Capacity (FEV1/VC) was ≤ 0·70. A reversible airway obstruction was diagnosed if the bronchodilation test was positive (ΔFEV1 >12% and >200 mL). If there was no bronchial obstruction, BP was performed. Originally, we wanted to use the Lower Limits of Normal (LLN) values for interpretation, but these have not been regularly implemented in the reporting documents.

Bronchial provocation

BP was performed to determine bronchial hyperresponsiveness (BHR) to methacholine according to the 1-concentration-4-step dosimeter protocol. This yields similar results as the ATS multi-concentration protocol but offers advantages in clinical practice. The test was considered positive (indicating BHR), if FEV1 decreased by at least 20% after inhalation of a maximum cumulative methacholine dose of 960 µg, and/or if specific airway resistance (sRaw) increased simultaneously by at least 100% and to at least 2·0 kPa*s, and/or if airway resistance (Raw) increased simultaneously by at least 100% and to at least 0·5 kPa*s/L. Previous studies have demonstrated the superiority of WBP over spirometry for diagnostic decision making.,

Decision making regarding the diagnosis of asthma

A committee of experts (AS, member of the author board of the NVL Asthma; RAJ, Senior Scientist for Respiratory Diseases, Occupational Medicine, Ludwig-Maximilians-University, Munich; KS, member of the author board of the NVL Asthma, Medical Director of the Rehabilitation Clinic for Pneumology Bad Reichenhall) reviewed each diagnosis in consideration of the patient's medical history, WBP and BP results. The committee was blinded to the FeNO results. The diagnostic decisions made by pneumologists and committee were based on the technical characteristics mentioned above in combination with the clinical pattern of patients and the course of disease within twelve weeks, which occasionally allowed a deviation from strict technical standard values.

Decision making regarding ICS responsiveness

Regarding the ICS responsiveness, at least one of the following criteria at t2 had to be fulfilled:

Increase of FEV1 from baseline (t1) by >12% and by >200 mL (objective criterion).

Increase of provocation dose during BP tests by at least one step (objective criterion).

Improvement by 0.5 score points in the ACQ (subjective criterion).

Sensitivity analysis was performed by inclusion of FeNO as an additional objective criterion according to Martin et al. in terms of a decrease of FeNO by ≥20% for baseline values >50 ppb, or a decrease of ≥10 ppb for baseline values ≤50 ppb. Then ICS responsiveness was defined as follows: either a combination of two objective criteria or the combination of one of the objective criteria and the subjective criterion (ACQ improvement).

Statistical analysis

The aim was to evaluate the diagnostic accuracy of FeNO for diagnosing asthma according to the following hypotheses:

Primary hypothesis: The expected sensitivity of 0·35 at the cut-off >50 ppb is significantly larger than 20%, and the expected specificity of 0·95 is significantly larger than 90%. This cut-off was chosen on basis of our previous study and systematic review.

Secondary hypothesis: If the clinical symptoms ‘allergic rhinitis’ and ‘wheezing’ are present, the positive predictive value (PPV) of FeNO >33 ppb is at least 0·70 (validation of the diagnostic algorithm).

Further secondary hypothesis: The PPV of FeNO >33 ppb for ICS responsiveness is at least 0·70.

The two primary endpoints of the primary hypothesis (sensitivity and specificity of FeNO) were each tested confirmatory on a two-sided 5% significance level. A hierarchical test procedure was used to control the global type-I error probability at a 5% significance level and therefore to control for the multiple testing problem. Using exact binomial tests, the expected specificity of 95% was first tested against a reference value of 90% assumed under the null hypothesis.

In case of a significant test result, another confirmatory test of the expected sensitivity of 35% against a reference value of 20% was performed. These tests each achieved a power of 90%, given expected sample sizes of 195 patients without asthma diagnosis and 105 asthma patients (based on a previous study).

A validation of the diagnostic algorithm (FeNO, ‘Allergic Rhinitis’ and ‘Wheezing’) was performed by means of Wilcoxon (Mann–Whitney) rank sum tests with inclusion of all patients of the data set. With the sample sizes mentioned above, this test reached a power of 80% at a two-sided, exploratory 5% significance level to detect a diagnostic accuracy of the area under the curve (AUC)=0.60 vs. AUC=0.5.

In accordance with the secondary hypotheses regarding ICS responsiveness, exploratory testing of the PPV values was performed by exact binomial tests on two-sided 5% significance levels against a reference value of 0·70. This analysis was performed only in patients who reported that they inhaled ICS regularly until follow-up.

Patients participating in the study were characterized by descriptive statistics (mean values, standard deviations (SD), medians, minimum, maximum; absolute and relative frequencies). AUCs were calculated for Receiver Operating Characteristics curves (ROC), which were used to quantify and display the diagnostic performance. Sensitivity analyses using reference equations for ROC calculation were applied to control for the potential influencing factors gender, age, height, allergy, smoking, and infection., For measures of diagnostic accuracy as well as for PPV and the negative predictive value (NPV), corresponding 95% CIs were calculated. The Youden Index = Sensitivity + Specificity – 1 was calculated as another measure of diagnostic accuracy. Fagan nomograms are provided for the PPV and NPV to enable the exploration of post-test probabilities depending on the population specific prevalence.

Role of the funding source

Circassia Germany did not play any role in the design of the study, data evaluation and interpretation of the results. All authors confirm that they had full access to all the data in the study and accept responsibility to submit for publication.

Results

Study population

A total of 479 patients were invited from July 2020 to August 2020, and from March 2021 to July 2021 to participate in the study (Figure 1). The follow-up was completed on January 21, 2022. Finally, 375 patients could be included at baseline. 67 (17·9%) were lost to follow-up because we could not reach them by phone or did not receive sufficient information to derive a final diagnosis. 308 patients with complete follow-up were recruited, 186 (60·4%) were female, average age was 44·7 years (standard deviation 16·7). 180 patients received follow-up with WBP and BP as appropriate, and 128 received follow-up by telephone interview. Follow-up was performed an average of 109 days later (standard deviation 34 days). According to the results of WBP and BP, and including the information about the course of disease during the 3 months, 161 patients (52·3%) had asthma [90 (55·9%) female], 6 (1·9%) COPD [4 (66·7%) female], and 141 (45·8%) no obstructive airway disease [92 (65·2%) female] (Table 1). Concerning the latter, no respiratory disease was found in 95 (30·8%) patients, 11 had gastro-esophageal reflux, 7 post-infectious BHR, 5 chronic sinusitis, 4 post-COVID, 3 ACE-inhibitor-induced cough, 3 vocal cord dysfunction, 3 musculoskeletal chest pain, 3 chronic bronchitis, 2 coronary artery disease, 1 acute bronchitis, 1 hyperventilation syndrome, 1 mycoplasma infection, 1 obstructive sleep apnea, 1 somatoform disorder of the respiratory system.

Figure 1

Flowchart of patient inclusion.

(OAD = obstructive airway disease; BP = bronchial provocation).

Overall, 285 patients (92·5%) received BP, and 23 (7·4%) bronchodilation testing (Figure 2). The diagnosis of asthma was based in 108 patients (35·1%) on BP (5 patients were classified as ´no asthma´ despite positive BP); and in 10 (3·2%) based on bronchodilation tests. On the basis of the follow-up, diagnostic categories changed in 43 patients (14·0%) from ´no asthma´ to ´asthma´; and 5 patients (1·6%) with positive BP changed from ´asthma to ´no asthma´. Patients with asthma had on average the highest FeNO values and ACQ scores; and had a slightly lower FEV1 than patients without airway obstruction (Table 1). COPD patients had on average the lowest FEV1. The most common symptoms of the asthma patients were wheezing and allergic rhinitis. In the lost follow-up group, 33 patients (52·4%) received the diagnosis asthma at baseline (T1). There were no significant differences compared to completers in prevalence of asthma (p = 1·0), age (p = 0·563), sex (p = 0·146), ACQ score (p = 0·857), FEV1 (p = 0·492), FEV1/VC (p = 0·452), and FeNO (p = 0·993).

Figure 2

Flowchart of the diagnostic work-up.

(OAD, obstructive airway disease; COPD, chronic obstructive pulmonary disease).

Diagnostic accuracy of FeNO

The results of the ROC analyses are depicted in Figure 3 A. The AUC comprising all 308 patients was 0·718 (95% CI 0·661–0·775; p < 0·001). The sensitivity at FeNO >50 ppb was 0·24 (95% CI 0·18–0·32), specificity 0·99 (0·95–1·0), PPV 0·95 (0·83–0·99), and NPV 0·54 (0·48–0·60) (Table 2; all cut-off points are presented in Supplement Table 1). 41 patients (13·3%) had FeNO >50 ppb. Concerning the primary hypothesis, the estimated specificity was significantly higher than the reference value of 90% (p < 0·001), while a comparison of the sensitivity to the reference value of 20% did not reach statistical significance (p = 0·20). Of note, the PPV was already above 0·70 at a cut-off >22 ppb; and the highest Youden index was reached at a cut-off >31 ppb. In the sensitivity analysis, the AUC using the reference equation of Dressel et al. was 0·718 (95% CI 0·661–0·774; p < 0·001), and thus nearly identical to the primary analysis. The AUC using the equation of Karrasch et al. was 0·713 (95% CI 0·656–0·770; p < 0·001).

Figure 3

Receiver Operator Characteristics (ROC) curves.

A) FeNO measurement in the diagnosis of asthma: FeNO measurement alone and combination of FeNO + wheezing + allergic rhinitis; B) Prediction of steroid responsiveness with FeNO: conservative estimation and sensitivity analysis; (FeNO = fractional exhaled nitric oxide; AUC = area under the curve; ICS = inhaled corticosteroid; 95% CI = 95% confidence interval).

Table 2

Comparison of the test characteristics at different cut-off points.

Patient group	FeNO	Sensitivity [%] (95%CI)	Specifivity [%] (95%CI)	PPV [%] (95%CI)	NPV [%] (95%CI)	Youden	n
Primary	>70	0·15 (0·10-0·21)	0·99 (0·95-1·00)	0·92 (0·75-0·99)	0·51 (0·45-0·57)	0·14	26
outcome:	>50	0·24 (0·18-0·32)	0·99 (0·95-1·00)	0·95 (0·83-0·99)	0·54 (0·48-0·60)	0·23	41
diagnostic	>40	0·32 (0·25-0·39)	0·97 (0·93-0·99)	0·93 (0·82-0·98)	0·57 (0·50-0·63)	0·29	55
accuracy	>37	0·34 (0·27-0·42)	0·96 (0·91-0·98)	0·90 (0·80-0·96)	0·57 (0·51-0·63)	0·30	61
of FeNO	>35	0·37 (0·29-0·45)	0·95 (0·90-0·98)	0·89 (0·79-0·96)	0·58 (0·51-0·64)	0·32	66
	>34	0·38 (0·30-0·46)	0·95 (0·90-0·98)	0·90 (0·80-0·96)	0·58 (0·52-0·65)	0·33	68
n = 308	>33	0·40 (0·32-0·48)	0·93 (0·87-0·96)	0·85 (0·75-0·92)	0·58 (0·52-0·65)	0·32	75
	>32	0·42 (0·34-0·50)	0·93 (0·87-0·96)	0·86 (0·76-0·93)	0·59 (0·52-0·66)	0·34	78
Pretest	>31	0·43 (0·36-0·52)	0·93 (0·87-0·96)	0·86 (0·77-0·93)	0·60 (0·53-0·66)	0·36	81
probability	>30	0·44 (0·36-0·52)	0·91 (0·85-0·95)	0·85 (0·75-0·91)	0·60 (0·53-0·66)	0·35	84
= 52%	>25	0·52 (0·44-0·59)	0·82 (0·75-0·88)	0·76 (0·67-0·84)	0·61 (0·54-0·68)	0·34	109
	>22	0·57 (0·49-0·65)	0·75 (0·67-0·82)	0·71 (0·63-0·79)	0·61 (0·54-0·69)	0·32	129
	>21	0·60 (0·52-0·67)	0·70 (0·62-0·77)	0·69 (0·60-0·76)	0·61 (0·54-0·69)	0·30	140
	>20	0·65 (0·57-0·73)	0·67 (0·59-0·75)	0·69 (0·61-0·76)	0·64 (0·56-0·71)	0·33	153
	>16	0·73 (0·66-0·80)	0·50 (0·42-0·59)	0·62 (0·54-0·69)	0·63 (0·54-0·72)	0·24	191
	>12	0·86 (0·79-0·91)	0·31 (0·24-0·39)	0·58 (0·51-0·64)	0·67 (0·54-0·78)	0·17	239
Validation of	>50	0·27 (0·15-0·41)	1·00 (0·80-1·00)	1·00 (0·75-1·00)	0·32 (0·20-0·46)	0·27	13
the algorithm:	>35	0·43 (0·29-0·58)	1·00 (0·80-1·00)	1·00 (0·84-1·00)	0·38 (0·24-0·53)	0·43	21
patients with	>34	0·45 (0·31-0·60)	1·00 (0·80-1·00)	1·00 (0·85-1·00)	0·39 (0·24-0·55)	0·45	22
wheezing and	>33	0·49 (0·34-0·64)	0·88 (0·64-0·99)	0·92 (0·75-0·99)	0·38 (0·23-0·54)	0·37	26
allergic rhinitis	>32	0·51 (0·36-0·66)	0·88 (0·64-0·99)	0·93 (0·76-0·99)	0·38 (0·23-0·55)	0·39	27
n = 66	>31	0·53 (0·38-0·67)	0·88 (0·64-0·99)	0·93 (0·76-0·99)	0·39 (0·24-0·57)	0·41	28
	>30	0·55 (0·40-0·69)	0·88 (0·64-0·99)	0·93 (0·77-0·99)	0·41 (0·25-0·58)	0·43	29
Pretest	>28	0·55 (0·40-0·69)	0·82 (0·57-0·96)	0·90 (0·73-0·98)	0·39 (0·23-0·57)	0·37	30
probability	>26	0·57 (0·42-0·71)	0·82 (0·57-0·96)	0·90 (0·74-0·98)	0·40 (0·24-0·58)	0·39	31
= 74%	>25	0·59 (0·44-0·73)	0·76 (0·50-0·93)	0·88 (0·72-0·97)	0·39 (0·23-0·58)	0·36	33
	>20	0·71 (0·57-0·83)	0·71 (0·44-0·90)	0·88 (0·73-0·96)	0·46 (0·27-0·67)	0·42	40

FeNO = Fractional exhaled nitric oxide; PPV = positive predictive value; NPV = negative predictive value.

Regarding the clinical decision rule including the symptoms ‘wheezing’ (yes/no) and ‘allergic rhinitis’ (yes/no), the AUC, which again includes all 308 patients, was 0·777 (0·762–0·829) (Figure 3 A). In the 66 patients with ‘wheezing’ and ‘allergic rhinitis´, the sensitivity at FeNO >33 ppb was 0·49 (0·34–0·64), specificity 0·88 (0·64–0·99), PPV 0·92 (0·75–0·99), and NPV 0·38 (0·23–0·54) (Table 2; all cut-off points are presented in Supplement Table 2). 26 patients (39·4%) had FeNO >33 ppb. With respect to the secondary hypothesis, the estimated PPV was significantly higher than the postulated value of 70% (p = 0·001). The highest sum of sensitivity and specificity was reached at a cut-off >34 ppb. The Fagan Nomograms are depicted in Figure 4.

Figure 4

Fagan's nomogram for assessment of FeNO for diagnosing asthma.

A) FeNO measurement alone; B) Combination of FeNO + wheezing + allergic rhinitis (FeNO = fractional exhaled nitric oxide).

ICS responsiveness estimated by FeNO

In total, 126 patients were prescribed ICS (106 with final diagnosis of asthma,) but only 83 were using ICS until the follow-up investigation according to self-reported data. Of these, all criteria including FeNO measurements and complete ACQ were fulfilled in 68 patients at baseline and follow-up (57 with final diagnosis of asthma). Using the conservative reference standard without FeNO measurement results, AUC was 0·524 (0·375–0·674; p = 0·748) (Figure 3B). Concerning the secondary endpoint according to our hypothesis, a comparison of the resulting PPV = 0·72 at the cut-off >33ppb against the postulated value of 70% did not reach statistical significance (p = 1·0). Accordingly, we found no useful cut-off value to predict ICS responsiveness. In the sensitivity analysis with inclusion of FeNO following the approach by Martin et al., AUC was 0·708 (0·576–0·840; p = 0·004) (Figure 3B). At the cut-off >43 ppb, sensitivity to predict ICS responsiveness was 0·55 (95%CI 0·36–0·74), specificity 0·82 (0·66–0·92), PPV 0·70 (0·47–0·87), and NPV 0·71 (0·56–0·84) (Table 3). The respective Fagan Nomograms are depicted in Figure 5.

Table 3

FeNO cut-off values to predict responsiveness to inhaled corticosteroids.

Patient group	FENO	Sensitivity [%] (95%CI)	Specifivity [%] (95%CI)	PPV [%] (95%CI)	NPV [%] (95%CI)	Youden	n
Conservative analysis	>52	0·31 (0·19-0·46)	0·80 (0·56-0·94)	0·79 (0·54-0·94)	0·33 (0·20-0·48)	0·11	19
- at least one:	>48	0·31 (0·19-0·46)	0·75 (0·51-0·91)	0·75 (0·51-0·91)	0·31 (0·19-0·46)	0·06	20
• Δ FEV1 > 12%	>46	0·31 (0·19-0·46)	0·70 (0·46-0·88)	0·71 (0·48-0·89)	0·30 (0·17-0·45)	0·01	21
• Δ Bronchial provocation 1 step	>43	0·33 (0·20-0·48)	0·65 (0·41-0·85)	0·70 (0·47-0·87)	0·29 (0·16-0·44)	-0·02	23
• Δ ACQ > 0·5	>39	0·33 (0·20-0·48)	0·60 (0·36-0·81)	0·67 (0·45-0·84)	0·27 (0·15-0·43)	-0·07	24
	>36	0·35 (0·22-0·51)	0·60 (0·36-0·81)	0·68 (0·46-0·85)	0·28 (0·15-0·44)	-0·05	25
n=68	>35	0·40 (0·26-0·55)	0·60 (0·36-0·81)	0·70 (0·50-0·86)	0·29 (0·16-0·46)	0·00	27
ICS responder n=48	>34	0·42 (0·28-0·57)	0·60 (0·36-0·81)	0·71 (0·51-0·87)	0·30 (0·17-0·47)	0·02	28
(Pretest probability = 71%)	>33	0·44 (0·29-0·59)	0·60 (0·36-0·81)	0·72 (0·53-0·87)	0·31 (0·17-0·48)	0·04	29
	>32	0·50 (0·35-0·65)	0·60 (0·36-0·81)	0·75 (0·57-0·89)	0·33 (0·19-0·51)	0·10	32
	>31	0·50 (0·35-0·65)	0·55 (0·32-0·77)	0·73 (0·54-0·87)	0·31 (0·17-0·49)	0·05	33
	>28	0·52 (0·37-0·67)	0·55 (0·32-0·77)	0·74 (0·56-0·87)	0·32 (0·17-0·51)	0·07	34
	>20	0·69 (0·54-0·81)	0·45 (0·23-0·68)	0·75 (0·60-0·87)	0·38 (0·19-0·59)	0·14	44
Sensitivity analysis	>52	0·52 (0·33-0·71)	0·90 (0·76-0·97)	0·79 (0·54-0·94)	0·71 (0·57-0·83)	0·41	19
- two of any objective criteria:	>48	0·52 (0·33-0·71)	0·87 (0·73-0·96)	0·75 (0·51-0·91)	0·71 (0·56-0·83)	0·39	20
• Δ FEV1 > 12%	>46	0·52 (0·33-0·71)	0·85 (0·69-0·94)	0·71 (0·48-0·89)	0·70 (0·55-0·83)	0·36	21
• Δ Bronchial provocation 1 step	>43	0·55 (0·36-0·74)	0·82 (0·66-0·92)	0·70 (0·47-0·87)	0·71 (0·56-0·84)	0·37	23
• Δ FeNO	>39	0·55 (0·36-0·74)	0·79 (0·64-0·91)	0·67 (0·45-0·84)	0·70 (0·55-0·83)	0·35	24
or	>36	0·55 (0·36-0·74)	0·77 (0·61-0·89)	0·64 (0·43-0·82)	0·70 (0·54-0·83)	0·32	25
• One objective criterion	>35	0·55 (0·36-0·74)	0·72 (0·55-0·85)	0·59 (0·39-0·78)	0·68 (0·52-0·82)	0·27	27
• and Δ ACQ > 0·5	>34	0·59 (0·39-0·76)	0·72 (0·55-0·85)	0·61 (0·41-0·78)	0·70 (0·53-0·83)	0·30	28
	>33	0·59 (0·39-0·76)	0·69 (0·52-0·83)	0·59 (0·39-0·76)	0·69 (0·52-0·83)	0·28	29
n=68	>32	0·66 (0·46-0·82)	0·67 (0·50-0·81)	0·59 (0·41-0·76)	0·72 (0·55-0·86)	0·32	32
ICS responder n=29	>31	0·66 (0·46-0·82)	0·64 (0·47-0·79)	0·58 (0·39-0·75)	0·71 (0·54-0·85)	0·30	33
(Pretest probability = 43%)	>28	0·69 (0·49-0·85)	0·64 (0·47-0·79)	0·59 (0·41-0·75)	0·74 (0·56-0·87)	0·33	34
	>20	0·79 (0·60-0·92)	0·46 (0·30-0·63)	0·52 (0·37-0·68)	0·75 (0·53-0·90)	0·25	44

FeNO = Fractional exhaled nitric oxide; PPV = positive predictive value; NPV = negative predictive value; FEV1 = Forced Expiratory Volume in the first second; ACQ = Asthma Control Questionnaire; ICS = inhaled corticosteroids.

Figure 5

Fagan's nomogram for prediction of responsiveness to inhaled corticosteroids.

A) conservative estimation; B) sensitivity analysis.

Discussion

To our knowledge, this is the first confirmatory study in ambulatory care using a priori determined cut-off values of FeNO in terms of hypothesis testing. In a large study population comprising 308 patients with 3-month follow up, we found that FeNO measurement allowed a reliable ruling-in but not ruling-out of an asthma diagnosis. Ruling-in was particularly efficient in combination with wheezing and allergic rhinitis as clinical symptoms. However, we could only partially verify the usefulness of FeNO measurement in predicting ICS responsiveness.

A systematic review showed a promising diagnostic accuracy of FeNO, with an AUC=0.80, and an in-depth analysis found adequate cut-off values around 50 ppb. However, several studies identified lower cut-off values around 30 to 40 ppb for ruling-in the diagnosis of asthma, in terms of post hoc analysis. In the present study, the specificity of FeNO >50 ppb was 0·99 (95%CI 0·95–1·0), thus verifying our hypothesis, but the sensitivity of 0·24 (95%CI 0·18–0·32) was considerably lower. Nevertheless, this allowed a ruling-in of asthma with a PPV of 0·95 (95%CI 0·84–0·99), whereas ruling-out was not possible. Notwithstanding the confirmatory character of FeNO >50 ppb, lower cut-off values might be useful, as suggested by BTS and NICE guidelines,, but this is limited due to the post hoc analysis.

Beyond this limitation, patient selection is of utmost importance. The prevalence of asthma in our multi-center study was higher than in the previous single-centre study. This might be explained by the fact that the patients were mainly recruited in allergy seasons, thus leading to a higher pre-test probability of the disease, and consequently to higher positive predictive values. This patient selection with regard to higher allergen exposure might be inferred from the clustering of the symptoms ‘wheezing’ and ‘allergic rhinitis’ and fits well with the previously established clinical decision rule, which is now confirmed. The AUC=0·777 was now even higher than in the former study (AUC=0·754), showing a difference of 0·023 (95%CI -0·038 to 0·085). According to this, FeNO >33 ppb allows a reliable diagnosis of asthma when patients are reporting these two symptoms.

Regarding ICS responsiveness, we could not verify the predictive value of FeNO >33 ppb. This might have been due to our strict criteria excluding FeNO from the assessment of ICS responsiveness. We chose this strict definition because we expected that a ‘regression to the mean’ effect could lead to over-estimation of the predictive value when FeNO baseline values are used to predict a difference between follow-up and baseline FeNO measurement. However, a secondary analysis with inclusion of the baseline value, corresponding to the analysis by Martin et al., identified a reasonable PPV of 0·70 using FeNO >43 ppb to predict ICS responsiveness. This post hoc finding is close to the value of >47 ppb reported by Smith et al., and to the value of ≥40 ppb reported by Price et al. regarding ICS responsiveness in patients with unspecific respiratory symptoms.

The major strength of the study is the hypothesis-driven analysis with a large population of patients as required by the previous power calculation, which allowed a verification of the pre-defined cut-off values and previously established diagnostic algorithm. Thereby we found no significant influence of gender, age or height on the diagnostic accuracy of FeNO measurement. Another strength is the diagnostic work-up with WPB, and almost 93% of the patients with BP. While BP is not very common in routine care internationally, it is performed regularly in ambulatory practices of pneumologists in Germany, and the advantages of WBP for diagnostic decision making was shown previously., Therefore, this in-depth investigation in ambulatory care should ensure a broad generalisability of the results. A limitation is, that the number of patients without asthma was lower than expected in the sample size calculation. A power of >99% was achieved because of the high observed specificity of 99% for the test of the first primary hypothesis. Specificity in particular is most important for ruling in the disease, and the PPV has been shown to be very high. In contrast, a post-hoc analysis showed, that the power of the test regarding the second primary hypothesis was only 22%. This was despite the higher number of patients with asthma than expected, which is due to the low observed sensitivity of 24%. The low sensitivity may be a concern in terms of high-quality testing, as too many asthma cases may be missed if the diagnosis is not secured with bronchial provocation in case of a negative FeNO result. Therefore, there is some risk that FeNO measurement will become inefficient if used unselectively. On the other hand the diagnostic accuracy increased when ‘wheezing’ and ‘allergic rhinitis’ were present, and reasonable ruling-in was possible at FeNO >33 ppb, which was the case in 26 (39·4%) of 66 patients. Accordingly, FeNO measurement appears to be most efficient in combination with clinical signs and symptoms. Beyond that, a number of studies indicated that indirect bronchial challenges might be more specific for the diagnosis of asthma than methacholine challenges. Despite this, we included methacholine challenges as reference, as these are well introduced and common in clinical practice, in contrast to indirect challenges. It might be that relative to indirect challenges the sensitivity of FeNO will turn out to be higher than estimated in the present study, but this would only confirm the value of this measurement. Another limitation is that the vast majority of patients in the study were of Caucasian origin. Therefore, the results may not be applicable to non-caucasian people. A further limitation is that we could not recruit all patients personally for follow-up investigation. We tried to mitigate this problem through telephone interviews. In this way, we were able to verify the hypothetical cut-off values and to confirm the clinical rule for diagnostic decision making in patients with suspected asthma. Bias in relation to the lost to follow-up group of 67 patients seems unlikely, as no significant differences to the completers were found. Another limitation might be that a longer disease course could have been considered, such as a 12-month follow-up. However, according to the asthma guidelines, a period of 3 months is considered as sufficient for treatment decisions.2, 3, 4, 5 In further studies, repeated follow-up could be performed, ideally every 3 months up to 12 months, to assess the course of disease. Another limitation might be that patients’ reports on allergic rhinitis did not have objective verification, e.g. with nasal provocation tests. This, however, is the typical state of knowledge under the conditions of ambulatory care. Unfortunately, for organisational reasons on practice level, we were unable to perform FeNO follow-up measurements and to ensure full completion of questionnaires in all patients.

The most important limitation might be that we had to rely on patients’ assertions regarding ICS inhalation and could not objectively control for medication adherence. We could only rely on the information given by the patients in the questionnaire. This may partially explain why we could not verify the PPV of 0·70 at FeNO >33ppb for ICS responsiveness. Nevertheless, our results of the sensitivity analysis confirmed the relationship between ICS responsiveness and increased FeNO values, but with higher cut-off values than expected. These results are consistent with those of previous studies, and noticeably consistent with cut-off values for the diagnosis of asthma.

To conclude, FeNO values >50 ppb allow a valid ruling-in of an asthma diagnosis, whereas ruling-out of asthma is not possible with FeNO measurement only. A FeNO cut-off >33 ppb is adequate, when patients are also suffering from wheezing and allergic rhinitis. Thus, FeNO measurement appears more efficient for ruling-in asthma when used in combination with clinical signs and symptoms. The probability of ICS responsiveness is enhanced with increased FeNO cut-off values, which alleviates therapeutic decision making. Therefore, FeNO measurement may replace bronchial provocation in a distinct proportion of patients.

Contributors

AS, RAJ, BB, CK conceived the study. BB implemented and managed the study, and helped to recruit the patients. CK helped with recruitment. AS, RAJ and KS controlled the diagnosis within the committee of experts. AH performed the statistical analyses. AS and BB drafted the manuscript and contributed equally; all authors revised it critically for important intellectual content and approved the final version. AS, RAJ, BB, AH have accessed and verified the data. AS was responsible for the decision to submit the manuscript. All authors confirm that they had full access to all the data in the study and accept responsibility for the decision to submit for publication.

Data sharing statement

Data are available from the authors upon reasonable request.

Declaration of interests

AS is an external expert for the Federal Joint Committee (Gemeinsamer Bundesausschuss) with regard to the development of the disease management programs for asthma and COPD; he received regular remuneration for taking part in the sessions. AS and KS are members of the National Asthma Guideline Board (Nationale Versorgungsleitlinie Asthma). The other authors declare that no conflicts of interest exist.