Diagnostic accuracy of radiographs for detecting supraglottitis: a systematic review and meta-analysis.

Aim: To assess the diagnostic performance of lateral radiograph of the neck for supraglottitis in adults and children.
Methods: Electronic database searches (including PubMed, EMBASE, CINAHL, Web of Science, and WHO International Clinical Trials Registry Platform) were carried out through July 2014. Citations of included studies and recent narrative reviews were searched. Studies that compared lateral radiograph of the neck with a reference standard of direct/indirect laryngoscopy were included. Two reviewers independently assessed the methodological quality of included studies by Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2).
Results: Of 2,088 potentially relevant citations, two single-gate (cohort) studies and six two-gate (case-control) studies were identified. These included studies of design are at the moderate or high risk of bias in QUADAS-2. The pooled sensitivity from bivariate random-effects regression was 92.9% (95% confidence interval [CI], 88.5-95.9%) and the pooled specificity was 89.2% (95% CI, 85.9-91.9%), but the diagnostic value would be overestimated because of selection bias in the six two-gate studies. The sensitivity and specificity of the single-gate studies were 100.0% (95% CI, 92.2-100.0%) and 30.6% (95% CI, 15.5-35.6%) in children and 81.0% (95% CI, 78.2-93.2%) and 85.7% specificity (95% CI, 78.2-93.2%) in adults.
Conclusion: This study determines that there are insufficient studies of lateral neck radiograph for detecting supraglottitis. Lateral radiograph of the neck seems to have moderate accuracy for detecting supraglottitis. Further approximately unbiased studies are needed to obtain more valid and reliable estimates of test accuracy.

Introduction

Supraglottitis is an inflammation of the epiglottis and/or adjacent supraglottic structures (e.g. aryepiglottic folds and arytenoids), and is caused by bacterial, viral, and fungal infections as well as other agents. It can be rapidly fatal due to life‐threatening airway obstruction. Patient with acute supraglottitis at high risk of airway obstruction should have direct visualization of the epiglottitis and should be monitored closely in the intensive care unit or other wards where airway management is immediately available.1 The prognosis of supraglottitis is good if adequate airway management is provided.

Direct visualization of the epiglottis by nasopharyngeal laryngoscopy or indirect laryngoscopy is the gold standard for diagnosing supraglottitis. Lateral soft‐tissue radiograph of the neck is a classic imaging tool and has long been used to detect supraglottitis.2 Other imaging methods including computed tomography and ultrasonography were also used.3, 4 Computed tomography has a greater sensitivity compared to lateral neck radiograph,4, 5 but is less easy to access in a clinical setting. Recently, neck ultrasonography was evaluated as a diagnostic method for supraglottitis.3 However, the accuracy of the ultrasonographic imaging has not yet been evaluated sufficiently. Therefore, lateral soft‐tissue radiographs of the neck are used frequently to detect supraglottitis as they are readily obtained and the equipment is available in most clinical settings.5, 6

The radiographic findings of supraglottitis are a thickened epiglottis (thumb sign),7 swelling of the aryepiglottis, and a poorly defined vallecular air pocket (vallecula sign).8 In diagnostic accuracy evaluations, patients are enrolled in either a “cohort‐type study” (single‐gate study), which uses a single set of inclusion criteria, or a “case–control‐type” study (two‐gate study), which uses different sets of criteria. Most assessments of the utility of lateral neck radiographs for supraglottitis have been case–control‐type studies, despite their high risk of overestimating test characteristics. Additionally, previous studies calculating diagnostic performance of radiographs in supraglottitis had small sample sizes.7, 8 The primary objective of this systematic review is to assess the diagnostic performance of lateral radiograph in screening of the neck for supraglottitis including both single‐ and two‐gate studies.

Methods

We carried out a systematic review and meta‐analysis of patients with supraglottitis who were diagnosed using laryngoscopy (direct/indirect laryngoscopy and nasopharyngeal laryngoscopy). Standard guidelines for the systematic review of diagnostic studies were used.9, 10, 11 The protocol of this review was registered with PROSPERO (International Prospective Register of Systematic Reviews: http://www.crd.york.ac.uk/PROSPERO/) under registration number CRD42014013007.

Search strategy

The following electronic databases were searched: PubMed (up to July 2014), CINAHL (up to July 2014), Web of Science (up to July 2014), and EMBASE (up to July 2014). The reference lists of all included publications and recent narrative reviews12, 13 were searched. To include unpublished trials, we used the WHO International Clinical Trials Registry Platform (http://www.who.int/ictrp/en/). A medical librarian (H.G.) constructed an electronic search strategy that contained search terms for disease condition (supraglottitis) and an index test (lateral neck radiograph) without a methodology search filter to identify diagnostic test accuracy. Both published and unpublished studies in all languages were included in this review. The search strategies and search results of each database were registered at PROSPERO.

Eligibility criteria

Studies that assessed the accuracy of radiography for epiglottitis according to accepted reference standards, defined as diagnosis by nasopharyngeal laryngoscopy or indirect/direct laryngoscopy, were included. If the study used health personnel or patients who were examined by radiographs for other diseases (e.g., foreign‐body detection) rather than for supraglottitis, it was included despite the lack of a laryngoscopic reference standard. The included studies were well‐defined single‐gate studies with sufficient data to calculate true‐positive, false‐positive, false‐negative, and true‐negative results. Two‐gate studies were included if the number of single‐gate studies were insufficient. Studies including five or less supraglottitis patients were excluded.

Study selection and data extraction

Two independent authors (Ta.F. and T.M.) reviewed all titles and abstracts identified by electronic and other searches. Full‐text papers of all studies that were identified as possibly eligible by at least one of the two review authors were retrieved and reviewed by two independent reviewers to ensure that they fulfilled the inclusion criteria. Any disagreement was resolved by discussion or by consulting a third reviewer (H.T.).

Two authors (Ta.F. and H.T.) independently extracted data from the eligible studies. Disagreements before data synthesis were resolved by discussion or in consultation with a third review author (To.F.). If information was lacking either to construct the main 2 × 2 table or for one of the prespecified subgroups, we contacted the study authors.

We also extracted the characteristics of the study patients, including population (children or adults), study setting, radiographic abnormalities (e.g., thumb sign or vallecula sign). In mixed study populations with separate results for children and adults, we used the cut‐off set by the author. If the study calculated diagnostic accuracy using a mixed population of adults and children, we recalculated the diagnostic accuracy of each group, using <18 years as the cut‐off age.

Assessment of study quality

Two authors (Ta.F. and H.T.) independently assessed the methodological quality of the included studies using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS‐2) criteria.14 The inter‐rater agreement was 84.7%. Disagreements were resolved by consensus with a third author (To.F.). The QUADAS‐2 is structured with the following four domains: patient selection, index test, reference standard, and flow and timing. Each domain was assessed in terms of risk of bias.

Methods of measurement

Sensitivity and specificity with 95% confidence intervals (CI) were calculated from the 2 × 2 tables (true‐positive, false‐positive, false‐negative, and true‐negative results). When false negatives or false positives are zero in a study, we added 0.5 to all cells of 2 × 2 tables in calculating the 95% CI of sensitivity and specificity, and likelihood ratio.

Using meta‐analysis, we calculated pooled sensitivity and specificity using extracted 2 × 2 tables from included studies. To avoid double counting of the results from the same patients, only one 2 × 2 table from each of the retrieved articles was included. When a study used multiple thresholds, we included the one with the highest sensitivity because lateral radiographs of the neck are used to rule out supraglottitis. Pooled sensitivity and specificity was estimated using the bivariate random effects model. Hierarchical summary receiver operating characteristic (HSROC) curves were used for data synthesis.

Three of the included studies calculated sensitivity and specificity using the number of radiographic interpretations as the denominator.8, 15, 16 For example, in a report by Ducic et al., 15 practitioners interpreted 26 radiographs of the supraglottitis and sensitivity was calculated using the number of radiographic interpretations (epiglottitis group = 390) as the denominator. In their study, 383 interpretations were true‐positives and the sensitivity was 98.2% (383/390). For data synthesis, we tried to provide original patient numbers for data synthesis to avoid calculating method heterogeneity. Then, we recalculated the number of true‐positive patients using sample size (epiglottitis = 26) and sensitivity (98.2%). Thus, the calculated number of true‐positive patients in the Ducic et al. report was 25.53 (26 × 0.982). Using the same method, we recalculated the number of false‐positive, false‐negative, and true‐negative patients. These numbers were rounded to integral numbers for further analysis.

In this review, the following were considered as potential sources of heterogeneity: type of study (single‐ or two‐gate study), index test (blind or not), reference standard (blind or not), and flow and timing (adequate or not). All analyses were carried out using Meta‐DiSc software.17

Results

Study selection

Of the 2088 potential citations, 123 articles were eligible for a full‐text review. After full‐text review, we identified two single‐gate studies (children = 1, adults = 1) and six two‐gate studies (children = 1, adults = 3, mixed = 2). After full‐text screening, 115 articles were excluded. The reasons and article titles are listed in the Data S1. Figure 1 shows the process of study selection. The characteristics of the included studies are detailed in Table 1.

Figure 1

Study flow diagram of assessment of published work regarding the diagnostic performance of lateral radiograph of the neck for supraglottitis.

Table 1

Main characteristics of reviewed studies regarding the use of radiographs of the neck for detecting supraglottitis

Author (publication year)	Study design	Population		Radiograph interpretation	Evaluated radiographic abnormality
Stankiewicz et al. (1985)15	Retrospective	Mixed	Supraglottitis: 14 supraglottitis patients Control: 30 croup patients	Forty‐four radiographs were interpreted in the emergency room (study 1). Six radiologists interpreted 41 of 44 radiographs (study 2)	Any
Rothrock et al. (1990)16	Retrospective	Mixed	Supraglottitis: 31 supraglottitis patients (25 children and 6 adults aged 6 months–61 years) Control: 31 age‐ and sex‐matched croup, pharyngitis, and dysphagia patients	Three senior emergency medicine residents interpreted the radiographs	EW/C3W >0.5 AEW/C3W >0.35 HPW/C3W >0.8
John et al. (1994)25	Retrospective	Child	Supraglottitis: 38 supraglottitis patients (age, 8 months–5 years) Control: 100 croup patients (age, 6 months–11 years) and another 100 patients (age, 2 months–12 years)	Aryepiglottic fold size was measured at the midpoint of the folds (AEW1), behind the epiglottitis (AEW2), and at the base of the folds (AEW3)	AEW1a AEW2 AEW3
Nemzek et al. (1995)26	Retrospective	Adult	Supraglottitis: 27 supraglottitis patients (age, 28–81 years) Control: 15 age‐matched, chronic musculoskeletal pain patients (age, 24–79 years)		EW/C4W > 0.5 AEWb ≥2 + HPW/C4W >1.5
Ducic et al. (1997)8	Retrospective	Adult	Supraglottitis: 26 supraglottitis patients (mean age, 44 ± 18.5 years) Control: 12 minor cervical trauma patients and 14 with suspected foreign bodies (mean age, 40 ± 19.5 years)	Staff emergency physicians (4), otolaryngology residents (3), radiology residents (4), and senior medical students (4) interpreted 56 radiographs	Vallecula sign (absence: deep and well‐defined vallecula)
Yong et al. (2001)18	Retrospective	Adult	Supraglottitis: 30 supraglottitis patients (age, 25–62 years) Control: 30 age‐ and sex‐matched normal adults (age, 23–61 years)	Two otolaryngologists and two diagnostic radiologists measured the area of interest on each radiograph twice at different times. The measurements made by each reviewer were averaged	EW >7.0 mm AEW >4.5 mm
Cohort‐type studies
Ragosta et al. (1997)22	Retrospective	Child	Fifty‐nine patients (23 supraglottitis and 36 non‐supraglottitis patients) with referring physician's diagnosis of possible/probable supraglottitis	Referring physician interpreted radiographs	Any
Fujiwara et al. (2014)19	Prospective	Adult	105 patients (21 supraglottitis and 84 non‐supraglottitis patients; mean age, 42.8 ± 18.1 years) who had radiographs for supraglottitis detection	One emergency physician and one radiologist independently interpreted the radiographs, with disagreement resolved by discussion	Thumb sign Vallecula sign

John et al. measured aryepiglottic width (AEW) at the midpoint of the fold (AEW1), behind the epiglottis (AEW2), and at the base of the fold (AEW3).

Nemzek et al. graded AEW on the following scale: 1+, slightly swollen; 2+, moderately swollen; 3+, markedly swollen; 4+, massively swollen.

C3W, third cervical vertebral body width; C4W, fourth cervical vertebral body width; EW, epiglottic width; HPW, hypopharyngeal airway width.

Methodological quality

Quality assessment using the QUADAS‐2 tool is shown in Figure 2. Because six studies of the eight included studies used the two‐gate study design, the patient selection domain and flow and timing domain had a high risk of bias and of invalid applicability. In five of the eight studies included, the index test was interpreted without the results of the reference standard.8, 15, 16, 18, 19 Due to the nature of the research, laryngoscopic evaluation was carried out after the index test. Thus, there was a potential influence of prior knowledge of the index test results and the reference standard domain had a high risk of bias.

Figure 2

Risk of bias summary: review authors’ judgement about each risk of bias item for each included study of the use of lateral radiograph of the neck for detecting supraglottitis. For risk of bias summary, Quality Assessment of Diagnostic Accuracy Studies 2 criteria were used.

Diagnostic accuracy

Figure 3 presents forest plots of the sensitivity and specificity of the included studies. Sensitivity ranged from 42.4% to 100.0% and specificity ranged from 75.8 to 100.0%. The ranges of specificity in included studies were relatively narrow, the sensitivity of included studies were varied.

Figure 3

Sensitivity and specificity of included studies regarding the use of radiographs for detecting supraglottitis. The 95% confidence intervals (CI) were calculated from 2 × 2 tables. When false negatives (FN) or false positives (FP) were zero in a study, we added 0.5 to all cells of 2 × 2 tables. TN, true negative; TP, true positive.

The pooled sensitivity from bivariate random‐effects regression was 92.9% (95% CI, 88.5–95.9%) and the pooled specificity was 89.2% (95% CI, 85.9–91.9%). Figure 4 shows the HSROC curve of lateral radiographs of the neck for supraglottitis.

Figure 4

Hierarchical summary receiver operating characteristic (HSROC) curve of lateral radiographs of the neck for detection of supraglottitis. The HSROC curve, summary operating point (gray square), and the 95% confidence contour for the summary point are shown. Gray and white circles correspond to individual study estimates.

Investigation of heterogeneity

We calculated diagnostic odds ratio (DOR) (DOR = positive likelihood ratio / negative likelihood ratio) to investigate heterogeneity. The DOR of potential sources of heterogeneity was calculated; 414.3 (95% CI, 19.9–8625.1) DOR in two‐gate studies and 24.8 (95% CI, 7.9–77.9) DOR in single‐gate studies, 149.7 (95% CI, 10.1–2227.7) DOR in index test blind and 243.4 DOR (95% CI, 18.3–3234.0) in index test open. We could not assess the heterogeneity of reference standard (blind or not) and flow and timing (adequate or not), because all included studies used unblinded reference standards and inadequate flow and timing.

Discussion

This systematic review showed that the available evidence of the accuracy of lateral radiographs of the neck for supraglottitis was limited. In this review, calculated pooled sensitivity and specificity of the radiographs included two‐gate studies. Heterogeneity investigation showed that the DOR of two‐gate study was 16.7‐fold that of single‐gate studies. In the diagnostic accuracy test, two‐gate design was the most overestimating factor and tended to have higher DOR compared with single‐gate studies.20, 21 The overestimation of test characteristics of the two‐gate type of studies was widely evaluated and accepted.21 Overall, included studies suffered from various types of potential bias, making it difficult to formulate definitive conclusions about the accuracy of lateral neck radiographs and their usefulness.

Of the two identified single‐gate studies, one study enrolled children and another enrolled adults. Ragosta et al. retrospectively reviewed medical records and included child patients with possible/probable supraglottitis diagnosed by a physician.22 Seventy‐three children were reviewed and 59 children underwent lateral radiographs of the neck. Supraglottitis was diagnosed by direct visualization. The sensitivity and specificity of the radiographs were 100.0% (95% CI, 92.2–100.0%) and 30.6% (95% CI, 15.5–35.6%) respectively. Fujiwara et al. prospectively enrolled 105 adult patients who had undergone lateral neck radiographs to evaluate supraglottitis.19 One radiologist and one emergency physician, who were blind to patients’ symptoms and laryngoscopic evaluations, interpreted lateral neck radiographs. The sensitivity and specificity of lateral radiographs of the neck were 81.0% (95% CI, 78.2–93.2%) and 85.7% (95% CI, 78.2–93.2%), respectively.

This systematic review has several limitations. First, we identified only two small single‐gate studies. We also identified six two‐gate studies, which used croup, minor trauma, or healthy persons as control groups. However, heterogeneity investigation suggested potential bias and it is difficult to generalize pooled diagnostic values obtained from this systematic review to the clinical setting. Second, many of the included studies lacked details of disease severity, even though the ability of lateral neck radiograph to detect supraglottitis depends on the severity of the edema in the supraglottic tissue.19 Recently, nasopharyngeal laryngoscopy, which is more informative than indirect laryngoscopy, has been in widespread use in the clinical setting. Based on nasopharyngeal laryngoscopic evaluation, some classification tools of supraglottitis have been developed.23, 24 These proposed classifications were published later than many of the studies included in our systematic review. Thus, many of the studies included here lacked these classifications. Further studies should assess the severity of supraglottitis when calculating the diagnostic value of lateral neck radiographs. Third, we carried out a meta‐analysis of included studies, but the radiographic abnormalities varied between them. The heterogeneity would have affected the pooled diagnostic value. Finally, a limitation of the included studies is the methodological problem of repeated measurement. Three of the included studies calculated sensitivity and specificity using the number of radiograph interpretations rather than the number of patients as the denominator.8, 15, 16 These studies tend to have a narrow 95% CI, although these studies enrolled similar patient numbers to the other five studies.

Conclusion

This systematic review cannot draw a definitive conclusion regarding the accuracy of radiographs in supraglottitis because of insufficient available evidence. Until now, the lateral soft‐tissue radiograph has been often used to rule out supraglottitis. We would make better use of radiographs for supraglottitis with knowledge of potential bias in previous studies. Additionally, the diagnostic value of radiographs would be lower when inexperienced individuals examine and evaluate. Further good‐quality studies will be needed to clarify the diagnostic value of radiographs in patients with suspected supraglottitis.

Conflict of Interest

None Declared.

Data S1 Title list of 115 articles regarding the use of radiographs for detecting supraglottitis excluded from this study during the full‐text screening process. Reasons for exclusion are shown.

Click here for additional data file.