Variability and accuracy in interpretation of consolidation on chest radiography for diagnosing pneumonia in children under 5 years of age.

BACKGROUND: Consolidation on chest radiography is widely used as the reference standard for defining pneumonia and variability in interpretation is well known but not well explored or explained.
METHODS: Three pediatric sub-specialists (infectious diseases, radiology and respiratory medicine) viewed 3,033 chest radiographs in children aged under 5 years of age who presented to one Emergency Department (ED) with a febrile illness. Radiographs were viewed blind to clinical information about the child and blind to findings of other readers. Each chest radiograph was identified as positive or negative for consolidation. Percentage agreement and kappa scores were calculated for pairs of readers. Prevalence of consolidation and reader sensitivity/specificity was estimated using latent class analysis.
RESULTS: Using the majority rule, 456 (15%) chest radiographs were positive for consolidation while the latent class estimate was 17%. The radiologist was most likely (21.3%) and respiratory physician least likely (13.7%) to diagnose consolidation. Overall percentage agreement for pairs of readers was 85-90%. However, chance corrected agreement between the readers was moderate, with kappa scores 0.4-0.6 and did not vary with patient characteristics (age, gender, and presence of chronic illness). Estimated sensitivity ranged from 0.71 to 0.81 across readers, and specificity 0.91 to 0.98.
CONCLUSIONS: Overall agreement for identification of consolidation on chest radiographs was good, but agreement adjusted for chance was only moderate and did not vary with patient characteristics. Clinicians need to be aware that chest radiography is an imperfect test for diagnosing pneumonia and has considerable variability in its interpretation.