Interobserver diagnostic variability at "moderate" agreement levels could significantly change the prognostic estimates of clinicopathologic studies: evaluation of the problem using evidence from patients with diffuse lung disease.

Does interobserver diagnostic variability (IODV) influence the accuracy of prognostic estimates of clinicopathologic studies? "Best evidence" from usual interstitial pneumonia (UIP) and nonspecific interstitial pneumonia (NSIP) patients was used to investigate the effects of IODV. Systematic literature review identified studies of UIP and NSIP providing "best evidence." Survival proportions from studies were compared using chi(2) and meta-analysis. Interobserver diagnostic variability was simulated in the data arbitrarily at 5% to 30% intervals. The various "diagnoses" were evaluated with kappa, and chi(2) statistics were used to evaluate the interobserver agreement and compare survival proportions. The survival proportions for UIP and NSIP patients in 7 retrospective level III studies ranged from 11% to 58% and 39% to 100%, respectively. Analysis of simulation results with kappa and chi(2) statistics showed that IODV greater than 10% resulted in significantly different survival proportion estimations. Interobserver diagnostic variability at moderate agreement levels significantly influences prognostic estimates. Evaluation and minimization of IODV in future clinicopathologic studies are indicated. Copyright 2010 Elsevier Inc. All rights reserved.