Elana Thau1, Manuela Perez2, Martin V Pusic3, Martin Pecaric4, David Rizzuti5, Kathy Boutis1. 1. Department of Pediatrics Division of Emergency Medicine Hospital for Sick Children and the University of Toronto Toronto Ontario Canada. 2. Department of Medical Imaging Hospital for Sick Children and the University of Toronto Toronto Ontario Canada. 3. Department of Pediatrics Harvard Medical School Boston Massachusetts USA. 4. Contrail Consulting Services Inc Toronto Ontario Canada. 5. Schulich School of Medicine & Dentistry Western University London Ontario Canada.
Abstract
OBJECTIVES: Using a sample of pediatric chest radiographs (pCXR) taken to rule out pneumonia, we obtained diagnostic interpretations from physicians and used learning analytics to determine the radiographic variables and participant review processes that predicted for an incorrect diagnostic interpretation. METHODS: This was a prospective cross-sectional study. A convenience sample of frontline physicians with a range of experience levels interpreted 200 pCXR presented using a customized online radiograph presentation platform. Participants were asked to determine absence or presence (with respective location) of pneumonia. The pCXR were categorized for specific image-based variables potentially associated with interpretation difficulty. We also generated heat maps displaying the locations of diagnostic error among normal pCXR. Finally, we compared image review processes in participants with higher versus lower levels of clinical experience. RESULTS: We enrolled 83 participants (20 medical students, 40 postgraduate trainees, and 23 faculty) and obtained 12,178 case interpretations. Variables that predicted for increased pCXR interpretation difficulty were pneumonia versus no pneumonia (β = 8.7, 95% confidence interval [CI] = 7.4 to 10.0), low versus higher visibility of pneumonia (β = -2.2, 95% CI = -2.7 to -1.7), nonspecific lung pathology (β = 0.9, 95% CI = 0.40 to 1.5), localized versus multifocal pneumonia (β = -0.5, 95% CI = -0.8 to -0.1), and one versus two views (β = 0.9, 95% CI = 0.01 to 1.9). A review of diagnostic errors identified that bony structures, vessels in the perihilar region, peribronchial thickening, and thymus were often mistaken for pneumonia. Participants with lower experience were less accurate when they reviewed one of two available views (p < 0.0001), and accuracy of those with higher experience increased with increased confidence in their response (p < 0.0001). CONCLUSIONS: Using learning analytics, we identified actionable learning opportunities for pCXR interpretation, which can be used to allow for a customized weighting of which cases to practice. Furthermore, experienced-novice comparisons revealed image review processes that were associated with greater diagnostic accuracy, providing additional insight into skill development of image interpretation.
OBJECTIVES: Using a sample of pediatric chest radiographs (pCXR) taken to rule out pneumonia, we obtained diagnostic interpretations from physicians and used learning analytics to determine the radiographic variables and participant review processes that predicted for an incorrect diagnostic interpretation. METHODS: This was a prospective cross-sectional study. A convenience sample of frontline physicians with a range of experience levels interpreted 200 pCXR presented using a customized online radiograph presentation platform. Participants were asked to determine absence or presence (with respective location) of pneumonia. The pCXR were categorized for specific image-based variables potentially associated with interpretation difficulty. We also generated heat maps displaying the locations of diagnostic error among normal pCXR. Finally, we compared image review processes in participants with higher versus lower levels of clinical experience. RESULTS: We enrolled 83 participants (20 medical students, 40 postgraduate trainees, and 23 faculty) and obtained 12,178 case interpretations. Variables that predicted for increased pCXR interpretation difficulty were pneumonia versus no pneumonia (β = 8.7, 95% confidence interval [CI] = 7.4 to 10.0), low versus higher visibility of pneumonia (β = -2.2, 95% CI = -2.7 to -1.7), nonspecific lung pathology (β = 0.9, 95% CI = 0.40 to 1.5), localized versus multifocal pneumonia (β = -0.5, 95% CI = -0.8 to -0.1), and one versus two views (β = 0.9, 95% CI = 0.01 to 1.9). A review of diagnostic errors identified that bony structures, vessels in the perihilar region, peribronchial thickening, and thymus were often mistaken for pneumonia. Participants with lower experience were less accurate when they reviewed one of two available views (p < 0.0001), and accuracy of those with higher experience increased with increased confidence in their response (p < 0.0001). CONCLUSIONS: Using learning analytics, we identified actionable learning opportunities for pCXR interpretation, which can be used to allow for a customized weighting of which cases to practice. Furthermore, experienced-novice comparisons revealed image review processes that were associated with greater diagnostic accuracy, providing additional insight into skill development of image interpretation.
Authors: Kathy Boutis; Stefan Cano; Martin Pecaric; T Bram Welch-Horan; Brooke Lampl; Carrie Ruzal-Shapiro; Martin Pusic Journal: Can Med Educ J Date: 2016-03-31