OBJECTIVE: The purpose of this study was to show the value of automated radiology report comparison and analysis in resident education by providing qualitative and quantitative feedback on the discrepancies between preliminary and finalized reports. MATERIALS AND METHODS: Anonymous surveys on dictation practices and the process of reviewing reports were completed by consenting radiology residents and faculty. All 277 reports obtained across all modalities during the 4-week study were retrieved from the dictation server in both their preliminary and finalized states, for a total of 544 reports. Disparities between these reports were automatically compared side by side and were categorized according to clinical relevance, report quality, or report structure. The frequency of report corrections was compared between junior (postgraduate years [PGYs] 2 and 3) and senior (PGYs 4 and 5) residents. Residents were surveyed regarding the usefulness of the feedback. RESULTS: Eighty-six reports (31%) were verified as unchanged, with no statistically significant difference noted between junior and senior residents (33.2% and 25.9%, respectively; p = 0.03). Of the 370 discrepancies noted in the 191 edited reports, 81 (21.9%) were discrepancies in clinically relevant findings; 106 (28.6%) were discrepancies in report quality; and 183 (49.5%) were discrepancies in report structure, syntax, or both. Although senior residents had a lower rate of discrepancies in the clinical relevance category than did junior residents (12.8% and 26.5%; p = 0.004), they had a higher rate of discrepancies in the report quality category (58.4% and 44.9%; p = 0.02). Surveys of both residents and faculty showed strong support for the project. CONCLUSION: Categorization of corrections was deemed useful by residents and can be helpful in assessing elements of reporting accuracy for individual feedback. Quantitative report comparison and analysis show promise in tailoring resident education at the programmatic level as cumulative data are gathered and trends are analyzed.
OBJECTIVE: The purpose of this study was to show the value of automated radiology report comparison and analysis in resident education by providing qualitative and quantitative feedback on the discrepancies between preliminary and finalized reports. MATERIALS AND METHODS: Anonymous surveys on dictation practices and the process of reviewing reports were completed by consenting radiology residents and faculty. All 277 reports obtained across all modalities during the 4-week study were retrieved from the dictation server in both their preliminary and finalized states, for a total of 544 reports. Disparities between these reports were automatically compared side by side and were categorized according to clinical relevance, report quality, or report structure. The frequency of report corrections was compared between junior (postgraduate years [PGYs] 2 and 3) and senior (PGYs 4 and 5) residents. Residents were surveyed regarding the usefulness of the feedback. RESULTS: Eighty-six reports (31%) were verified as unchanged, with no statistically significant difference noted between junior and senior residents (33.2% and 25.9%, respectively; p = 0.03). Of the 370 discrepancies noted in the 191 edited reports, 81 (21.9%) were discrepancies in clinically relevant findings; 106 (28.6%) were discrepancies in report quality; and 183 (49.5%) were discrepancies in report structure, syntax, or both. Although senior residents had a lower rate of discrepancies in the clinical relevance category than did junior residents (12.8% and 26.5%; p = 0.004), they had a higher rate of discrepancies in the report quality category (58.4% and 44.9%; p = 0.02). Surveys of both residents and faculty showed strong support for the project. CONCLUSION: Categorization of corrections was deemed useful by residents and can be helpful in assessing elements of reporting accuracy for individual feedback. Quantitative report comparison and analysis show promise in tailoring resident education at the programmatic level as cumulative data are gathered and trends are analyzed.