David W Swenson1, Grayson L Baird2, David C Portelli3, Martha B Mainiero4, Jonathan S Movson4. 1. Department of Diagnostic Imaging, Rhode Island Hospital/Hasbro Children's Hospital, Warren Alpert Medical School of Brown University, 593 Eddy St, Providence, RI, 02903, USA. swenson.david.w@gmail.com. 2. Department of Diagnostic Imaging and Lifespan Biostatistics Core, Rhode Island Hospital, 593 Eddy St, Providence, RI, 02903, USA. 3. Department of Emergency Medicine, Rhode Island Hospital, Warren Alpert Medical School of Brown University, 593 Eddy St, Providence, RI, 02903, USA. 4. Department of Diagnostic Imaging, Rhode Island Hospital/Hasbro Children's Hospital, Warren Alpert Medical School of Brown University, 593 Eddy St, Providence, RI, 02903, USA.
Abstract
PURPOSE: The purpose of this study was to describe a new, broadly applicable radiology report categorization (RADCAT) system that was developed collaboratively between radiologists and emergency department (ED) physicians, and to establish its usability and performance by interobserver variation. METHODS: In collaboration with our ED colleagues, we developed the RADCAT system for all imaging studies performed in our level-1 trauma center, including five categories that span the spectrum of normal through emergent life-threatening findings. During a pilot phase, four radiologists used the system real-time to categorize a minimum of 400 reports in the ED. From this pool of categorized studies, 58 reports were then selected semi-randomly, de-identified, stripped of their original categorization, and recategorized based on the narrative radiology report by 12 individual reviewers (6 radiologists, and 6 ED physicians). Interobserver variation between all reviewers, radiologists only, and ED physicians only was calculated using Cohen's Kappa statistic and Kendall's coefficient of concordance. RESULTS: Altogether, agreement among radiologists and ED physicians was substantial (κ = 0.73, p < 0.0001) and agreement for each category was substantial (all κ > 0.60, p < 0.0001). The lowest agreement was observed with RADCAT-3 (κ > 0.61, p < 0.0001) and the highest agreement with RADCAT-1 (κ > 0.85, p < 0.0001). A high trend in agreement was observed for radiologists and ED physicians and their combination (all W > 0.90, p < 0.0001). CONCLUSIONS: Our RADCAT system is understandable between radiologists and ED physicians for categorizing a wide range of imaging studies, and warrants further assessment and validation. Based upon these pilot results, we plan to adopt this RADCAT scheme and further assess its performance.
PURPOSE: The purpose of this study was to describe a new, broadly applicable radiology report categorization (RADCAT) system that was developed collaboratively between radiologists and emergency department (ED) physicians, and to establish its usability and performance by interobserver variation. METHODS: In collaboration with our ED colleagues, we developed the RADCAT system for all imaging studies performed in our level-1 trauma center, including five categories that span the spectrum of normal through emergent life-threatening findings. During a pilot phase, four radiologists used the system real-time to categorize a minimum of 400 reports in the ED. From this pool of categorized studies, 58 reports were then selected semi-randomly, de-identified, stripped of their original categorization, and recategorized based on the narrative radiology report by 12 individual reviewers (6 radiologists, and 6 ED physicians). Interobserver variation between all reviewers, radiologists only, and ED physicians only was calculated using Cohen's Kappa statistic and Kendall's coefficient of concordance. RESULTS: Altogether, agreement among radiologists and ED physicians was substantial (κ = 0.73, p < 0.0001) and agreement for each category was substantial (all κ > 0.60, p < 0.0001). The lowest agreement was observed with RADCAT-3 (κ > 0.61, p < 0.0001) and the highest agreement with RADCAT-1 (κ > 0.85, p < 0.0001). A high trend in agreement was observed for radiologists and ED physicians and their combination (all W > 0.90, p < 0.0001). CONCLUSIONS: Our RADCAT system is understandable between radiologists and ED physicians for categorizing a wide range of imaging studies, and warrants further assessment and validation. Based upon these pilot results, we plan to adopt this RADCAT scheme and further assess its performance.
Authors: Elizabeth Lazarus; Martha B Mainiero; Barbara Schepps; Susan L Koelliker; Linda S Livingston Journal: Radiology Date: 2006-03-28 Impact factor: 11.105
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Shawn G Anthony; Luciano M Prevedello; Maria M Damiano; Tejal K Gandhi; Peter M Doubilet; Steven E Seltzer; Ramin Khorasani Journal: Radiology Date: 2011-04-05 Impact factor: 11.105
Authors: Matthew S Davenport; Shokoufeh Khalatbari; Peter S C Liu; Katherine E Maturen; Ravi K Kaza; Ashish P Wasnik; Mahmoud M Al-Hawary; Daniel I Glazer; Erica B Stein; Jeet Patel; Deepak K Somashekar; Benjamin L Viglianti; Hero K Hussain Journal: Radiology Date: 2014-02-18 Impact factor: 11.105
Authors: Nicholas Petruzzi; Donald Mitchell; Flavius Guglielmo; Patrick O'Kane; Sandeep Deshmukh; Christopher Roth; Conor Shortt; Stephen Balfour; Laurence Parker Journal: Acad Radiol Date: 2013-03-28 Impact factor: 3.173
Authors: Berrend G Muller; Joanna H Shih; Sandeep Sankineni; Jamie Marko; Soroush Rais-Bahrami; Arvin Koruthu George; Jean J M C H de la Rosette; Maria J Merino; Bradford J Wood; Peter Pinto; Peter L Choyke; Baris Turkbey Journal: Radiology Date: 2015-06-18 Impact factor: 11.105
Authors: Brady J McKee; Shawn M Regis; Andrea B McKee; Sebastian Flacke; Christoph Wald Journal: J Am Coll Radiol Date: 2014-08-28 Impact factor: 5.532