Shahein H Tajmir1,2, Hyunkwang Lee3,4, Randheer Shailam3,5, Heather I Gale6, Jie C Nguyen7, Sjirk J Westra3,5, Ruth Lim3,5, Sehyo Yune3,5, Michael S Gee3,5, Synho Do3,5. 1. Department of Radiology, Massachusetts General Hospital, Boston, MA, USA. shahein@stajmir.com. 2. Harvard Medical School, Boston, MA, USA. shahein@stajmir.com. 3. Department of Radiology, Massachusetts General Hospital, Boston, MA, USA. 4. Harvard John A. Paulson School of Engineering and Applied Sciences, Cambridge, MA, USA. 5. Harvard Medical School, Boston, MA, USA. 6. The Billings Clinic, Billings, MT, USA. 7. Children's Hospital of Philadelphia, Philadelphia, PA, USA.
Abstract
OBJECTIVE: Radiographic bone age assessment (BAA) is used in the evaluation of pediatric endocrine and metabolic disorders. We previously developed an automated artificial intelligence (AI) deep learning algorithm to perform BAA using convolutional neural networks. We compared the BAA performance of a cohort of pediatric radiologists with and without AI assistance. MATERIALS AND METHODS: Six board-certified, subspecialty trained pediatric radiologists interpreted 280 age- and gender-matched bone age radiographs ranging from 5 to 18 years. Three of those radiologists then performed BAA with AI assistance. Bone age accuracy and root mean squared error (RMSE) were used as measures of accuracy. Intraclass correlation coefficient evaluated inter-rater variation. RESULTS: AI BAA accuracy was 68.2% overall and 98.6% within 1 year, and the mean six-reader cohort accuracy was 63.6 and 97.4% within 1 year. AI RMSE was 0.601 years, while mean single-reader RMSE was 0.661 years. Pooled RMSE decreased from 0.661 to 0.508 years, all individually decreasing with AI assistance. ICC without AI was 0.9914 and with AI was 0.9951. CONCLUSIONS: AI improves radiologist's bone age assessment by increasing accuracy and decreasing variability and RMSE. The utilization of AI by radiologists improves performance compared to AI alone, a radiologist alone, or a pooled cohort of experts. This suggests that AI may optimally be utilized as an adjunct to radiologist interpretation of imaging studies to improve performance.
OBJECTIVE: Radiographic bone age assessment (BAA) is used in the evaluation of pediatric endocrine and metabolic disorders. We previously developed an automated artificial intelligence (AI) deep learning algorithm to perform BAA using convolutional neural networks. We compared the BAA performance of a cohort of pediatric radiologists with and without AI assistance. MATERIALS AND METHODS: Six board-certified, subspecialty trained pediatric radiologists interpreted 280 age- and gender-matched bone age radiographs ranging from 5 to 18 years. Three of those radiologists then performed BAA with AI assistance. Bone age accuracy and root mean squared error (RMSE) were used as measures of accuracy. Intraclass correlation coefficient evaluated inter-rater variation. RESULTS: AI BAA accuracy was 68.2% overall and 98.6% within 1 year, and the mean six-reader cohort accuracy was 63.6 and 97.4% within 1 year. AI RMSE was 0.601 years, while mean single-reader RMSE was 0.661 years. Pooled RMSE decreased from 0.661 to 0.508 years, all individually decreasing with AI assistance. ICC without AI was 0.9914 and with AI was 0.9951. CONCLUSIONS: AI improves radiologist's bone age assessment by increasing accuracy and decreasing variability and RMSE. The utilization of AI by radiologists improves performance compared to AI alone, a radiologist alone, or a pooled cohort of experts. This suggests that AI may optimally be utilized as an adjunct to radiologist interpretation of imaging studies to improve performance.
Entities:
Keywords:
Augmented intelligence; Bone age; Machine learning; Pediatric; Radiographs
Authors: Lizhe Xie; Wen Tang; Iman Izadikhah; Zhenqi Zhao; Yang Zhao; Hu Li; Bin Yan Journal: Int J Comput Assist Radiol Surg Date: 2022-01-04 Impact factor: 2.924
Authors: Ian Pan; Grayson L Baird; Simukayi Mutasa; Derek Merck; Carrie Ruzal-Shapiro; David W Swenson; Rama S Ayyala Journal: Radiol Artif Intell Date: 2020-07-29
Authors: Safwan S Halabi; Luciano M Prevedello; Jayashree Kalpathy-Cramer; Artem B Mamonov; Alexander Bilbily; Mark Cicero; Ian Pan; Lucas Araújo Pereira; Rafael Teixeira Sousa; Nitamar Abdala; Felipe Campos Kitamura; Hans H Thodberg; Leon Chen; George Shih; Katherine Andriole; Marc D Kohli; Bradley J Erickson; Adam E Flanders Journal: Radiology Date: 2018-11-27 Impact factor: 29.146