Changhwan Lee1, Yeesuk Kim2, Young Soo Kim3, Jongseong Jang4. 1. 1 Department of Biomedical Engineering, Hanyang University, Seoul, Korea. 2. 2 Department of Orthopedic Surgery, College of Medicine, Hanyang University, Seoul, Korea. 3. 3 Institute of Innovative Surgical Technology, Hanyang University, Seoul, Korea. 4. 4 Center for Integration of Advanced Medicine Life Science Innovative Technologies, Department of Advanced Medical Initiatives, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka-shi, Fukuoka-ken 812-8582, Japan.
Abstract
OBJECTIVE: Radiology reports are rich resources for biomedical researchers. Before utilization of radiology reports, experts must manually review these reports to identify the categories. In fact, automatically categorizing electronic medical record (EMR) text with key annotation is difficult because it has a free-text format. To address these problems, we developed an automated system for disease annotation. MATERIALS AND METHODS: Reports of musculoskeletal radiography examinations performed from January 1, 2016, through December 31, 2016, were exported from the database of Hanyang University Medical Center. After sentences not written in English and sentences containing typos were excluded, 3032 sentences were included. We built a system that uses a recurrent neural network (RNN) to automatically identify fracture and nonfracture cases as a preliminary study. We trained and tested the system using orthopedic surgeon-classified reports. We evaluated the system for the number of layers in the following two ways: the word error rate of the output sentences and performance as a binary classifier using standard evaluation metrics including accuracy, precision, recall, and F1 score. RESULTS: The word error rate using Levenshtein distance showed the best performance in the three-layer model at 1.03%. The three-layer model also showed the highest overall performance with the highest precision (0.967), recall (0.967), accuracy (0.982), and F1 score (0.967). CONCLUSION: Our results indicate that the RNN-based system has the ability to classify important findings in radiology reports with a high F1 score. We expect that our system can be used in cohort construction such as for retrospective studies because it is efficient for analyzing a large amount of data.
OBJECTIVE: Radiology reports are rich resources for biomedical researchers. Before utilization of radiology reports, experts must manually review these reports to identify the categories. In fact, automatically categorizing electronic medical record (EMR) text with key annotation is difficult because it has a free-text format. To address these problems, we developed an automated system for disease annotation. MATERIALS AND METHODS: Reports of musculoskeletal radiography examinations performed from January 1, 2016, through December 31, 2016, were exported from the database of Hanyang University Medical Center. After sentences not written in English and sentences containing typos were excluded, 3032 sentences were included. We built a system that uses a recurrent neural network (RNN) to automatically identify fracture and nonfracture cases as a preliminary study. We trained and tested the system using orthopedic surgeon-classified reports. We evaluated the system for the number of layers in the following two ways: the word error rate of the output sentences and performance as a binary classifier using standard evaluation metrics including accuracy, precision, recall, and F1 score. RESULTS: The word error rate using Levenshtein distance showed the best performance in the three-layer model at 1.03%. The three-layer model also showed the highest overall performance with the highest precision (0.967), recall (0.967), accuracy (0.982), and F1 score (0.967). CONCLUSION: Our results indicate that the RNN-based system has the ability to classify important findings in radiology reports with a high F1 score. We expect that our system can be used in cohort construction such as for retrospective studies because it is efficient for analyzing a large amount of data.
Entities:
Keywords:
automatic annotation; deep learning; natural language processing; radiology reports; recurrent neural network
Authors: Timothy L Chen; Max Emerling; Gunvant R Chaudhari; Yeshwant R Chillakuru; Youngho Seo; Thienkhai H Vu; Jae Ho Sohn Journal: J Biomed Inform Date: 2020-12-15 Impact factor: 6.317
Authors: Arlene Casey; Emma Davidson; Michael Poon; Hang Dong; Daniel Duma; Andreas Grivas; Claire Grover; Víctor Suárez-Paniagua; Richard Tobin; William Whiteley; Honghan Wu; Beatrice Alex Journal: BMC Med Inform Decis Mak Date: 2021-06-03 Impact factor: 2.796
Authors: Joanna F Dipnall; Richard Page; Lan Du; Matthew Costa; Ronan A Lyons; Peter Cameron; Richard de Steiger; Raphael Hau; Andrew Bucknill; Andrew Oppy; Elton Edwards; Dinesh Varma; Myong Chol Jung; Belinda J Gabbe Journal: PLoS One Date: 2021-09-23 Impact factor: 3.240