Yang Chen1, Xiaofeng Ren, Guo-Qiang Zhang, Rong Xu. 1. Department of Electrical Engineering and Computer Science, Case Western Reserve University School of Engineering, Cleveland, Ohio, USA.
Abstract
BACKGROUND: Visual information is a crucial aspect of medical knowledge. Building a comprehensive medical image base, in the spirit of the Unified Medical Language System (UMLS), would greatly benefit patient education and self-care. However, collection and annotation of such a large-scale image base is challenging. OBJECTIVE: To combine visual object detection techniques with medical ontology to automatically mine web photos and retrieve a large number of disease manifestation images with minimal manual labeling effort. METHODS: As a proof of concept, we first learnt five organ detectors on three detection scales for eyes, ears, lips, hands, and feet. Given a disease, we used information from the UMLS to select affected body parts, ran the pretrained organ detectors on web images, and combined the detection outputs to retrieve disease images. RESULTS: Compared with a supervised image retrieval approach that requires training images for every disease, our ontology-guided approach exploits shared visual information of body parts across diseases. In retrieving 2220 web images of 32 diseases, we reduced manual labeling effort to 15.6% while improving the average precision by 3.9% from 77.7% to 81.6%. For 40.6% of the diseases, we improved the precision by 10%. CONCLUSIONS: The results confirm the concept that the web is a feasible source for automatic disease image retrieval for health image database construction. Our approach requires a small amount of manual effort to collect complex disease images, and to annotate them by standard medical ontology terms.
BACKGROUND: Visual information is a crucial aspect of medical knowledge. Building a comprehensive medical image base, in the spirit of the Unified Medical Language System (UMLS), would greatly benefit patient education and self-care. However, collection and annotation of such a large-scale image base is challenging. OBJECTIVE: To combine visual object detection techniques with medical ontology to automatically mine web photos and retrieve a large number of disease manifestation images with minimal manual labeling effort. METHODS: As a proof of concept, we first learnt five organ detectors on three detection scales for eyes, ears, lips, hands, and feet. Given a disease, we used information from the UMLS to select affected body parts, ran the pretrained organ detectors on web images, and combined the detection outputs to retrieve disease images. RESULTS: Compared with a supervised image retrieval approach that requires training images for every disease, our ontology-guided approach exploits shared visual information of body parts across diseases. In retrieving 2220 web images of 32 diseases, we reduced manual labeling effort to 15.6% while improving the average precision by 3.9% from 77.7% to 81.6%. For 40.6% of the diseases, we improved the precision by 10%. CONCLUSIONS: The results confirm the concept that the web is a feasible source for automatic disease image retrieval for health image database construction. Our approach requires a small amount of manual effort to collect complex disease images, and to annotate them by standard medical ontology terms.
Authors: Samuel G Armato; Geoffrey McLennan; Michael F McNitt-Gray; Charles R Meyer; David Yankelevitz; Denise R Aberle; Claudia I Henschke; Eric A Hoffman; Ella A Kazerooni; Heber MacMahon; Anthony P Reeves; Barbara Y Croft; Laurence P Clarke Journal: Radiology Date: 2004-09 Impact factor: 11.105
Authors: Pedro F Felzenszwalb; Ross B Girshick; David McAllester; Deva Ramanan Journal: IEEE Trans Pattern Anal Mach Intell Date: 2010-09 Impact factor: 6.226
Authors: David B Keator; J S Grethe; D Marcus; B Ozyurt; S Gadde; Sean Murphy; S Pieper; D Greve; R Notestine; H J Bockholt; P Papadopoulos Journal: IEEE Trans Inf Technol Biomed Date: 2008-03
Authors: Matthew S Simpson; Daekeun You; Md Mahmudur Rahman; Sameer K Antani; George R Thoma; Dina Demner-Fushman Journal: AMIA Annu Symp Proc Date: 2012-11-03