Haksoo Kim1, James I Monroe2, Simon Lo3, Min Yao3, Paul M Harari4, Mitchell Machtay3, Jason W Sohn3. 1. Proton Therapy Center, National Cancer Center, 323 Ilsan-ro, Ilsandong-gu, Goyang-si, Gyeonggi-do 410-769, South Korea. 2. Department of Radiation Oncology, School of Medicine, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, Ohio 44106 and St. Anthony's Medical Center, 10010 Kennerly Road, St. Louis, Missouri 63128. 3. Department of Radiation Oncology, School of Medicine, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, Ohio 44106 and University Hospitals of Cleveland, 11100 Euclid Avenue, Cleveland, Ohio 44106. 4. Department of Human Oncology, University of Wisconsin School of Medicine and Public Health, 600 Highland Avenue K4/336, Madison, Wisconsin 53792.
Abstract
PURPOSE: A quantitative and objective metric, the medical similarity index (MSI), has been developed for evaluating the accuracy of a medical image segmentation relative to a reference segmentation. The MSI uses the medical consideration function (MCF) as its basis. METHODS: Currently, no indices provide quantitative evaluations of segmentation accuracy with medical considerations. Variations in segmentation can occur due to individual skill levels and medical relevance--curable or palliative intent, boundary uncertainty due to volume averaging, contrast levels, spatial resolution, and unresolved motion all affect the accuracy of a patient segmentation. Current accuracy measuring indices are not medically relevant. For example, undercontouring the tumor volume is not differentiated from overcontouring tumor. Dice similarity coefficient (DSC) and Hausdorff distance (HD) are two similarity measures often used. However, these metrics consider only geometric difference without considering medical implications. Two segments (under- vs overcontouring tumor) with similar DSC and HD measures could produce significantly different medical treatment results. The authors are proposing a MSI involving a user-defined MCF derived from an asymmetric Gaussian function. The shape of the MCF can be determined by a user, reflecting the anatomical location and characteristics of a particular tissue, organ, or tumor type. The peak of MCF is set along the reference contour; the inner and outer slopes are selected by the user. The discrepancy between the test and reference contours is calculated at each pixel by using a bidirectional local distance measure. The MCF value corresponding to that distance is summed and averaged to produce the MSI. Synthetic segmentations and clinical data from a 15 multi-institutional trial for a head-and-neck case are scored and compared by using MSI, DSC, and Hausdorff distance. RESULTS: The MSI was shown to reflect medical considerations through the choice of MCF penalties for under- and overcontouring. Existing similarity scores were either insensitive to medical realities or simply inaccurate. CONCLUSIONS: The medical similarity index, a segmentation evaluation metric based on medical considerations, has been proposed, developed, and tested to incorporate clinically relevant considerations beyond geometric parameters alone.
PURPOSE: A quantitative and objective metric, the medical similarity index (MSI), has been developed for evaluating the accuracy of a medical image segmentation relative to a reference segmentation. The MSI uses the medical consideration function (MCF) as its basis. METHODS: Currently, no indices provide quantitative evaluations of segmentation accuracy with medical considerations. Variations in segmentation can occur due to individual skill levels and medical relevance--curable or palliative intent, boundary uncertainty due to volume averaging, contrast levels, spatial resolution, and unresolved motion all affect the accuracy of a patient segmentation. Current accuracy measuring indices are not medically relevant. For example, undercontouring the tumor volume is not differentiated from overcontouring tumor. Dice similarity coefficient (DSC) and Hausdorff distance (HD) are two similarity measures often used. However, these metrics consider only geometric difference without considering medical implications. Two segments (under- vs overcontouring tumor) with similar DSC and HD measures could produce significantly different medical treatment results. The authors are proposing a MSI involving a user-defined MCF derived from an asymmetric Gaussian function. The shape of the MCF can be determined by a user, reflecting the anatomical location and characteristics of a particular tissue, organ, or tumor type. The peak of MCF is set along the reference contour; the inner and outer slopes are selected by the user. The discrepancy between the test and reference contours is calculated at each pixel by using a bidirectional local distance measure. The MCF value corresponding to that distance is summed and averaged to produce the MSI. Synthetic segmentations and clinical data from a 15 multi-institutional trial for a head-and-neck case are scored and compared by using MSI, DSC, and Hausdorff distance. RESULTS: The MSI was shown to reflect medical considerations through the choice of MCF penalties for under- and overcontouring. Existing similarity scores were either insensitive to medical realities or simply inaccurate. CONCLUSIONS: The medical similarity index, a segmentation evaluation metric based on medical considerations, has been proposed, developed, and tested to incorporate clinically relevant considerations beyond geometric parameters alone.
Authors: Mathieu Hatt; John A Lee; Charles R Schmidtlein; Issam El Naqa; Curtis Caldwell; Elisabetta De Bernardi; Wei Lu; Shiva Das; Xavier Geets; Vincent Gregoire; Robert Jeraj; Michael P MacManus; Osama R Mawlawi; Ursula Nestle; Andrei B Pugachev; Heiko Schöder; Tony Shepherd; Emiliano Spezi; Dimitris Visvikis; Habib Zaidi; Assen S Kirov Journal: Med Phys Date: 2017-05-18 Impact factor: 4.071