Zhuangzhuang Zhang1, Tianyu Zhao2, Hiram Gay2, Weixiong Zhang3, Baozhou Sun2. 1. Department of Computer Science and Engineering, Washington University, One Brookings Drive, Campus Box 1045, St. Louis, MO, 63130, USA. 2. Department of Radiation Oncology, Washington University School of Medicine, 4921 Parkview Place, Campus Box 8224, St. Louis, MO, 63110, USA. 3. Department of Computer Science and Engineering, Department of Genetics, Washington University, One Brookings Drive, Campus Box 1045, St. Louis, MO, 63130, USA.
Abstract
PURPOSE: The research is to develop a novel CNN-based adversarial deep learning method to improve and expedite the multi-organ semantic segmentation of CT images and to generate accurate contours on pelvic CT images. METHODS: Planning CT and structure datasets for 120 patients with intact prostate cancer were retrospectively selected and divided for tenfold cross-validation. The proposed adversarial multi-residual multi-scale pooling Markov random field (MRF) enhanced network (ARPM-net) implements an adversarial training scheme. A segmentation network and a discriminator network were trained jointly, and only the segmentation network was used for prediction. The segmentation network integrates a newly designed MRF block into a variation of multi-residual U-net. The discriminator takes the product of the original CT and the prediction/ground-truth as input and classifies the input into fake/real. The segmentation network and discriminator network can be trained jointly as a whole, or the discriminator can be used for fine-tuning after the segmentation network is coarsely trained. Multi-scale pooling layers were introduced to preserve spatial resolution during pooling using less memory compared to atrous convolution layers. An adaptive loss function was proposed to enhance the training on small or low contrast organs. The accuracy of modeled contours was measured with the dice similarity coefficient (DSC), average Hausdorff distance (AHD), average surface Hausdorff distance (ASHD), and relative volume difference (VD) using clinical contours as references to the ground-truth. The proposed ARPM-net method was compared to several state-of-the-art deep learning methods. RESULTS: ARPM-net outperformed several existing deep learning approaches and MRF methods and achieved state-of-the-art performance on a testing dataset. On the test set with 20 cases, the average DSC on the prostate, bladder, rectum, left femur, and right femur were 0.88 ( ± 0.11), 0.97 ( ± 0.07), 0.86 ( ± 0.12), 0.97 ( ± 0.01), and 0.97 ( ± 0.01), respectively. The average HD (mm) on these organs were 1.58 ( ± 1.77), 1.91 ( ± 1.29), 3.14 ( ± 2.39), 1.76 ( ± 1.57), and 1.92 ( ± 1.01). The average surface HD (mm) on these organs are 2.11 ( ± 2.03), 2.36 ( ± 2.43), 3.05 ( ± 2.11), 1.99 ( ± 1.66), and 2.00 ( ± 2.07). CONCLUSION: ARPM-net was designed for the automatic segmentation of pelvic CT images. With adversarial fine-tuning, ARPM-net produces state-of-the-art accurate contouring of multiple organs on CT images and has the potential to facilitate routine pelvic cancer radiation therapy planning process.
PURPOSE: The research is to develop a novel CNN-based adversarial deep learning method to improve and expedite the multi-organ semantic segmentation of CT images and to generate accurate contours on pelvic CT images. METHODS: Planning CT and structure datasets for 120 patients with intact prostate cancer were retrospectively selected and divided for tenfold cross-validation. The proposed adversarial multi-residual multi-scale pooling Markov random field (MRF) enhanced network (ARPM-net) implements an adversarial training scheme. A segmentation network and a discriminator network were trained jointly, and only the segmentation network was used for prediction. The segmentation network integrates a newly designed MRF block into a variation of multi-residual U-net. The discriminator takes the product of the original CT and the prediction/ground-truth as input and classifies the input into fake/real. The segmentation network and discriminator network can be trained jointly as a whole, or the discriminator can be used for fine-tuning after the segmentation network is coarsely trained. Multi-scale pooling layers were introduced to preserve spatial resolution during pooling using less memory compared to atrous convolution layers. An adaptive loss function was proposed to enhance the training on small or low contrast organs. The accuracy of modeled contours was measured with the dice similarity coefficient (DSC), average Hausdorff distance (AHD), average surface Hausdorff distance (ASHD), and relative volume difference (VD) using clinical contours as references to the ground-truth. The proposed ARPM-net method was compared to several state-of-the-art deep learning methods. RESULTS: ARPM-net outperformed several existing deep learning approaches and MRF methods and achieved state-of-the-art performance on a testing dataset. On the test set with 20 cases, the average DSC on the prostate, bladder, rectum, left femur, and right femur were 0.88 ( ± 0.11), 0.97 ( ± 0.07), 0.86 ( ± 0.12), 0.97 ( ± 0.01), and 0.97 ( ± 0.01), respectively. The average HD (mm) on these organs were 1.58 ( ± 1.77), 1.91 ( ± 1.29), 3.14 ( ± 2.39), 1.76 ( ± 1.57), and 1.92 ( ± 1.01). The average surface HD (mm) on these organs are 2.11 ( ± 2.03), 2.36 ( ± 2.43), 3.05 ( ± 2.11), 1.99 ( ± 1.66), and 2.00 ( ± 2.07). CONCLUSION: ARPM-net was designed for the automatic segmentation of pelvic CT images. With adversarial fine-tuning, ARPM-net produces state-of-the-art accurate contouring of multiple organs on CT images and has the potential to facilitate routine pelvic cancer radiation therapy planning process.
Authors: Neeraja Konuthula; Francisco A Perez; A Murat Maga; Waleed M Abuzeid; Kris Moe; Blake Hannaford; Randall A Bly Journal: Int J Comput Assist Radiol Surg Date: 2021-05-19 Impact factor: 3.421