PURPOSE: The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI-ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies. METHODS: We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa. RESULTS: Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions. CONCLUSION: We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.
PURPOSE: The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI-ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies. METHODS: We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa. RESULTS: Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions. CONCLUSION: We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.
Entities:
Keywords:
Deep learning; Image-guided biopsy; Information fusion; Magnetic resonance imaging; Multimodality training; Prostate cancer detection; Temporal enhanced ultrasound
Authors: Farhad Imani; Mahdi Ramezani; Saman Nouranian; Eli Gibson; Amir Khojaste; Mena Gaed; Madeleine Moussa; Jose A Gomez; Cesare Romagnoli; Michael Leveridge; Silvia Chang; Aaron Fenster; D Robert Siemens; Aaron D Ward; Parvin Mousavi; Purang Abolmaesumi Journal: IEEE Trans Biomed Eng Date: 2015-02-16 Impact factor: 4.538
Authors: Shekoofeh Azizi; Sharareh Bayat; Pingkun Yan; Amir Tahmasebi; Guy Nir; Jin Tae Kwak; Sheng Xu; Storey Wilson; Kenneth A Iczkowski; M Scott Lucia; Larry Goldenberg; Septimiu E Salcudean; Peter A Pinto; Bradford Wood; Purang Abolmaesumi; Parvin Mousavi Journal: Int J Comput Assist Radiol Surg Date: 2017-06-20 Impact factor: 2.924
Authors: Mehdi Moradi; Purang Abolmaesumi; D Robert Siemens; Eric E Sauerbrei; Alexander H Boag; Parvin Mousavi Journal: IEEE Trans Biomed Eng Date: 2008-12-02 Impact factor: 4.538
Authors: Alireza Mehrtash; Alireza Sedghi; Mohsen Ghafoorian; Mehdi Taghipour; Clare M Tempany; William M Wells; Tina Kapur; Parvin Mousavi; Purang Abolmaesumi; Andriy Fedorov Journal: Proc SPIE Int Soc Opt Eng Date: 2017-03-03
Authors: Ozan Oktay; Enzo Ferrante; Konstantinos Kamnitsas; Mattias Heinrich; Wenjia Bai; Jose Caballero; Stuart A Cook; Antonio de Marvao; Timothy Dawes; Declan P O'Regan; Bernhard Kainz; Ben Glocker; Daniel Rueckert Journal: IEEE Trans Med Imaging Date: 2017-09-26 Impact factor: 10.048
Authors: Geoffrey A Sonn; Edward Chang; Shyam Natarajan; Daniel J Margolis; Malu Macairan; Patricia Lieu; Jiaoti Huang; Frederick J Dorey; Robert E Reiter; Leonard S Marks Journal: Eur Urol Date: 2013-03-17 Impact factor: 20.096
Authors: Hashim U Ahmed; Ahmed El-Shater Bosaily; Louise C Brown; Rhian Gabe; Richard Kaplan; Mahesh K Parmar; Yolanda Collaco-Moraes; Katie Ward; Richard G Hindley; Alex Freeman; Alex P Kirkham; Robert Oldroyd; Chris Parker; Mark Emberton Journal: Lancet Date: 2017-01-20 Impact factor: 79.321
Authors: Jo Schlemper; Ozan Oktay; Michiel Schaap; Mattias Heinrich; Bernhard Kainz; Ben Glocker; Daniel Rueckert Journal: Med Image Anal Date: 2019-02-05 Impact factor: 8.545
Authors: Alireza Mehrtash; Tina Kapur; Clare M Tempany; Purang Abolmaesumi; William M Wells Journal: Proc IEEE Int Symp Biomed Imaging Date: 2021-05-25