Jaehee Chun1,2, Hao Zhang1, H Michael Gach1,3, Sven Olberg1,4, Thomas Mazur1, Olga Green1, Taeho Kim1, Hyun Kim1, Jin Sung Kim2, Sasa Mutic1, Justin C Park1,4. 1. Department of Radiation Oncology, Washington University in St. Louis, St Louis, MO, 63110, USA. 2. Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, South Korea. 3. Departments of Radiology and Biomedical Engineering, Washington University in St. Louis, St Louis, MO, 63110, USA. 4. Department of Biomedical Engineering, Washington University in St. Louis, St Louis, MO, 63110, USA.
Abstract
PURPOSE: Deep learning (DL)-based super-resolution (SR) reconstruction for magnetic resonance imaging (MRI) has recently been receiving attention due to the significant improvement in spatial resolution compared to conventional SR techniques. Challenges hindering the widespread implementation of these approaches remain, however. Low-resolution (LR) MRIs captured in the clinic exhibit complex tissue structures obfuscated by noise that are difficult for a simple DL framework to handle. Moreover, training a robust network for a SR task requires abundant, perfectly matched pairs of LR and high-resolution (HR) images that are often unavailable or difficult to collect. The purpose of this study is to develop a novel SR technique for MRI based on the concept of cascaded DL that allows for the reconstruction of high-quality SR images in the presence of insufficient training data, an unknown translation model, and noise. METHODS: The proposed framework, based on the concept named cascaded deep learning, consists of three components: (a) a denoising autoencoder (DAE) trained using clinical LR noisy MRI scans that have been processed with a nonlocal means filter that generates denoised LR data; (b) a down-sampling network (DSN) trained with a small amount of paired LR/HR data from volunteers that allows for the generation of perfectly paired LR/HR data for the training of a generative model; and (c) the proposed SR generative model (p-SRG) trained with data generated by the DSN that maps from LR inputs to HR outputs. After training, LR clinical images may be fed through the DAE and p-SRG to yield SR reconstructions of the LR input. The application of this framework was explored in two settings: 3D breath-hold MRI axial SR reconstruction from LR axial scans (<3 sec/vol) and in the enhancement of the spatial resolution of LR 4D-MRI acquisitions (0.5 sec/vol). RESULTS: The DSN produces LR scans from HR inputs with a higher fidelity to true, LR clinical scans compared to conventional k-space down-sampling methods based on the metrics of root mean square error (RMSE) and structural similarity index (SSIM). Furthermore, HR outputs generated by the p-SRG exhibit improved scores in the peak signal-to-noise ratio, normalized RMSE, SSIM, and in the blind/reference-less image spatial quality evaluator assessment compared to conventional approaches to MRI SR. CONCLUSIONS: The robust, SR reconstruction method for MRI based on the novel cascaded deep learning framework is an end-to-end method for producing detail-preserving SR reconstructions from noisy, LR clinical MRI scans. Fourfold enhancements in spatial resolution facilitate target delineation and motion management during radiation therapy, enabling precise MRI-guided radiation therapy with 3D LR breath-hold MRI and 4D-MRI in a clinically feasible time frame.
PURPOSE: Deep learning (DL)-based super-resolution (SR) reconstruction for magnetic resonance imaging (MRI) has recently been receiving attention due to the significant improvement in spatial resolution compared to conventional SR techniques. Challenges hindering the widespread implementation of these approaches remain, however. Low-resolution (LR) MRIs captured in the clinic exhibit complex tissue structures obfuscated by noise that are difficult for a simple DL framework to handle. Moreover, training a robust network for a SR task requires abundant, perfectly matched pairs of LR and high-resolution (HR) images that are often unavailable or difficult to collect. The purpose of this study is to develop a novel SR technique for MRI based on the concept of cascaded DL that allows for the reconstruction of high-quality SR images in the presence of insufficient training data, an unknown translation model, and noise. METHODS: The proposed framework, based on the concept named cascaded deep learning, consists of three components: (a) a denoising autoencoder (DAE) trained using clinical LR noisy MRI scans that have been processed with a nonlocal means filter that generates denoised LR data; (b) a down-sampling network (DSN) trained with a small amount of paired LR/HR data from volunteers that allows for the generation of perfectly paired LR/HR data for the training of a generative model; and (c) the proposed SR generative model (p-SRG) trained with data generated by the DSN that maps from LR inputs to HR outputs. After training, LR clinical images may be fed through the DAE and p-SRG to yield SR reconstructions of the LR input. The application of this framework was explored in two settings: 3D breath-hold MRI axial SR reconstruction from LR axial scans (<3 sec/vol) and in the enhancement of the spatial resolution of LR 4D-MRI acquisitions (0.5 sec/vol). RESULTS: The DSN produces LR scans from HR inputs with a higher fidelity to true, LR clinical scans compared to conventional k-space down-sampling methods based on the metrics of root mean square error (RMSE) and structural similarity index (SSIM). Furthermore, HR outputs generated by the p-SRG exhibit improved scores in the peak signal-to-noise ratio, normalized RMSE, SSIM, and in the blind/reference-less image spatial quality evaluator assessment compared to conventional approaches to MRI SR. CONCLUSIONS: The robust, SR reconstruction method for MRI based on the novel cascaded deep learning framework is an end-to-end method for producing detail-preserving SR reconstructions from noisy, LR clinical MRI scans. Fourfold enhancements in spatial resolution facilitate target delineation and motion management during radiation therapy, enabling precise MRI-guided radiation therapy with 3D LR breath-hold MRI and 4D-MRI in a clinically feasible time frame.
Authors: Bryan M Li; Leonardo V Castorina; Maria Del C Valdés Hernández; Una Clancy; Stewart J Wiseman; Eleni Sakka; Amos J Storkey; Daniela Jaime Garcia; Yajun Cheng; Fergus Doubal; Michael T Thrippleton; Michael Stringer; Joanna M Wardlaw Journal: Front Comput Neurosci Date: 2022-08-25 Impact factor: 3.387