Obtaining dual-energy computed tomography (CT) information from a single-energy CT image for quantitative imaging analysis of living subjects by using deep learning.

Computed tomographic (CT) is a fundamental imaging modality to generate cross-sectional views of internal anatomy in a living subject or interrogate material composition of an object, and it has been routinely used in clinical applications and nondestructive testing. In a standard CT image, pixels having the same Hounsfield Units (HU) can correspond to different materials, and it is therefore challenging to differentiate and quantify materials. Dual-energy CT (DECT) is desirable to differentiate multiple materials, but the costly DECT scanners are not widely available as single-energy CT (SECT) scanners. Recent advancement in deep learning provides an enabling tool to map images between different modalities with incorporated prior knowledge. Here we develop a deep learning approach to perform DECT imaging by using the standard SECT data. The end point of the approach is a model capable of providing the high-energy CT image for a given input low-energy CT image. The feasibility of the deep learning-based DECT imaging method using a SECT data is demonstrated using contrast-enhanced DECT images and evaluated using clinical relevant indexes. This work opens new opportunities for numerous DECT clinical applications with a standard SECT data and may enable significantly simplified hardware design, scanning dose and image cost reduction for future DECT systems.

Introduction

Pixel number in a standard single-energy computed tomography (SECT) image represents effective linear attenuation coefficient and it is an averaged contribution of all materials or chemical elements in the pixel. It therefore does not give a unique description for any given material, and pixels having the same CT numbers can represent materials with different elemental compositions, making the differentiation and quantification of materials extremely challenging. Dual-energy CT (DECT) scans the object using two different energy spectra and is able to take advantage of the energy dependence of the linear attenuation coefficients to yield material-specific images.[1-20] This enables DECT to be applied in several emerging clinical applications, including virtual monoenergetic imaging, automated bone removal in CT angiography, perfused blood volume, virtual noncontrast-enhanced images, urinary stone characterization and so on.[21]

In practice, leading industrial CT vendors have used different techniques to acquire dual-energy data. For examples, GE Healthcare scanners use rapid switching of x-ray tube potential to acquire alternate projection measurements at low and high-energy spectra.[11,22] This technique requires the transition time from low to high tube potential between consecutive views to be less than a millisecond while separating the spectra as much as possible, which is very technically challenging. Siemens Healthineers scanners use dual x-ray sources and two data acquisition systems, both of which are mounted on the same gantry.[7] Thus, the dual-source scanner cost is much higher than the standard SECT scanner. Philips Healthcare scanner acquires DECT projection data using a layered detector, with which the low-energy data and the high-energy data are collected by the front detector layer and the back-detector layer, respectively.[6,23] All the DECT data acquisition techniques have posed a significant burden on CT system hardware. Hence, DECT scanners are not widely available as SECT scanners, especially for less-developed regions. In addition to the increased complexity of the imaging system and cost, DECT may also increase the radiation dose to patients due to the additional CT scan. In this work, we demonstrate a standard DECT imaging using images acquired by a SECT scanner is feasible by leveraging from the state-of-the-art deep learning technique and seamless integration of prior DECT knowledge in the deep learning model training process. It is intriguing to note that the proposed method enables DECT clinical applications (such as iodine quantification, virtual noncontrast-enhanced imaging) to be performed using a SECT data, which has potential to provide a fundamental paradigm for DECT imaging applications for less developed regions where only SECT scanners are available and high-end DECT scanners are not affordable.

Deep neural network has recently attracted much attention for its unprecedented ability to learn complex relationships and incorporate existing knowledge into the inference model through feature extraction and representation learning.[24] The method has found widespread applications in biomedicine.[25-32] Here we introduce a hierarchical neural network for DECT imaging with a SECE data and demonstrate the superior performance of the deep learning-based DECT method using a popular DECT clinical application. The essences of our approach are the extraction of the intrinsic features between the differences of the high- and low-energy CT images by exclusion of the image noise, and the construction of a robust encoder/decoder network architecture to map the differences. By using the network architecture, we incorporate the intrinsic differences learned from paired DECT images acquired from commercial scanners into a deep learning model for the high-energy CT image prediction in the subsequent new low-energy CT image input.

Methods

In this study, we aim to train a deep learning model to transform low-energy CT image I to high-energy CT image I using routinely available paired DECT images. To this end, we first use a fully convolutional network derived from ResNet to significantly reduce the image noise.[33] This procedure is performed in an ene-to-end fashion to provide low noise DECT images I and I. With these images, instead of directly mapping the high-energy CT image from the low-energy CT image, an independent mapping convolutional neural network (CNN), which we call DECT-CNN, is trained to learn an difference I image between I and I for the given input I. Then a predicted high-energy CT image I is calculated as the summation of the origianl low-energy CT image I and the predicted difference image I during inference procedure. The DECT-CNN model is based on an U-Net-type architecture and the mean-squared-error is used as the loss function. During training, the loss function is minimized using the adaptive moment estimation (ADAM) algorithm. Weights in the convolutional kernels are updated using back-propagation method.

The proposed deep learning-based DECT imaging approach was evaluated using a popular DE application: virtual non-contrast (VNC) imaging and iodine contrast agent quantification. The VNC images and the iodine maps were decomposed using a statistically optimal image-domain material decomposition algorithm.[34] To assess the accuracy of the approach, we retrospectively acquired DECT images from 22 patients who received contrast-enhanced abdomen CT scan. Quantitative comparisons between the original high-energy CT images and the predicted high-energy CT images were performed using clinically relevant HU values for different types of tissues. Material-specific images (VNC images and iodine maps) quantification obtained from the original DECT images and the deep learning-predicted DECT images were also compared and quantitatively evaluated using HU value and noise level in region-of-interests (ROIs).

Results

We found that the mapping CNN yields inferior high-energy CT images with original noisy low-energy CT images as input, indicating the CNN wastes its ability of expression on mapping the image difference between high- and low-energy levels with the presence of noise. Fig. 1 shows an example of the DECT images with and without noise reduction. As can be seen, the denoised images provide significantly improved quality compared to the raw CT images. There are no anatomical structure information in the difference images, suggesting the spatial resolution of the denoised CT images is well preserved after noise reduction. Quantitative assessments of the images with and without noise reduction using ROIs on different types of tissues are shown in Table. 1. The HU accuracy of the images is well preserved after noise reduction. The HU differences between the original DECT images and noise reduced DECT images are smaller than 3 HU, while the standard deviations of the ROIs suggest the noise has significantly reduced.

Fig. 1.

Low- and high-energy CT images with and without noise reduction. The difference images in column three are obtained by subtracting the denoised images from the raw images. All images are displayed in (C=0HU and W=500HU).

Table 1.

Quantitative analysis of the DECT images with and without noise reduction. Region-of-interests (ROIs) assessments on different tissues (arota, liver, spine, and stomach) show the HU accuracy of the CT images is well preserved, while the noise is significantly reduced after noise reduction.

ROIs		HUraw	HUdenoised	ΔHU	stdraw	stddenoised
Arota	100 kV	314.5	312.7	1.8	35.4	6.8
	140 kV	166.7	164.8	1.9	33.0	7.6
Liver	100 kV	71.8	69.5	2.3	31.6	6.8
	140 kV	60.5	63.4	−2.9	24.8	5.4
Spine	100 kV	214.6	214.7	−0.1	39.7	20.7
	140 kV	149.5	149.0	0.5	32.2	13.6
Stomach	100 kV	2.2	2.6	−0.4	36.7	8.1
	140 kV	−1	−1	0	38.8	7.7

With noise significantly reduced DECT images, the CNN can appreciate the ingenuous CT image difference at different energy levels and ultimately yield superior high-energy CT image. Fig. 2 shows original DECT images, deep learning predicted high-energy CT images as well as the differences images between the predicted images and their corresponding original images for three patients. As can be seen, the predicted 140kV high-energy CT images are highly consistent with original 140 kV images. The difference images show marginal anatomical structures and suggest the spatial resolution is greatly preserved in the deep learning predicted 140 kV images. Quantitative evaluation using clinical relevant metrics shows the HU values of the original images and the deep learning predicted images are very close to each other. The HU difference between the predicted and original high-energy CT images are 1.94 HU, 3.32 HU, 1.83 HU and 1.10 HU for ROIs on spine, aorta, liver and stomach, respectively (Fig. 3).

Fig. 2.

Original low- (1st column) and high-energy (2nd column) DECT images and predicted high-energy CT images (3rd column), and difference images (4th column) between the predicted and original 140 kV images. All images are displayed in (C=0HU and W=500HU).

Fig. 3.

Quantitative measurement of the 140 kV images using ROIs on spine (a), aorta (b), liver (c) and stomach (d) for different patients. The deep learning (DL) predicted images are highly consistent with the original CT images.

Fig. 4 shows the three-dimensional VNC images and iodine maps obtained from the original 100 kV/140 kV DECT images and the deep learning-based DECT images. As can be seen, the deep learning-based DECT approach provides high quality VNC and iodine maps. Since material decomposition uses matrix inversion and yields amplified image noise, the noise levels in VNC images and iodine maps obtained from the original DECT images are much higher than the 100 kV images. Due to the noise correlation between the predicted high-energy CT images and the original low-energy CT images, deep learning-based DECT imaging provides noise significantly reduced VNC images and iodine maps. The HU differences between VNC images obtained from original DECT and deep learning DECT are 4.10 HU, 3.75 HU, 2.33 HU and 2.92 HU for ROIs on spine, aorta, liver and stomach, respectively. The aorta iodine quantification differences between iodine maps obtained from original DECT and deep learning DECT images are 0.2%, 0.5%, and 1.7% for the three patients, respectively, suggesting high consistency between the predicted and the original high-energy CT images. More importantly, the noise of the iodine maps obtained from the deep leaning predicted DECT images are about 6-fold smaller than that obtained from the original DECT images.

Fig. 4.

Illustration of the contrast-enhanced 100 kV CT images (C=0HU and W=500HU) and VNC images (C=0HU and W=500HU) and iodine maps (C=0.6 and W=1.2) obtained using original DECT images and deep learning (DL)-based DECT images in transversal, coronal and sagittal views.

Discussion and Conclusion

Since we have used an image domain material decomposition method, artifacts in the C-T images, such as beam hardening artifacts, scatter artifacts, may reduce the accuracy of the material-specific images. This can impact material decomposition using both the original DECT images and the deep learning DECT images. Hence, in the cases where artifacts significantly reduce the HU accuracy of the CT images, beam hardening correction[20] and scatter correction[35-37] methods can be employed to reduce the CT artifacts and further to enhance the material decomposition accuracy.

Classical U-Net uses cross entropy loss and soft-max for classification and segmentation problem (i.e., 0–1 labelling). Here we are using mean-squared-error because the dual-energy residual mapping is not a classification or segmentation problem. Instead, the encoder-decoder architecture is employed to incorporate the dual-energy residual into multi-resolution multi-scale feature representation. The output of the network is a 1 × 1 convolutional layer which yields a final feature that has the spatial shape as the DECT residual image. The decoded final feature was then compared to the training label (ground truth) using the mean-squared-error. The parameters of the network are updated using backpropagation during training procedure.

The difference of dual-energy attenuation properties of different types of tissues can be characterized by the residual images of the DECT images. By using the residual images and the corresponding low-energy CT images (with consistent anatomical information) to train the deep learning model, the model can learn where the organs are in a normal body and what is the dual-energy attenuation properties. The trained model can then adjust the pixel value of the low-energy CT (100kV) according its location to reflect the attenuation property at the high-energy (140 kV). In this study, all patients were performed contrast-enhanced CT scans, so one of the distinct features of the model learned is the attenuation property of the iodine contrast agent at the dual-energy scenarios, which can be used to map the dual-energy residual during the inference process.

This study demonstrates that highly accurate DECT imaging with single low-energy data is achievable by using a deep learning approach. The proposed algorithm shows superior and reliable performance on the clinical datasets, and provides clinically valuable high quality VNC and iodine maps. Compared to the current standard DECT techniques, the proposed method can significantly simplify the DECT system design, reduce the scanning dose by using only a single kV data acquisition, and reduce the noise level of material decomposition by taking advantage of the noise correlation of the deep learning derived DECT images. The strategy reduces the DECT imaging cost and may find widespread clinical applications, including cardiac imaging, angiography, perfusion imaging and urinary stone characterization and so on.