Validation of Deep-Learning Image Reconstruction for Low-Dose Chest Computed Tomography Scan: Emphasis on Image Quality and Noise.

OBJECTIVE: Iterative reconstruction degrades image quality. Thus, further advances in image reconstruction are necessary to overcome some limitations of this technique in low-dose computed tomography (LDCT) scan of the chest. Deep-learning image reconstruction (DLIR) is a new method used to reduce dose while maintaining image quality. The purposes of this study was to evaluate image quality and noise of LDCT scan images reconstructed with DLIR and compare with those of images reconstructed with the adaptive statistical iterative reconstruction-Veo at a level of 30% (ASiR-V 30%).
MATERIALS AND METHODS: This retrospective study included 58 patients who underwent LDCT scan for lung cancer screening. Datasets were reconstructed with ASiR-V 30% and DLIR at medium and high levels (DLIR-M and DLIR-H, respectively). The objective image signal and noise, which represented mean attenuation value and standard deviation in Hounsfield units for the lungs, mediastinum, liver, and background air, and subjective image contrast, image noise, and conspicuity of structures were evaluated. The differences between CT scan images subjected to ASiR-V 30%, DLIR-M, and DLIR-H were evaluated.
RESULTS: Based on the objective analysis, the image signals did not significantly differ among ASiR-V 30%, DLIR-M, and DLIR-H (p = 0.949, 0.737, 0.366, and 0.358 in the lungs, mediastinum, liver, and background air, respectively). However, the noise was significantly lower in DLIR-M and DLIR-H than in ASiR-V 30% (all p < 0.001). DLIR had higher signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) than ASiR-V 30% (p = 0.027, < 0.001, and < 0.001 in the SNR of the lungs, mediastinum, and liver, respectively; all p < 0.001 in the CNR). According to the subjective analysis, DLIR had higher image contrast and lower image noise than ASiR-V 30% (all p < 0.001). DLIR was superior to ASiR-V 30% in identifying the pulmonary arteries and veins, trachea and bronchi, lymph nodes, and pleura and pericardium (all p < 0.001).
CONCLUSION: DLIR significantly reduced the image noise in chest LDCT scan images compared with ASiR-V 30% while maintaining superior image quality.

INTRODUCTION

Lung cancer is the leading cause of cancer-related death in several countries (1). Thus, disease diagnosis at an asymptomatic stage when it can be controlled or treated is desirable. Annual low-dose computed tomography (LDCT) scan screening could significantly reduce lung cancer mortality among high-risk patients (234). With its increasing clinical use, the greater cumulative burden of annual radiation exposure has led the development of various strategies to accommodate LDCT scan of the chest (5). However, despite refinements in CT hardware technology, further dose reduction has been limited due to the presence of image noise in traditional reconstruction methods, including filtered back projection (FBP). Hence, various vendors have developed several iterative reconstruction methods (6). Regardless of type, iterative reconstruction methods deliver lower image noise, artifacts, or both at lower radiation dose than FBP-based image reconstruction methods (6). However, the smoothing artifact imparts a plastic-like, blotchy image appearance that has been observed in all iterative reconstructions particularly at higher levels. The modified appearance is considered a limitation of these techniques, and it affects the evaluation of CT scan images and, arguably, the interpretation of imaging findings (7).

Recently, the use of deep learning-based image reconstruction (DLIR) methods using deep convolutional neural networks has been proposed to facilitate dose reduction while maintaining the image quality and diagnostic performance of CT scan (8910111213141516). We hypothesized that DLIR could maintain the diagnostic image quality of chest LDCT scan. Moreover, a commercially available DLIR (TrueFidelity, GE Healthcare) was compared with adaptive statistical iterative reconstruction-Veo at a level of 30% (ASiR-V 30%) for the objective and subjective image assessment of chest LDCT scan. The purposes of this study was to evaluate image quality and noise of LDCT scan images reconstructed with DLIR and compare with those of images reconstructed with the ASiR-V 30%.

MATERIALS AND METHODS

Study Population

Initially, 60 patients, all of whom underwent consecutive chest LDCT scan (Revolution CT, GE Healthcare) in December 2019, were enrolled. All patients underwent LDCT scan for lung cancer screening. Two patients were excluded due to severe emphysema (n = 1) and motion artifact (n = 1). Thus, 58 patients were finally included in the current study. The characteristics of the patients, such as age and sex, were also assessed. This retrospective study was approved by our Institutional Review Board (Veterans Health Service Medical Center, IRB file No. 2020-02-024), and the need for informed consent for the use of existing CT scan images, including raw data, was waived.

CT Scan Image Acquisition

All patients underwent scanning on a 512-slice CT scanner (Revolution CT) while in supine position with arms raised above the shoulders to prevent artifacts. The patients were provided instructions to prevent any voluntary motion and to cautiously follow the breath-hold instructions. All LDCT scan images were obtained without the use of contrast medium. The scanning protocols were as follows: individual detector width, 0.625 mm; gantry rotation time, 0.35 seconds; beam pitch, 1.531:1; voltage, 120 kV; and tube current, 65 mA.

The LDCT scan datasets were reconstructed with ASiR-V 30% (applying standard algorithm) and experimentally using DLIR (TrueFidelity) at medium and high levels (DLIR-M and DLIR-H, respectively) (applying standard algorithm). All axial CT scan images were reconstructed with a 30–40-cm field of view and 2.5-mm section thickness. In this study, the DLIR technique uses deep convolutional neural network-based models to pattern high-dose FBP image texture with decreased noise and enhanced signals from millions of trained parameters (17). All scan data were directly displayed on the picture archiving and communication systems (PACS) (Marosis M-view 4.5; Marotech) workstation monitors, and the full functionality of the PACS software was made available to the participating radiologists (e.g., window and level settings and measurements).

To assess radiation exposure, we reviewed the CT dose index (CTDIvol) and the dose-length product (DLP) recorded as Digital Imaging and Communications in Medicine data. Moreover, the effective dose and size-specific dose estimate (SSDE) were calculated. The estimated effective dose was calculated as DLP multiplied by a k-factor of 0.014 mSv·mGy−1·cm−1 for the chest (18). The SSDE is an index in which the CTDI is corrected by the body habitus (19). Size-dependent conversion factors were obtained according to the American Association of Physicists in Medicine Report 204 (20); they were based on the sum of the anteroposterior and lateral dimensions of chest CT scan at the level of the superior portion of the breast of each patient.

Objective Image Analysis

One radiologist with 2 years of experience in radiology performed an objective image analysis of the axial images. Standardized 20-mm-diameter circular regions of interest (ROIs) were used to record signal and noise, which represented mean attenuation value and standard deviation (SD) in Hounsfield units (HU) for the lungs, mediastinum, liver, and background air for chest LDCT scan in ASiR-V 30%, DLIR-M, and DLIR-H image sets (21). Lung measurements were obtained from the lower lobes toward the periphery, mediastinal measurements from the left ventricle at the level of the coronary sinus, and liver measurements from the liver avoiding the blood vessels and biliary tree. Background air measurements were defined as the SD of air external and anterior to the patient at the sternomanubrial junction (22). Moreover, the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) were calculated in all three image sets.

SNR was calculated as follows: SNR = HUROI / SDROI (23).

CNR between the lung and mediastinum was calculated as follows: CNR = 2 (HU(24)

Subjective Image Analysis

Two radiologists with 2 and 13 years of experience, respectively, in chest CT scan performed a subjective image analysis. The radiologists were blinded to the patients' data and image reconstruction techniques, and they examined the images in random order using the PACS. CT scan was graded on axial images with data sets displayed on standard windows, and windowing was allowed as in routine reporting conditions. Each reader individually and randomly assessed the subjective image contrast and noise. The images were graded on a scale of 1–5 (Table 1). The conspicuity of major structures, including the pulmonary arteries, pulmonary veins, trachea and bronchi, lymph nodes, pleura, and pericardium, were graded on a scale of 1–5 for all CT scan images reconstructed with ASiR-V 30%, DLIR-M, and DLIR-H (Table 1) (21).

Table 1

Subjective Image Assessment

Rating	Image Contrast	Image Noise	Conspicuity of Contrast
5	Excellent	Unacceptable	Excellently visualized
4	Above average	Above average	Better than average
3	Acceptable	Average	Average
2	Suboptimal	Below average	Suboptimal
1	Poor	Minimal	Cannot identify

Statistical Analysis

Data were recorded in Excel (Microsoft Office 2010) and were analyzed with the Statistical Package for the Social Sciences software version 18.0 (IBM Corp.). The objective data are expressed as mean ± SD. The differences among the CT scan images subjected to ASiR-V 30%, DLIR-M, and DLIR-H were evaluated. The three reconstructions were compared using one-way analysis of variance, and post-hoc pairwise comparisons were adjusted for multiple comparisons using the Bonferroni correction. A p value < 0.05 was considered statistically significant.

For the subjective analysis, we calculated the interobserver agreement using the kappa statistic to evaluate the agreement between the two readers. A kappa statistic of 0.81–1.00 indicates an excellent agreement; 0.61–0.80, substantial agreement; 0.41–0.60, moderate agreement; 0.21–0.40, fair agreement; and 0.00–0.20, poor agreement (25).

RESULTS

Basic Characteristics of the Participants and Radiation Dose

Of 58 consecutive patients, 56 (96.6%) were men and 2 (3.4%) women, and the mean age of the participants was 72 (age range: 53–92) years. As for the radiation dosage, the mean CTDIvol, DLP, effective dose, and SSDE values were 1.07 ± 0 mGy, 53.9 ± 2.3 mGy*cm, 0.75 ± 0.03 mSv, and 0.69 ± 0.05 mGy, respectively.

Objective Analysis

The objective image analysis results are presented in Table 2. The image signal did not significantly differ across ASiR-V 30%, DLIR-M, and DLIR-H; however, the mean signal values of DLIR images were more likely to be higher than those of ASiR-V 30% images. The other parameters, including image noise, SNR, and CNR, differed significantly. Regarding image noise, the lung values did not significantly differ between DLIR-M and DLIR-H (p = 0.837). However, they were significantly higher in ASiR-V 30% and significantly lower in DLIR-H (ASiR-V 30% vs. DLIR-M, p = 0.018 and ASiR-V 30% vs. DLIR-H, p < 0.001, respectively) (Fig. 1). The noise in the mediastinum, liver, and background air significantly differed across the three different reconstructions, and it was significantly higher in ASiR-V 30% and significantly lower in DLIR-H (ASiR-V 30% vs. DLIR-M, ASiR-V 30% vs. DLIR-H, and DLIR-M vs. DLIR-H, all p < 0.001) (Fig. 2). The SNR in the lung was significantly different between ASiR-V 30% and DLIR-H (p = 0.025). However, it did not significantly differ between ASiR-V 30% and DLIR-M (p = 0.248) and DLIR-M and DLIR-H (p = 1.000). The SNR in the mediastinum and liver significantly differed across the three different reconstructions, and it was higher in DLIR-H and lower in ASiR-V 30% (ASiR-V 30% vs. DLIR-M, ASiR-V 30% vs. DLIR-H, and DLIR-M vs. DLIR-H, all p = 0.001, < 0.001, and < 0.001, respectively). The CNR in the lung significantly differed between ASiR-V 30% and DLIR-M (p = 0.002) and ASiR-V 30% and DLIR-H (p < 0.001). However, it was not significantly different between DLIR-M and DLIR-H (p = 0.519). The CNR in the mediastinum and liver significantly differed across the three different reconstructions, and it was higher in DLIR-H and lower in ASiR-V 30% (ASiR-V 30% vs. DLIR-M, ASiR-V 30% vs. DLIR-H, and DLIR-M vs. DLIR-H, all p < 0.001, respectively).

Table 2

Objective Image Analysis Results

Variables	ASiR-V 30%	DLIR-M	DLIR-H	P	P
					ASiR-V vs. DLIR-M	ASiR-V vs. DLIR-H	DLIR-M vs. DLIR-H
Singnal (HU)
Lung	-864.9 ± 45.4	-867.0 ± 43.0	-867.3 ± 43.0	0.949	1.000	1.000	1.000
Mediastinum	45.6 ± 6.9	46.4 ± 6.5	46.3 ± 6.4	0.737	1.000	1.000	1.000
Liver	62.2 ± 7.7	63.0 ± 7.4	67.4 ± 35.2	0.366	1.000	0.568	0.790
Air	-966.0 ± 262.5	-1001.7 ± 12.3	-1000.5 ± 3.7	0.358	0.621	0.669	1.000
Noise (HU)
Lung	34.9 ± 8.1	30.3 ± 9.5	28.5 ±9.1	< 0.001*	0.018*	< 0.001*	0.837
Mediastinum	22.8 ± 3.3	14.0 ± 2.3	9.1 ± 1.5	< 0.001*	< 0.001*	< 0.001*	< 0.001*
Liver	26.5 ± 2.7	16.9 ± 2.2	12.1 ± 6.3	< 0.001*	< 0.001*	< 0.001*	< 0.001*
Air	12.7 ± 1.6	6.6 ± 1.1	3.8 ± 0.6	< 0.001*	< 0.001*	< 0.001*	< 0.001*
SNR
Lung	27.0 ± 9.1	32.7 ± 16.7	35.7 ± 23.9	0.027*	0.248	0.025*	1.000
Mediastinum	2.0 ± 0.4	3.4 ± 0.8	5.2 ± 1.1	< 0.001*	0.001*	< 0.001*	< 0.001*
Liver	2.4 ± 0.4	3.8 ± 0.7	5.6 ± 0.8	< 0.001*	< 0.001*	< 0.001*	< 0.001*
CNR
Lung	28.5 ± 13.9	43.6 ± 28.3	49.4 ± 24.4	< 0.001*	0.002*	< 0.001*	0.519
Mediastinum	3405.5 ± 1059.3	9865.1 ± 3549.9	24012.9 ± 8102.8	< 0.001*	< 0.001*	< 0.001*	< 0.001*
Liver	2676.2 ± 522.1	7295.4 ± 2548.5	16016.5 ± 3910.9	< 0.001*	< 0.001*	< 0.001*	< 0.001*

Data given is mean ± SD. *p < 0.05. ASiR-V = adaptive statistical iterative reconstruction-Veo, CNR= contrast-to-noise ratio, DLIR-H = deep-learning image reconstruction at high level, DLIR-M = deep-learning image reconstruction at medium level, HU = Hounsfield units, SD = standard deviation, SNR = signal-to-noise ratio

Fig. 1

Comparison of low-dose chest CT scan in axial lung window images of lung in 73-year-old man.

Reconstruction was performed with ASiR-V 30% (A), DLIR-M, (B) and DLIR-H (C). Signal did not significantly vary across different reconstructions. However, image noise of DLIR images was lower than that of ASiR-V 30% images (ASiR-V 30% vs. DLIR-M, p = 0.018 and ASiR-V 30% vs. DLIR-H, p < 0.001, respectively). Image noise in lung did not significantly differ between DLIR-M and DLIR-H (p = 0.837). ASiR-V = adaptive statistical iterative reconstruction-Veo, CT = computed tomography, DLIR = deep-learning image reconstruction, DLIR-H = DLIR at high levels, DLIR-M = DLIR at medium levels

Fig. 2

Comparison of low-dose chest CT scan in axial soft tissue window images of mediastinum in 73-year-old man.

Reconstruction was performed with ASiR-V 30% (A), DLIR-M (B), and DLIR-H (C). Signal did not significantly vary across different reconstructions. However, image noise of DLIR images was lower than that of ASiR-V 30% images (ASiR-V 30% vs. DLIR-M, ASiR-V 30% vs. DLIR-H, and DLIR-M vs. DLIR-H, all p < 0.001).

Subjective Analysis

The subjective image analysis results are presented in Table 3. DLIR-M (5.0) and DLIR-H (5.0) had better subjective image contrast and noise than ASiR-V 30% (4.1), and the scores significantly differed between ASiR-V 30% and DLIR-M (p < 0.001) and ASiR-V 30% and DLIR-H (p < 0.001). However, the difference between DLIR-M and DLIR-H was not statistically significant, and the mean score of DLIR-M was slightly higher. The subjective image noise had similar results (ASiR-V 30% vs. DLIR-M, ASiR-V 30% vs. DLIR-H, all p < 0.001 and DLIR-M vs. DLIR-H, p = 1.000, respectively). For major structure conspicuity, the DLIR-H images yielded the highest scores in all major structures (pulmonary arteries, pulmonary veins, trachea and bronchi, lymph nodes, pleura, and pericardium); the scores between ASiR-V 30% and DLIR-M (p = 0.030, 0.013, < 0.001, < 0.001, and < 0.001, respectively) and ASiR-V 30% and DLIR-H (p < 0.001) significantly differed. However, the scores for DLIR-M and DLIR-H were comparable in all major structures (p = 1.000, 0.065, 0.948, and 0.462, respectively) except the pleura and pericardium (p = 0.003), which had equal level of conspicuity (Table 3). The interobserver agreement between the two readers was substantial (kappa value of 0.70).

Table 3

Subjective Image Analysis Results

Variables	ASiR-V 30%	DLIR-M	DLIR-H	P	P
					ASiR-V vs. DLIR-M	ASiR-V vs. DLIR-H	DLIR-M vs. DLIR-H
Subjective image contrast	4.1 ± 0.3	5.0 ± 0.1	5.0 ± 0.2	< 0.001*	< 0.001*	< 0.001*	1.000
Subjective image noise	2.1 ± 0.3	1.0 ± 0.2	1.0 ± 0.0	< 0.001*	< 0.001*	< 0.001*	1.000
Conspicuity of structures
Pulmonary arteries	4.9 ± 0.3	5.0 ± 0.2	5.0 ± 0.1	< 0.001*	0.030*	< 0.001*	1.000
Pulmonary veins	4.7 ± 0.5	4.9 ± 0.3	5.0 ± 0.2	< 0.001*	0.013*	< 0.001*	0.065
Trachea and bronchi	4.8 ± 0.4	5.0 ± 0.2	5.0 ± 0.1	< 0.001*	< 0.001*	< 0.001*	0.948
Lymph nodes	4.2 ± 0.4	4.9 ± 0.4	5.0 ± 0.2	< 0.001*	< 0.001*	< 0.001*	0.462
Pleura and pericardium	4.1 ± 0.3	4.8 ± 0.4	4.9 ± 0.3	< 0.001*	< 0.001*	< 0.001*	0.003*

Data are presented as mean ± SD. *p < 0.05.

DISCUSSION

This study showed that DLIR reconstruction yielded chest LDCT scan images with significantly higher SNR and CNR and lower noise than ASiR-V 30%. The subjective overall image quality score of DLIR was significantly better than that of ASiR-V 30%. Thus, DLIR can yield a better image quality than ASiR-V 30%.

Recently, several clinical studies on deep convolutional neural network-based reconstruction techniques have reported that DLIR yields favorable noise texture with superior image quality alone and a significantly reduced image noise in coronary CT angiography (91314) and abdominal CT scan (8121516). Although the locations where CT scan was applied differed, some results were in accordance with and reinforced the results of our study. A recent study using DLIR was conducted using low-dose chest CT scan. The lesion detection rate and image quality of low-dose DLIR images were assessed and compared with those of low- and standard-dose iterative reconstruction images from Canon Medical Systems. Results showed that DLIR images had a better image quality than iterative reconstruction images. Moreover, the lesion detection rate between these images and standard-dose iterative reconstruction images were comparable, and this finding supports the finding of our study (12). Although there is a similar report on chest CT scan, the sample size of the current study (n = 58) was relatively larger than that of the previous study (n = 22), and the DLIR technique used in this study was different from that used in the previous report. Hence, we believe that the findings described herein are significant. The details on the differences in the techniques are presented in the proceeding paragraphs.

In this study, the image signal did not significantly vary across ASiR-V 30%, DLIR-M, and DLIR-H. However, the mean signal values of DLIR images were more likely to be higher than those of ASiR-V 30% images. The DLIR used in this study input a low-dose sinogram through the deep neural network, and the output image was compared to a ground truth image (FBP of the same data). These two images were compared based on multiple parameters, such as image noise, contrast resolution, contrast detectability, and noise texture. The output image shows the differences in the network via backpropagation, which then strengthens some equations and weakens others. This process is repeated until the accuracy between the output image and the ground truth image is detected (17). In deep learning, the training target determines the output. FBP is a mathematically accurate reconstruction algorithm developed under the best data acquisition and reconstruction conditions. Moreover, our DLIR engine with deep convolutional neural networks was involved in the raw data acquisition phase, which minimizes the loss of signal. Most recently published papers on commercialized DLIR engine used training pairs presented as hybrid-iterative reconstruction images and high-dose model-based iterative reconstruction images. Deep convolutional neural networks in previous studies were involved only after raw data acquisition, particularly in the denoising phase of reconstructed image alone (810121314). The DLIR technique used in our study reflected a more ideal projection data than that in previous studies. Thus, we believe that our results support these data.

A recent study conducted by Jensen et al. (15) who examined the value of DLIR algorithm offered by the same vendor in our study (TrueFidelity) in the oncologic evaluation of contrast-enhanced abdominal CT scans was published. Similar to our study, their study evaluated how the objective and subjective analyses of image quality differed when the ASiR-V 30% and the DLIR algorithms were applied to each abdomen CT image. Results revealed that DLIR improved the image quality and lesion diagnostic confidence of contrast-enhanced oncologic CT scans of the abdomen relative to that of our standard ASiR-V 30%. Based on these results, the improvement in CT scan image quality using the DLIR technique may lead to a higher lesion diagnostic confidence in chest CT scan. That is, patients can have a more accurate diagnosis in equivocal situations based on the morphological features of a lesion. Therefore, the DLIR algorithm may offer a stronger foundation in which radiologists can establish a more accurate diagnosis during the interpretation of chest CT scan findings.

CNR measures signal amplitude in the presence of noise, independent of size in a homogeneous object. SNR incorporates size and shape to describe object conspicuity and can be used for objects that are not homogeneous (26). Both CNR and SNR are measures of image quality. A high SNR is important for the detection of small lesions, and a high CNR is required to distinguish any lesions from the background parenchyma. The conservation of the SNR and CNR at low doses is required to maintain the diagnostic image quality. In our study, the measured CNR of DLIR-M and DLIR-H was significantly higher than that of ASiR-V 30% (Table 2). In particular, when measuring the lungs, mediastinum, and liver, the SNR of DLIR-H was higher (45.74, 5.20, 5.61, respectively) than that of ASiR-V 30% (26.98, 2.04, and 2.37, respectively). The Rose criterion indicates that an SNR ≥ 5 will usually allow object detection, with decreased detection, as it approaches zero (27). The SNR values of DLIR-H were > 5 in all locations, thereby indicating a high object detectability regardless of lesion location in chest LDCT scan.

The current study had some limitations. The study population was relatively small, and our investigation was retrospective in nature. Moreover, the study was carried out at a single institution. Therefore, the findings were considered preliminary. An extremely large sample size might reject the null hypotheses with clinically negligible differences, leading to the possibility that what is insignificant may become significant (28). However, the difference in image noise between DLIR and ASiR-V 30% was larger than the SD of each reconstruction method, indicating that a significant difference in our data was not clinically negligible even though the patient population was slightly larger than that required based on the power calculation. In addition, the high-spatial-frequency algorithm images of our patients were not assessed and, consequently, were not included in this study. Finally, this study focuses on the assessment of image quality, and the clinical images were evaluated only at a single radiation dose per patient. Because the benefit of denoising increases as radiation doses are reduced, objective and subjective image analyses must be conducted at various radiation dose levels. Therefore, further studies should be performed to analyze task-specific accuracy at varying radiation doses. Moreover, large-scale studies of general patient cohorts must be conducted.

In conclusion, the image noise, image quality, SNR, and CNR of chest LDCT scan images improve with DLIR compared with ASiR-V 30%. Thus, the use of DLIR is feasible in clinical practice, and this method is beneficial, as it yields less image noise while maintaining a favorable noise texture for lung cancer screening or diagnosis and follow-up with LDCT scan.