Automated Lung Segmentation from Computed Tomography Images of Normal and COVID-19 Pneumonia Patients.

Background: Automated image segmentation is an essential step in quantitative image analysis. This study assesses the performance of a deep learning-based model for lung segmentation from computed tomography (CT) images of normal and COVID-19 patients.
Methods: A descriptive-analytical study was conducted from December 2020 to April 2021 on the CT images of patients from various educational hospitals affiliated with Mashhad University of Medical Sciences (Mashhad, Iran). Of the selected images and corresponding lung masks of 1,200 confirmed COVID-19 patients, 1,080 were used to train a residual neural network. The performance of the residual network (ResNet) model was evaluated on two distinct external test datasets, namely the remaining 120 COVID-19 and 120 normal patients. Different evaluation metrics such as Dice similarity coefficient (DSC), mean absolute error (MAE), relative mean Hounsfield unit (HU) difference, and relative volume difference were calculated to assess the accuracy of the predicted lung masks. The Mann-Whitney U test was used to assess the difference between the corresponding values in the normal and COVID-19 patients. P<0.05 was considered statistically significant.
Results: The ResNet model achieved a DSC of 0.980 and 0.971 and a relative mean HU difference of -2.679% and -4.403% for the normal and COVID-19 patients, respectively. Comparable performance in lung segmentation of normal and COVID-19 patients indicated the model's accuracy for identifying lung tissue in the presence of COVID-19-associated infections. Although a slightly better performance was observed in normal patients.
Conclusion: The ResNet model provides an accurate and reliable automated lung segmentation of COVID-19 infected lung tissue.A preprint version of this article was published on arXiv before formal peer review (https://arxiv.org/abs/2104.02042). Copyright: © Iranian Journal of Medical Sciences.

What’s Known

Computed tomography (CT) is a complementary procedure in the diagnosis and management of COVID-19. Quantitative analysis of CT images provides key information about lesion size and disease severity.

The accuracy of quantitative analyses depends on the reliability of the image segmentation method.

What’s New

The residual network (ResNet) model provides acceptable lung segmentation from CT images of COVID-19 patients compared to normal patients, despite the presence of infections.

The model provides reliable lung segmentation for the quantitative image analysis of pneumonia lesions.

Introduction

In December 2019, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus emerged in Wuhan (China) and rapidly spread throughout the world. By infecting the respiratory tract, the virus causes a respiratory disease called coronavirus disease 2019 (COVID-19). The most common screening method to detect COVID-19 is the real-time polymerase chain reaction (RT-PCR). However, high false-negative RT-PCR results in the case of low viral load in test specimens undermine the accuracy of the test. Chest X-ray and computed tomography (CT) are used as complementary and faster screening methods for the early detection of COVID-19. Some studies have reported that CT imaging outperforms X-ray in providing more structural/anatomical details of the lungs. Chest CT images have a sensitivity of 0.97 in diagnosing COVID-19, enabling the detection of radiological patterns such as bilateral and peripheral ground-glass opacities and patchy consolidations in the lungs of infected patients. Moreover, quantitative analysis of CT images provides key information about lesions size and disease severity for which reliable lung segmentation from CT images is an essential prerequisite. In addition, automated lung segmentation facilitates quantitative image analysis by removing unnecessary regions in chest CT images.

Different approaches for lung segmentation have been used, including manual segmentation, rule-based, atlas-based, machine learning-based as well as hybrid techniques. Manual segmentation is no longer an option, since it is too time-consuming and labor-intensive, particularly in situations where the health system is already overloaded. Other conventional methods such as atlas-based or intensity-based algorithms provide acceptable results in normal patients or those with mild diseases. However, their implementation is inefficient in diseases such as COVID-19 where the infection alters common patterns/structures of the lung. In the search for a solution, recent research studies have evaluated the use of deep learning models for lung and lesion segmentation and demonstrated a promising performance of convolutional neural networks in distinguishing the lungs from the chest wall.

A number of studies have used common deep learning architectures (e.g., U-Net, 3D U-Net, UNet++, and V-Net) for the segmentation of COVID-19 infected lung tissue. A previous study reported the development and evaluation of state-of-the-art algorithms for cases with insufficient annotated image datasets using transfer learning or weakly-annotated datasets. Due to the presence of significant abnormalities in the lungs of COVID-19 patients, lung segmentation (i.e., distinguishing lung boundaries from infection) is challenging. Whereas in normal patients, there is a distinct contrast between lung tissue and chest wall. Most of the automated COVID-19 diagnostic models utilize lung segmentation as a pre-processing step. In an unsupervised framework for the detection of COVID-19 lung lesions, Yao and colleagues used a U-Net model (relying on a 2D U-Net as the backbone of the model) for lung segmentation with a Dice similarity coefficient (DSC) score of >0.98 in both Coronacases (https://coronacases.org) and Radiopedia (https://radiopaedia.org) datasets. In a weakly-supervised lesion localization for COVID-19 classification, Wang and colleagues used a pre-trained U-Net for lung segmentation as an input to their novel Network to Detect COVID-19 (DeCoVNet). They used lung masks generated by an unsupervised method to train the U-Net. Zhao and colleagues proposed a novel spatial- and channel-wise coarse-to-fine attention network (SCOAT-Net), built upon the UNet++ architecture, to address the challenges associated with the segmentation of COVID-19 infected lung tissue from CT images. The reported results from the model were encouraging with acceptable generalizability. Oulefki and colleagues reported the design and evaluation of a tool for automated segmentation of COVID-19 infected lung tissue using a novel image contrast enhancement algorithm along with an improved image-dependent multilevel thresholding method.

The present study assessed the efficiency of the ResNet model in automated lung segmentation from CT images of patients with COVID-19 in comparison with normal patients. For accuracy purposes, a dataset of normal patients was also used to study the impact of lung abnormalities caused by COVID-19 on lung segmentation.

Patients and Methods

Dataset

A descriptive-analytical study was conducted from December 2020 to April 2021 on CT images of patients who were referred to the radiology departments of various educational hospitals affiliated with Mashhad University of Medical Sciences (Mashhad, Iran). The right for access to data was approved by Mashhad University of Medical Sciences (ethical code: IR.MUMS.REC.1399.515). Written informed consent was obtained from all participants.

The dataset consisted of chest CT images from 1,200 patients with confirmed COVID-19 (a positive RT-PCR test) and 120 normal patients. The latter included patients with suspected respiratory diseases, a negative RT-PCR test, and CT images not showing any lesions or lung abnormalities. Excluded were those patients with a positive RT-PCR test whose CT images did not reveal any respiratory involvement and those with lung lesions or infections unrelated to COVID-19, such as nodular lung lesions, bronchiectasis, chronic obstructive pulmonary disease, mediastinal lymphadenopathy, or atelectasis.

CT images were acquired using a Somatom Spirit Dual Slice CT scanner (Siemens, Germany) with a tube voltage of 130 kVp, tube current-exposure time of 48 mAs, rotation time of 0.8 s, and slice thickness of 5 mm. To generate ground truth masks of the lungs, CT images were segmented semi-automatically using Pulmonary Toolkit software. The software is a non-commercial open-source MATLAB-based platform developed for the visualization and analysis of 3D clinical lung images with the capability of automatic lung and airway segmentation. The resulting binary masks were manually corrected/verified under the supervision of a radiologist to avoid potential errors. Prior to the training of the network, all images were cropped to eliminate areas outside of the lung volume and resized to a matrix size of 296×216 voxels using a linear interpolation algorithm. Then, Hounsfield units (HU) were scaled to an intensity range between 0 and 1.3.

Implementation

The ResNet model implemented in NiftyNet was used for automated lung segmentation. NiftyNet is an open-source platform built upon TensorFlow that consists of common convolutional neural networks used in medical imaging. The ResNet model comprises 20 convolutional layers, of which every two layers are connected by residual connections (figure 1). In this network, dilation factors of one, two, and four are applied to the convolutional kernels to extract low-level, mid-level, and high-level features from input images, respectively. Moreover, a fully connected SoftMax layer is embedded as the final output layer of the network.

Figure 1

Architectural representation of the ResNet model.

Of the 1,200 chest CT images included in the study, 1,080 images and their corresponding masks were randomly selected to train the network. The remaining 120 images were used for external validation. To eliminate any overlap, the dataset was divided into training and test groups based on patients’ data (not two-dimensional slices). To prevent overfitting, 5% of the training datasets were used for validation of the model during the training phase. No significant difference was found between the training loss and validation loss.

Training of the deep learning model was performed on two-dimensional slices using settings such as learning rate: 0.02, optimizer: Adam, loss function: Dice-NS (Dice no-square), decay: 0.0001, batch size: 17, and weighted regression type: L2 norm.

Dice-NS was calculated by differentiating DSC (Eq. 1) with respect to the i voxel of the predicted lung masks.

(1)

where I and I denote the reference and predicted lung masks.

Evaluation Metrics

To evaluate the performance of the deep learning model, predicted and ground truth lung segmentations were compared to the external test dataset that included 120 COVID-19 and 120 normal patients. The assessment was performed by calculating DSC (Eq. 1), Jaccard index (JC) (Eq. 2), mean error (ME) (Eq. 3), and mean absolute error (MAE) (Eq. 4) within the estimated lung region.

(2)

(3)

(4)

where I and I denote the reference and predicted lung masks. V and i indicate the total number of voxels in the lung area and the index of voxels in I and I images, respectively.

Furthermore, false-positive rates, false-negative rates, relative mean CT number (HU) difference, absolute relative mean HU difference, relative volume difference, and absolute relative volume difference metrics were calculated between the reference and predicted lung volumes.

A receiver operating characteristic (ROC) analysis demonstrates the ability of a binary classifier system. Therefore, for a more accurate evaluation of the ResNet model, the ROC curve was drawn for both normal and COVID-19 datasets. To this end, voxel-wise lung probability maps were generated for external validation datasets. Then, separate ROC curves for normal and COVID-19 patients were drawn by plotting sensitivity (true-positive rate) against specificity (false-positive rate) at various threshold values defined for lung tissue masks.

Statistical Analysis

The distribution of calculated metrics in normal and COVID-19 patients was analyzed using Shapiro-Wilk and Kolmogorov-Smirnov normalization tests. The results showed a non-normal distribution of data across all metrics. Therefore, the Mann-Whitney U test was used to assess the difference between the corresponding values in the normal and COVID-19 datasets. All data were analyzed using SPSS software, version 16.0 (IBM Corp. Armonk, NY, USA). P values less than 0.05 were considered statistically significant.

Results

Representative results of lung segmentation for the normal and COVID-19 patients are presented in figures 2 and 3. Figure 2 shows a good match between the reference and predicted masks for both normal and infected lung tissues, indicating a promising performance of the ResNet model in detecting lung boundaries. Figure 3 depicts minor segmentation errors in two cases where the model was not successful in defining an accurate margin for the lung and exclude the bronchi from the segmented area (outlier report). The miss-segmentation error is more noticeable in COVID-19 patients due to the similar image intensity of the chest wall and severe infections, making accurate lung boundary detection very challenging.

Figure 2

Axial views of (A) ground truth lung boundaries in normal patients and (B) corresponding predicted lung boundaries. (C) Ground truth lung boundaries in COVID-19 patients and (D) corresponding predicted lung boundaries.

Figure 3

Representation of an outlier report showing a case with noticeable errors, including axial views of (A) ground truth lung boundaries in normal patients and (B) corresponding predicted lung boundaries. (C) Ground truth lung boundaries in COVID-19 patients and (D) corresponding predicted lung boundaries.

Table 1 shows a statistical analysis of some quantitative metrics for lung segmentation, including DSC, JC, ME, MAE, false-positive rates, false-negative rates, mean HU difference within the lung mask, and volume difference calculated for external validation datasets. Overall, the ResNet model showed better performance in lung segmentation of normal patients compared to COVID-19 patients due to the high-density infections close to the chest wall.

Table 1

Statistics of quantitative metrics calculated between the reference and predicted lung masks in normal and COVID-19 patients

Parameter	Normal					COVID-19					P value*
	Min	Max	Mean±SD	Median	IQR	Min	Max	Mean±SD	Median	IQR
Dice coefficient	0.970	0.985	0.980±0.003	0.982	0.003	0.903	0.986	0.971±0.017	0.976	0.013	0.23
Jaccard index	0.942	0.971	0.962±0.007	0.965	0.006	0.823	0.973	0.938±0.040	0.953	0.025	0.23
Mean error	-0.031	0.015	-0.015±0.009	-0.018	0.011	-0.155	0.042	-0.024±0.042	-0.12	0.030	0.93
Mean absolute error	0.028	0.057	0.037±0.007	0.035	0.006	0.026	0.176	0.061±0.040	0.046	0.025	0.25
False-positive rate	0.015	0.223	0.059±0.043	0.050	0.049	0.033	0.280	0.076±0.043	0.067	0.033	0.20
False-negative rate	0.020	0.036	0.026±0.003	0.026	0.005	0.011	0.167	0.044±0.040	0.027	0.024	0.18
Relative mean HU difference (%)	-3.673	-2.020	-2.679±0.382	-2.658	0.523	-16.799	-1.183	-4.403±4.097	-2.704	-2.345	0.18
Absolute relative mean HU difference (%)	0	3.475	0.828±1.318	0	2.442	0.921	10.894	3.253±3.106	0.046	2.123	0.002
Relative volume difference (%)	-4.726	29.524	2.405±7.359	0.305	6.526	-12.666	90.561	5.928±17.261	1.014	8.211	0.23
Absolute relative volume difference (%)	1.960	33.164	7.875±6.548	4.799	4.914	2.199	91.486	12.743±16.384	6.025	10.602	0.08

Mann-Whitney U test, IQR: Interquartile range

The mean DSC and JC values in normal patients were 0.980±0.003 and 0.962±0.007, and in COVID-19 patients were 0.971±0.017 and 0.938±0.040, respectively. The ResNet model achieved ME of -0.015±0.009 HU and -0.024±0.042 HU, and MAE of 0.037±0.007 HU and 0.061±0.040 HU within the lung masks for the normal and COVID-19 patients, respectively. Moreover, comparable measures of false-positive and false-negative rates were obtained for the two datasets. Quantitative image assessment revealed relative mean HU differences of -2.679±0.382% and -4.403±4.097%, and relative volume differences of 2.405±7.359% and 5.928±17.261% for the lung tissue in the normal and COVID-19 patients, respectively. The P values reported in table 1 are greater than 0.05 for all metrics except for the absolute relative mean HU difference, thus there was no significant difference in parameters between the normal and COVID patients. This indicates an acceptable and similar performance of the ResNet model in both patient groups.

The ROC curves for the ResNet model are shown in figure 4. The area under the curve (AUC) of 0.96 and 0.92 was achieved for the normal and COVID-19 datasets, respectively.

Figure 4

Result of receiver operating characteristic (ROC) analysis using different thresholds for probability maps in the lungs of normal and COVID-19 patients. AUC: Area under the curve

Discussion

The ResNet model provided an accurate and reliable lung segmentation in both normal and COVID-19 patients, indicating its capability to detect infected lung tissue. Recent studies suggest that chest CT imaging findings play a significant role in the diagnosis and management of COVID-19. Accurate lung segmentation is a crucial step for the calculation of quantitative indices, determining lung engagement, and disease severity. Current segmentation methods, which have shown satisfactory performance in normal or mild lung diseases, are either time-consuming or pose serious challenges in the segmentation of COVID-19 infected lung tissue due to low-intensity contrast between infections and normal tissues. Deep learning-based models have been widely used as a reliable tool to assist clinicians in fast and efficient segmentation of lung and lesions in COVID-19 patients.

Gerard and colleagues developed a segmentation algorithm called LobeNet aiming at the prediction of the left and right lung regions in the presence of diffuse opacities and consolidations. A quantitative assessment of LobeNet performance in 87 COVID-19 patients showed an average DSC score of 0.985. Moreover, Ma and colleagues reported DSC scores of 0.973 and 0.977 from a 3D U-Net for the segmentation of the left and right lungs from CT images of COVID-19 patients, respectively. Tilborghs and colleagues compared the performance of various deep learning algorithms using a multicenter COVID-19 dataset. They concluded that combining different methods could improve the segmentation performance and increase DSC up to 0.987. The ResNet model used in our study achieved average DSC scores of 0.980±0.003 and 0.971±0.017 in normal and COVID-19 external test datasets, respectively. This result is in line with the findings of previous studies, which indicate good agreement between the predicted and ground truth lung masks.

Yan and colleagues developed a new network called COVID-SegNet for the segmentation of CT images of COVID-19 infected lung regions. They compared the proposed network against other state-of-the-art models such as FCN, U-Net, V-Net, and UNet++. The COVID-SegNet achieved a DSC, sensitivity, and precision of 0.865, 0.986, and 0.983, respectively. In another study, conducted by Trivizakis and colleagues, the U-Net architecture, one of the most common image segmentation architectures, was used for the segmentation of COVID-19 infected lung tissue. They reported a DSC of 0.950, a sensitivity of 0.920, and a specificity of 0.875. Muller and colleagues performed a similar analysis using 3D U-Net for lung segmentation using a cross-validation scheme to reduce/avoid the risk of overfitting. Their approach led to lung segmentation with a DSC of 0.956, sensitivity of 0.956, and specificity of 0.998. In our study, we obtained a false-positive rate of 0.059 and 0.076, and a false-negative rate of 0.026 and 0.044 for the normal and COVID-19 patients, respectively. These are comparable with the results reported in the literature, demonstrating a promising performance of the ResNet model.

Suri and colleagues proposed a COVID-19 Lung Image Analysis System (COVLIAS) consisting of hybrid deep learning models such as VGG-SegNet and ResNet-SegNet for lung segmentation. A validation framework with a 40:60 ratio between training and testing datasets was used on CT images of 72 patients. COVLIAS yielded maximum AUC, JC, and DSC of 0.98, 0.93, and 0.96, respectively. Zhao and colleagues develop an automated method to segment pulmonary parenchyma in chest CT images and analyzed texture features of the segmented regions in COVID-19 patients. They integrated a three-dimensional V-Net with a shape deformation module using a spatial transform network. The segmentation method achieved a DSC of 0.9796, sensitivity of 0.9840, and specificity of 0.9954. In our study, the ResNet model achieved an AUC of 0.96 and 0.92 for the normal and COVID patients, respectively. These results are comparable to those reported in previous studies, indicating great promise in facilitating automated lung segmentation. The results of the ResNet model in normal and COVID-19 patients showed high accuracy in terms of relative mean HU difference (-2.679% and -4.403%) and relative volume difference (2.405% and 5.928%), respectively. Overall, the performance of the ResNet model in COVID-19 patients, despite the presence of infections and subsequent changes in image intensity contrasts, is comparable to its performance in normal patients.

To the best of our knowledge, this is the first study that compared the results of lung segmentation in normal and COVID-19 patients using the ResNet model. Other studies conducted in this field have focused on implementing U-Net or networks with an architecture similar to U-Net for lung segmentation in COVID-19 patients. The ResNet model provided promising results for lung segmentation in both patients groups. Deep convolutional neural networks apply more convolutional kernels to an image, leading to the extraction of features with high complexities. However, due to issues related to vanishing gradient, training of deep networks would be challenging using too deep networks. Deep residual networks, consisting of several residual or shortcut connections, were introduced by He and colleagues to address gradient vanishing issues during the training phase of the deep neural networks and to reduce computational cost. More importantly, the ResNet model uses dilated convolution without applying down-sampling or max-pooling. Whereas other architectures tend to process the input image at lower resolution levels to avoid extra processing burden. The ResNet model processes input images at their original resolution without any resolution degradation at any layer. Therefore, this model is capable of extracting discriminative features with remarkable robustness.

The main limitation of the study was the lack of multicenter datasets to evaluate the sensitivity of the ResNet model in terms of variations in image quality and acquisition parameters. In order to develop a comprehensive framework suitable for clinical practice, it is recommended to evaluate the performance of the ResNet model in lesion segmentation. This allows distinguishing COVID-19 lesions from other pneumonia lesions and diagnosing subtypes of COVID-19 pneumonia.

Conclusion

The ResNet model provides an accurate and reliable automated lung segmentation of COVID-19 infected lung tissue. The model achieved very promising results with DSC scores of 0.980 and 0.971 in normal and COVID-19 external test datasets, respectively. This model allows accurate quantitative image analysis and diagnosis for the effective treatment of COVID-19 patients. In future studies, it is recommended to investigate the performance of the ResNet model for lung lobe segmentation. The model can also be used for the detection of weakly-supervised COVID-19 lesions.

Authors’ Contribution

H.A: Conceptualization, Supervision and, Methodology; F.Gh and S.M: Use of Software, Formal analysis, Data curation, Visualization; N.A.G, S.J.Sh.R, P.L, and S.M.T: Dataset search; P.L: CT Image report; P.L and S.M.T: Investigation; F.Gh, S.M, N.A.G, S.J.Sh.R, and H.A: Writing—original draft preparation; F.Gh, S.M, P.L, S.M.T, and H.A: Revision. All authors have read and approved the final manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investi-gated and resolved.

Conflict of Interest

None declared.