Automatic segmentation of pulmonary lobes on low-dose computed tomography using deep learning.

BACKGROUND: To develop and validate a fully automated deep learning-based segmentation algorithm to segment pulmonary lobe on low-dose computed tomography (LDCT) images.
METHODS: This study presents an automatic segmentation of pulmonary lobes using a fully convolutional neural network named dense V-network (DenseVNet) on lung cancer screening LDCT images. A total of 160 LDCT cases for lung cancer screening (100 in the training set, 10 in the validation set, and 50 in the test set) was included in this study. Specifically, the template of pulmonary lobes (the right lung consists of three lobes, and the left lung consists of two lobes) were obtained from pixel-level annotations by semiautomatic segmentation platform. Then, the model was trained under the supervision of the LDCT training set. Finally, the trained model was used to segment the LDCT in the test set. Dice coefficient, Jaccard coefficient, and Hausdorff distance were adopted as evaluation metrics to verify the performance of our segmentation model.
RESULTS: In this study, the model achieved the accurate segmentation of each pulmonary lobe in seconds without the intervention of researchers. The testing set consisted 50 LDCT cases were used to evaluate the performance of the segmentation model. The all-lobes Dice coefficient of the test set was 0.944, the Jaccard coefficient was 0.896, and the Hausdorff distance was 92.908 mm.
CONCLUSIONS: The segmentation model based on LDCT can automatically and robustly and efficiently segment pulmonary lobes. It will provide effective location information and contour constraints for pulmonary nodule detection on LDCT images for lung cancer screening, which may have potential clinical application. 2021 Annals of Translational Medicine. All rights reserved.

Introduction

Worldwide, lung cancer remains the leading cause of cancer incidence and mortality (1). The majority of patients with lung cancer are diagnosed at an advanced tumor stage, and only a small number of patients can be detected at the early tumor stage. The research of the International Early Lung Cancer Action Project (I-ELCAP) showed that annual low-dose computed tomography (LDCT) screening allows at least 85% of lung cancers to be diagnosed at stage I, and the estimated 10-year survival rate of early-stage (stage I) lung cancer is 88% (2). Early diagnosis and treatment of lung cancer play an important role in improving lung cancer survival and reducing lung cancer mortality.

As LDCT plays a more important role in lung cancer screening, which leads to an increase in the workload of radiologists in identifying and diagnosing lung nodules, and causes the burden of radiologists and the increase of misdiagnosed rate. Computer-assisted diagnosis (CAD) methods have value to the radiologists in the early detection and diagnosis of suspicious nodules on the LDCT.

Segmentation of pulmonary lobes from LDCT images is a crucially important step for CAD to identify pulmonary nodules easier. This step aims to eliminate the other structures of non-existent pulmonary nodules, such as the mediastinum, thoracic wall, heart, abdominal organs from the lung. In order to achieve this goal, many researchers have applied various kind of theories to the field of pulmonary lobe segmentation and put forward a number of pulmonary lobe segmentation algorithms. In recent years, conventional lobes segmentation algorithms were based on identifying the three major pulmonary fissures. Previous studies have used anatomical information, including the fixed shape and position of the airways, vessels, and fissures as prior knowledge to generate the final segmentation of pulmonary lobes (3,4). However, the anatomic information of airway, blood vessel, and the fissure is not always reliable, so these methods could not achieve the satisfactory pulmonary segmentation result for all cases.

With the development of computer technology, deep learning has become the most promising technology. From the earliest LeNet (5) to AlexNet (6), VggNet (7), GoogleNet (8), ResNet (9), and the recent DenseNet (10), the performance of CNN models getting stronger and more mature. CNN models were widely used in medical images, include lesion detection, qualitative diagnosis, automatic generation of structured report, lesion extraction, organ delineation for radiotherapy, and so on (11-15).

End-to-end trained fully convolutional networks (FCN) were previously designed to solve image segmentation problems in computer vision (16-17), and has recently been widely applied in medical image segmentation. Gibson et al. (18) proposed a novel deep learning-based algorithm named dense V-network (DenseVNet) to automatic segment multi-organ on abdominal CT. Imran et al. (19) combined the ideas of the DenseVNet and progressive holistically nested networks to obtain a new architecture for pulmonary lobe segmentation on routine chest CT.

Other studies have successfully applied CNN models to their lobe segmentation framework on routine chest CT. Gerard et al. (20) present a pulmonary lobes segmentation pipeline consisting of a series of 3D convolutional neural networks. Xie et al. (21) used a multi-resolution approach with two cascaded CNNs to capture both global context and local details for pulmonary lobes segmentation. With the decrease of radiation dose, the image noise of LDCT was obviously higher than that of routine chest CT. Therefore, pulmonary lobes segmentation on LDCT becomes a laborious task.

This study was aimed to verify the feasibility of pulmonary lobes segmentation in LDCT images using deep learning. We present the following article in accordance with the STARD reporting checklist (available at http://dx.doi.org/10.21037/atm-20-5060).

Methods

Datasets

The data used in the research were LDCT images that were acquired at the opportunistic lung cancer screening cohort in our hospital from October 2018 to February 2019. The LDCT dataset totally contained 160 cases (75 males, 85 females; median age was 56 years). Each LDCT cases was obtained using 64-detector row scanners (GE LightSpeed VCT, GE Medical Systems Optima CT660, and GE Medical Systems Discovery CT750) at full inspiration. CT parameters were as follows: tube voltage, 120 kVp; tube current, 30 mAs or 30–250 mA by using the automatic milliampere technology (GE Medical Systems Discovery CT750), noise index 40; pitch, 0.984; rotation time, 0.5 s; the thin-slice reconstruction thickness was 1.25 mm, and the interval was 0.8 mm using a standard reconstruction algorithm. The scanning range was from the apex of the lung to the diaphragm. LDCT images were saved as the standard DICOM format with a resolution of 512 (width) by 512 (height) pixels.

The LDCT dataset were randomly divided into the training set, the validation set, and the independent test set, which contained 100 cases, 10 cases, and 50 cases respectively. The validation set was used for hyperparameter tuning and detecting for the best model.

Reference standard segmentations

LDCT images were processed with an open-source platform for analyzing medical image data (3D Slicer, version 4.6) (22). The ground truth of pulmonary lobe segmentation was marked by a radiologist (Z Zhang), under the supervision of a senior radiologist (N Wu) with more than 30 years of experience in chest diagnosis. Normally, the right lung is composed of upper, middle, and lower lobe, and the left lung is composed of upper and lower lobe, as shown in . The segmentation procedure included three mainly steps: lace a small number of fiducials on fissures, which were respectively left oblique fissure, right oblique fissure, and right horizontal fissure; select the 3D Gaussian filter with the smooth mode of filtering strength; and create the label map by the “Slow” method in order to achieve more accurate.

Figure 1

Schematic diagram of pulmonary lobes anatomy, showing the lobes and major fissures.

Before input to the network, all the images were preprocessing as follow: (I) resize matrix size to (256, 256, 256); (II) remove intensity value large than 1,000 or less than −1,000. The data augmentation method included random rotation within −10 to 10 degrees and scaling image size within 0.8 to 1.2 mm.

Deep learning-based segmentation network

A fully convolutional DenseVNet was used (18), implemented as part of NiftyNet (23), as shown in . The architecture consists of a downsampling, upsampling, and skip connection components. The network used the dense feature stack for learning and inference, which improvs the processing speed and accuracy. The input to each convolutional layer of the stack was the concatenated outputs from all preceding layers of the stack. The main advantage of this structure is that it significantly reduced parameters and improved performance through gradient propagation and features reuse (10). In the downsampling path, strided convolutional layers reduce the dimensionality of the input feature map and connect dense feature stacks. Single convolutional layers in the skip connections, followed by bilinear upsampling, transform the output feature maps from each stage of the encoder path to the original image size. Each dense feature stack performed 3D convolution (3×3×3) with a learned kernel, used the rectified linear unit (ReLU) nonlinearity as the activation function (24), attached to a batch normalization (BN) (25) and dropout (26) with probability of 0.5. The final layer in the network is a 1×1 convolution followed by a softmax nonlinearity function to predict the class label at each pixel. To train the DenseVNet, a Dice-based loss function (27) was used at each stage of the network. The model was trained on 1,000 epochs, the initial learning rate was set as 0.001, batch size 1, and the Adam optimization method was chosen.

Figure 2

The framework of the proposed DenseVNet for pulmonary lobes segmentation. The network generates feature maps at three different resolutions using a cascade of dense feature stacks and strided convolutional layers. These features maps are concatenated and processed by the subsequent convolutional layers. bilinear upsampling layers transform the feature maps to the original image resolution. BN, batch normalization.

Evaluation metrics

Each pulmonary lobe segmentation was compared to the ground truth segmentation by three metrics, which were described as follows:

where A and B were the algorithm and ground truth segmentations, D (a, b) as the Euclidian distance between boundary pixels of A and B. The Dice coefficient and the Jaccard coefficient are overlap-based metrics methods, which were used for comparing similarities in the volumes between our algorithm and ground truth segmentations of the LDCT cases. The Hausdorff distance presents the maximum distance of a set to the nearest point in the other set (28), reflects agreement between segmentation boundaries. The smaller the Hausdorff distance, the more agreement between segmentation boundaries.

Statistical analysis

The performance of the proposed DenseVNet and the U-net model was evaluated using three critical evaluation metrics: the Dice coefficient, the Jaccard coefficient, and the Hausdorff distance. Evaluation metrics were summarized as average. In the test set, Independent sample t-test was employed to compare the proposed DenseVNet and the U-net model. A P value less than 0.05 was considered significant. Statistical analyses were performed using SPSS 22.0 (SPSS Inc., Chicago, IL, USA).

The Bland Altman method (29) was used to measure the agreement between our algorithm and ground truth of the LDCT cases.

Ethics statement

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This retrospective study was approved by the medical ethics committee of National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College (NO.: 19-018/1840) and written informed consent was waived due to the retrospective nature of the study.

Results

Implementation

The network was implemented by the NiftyNet on the Tensorflow platform. Training and validation of our model were conducted on a local computing workstation, equipped with Intel Xeon CPU, NVIDIA Tesla V100 GPU, 64 GB of RAM. Training time by the proposed method was 20 hours. For reproducibility, we have shared our trained DenseVNet model config on GitHub (https://github.com/RainyRen/LungLobeSeg).

We selected the U-net model as a baseline model, which is a widely used network for biomedical image segmentation (30). The proposed DenseVNet model takes 3.1 seconds to segment lung lobes from one LDCT scan, which is approximately 2 times faster than the U-net model. The number of parameters in the DenseVNet model is only half that of the U-net model. Moreover, the DenseVNet model needed to allocate 6G graphics memory for model training, while the U-net model needed to allocate at least 14G graphics memory for model training.

Experimental results

The LDCT images were acquired from multi-detector computed tomography images. The in-plane spatial resolution was between 0.5 and 1 mm, and the reconstruction thickness was 1.25 mm. The thin-slice reconstruction is now common in clinical diagnosis, particularly for LDCT-based image analysis such as lung nodule detection.

To demonstrate the effectiveness, illustrated examples of segmentation result generated by our segmentation network. The original LDCT images (sagittal and coronal view) without and with overlaid segmentation were presented in parallel for comparison. The three-dimensional rendering of the segmentation result was also provided to give a more vivid understanding of pulmonary lobe segmentation. As demonstrated, the method had successfully separated the pulmonary lobes from the LDCT in all cases.

Figure 3

The DenseVNet segmentation results for one case in sagittal, coronal, and 3D views. Right upper lobe in yellow, right middle lobe in light blue, right lower lobe in purple, left upper lobe in light grey and left lower lobe in dark blue. DenseVNet, dense V-network.

The performance of all three metrics (Dice coefficient, Jaccard coefficient, and Hausdorff distance) on the test set were summarized in . The proposed method achieved the all-lobes Dice coefficient, Jaccard coefficient, and Hausdorff distance were 0.944, 0.896, and 92.908 mm, respectively. The DenseVNet had a higher Dice coefficient and Jaccard coefficient than the U-net, all statistically significant except for the left upper lobe and the left lower lobe. The right middle lobe segmentation was the least accurate for the DenseVNet and the U-net. The Hausdorff distance among pulmonary lobes showed high variability. The DenseVNet had a lower Hausdorff distance significant than the U-net on the right upper lobe and the left lower lobe, but other pulmonary lobes segmentation of the DenseVNet were significantly higher than the U-net.

Table 1

Performance comparison of the proposed DenseVNet and the U-net model on the pulmonary lobe segmentation test set for three metrics

Model	RUL	RML	RLL	LUL	LLL	All-lobes
Dice coefficient (range 0–1)
U-Net	0.920*	0.826*	0.930*	0.936	0.924	0.907*
DenseVNet	0.956*	0.923*	0.963*	0.943	0.935	0.944*
Jaccard coefficient (range 0–1)
U-Net	0.853*	0.711*	0.871*	0.880	0.860	0.835*
DenseVNet	0.916*	0.859*	0.928*	0.895	0.881	0.896*
Hausdorff distance (mm)
U-Net	126.832*	44.950*	72.420*	51.936*	67.118*	72.651*
DenseVNet	67.000*	62.455*	96.712*	198.947*	39.428*	92.908*

*, shows statistically significant finding. RUL, right upper lobe; RML, right middle lobe; RLL, right lower lobe; LUL, left upper lobe; LLL, left lower lobe; DenseVNet, dense V-network.

The result of the proposed segmentation model was very close to ground truth in every plot. The consistency of the left lobes was slightly inferior to that of the right lobes, as shown in .

Figure 4

Bland-Altman plots showed the agreement between the DenseVNet segmentation and ground truth segmentation. Within an interval between two dotted lines, 95% of cases in the dataset fall within the range of the mean ± double standard deviations. DenseVNet, dense V-network.

Discussion

LDCT imaging has become an efficient tool for the detection and following-up of pulmonary nodule. Combined with CAD as the second reader can significantly improve the sensitivity of radiologists to detect pulmonary nodules on CT scans, which can be used as the preferred assisted way for radiologists to detect pulmonary nodules (31). A CAD platform is needed to accurately detect pulmonary nodule and quantify the severity of nodule.

Previously proposed automatic techniques for pulmonary lobe segmentation are sensitive to anatomic and disease variations, reconstruction kernels, and parameter settings, making it frequently requires manual correction by experienced radiologists, which is a barrier to application in a clinical workflow. Deep learning has become a popular method in medical image analysis, which can be automatically, time-saving, and accurately applied to clinical work, so we focused on the deep learning method to segment the pulmonary lobes on LDCT.

FCN is a commonly used deep learning method in the field of computer vision which is proposed to solve image segmentation problems in both natural and medical image analysis. Conventional convolutional neural networks usually used a pooling layer to down-sample the input image. However, the pooling layer will reduce the resolution of output feature maps, resulting in a smaller output feature map. The role of the upsampling layer is to bring back the resolution to the resolution of the previous layer. FCN is achieved by training end-to-end pixel-wise prediction performed by replacing fully connected layers as a series of upsampling layers after convolutional layers, which can prevent segmentation result with a far lower resolution than the original images, and ensure the accuracy of segmentation. Our study applied the DenseVNet on the pulmonary lobes segmentation task and the model can achieve accurate segmentation on LDCT

LDCT images with greater image noise and streak artifacts than routine-dose CT, and some fine anatomical structures (such as airways, vessels, and fissures) as basis of pulmonary lobes segmentation become blurring or undetectable due to the partial volume effect. It is difficult to achieve pulmonary lobes segmentation on LDCT images. There are few studies extended their segmentation method applied to LDCT images.

The average Dice coefficient of the per-lobes and the all-lobes were above 0.92 on the test set, which means the segmentation of pulmonary lobes on LDCT using the DenseVNet was equally performed of the previous study (19) applied on standard-dose CT. This result indicates that DenseVNet can accurately solve the segmentation of pulmonary lobes on LDCT, without interactive manual correction. The automatically generated segmentation of pulmonary lobes has the potential to support disease assessment and treatment planning in lung cancer screening.

Anatomical variation or partial volume effects between cases, especially with the right horizontal fissure, is an obstacle to the segmentation of the right middle lobe (32), Our model obtained good results of the right middle lobe with a Dice coefficient of 0.923, Jaccard coefficient of 0.859, and Hausdorff distance of 62.455 mm on the test set. Due to the LDCT scan without electrocardiographically (ECG)-gated, the consistency of both left lobes was more affected by cardiac motion.

Like several other papers on medical image segmentation, the Dice coefficient was adopted as the main indicator of segmentation algorithm accuracy, and different metrics were used to evaluate the results (33,34). The Dice coefficient and Jaccard coefficient accounted for only the number of correctly or misclassified voxels without reflecting their spatial distribution, so we added the Hausdorff distance to indicate the boundary error of segmentation results. Retrospectively visual inspection of the results revealed that it may achieve satisfying accuracy over most of images. The Hausdorff distance was sensitive to outliers, which was determined solely by the largest distance of boundary error. A small piece of area was misclassified into pulmonary lobes in some test set cases, which may result in an obvious increase in the Hausdorff distance, as shown in . We thought the misclassified area may be due to the Dice-based loss function was used on the proposed DenseVnet. When the loss function reached the minimum during the process of model training, the segmentation boundary has not fully converged.

Figure 5

The DenseVNet segmentation results for three cases. The red arrow indicated misclassified areas. DenseVNet, dense V-network.

There are some limitations to our work. The training set was collected in our hospital, which may not be representative of all lung cancer screening cases in other regions. The generalization capability of our model lacked further validation on multi-center datasets.

Conclusions

Deep learning can achieve the automatic segmentation of pulmonary lobes within lung fields on LDCT images. The all-lobes Dice coefficient of the test set was 0.944, the Jaccard coefficient of the test set was 0.896, and the Hausdorff distance of the test set was 92.908 mm. This deep learning algorithm for pulmonary lobes segmentation will probably benefit new research directions, such as assessing emphysema and pulmonary fibrosis, planning radiotherapy and surgical procedures, for lung cancer LDCT screening in the future.

The article’s supplementary files as