Automated Segmentation of Left Ventricular Myocardium on Cardiac Computed Tomography Using Deep Learning.

OBJECTIVE: To evaluate the accuracy of a deep learning-based automated segmentation of the left ventricle (LV) myocardium using cardiac CT.
MATERIALS AND METHODS: To develop a fully automated algorithm, 100 subjects with coronary artery disease were randomly selected as a development set (50 training / 20 validation / 30 internal test). An experienced cardiac radiologist generated the manual segmentation of the development set. The trained model was evaluated using 1000 validation set generated by an experienced technician. Visual assessment was performed to compare the manual and automatic segmentations. In a quantitative analysis, sensitivity and specificity were calculated according to the number of pixels where two three-dimensional masks of the manual and deep learning segmentations overlapped. Similarity indices, such as the Dice similarity coefficient (DSC), were used to evaluate the margin of each segmented masks.
RESULTS: The sensitivity and specificity of automated segmentation for each segment (1-16 segments) were high (85.5-100.0%). The DSC was 88.3 ± 6.2%. Among randomly selected 100 cases, all manual segmentation and deep learning masks for visual analysis were classified as very accurate to mostly accurate and there were no inaccurate cases (manual vs. deep learning: very accurate, 31 vs. 53; accurate, 64 vs. 39; mostly accurate, 15 vs. 8). The number of very accurate cases for deep learning masks was greater than that for manually segmented masks.
CONCLUSION: We present deep learning-based automatic segmentation of the LV myocardium and the results are comparable to manual segmentation data with high sensitivity, specificity, and high similarity scores.

INTRODUCTION

The delineation of left myocardial boundaries is important for the evaluation of myocardial wall thickening and obtaining functional data. Manual segmentation of the left ventricle (LV) is time-intensive (20–30 minutes); thus, semi-automated threshold-based segmentation software has been developed. However, the task remains time consuming, and intra- and inter-observer variability are not well demonstrated (12). Therefore, highly accurate automatic LV wall segmentation methods have been investigated using computed tomography (CT), magnetic resonance imaging (MRI), and echocardiography (234567). Currently, cardiac CT is widely used to detect coronary artery and valvular heart diseases, as well as assess LV wall thickness and cardiac function. Fully automated segmentation algorithms using CT have been investigated in previous studies (78910111213). These approaches used model-based methods. The LV can be seen as a donut with the same center. The Hough transform converts the location of each point to have a center and radius. Points in the LV have the same center and similar radii. The parameters of the Hough transform are related to the values of the center and radius. By using these parameters, we can extract the LV region. However, it is difficult to select the exact threshold values of these parameters. Other studies incorporated more adaptive selection methods for these parameters. Algorithms using a generalized Hough transform (1113) or parametric adaptation (81011) are useful in extracting cardiac structures, and clinical studies have shown that such algorithms are robust and can help realize accurate segmentation of the cardiac chambers.

Machine learning (ML) algorithms have also been proposed to perform automatic segmentation of the LV. A recent study of cardiac MRI demonstrated 98.7% accuracy in LV segmentation using convolutional neural network (CNN)-based localization (6). In this study, we aimed to develop a deep learning-based automated LV segmentation system using cardiac CT and evaluate the accuracy of the algorithm.

MATERIALS AND METHODS

Dataset

This study was conducted under approval of the hospital Institutional Review Board (approval number: 2018-0583) and the need for informed consent was waived due to the study's retrospective nature. In this study, 1109 patients with suspected coronary artery disease who were included in two previously published studies for the quantitative evaluation of coronary angiography (1415) were selected randomly. After excluding nine cases with image noise and low resolution (n = 4) or ML errors (n = 5), a dataset (multiphase cardiac CT with whole three-dimensional [3D] volume images of the LV) of 1100 subjects (median age, 63.0 years; interquartile range, 55.3–69.0 years; male:female = 839:261) with coronary artery disease was included. The CT data for 1100 subjects were obtained during the end-diastolic phase, which has the least motion artifacts.

First, to develop a fully automated deep learning algorithm using semantic segmentation methods based on a fully convolutional network (FCN), 50 subjects were selected randomly as a training set (Fig. 1). An experienced cardiac radiologist generated the manual segmentations of training set data. Twenty subjects were used in the validation set, and 30 cases were used as the internal test set. For external validation, automatic two-dimensional (2D) LV segmentation was performed in 1000 subjects using a model trained on the development set. Manual segmentation of LV was performed independently on multi-slice 2D CT images for these 1000 subjects by an experienced technician. That 2D stack of images was used to generate a 3D view of each mask. We then compared the manually drawn masks and masks obtained using ML.

Fig. 1

Diagram demonstrating use of data.

LV = left ventricle, ROI = region of interest

In addition, automatic segmentation was performed in 100 randomly selected subjects with end-systolic phase data through ML using the model trained on the development set of 100 CT images obtained in the diastolic phase. The 100 masks obtained via ML were evaluated by an experienced technician and corrected manually, if the masks were drawn incorrectly. The similarities of the two systolic mask sets, ML masks, and manually evaluated masks were then compared. The LV volumes and ejection fractions were calculated for the 100 systolic-diastolic paired cases.

CT Acquisition

Using a second-generation dual source CT scanner (Somatom Definition Flash, Siemens Healthineers, Erlangen, Germany), electrocardiography (ECG)-gated cardiac CT scanning was performed. A retrospective ECG-gated spiral scan with ECG-based tube current modulation was applied to multiphase of 0–90% of the R-R interval, which comprises a full dose pulsing window of 30–80% of the R-R interval. The tube current was reduced to 20% of the maximum outside the ECG pulsing window (16). A bolus of 70–90 mL of contrast material (iomeprol; Iomeron 400, Bracco Imaging S.p.A, Milan, Italy) was administered by a power injector (Stellant D, Medrad, Indianola, PA, USA) at 4.0 mL/s followed by 40 mL of saline. The bolus tracking method (region of interest, ascending aorta; attenuation threshold level: 100 HU; scan delay: 8 seconds) was employed to determine the scan time. In all CT scans, the tube voltage and tube current–exposure time product were adjusted according to each patient's body size. Scan parameters were as follows: tube voltage: 80, 100, or 120 kV; tube current–exposure time product: 185–380 mAs; collimation: 128 × 0.6 mm; and gantry rotation time: 280 seconds. The tube voltage and tube current–exposure time product were adjusted based on the patient's body size. The mean effective radiation dose of coronary computed tomography angiography was 9.3 ± 5.8 mSv. The detailed CT acquisition protocol is described in Supplementary Table 1. Note that a standard cardiac filter (B26f) was used for imaging reconstruction.

Manual Segmentation

Manual drawing was performed using the in-house software (A-view Cardiac, Asan Medical Center, Seoul, Korea) (Fig. 2). After selecting the center of the image, the image was rotated to demonstrate a four- or five-chamber view. The center of the LV cavity, basal myocardial border or mitral annulus, and endomyocardial and epimyocardial borders of the LV apex were selected as reference points. The mask was then generated semi-automatically by interpolation. Annotations were drawn in the short axis view of the heart by excluding the papillary muscles, trabeculae, LV cavity blood pool, and epicardial fat. Note that manual correction was performed if the generated mask did not correctly demarcate the LV margin. The drawn contours were propagated to other slices and adjusted when the drawn points were removed or newly placed.

Fig. 2

Semi-automatic manual LV segmentation method.

CNN Architecture and Training

It is well known that the CNN architecture, which is composed of multiple convolutional and pooling layers, is effective in image classification. In contrast to image classification, semantic segmentation provides not only the label of the object in the image, but also a pixel-by-pixel location of the given object. This semantic segmentation can be implemented using the CNN architecture by changing the fully connected layer to a fully convolutional one.

To mimic the manual segmentation procedure, we decomposed the short-axis view volume into multiple 2D slice images. For 2D segmentation, we used a FCN (17) with pretrained weights from VGG16 trained on the ImageNet database (18). We applied skip connections to the FCN-VGG16 model that combined hierarchical features from convolutional layers with different scales. By including skip connections, the model was able to fuse three predictions at 8-, 16-, and 32-pixel strides to refine the spatial precision of the output. We refer to this neural network model as the FCN-all-at-once-VGG16 model, and have used it for the proposed segmentation method. The schema for demonstrating FCN-all-at-once-VGG16 is shown in Figure 3. As a preprocessing step, the slice images were resampled to 256 × 256 pixels and converted to the red, green, blue (RGB) color format. The central image and nearby images with different displacements were merged into a single RGB image using three displacement values, i.e., 0, 1, and 2 slices. The FCN-all-at-once-VGG16 model was trained on each displacement setting using the preprocessed image pairs (i.e., the 24-bit RGB color image and 8-bit gray mask) from the training dataset. The three extracted masks were fused to generate a single resulting mask using the majority voting algorithm. These resulting masks were resampled to the original size and stacked to create a 3D volume. Here, we used a minibatch size of a single image and Adam optimization with initial learning rates of 10−4. The algorithm and an example case are available in a repository (https://github.com/leegaeun/Cardiac_Segmentation).

Fig. 3

Fully convolutional neural network architecture.

A. RGB data generation at various displacements. B. Results for each input image. FCN = fully convolutional network, ReLU = rectified linear unit, RGB = red, green, blue

Statistical Analysis

Various analytical methods were used to compare the manually drawn masks and ML-derived masks. The quantitative analysis involved calculating sensitivity and specificity according to the number of pixels where the two 3D masks of manual and ML segmentation overlap. True positive was defined as the number of pixels where the two masks overlapped. True negative was defined as the number of pixels not included in either mask. False positive was defined as the number of pixels in the ML masks that were not included in the manual mask, and false negative was defined as the number of pixels in the manual mask that were not included in the ML mask. However, it should be noted that this method is limited when the number of pixels indicative of true negative (i.e., image pixels other than the LV mask) is large. Therefore, we employed similarity algorithms, including the Dice similarity coefficient (DSC), Jaccard similarity coefficient (JSC), mean surface distance (MSD), and Hausdorff surface distance (HSD). The gold standard for the similarity analyses was manual segmentation. The four similarity algorithm formulas and diagrams to explain the four margin analysis methods are described in Supplementary Figure 1.

Additionally, the automatic segmentation of the LV myocardium was assessed qualitatively based on a 17-segment model, i.e., visually, from 100 randomly selected cases. The 2D slice short axis views of the apical, mid, and basal myocardial segments, as well as two-, three-, and four-chamber views, were assessed. We assigned “very accurate” if all contours of the LV wall were correctly demarcated. If one segment, e.g., the basal anteroseptal wall, was inaccurately drawn, the case was classified as “accurate.” If two segments were inaccurately drawn, the case was classified as “mostly accurate,” and if more than three segments were inaccurately drawn, the case was considered “inaccurate.”

RESULTS

Overall, automated segmentation data were comparable to manual segmentation data. The accuracy of ML for demarcation of the entire LV myocardium was 88.3%, and sensitivity and specificity were high (91.2% and 99.7%, respectively). The accuracies, sensitivities, and specificities of ML for demarcating apical, mid, and basal walls were similar. The sensitivity of ML was lowest for the basal wall (89.6%) (Table 1).

Table 1

LV Myocardium Segmentation Performance of Machine Learning

	Accuracy (%)	Sensitivity (%)	Specificity (%)
LV whole myocardium	88.3	91.2	99.7
Apical wall	88.6	93.7	100.0
Mid-wall	89.4	93.2	99.9
Basal wall	89.2	89.6	100.0

LV = left ventricle

The mean DSC between manual segmentation and ML masks was 88.3 ± 6.2% (minimum 78.1% and maximum 96.5%). The mean JSC was 79.5 ± 7.0%, MSD was 1.0 ± 2.4 mm, and HSD was 13.4 ± 12.2 mm (Table 2). The per-segment results for ML similarity coefficients and segmentation performance are listed in Table 3. The DSC in each segment ranged 85.6–91.4%. The sensitivity and specificity values for automated segmentation in each segment (1–16 segments) were also high (85.5–100.0%). Compared to other segments, DSCs were lower at the basal inferoseptal (85.6%) and apical inferior (85.9%) walls. Sensitivity was also lower at the basal inferoseptal walls (85.5%).

Table 2

Similarity Coefficient Results

Metrics	Mean ± SD	Explanation
DSC (%)	88.3 ± 6.2	100, when two masks are same
JSC (%)	79.5 ± 7.0	100, when two masks are same
MSD (mm)	1.0 ± 2.4	MSD between two masks
HSD (mm)	13.4 ± 12.2	Maximum surface distance between two masks

DSC = Dice similarity coefficient, HSD = Hausdorff surface distance, JSC = Jaccard similarity coefficient, MSD = mean surface distance, SD = standard deviation

Table 3

Machine Learning Per-Segment Similarity Coefficient and Segmentation Performance Results

Segment	1	2	3	4	5	6	7	8
DSC (%)	89.4	87.7	85.6	87.5	91.0	91.4	90.4	89.9
Sensitivity (%)	88.7	89.1	85.5	86.0	93.6	93.4	94.0	93.9
Specificity (%)	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0

The LV blood volumes were automatically computed by using morphological operations from the automatic and manual LV segmentation masks. The example case of blood volume calculation is shown in Supplementary Figures 2 and 3. The comparison results of the similarity coefficients, LV volumes, and ejection fraction are given in Supplementary Table 2. DSC and JSC in the systolic phase data were higher than those in the paired diastolic phase data were, and the ejection fraction values were different in 6.4 ± 9.2% between the automatic segmentation and manual segmentations.

In 100 cases selected from the 1000 diastolic phase data cases, all manual segmentation and ML masks for visual analysis were classified as very accurate to mostly accurate, and there were no inaccurate cases (Table 4). Interestingly, the number of very accurate cases in the ML masks was larger than that for manually segmented masks (53 vs. 31). Most cases that indicate the superior performance of ML were detected due to mistakenly drawn manual masks of the mitral annulus in the basal myocardium (Fig. 4).

Table 4

Qualitative Results from Visual Assessment of Automatic Segmentation of LV Myocardium from 100 Randomly Selected Cases

Grade	Manual	Deep Learning
1-very accurate	31	53
2-accurate	64	39
3-mostly accurate	15	8
4-inaccurate	0	0

Fig. 4

Segmentation examples.

A. Example of superior manual segmentation performance. Machine learning-selected mask includes coronary sinus (arrow) that should not be included as part of LV myocardium. B. Example of superior performance of machine learning segmentation. Right atrium is incidentally included in manual segmentation of LV myocardium (arrows). However, machine learning segmentation clearly demarcates border of basal myocardium (arrowheads).

DISCUSSION

We developed a new automatic LV segmentation algorithm by using a CNN. The scheme is based on 2.5 dimensions, which includes the central slice along with nearby slices. We incorporated the transfer learning strategy in which the parameters were initiated from the trained weights on ImageNet (18) and fine-tuned on our dataset. This strategy could permit a reduction in the number of training cases and achieve better performance.

We evaluated the similarity and segmentation performance of ML relative to manual segmentation of the LV of the heart and showed comparable performance of ML using a large data set. First, with ML, the DSC values were high, ranging from 85.9–94.0%, and the sensitivity and specificity values for the ML masks were very high.

LV function is an important predictor of outcomes for patients with myocardial dysfunction, and ventricular volume and myocardial mass are associated with myocardial diseases such as infiltrative diseases, hypertrophic cardiomyopathy, or ischemic cardiomyopathy due to coronary artery disease (1920). Accurate assessment of LV segmentation is essential for clinical purposes, and reproducible data are required to resolve the manual segmentation issue.

Cardiac CT data with retrospective ECG gating can be used for LV function analysis and the LV functional parameters showed good agreement with cine MRI and echocardiography (5212223242526272829). Cardiac CT provides morphologic and functional data when CT is used to evaluate coronary artery or heart valve disease, although CT has not been used as a first-line modality to assess LV function. Previous studies (5730) have reported that assessment of LV function using cardiac CT, with or without the use of semi-automated software tools, was feasible and reliable. However, the processing time with manual or commercial software tools requires significant time, and software tools sometimes do not optimally delineate the endo- and epi-myocardial borders. To include LV function analysis in routine clinical practice, reproducibility and time saving are crucial issues. Using cardiac CT data, retrospective studies to validate automatic ML segmentation in selective cardiac phases are feasible. Several previous studies have shown the usefulness of automatic ventricular segmentation in dozens of patients (79), and fully automated segmentation reduces the post-processing time to approximately one minute. In our study, we confirmed the performance of ML segmentation in a large patient population.

Moreover, during qualitative assessment, we observed that the number of very accurate cases was greater for ML masks than for manually segmented masks. This finding suggests that although manual segmentation is used as a reference standard in our study, there are more accurately drawn masks in the ML sets. This finding can be explained from the fact that the similarity coefficients of the ML and manually segmented masks were more similar for systolic phase datasets, which were initially generated automatically by ML and then modified based on visual assessment.

This work has several limitations. First, the segmentation method is applied to the transformed data to the view along the short axis, and the user performed the transformation manually. This makes the performance of the algorithm less robust. Second, in our study, there were five cases with ML errors. Among them, the fields-of-view of the CT data in three cases was quite different from that in the others. In these cases, the LVs were not located in the middle and the short-axis was not aligned well. Another case had motion blurring. The last one showed lower contrast enhancement in the LV region. The developed automatic segmentation algorithm had very low performance (DSC < 0.2) for these cases. Regarding future works, we are considering using 3D convolution to overcome the first limitation and to incorporate image augmentation scheme to address the second.

Using a large data set, we investigated deep learning-based automatic segmentation of the LV of the heart. The automated segmentation results were comparable to those provided by manual segmentation with a high Dice index. Based on visual analysis, automated LV segmentation using deep learning is superior to semi-automatic segmentation performed by an experienced technician. Compared to manual segmentation, automated LV segmentation can reduce the time required to obtain information regarding myocardial wall thickening and LV function, and can also improve the reproducibility of clinical assessments.