Machine Learning for Assessment of Coronary Artery Disease in Cardiac CT: A Survey.

Cardiac computed tomography (CT) allows rapid visualization of the heart and coronary arteries with high spatial resolution. However, analysis of cardiac CT scans for manifestation of coronary artery disease is time-consuming and challenging. Machine learning (ML) approaches have the potential to address these challenges with high accuracy and consistent performance. In this mini review, we present a survey of the literature on ML-based analysis of coronary artery disease in cardiac CT. We summarize ML methods for detection and characterization of atherosclerotic plaque as well as anatomically and functionally significant coronary artery stenosis.

1. Introduction

Diagnosis and monitoring of coronary artery disease (CAD) is increasingly based on non-invasive imaging with computed tomography (CT), allowing excellent visualization of the coronary arteries with high spatial resolution. Cardiac CT exams consist of hundreds of slices and the number of cardiac CT studies has been steadily increasing (1). This has led to an increased workload for medical professionals, which in combination with shortages of trained cardiac imagers (2) might lead to cardiac CT underuse in the clinic. Machine learning (ML) could offer a way to address these challenges and facilitate automatic cardiac CT analysis with consistent and accurate results. Furthermore, ML algorithms might enable an increased range of secondary diagnoses.

This survey provides an overview of ML algorithms for detection, characterization, and quantification of CAD in cardiac CT. We searched PubMed for articles related to ML-based assessment of CAD in cardiac CT published within the last 10 years (search strategy in Supplementary Materials) which led to inclusion of 59 studies. The structure of this survey is as follows. We provide a brief primer on ML in section 2. Applications of ML for automatic detection and characterization of atherosclerotic plaque are summarized in section 3. Studies focusing on ML for anatomical and functional evaluation of luminal stenosis are summarized in section 4. Finally, section 5 provides a discussion of outstanding challenges for transfer of ML algorithms into the clinic.

2. Machine Learning

Machine learning comes in many flavors, but most applications in cardiac CT use supervised learning. In supervised learning, a model is optimized to provide the correct labels as defined by the reference standard during training, and predict a label to new and unseen samples during testing.

Each sample can be described based on characteristics or features. Among the simplest ML algorithms are k-nearest neighbor (kNN) classifiers, which look for training samples with similar feature values to a test sample, and assign the test sample to the majority class among these training samples. Linear classifier (LC) models like support vector machines (SVM) aim to find a linear combination of features to separate samples in different classes. Alternatively, samples can be separated by thresholding feature values along a single axis. This is unlikely to lead to highly accurate classifiers, but by consecutively applying thresholds, a decision tree model can be built for more accurate classification.

ML performance can often be improved by combining predictions of multiple models. Ensembles (E) combine predictions of multiple simultaneously executed models, e.g., by averaging predictions of decision trees in a random forest (RF). In boosting (BO), models are applied consecutively and each model is trained to correct errors of its predecessors. Finally, artificial neural networks (ANNs) transform samples into targets through layers of trainable neurons, which are loosely based on biological neurons. While ANNs have been around since the 1950s, it has recently become possible to train networks that have many layers, i.e., deep learning. The success of deep learning in medical image analysis has been to a large extent due to the inclusion of trainable image filters in so-called convolutional neural networks (CNNs), which can be trained to extract valuable features from raw image data (3). For a more in-depth introduction to ML and deep learning, please refer to Jordan and Mitchell (4).

3. Atherosclerotic Plaque Detection, Characterization, and Quantification

CT offers a non-invasive alternative to e.g., catheter-guided X-ray angiography, optical coherence tomography, and intravascular ultrasound (IVUS) for atherosclerotic plaque visualization. Characterization and quantification of plaque in CT provide insight in different stages of CAD (5). In this section, we survey analysis of methods for calcified plaque (section 3.1) and non-calcified and mixed plaque (section 3.2). Reviewed papers are listed in Table 1.

Table 1

Publications related to analysis of (A) calcified and (B) non-calcified and mixed atherosclerotic plaque.

(A) Calcified plaque	CSCT	Chest CT	CCTA	Training	Testing	Classifier
de Vos et al. (6)	✓	✓		1,554	1,036	CNN
Cano-Espinosa et al. (7)		✓		4,973	1,000	CNN
Lessmann et al. (8)		✓		1,181	506	CNN
Yang et al. (9)	✓		✓	32	40	SVM
Wolterink et al. (10)			✓	150	100	CNN
Wolterink et al. (11)	✓			384	570	RF
Shahzad et al. (12)	✓			209	157	kNN
Išgum et al. (13)		✓		337	231	kNN, SVM
Sánchez et al. (14)	✓			200	76	kNN
Liu et al. (15)	✓			*	31	SVM
Kurkure et al. (16)	✓			100	105	SVM
Brunner et al. (17)	✓			*	30	SVM
(B) Non-calcified plaque	Detect	Characterize		Training	Testing	Classifier
Kolossváry et al. (18)		✓		*	7	LC
Masuda et al. (19)		✓		*	78	BO
Zreik et al. (20)	✓	✓		98	65	CNN
Zhao et al. (21)	✓	✓		*	18	SVM
Jawaid et al. (22)	✓			*	32	SVM
Wei et al. (23)	✓			*	83	LC
Yamak et al. (24)		✓		-	3	E
Kelm et al. (25)	✓	✓		*	229	RF
Zuluaga et al. (26)	✓			1/13†	14/2	SVM

Check marks in (A) indicate detection (Detect) or characterization (Characterize) of plaque, check marks in (B) indicate analysis in dedicated non-contrast-enhanced calcium scoring CT (CSCT), chest CT (Chest CT) or coronary CT angiography (CCTA) images. The number of patients included for method development (Training) and evaluation (Testing) are listed,

indicates cross-validation and - indicates training on non-patient data. The classifier with which the primary result was obtained is indicated (Classifier, see section 2 for abbreviations).

A total of 15 scans was divided into training sets ranging from 1 to 13 and respective test sets comprised of the remaining scans.

3.1. Calcified Plaque

Coronary artery calcification (CAC) quantification or scoring is typically performed in dedicated non-contrast-enhanced, ECG-triggered, calcium scoring CT images (CSCT). Using dedicated software, an expert identifies voxels with a density over 130 Hounsfield units (HU) in the coronary arteries. Identified CAC is then quantified according to its volume, density, or a combination of both (27). CAC cannot only be quantified in CSCT, but also in other kinds of CT images visualizing the heart, such as cardiac CT angiography (CCTA) and non-gated chest CT. Calcium scoring is not considered a difficult task for trained clinicians, but it is time-consuming when performed in large numbers of images. Hence, automatic ML-based methods have been proposed.

ML-based calcium scoring methods proposed prior to the advent of deep learning have focused on identification of CAC lesions among a large set of samples, i.e., groups of connected voxels above 130 HU. Samples are described with features such as size, shape, appearance and location to distinguish CAC from other candidate lesions such as calcifications in the aorta. Location features are of particular importance, as recognized by Liu et al. (15), Kurkure et al. (16), and Brunner et al. (17) who proposed a heart coordinate system. Similarly, Sánchez et al. (14) described candidate locations relative to anatomical landmarks. Išgum et al. (13) used multi-atlas registration to estimate the location of the coronary artery tree, while Shahzad et al. (12) and Wolterink et al. (11) estimated the location of three major coronary arteries for per-vessel calcium scoring. Yang et al. (9) extracted coronary artery centerlines in CCTA images and propagated these to CSCT images of the same patients to provide location features.

Deep learning-based methods have typically classified individual voxels instead of candidate lesions. Due to the extreme imbalance between numbers of CAC and background voxels in CT images, Wolterink et al. (10) proposed to use two CNNs, where one CNN identified candidate voxels in CCTA and the second CNN further discriminated among identified candidates. Similarly, Lessmann et al. (8) used two CNNs to identify calcified voxels in chest CT. Cano-Espinosa et al. (7) and de Vos et al. (6) avoid voxel-based classification altogether by directly regressing calcium scores in chest CT, enabling automatic scoring in less than a second.

Automatic CAC scoring methods have been validated in large data sets (28) and in other types of CT scans in which the heart is routinely visualized, such as attenuation correction images for PET-CT (29) and CT images acquired for radiotherapy treatment planning (30–32). Wolterink et al. presented a public data set with reference standard for standardized evaluation of CAC scoring in CSCT (33).

3.2. Non-calcified Plaque

Non-calcified plaque is typically lipid-rich and vulnerable to rupture, causing acute coronary syndrome (34). ML-based analysis methods in CCTA have been developed for detection or localization of non-calcified plaque, as well as characterization of lipid and fibrous plaque components.

Coronary artery localization by means of centerline extraction is a typical preprocessing step for ML-based plaque analysis. Traditionally, many automatic centerline extraction methods have been based on minimum cost paths between proximal and distal artery points (35, 36). ML has been used to verify automatic centerline extraction results with an RF (25) or CNN (37). Alternatively, centerlines can be iteratively extracted based on a single seed point. Wolterink et al. (38) showed how such a tracker can be guided by a 3D CNN that locally detects the artery orientation.

Coronary artery centerlines can be used to reconstruct CCTA volumes into images that allow better plaque visualization and identification. Zhao et al. (21), Jawaid et al. (22), Wei et al. (23), and Zuluaga et al. (26) used cross-sectional images along the coronary artery centerline to extract features describing the vessel wall shape and texture. In Jawaid et al. (22) and Wei et al. (23), these features were used in an SVM or linear classifier to determine whether the image contained non-calcified plaque. Similarly, Zuluaga et al. (26) used such features to train an SVM classifying lesion segments as either healthy or diseased, i.e., containing non-calcified or calcified plaque. Zhao et al. (21) trained an SVM to classify cross-sectional images as healthy or containing non-calcified, calcified, or mixed plaque. For the same task, Zreik et al. (20) trained a recurrent CNN that did not depend on hand-crafted feature extraction. Kelm et al. (25) used an RF classifier to classify whether non-calcified or calcified plaque was present along a coronary artery centerline segment.

Characterization of individual components in non-calcified plaque is a challenging task due to low-contrast boundaries between plaque components (39). Yamak et al. (24) exploited additional attenuation data provided by dual-energy CT to characterize plaque in manually determined regions of interest in axial slices. To validate their model in patient scans, manual CCTA annotations by an expert were used. However, obtaining reliable manual reference annotations for non-calcified plaque in CCTA is challenging. Kolossváry et al. (18) determined the reference standard in CCTA through registration of histology images to ex-vivo CCTA scans. Features were extracted for each cross-sectional image and lesions were classified into advanced or early stage atherosclerosis using a linear classifier. Alternatively, Masuda et al. (19) used an in-vivo IVUS-based reference standard to train a boosting classifier with histogram-based features distinguishing fibrous from lipid plaque in CCTA.

4. Coronary Stenosis Detection and Characterization

Non-invasive assessment of CAD-induced stenotic lesions in CT prior to invasive treatment may prevent unnecessary costs and complications (40). Therefore, CT images have long been used to assess the anatomical significance of lesions by a local measurement of luminal narrowing. However, determination of the functional significance of a lesion by taking physiology into account can better stratify patients in need of treatment (41). In this section, we review ML algorithms for the detection and quantification of anatomically (section 4.1) and functionally (section 4.2) significant stenosis. Reviewed papers are listed in Table 2.

Table 2

Publications related to (A) anatomically and (B) functionally significant stenosis detection.

	Structure		Patients
	Artery	Myocardium	Training	Testing	Classifier
(A) Anatomical significance
Lee et al. (42)	✓		412	136	CNN
Freiman et al. (43)	✓		*	90	CNN
Zreik et al. (20)	✓		98	65	CNN
Huang et al. (44)	✓		45	7	CNN
Kang et al. (45)	✓		*	42	SVM
Xiong et al. (46)		✓	*	140	BO
Mukhopadhyay et al. (47)		✓	*	27	ANN
Kelm et al. (25)	✓		*	229	RF
Zuluaga et al. (48)	✓		9	9	SVM
(B) Functional significance
Kumamaru et al. (49)	✓		*	131	CNN
Wang et al. (50)	✓		8	63	ANN
Hae et al. (51)		✓	932	279	BO
Dey et al. (52)	✓		*	254	BO
Zreik et al. (53)		✓	*	166	SVM
Han et al. (54)		✓	*	252	BO
Itu et al. (55)	✓		-	87	ANN

Check marks indicate arterial (Artery) or myocardial (Myocardium) analysis. The number of patients included for method development (Training) and evaluation (Testing) are listed,

indicates cross-validation and - indicates training on non-patient data. The classifier with which the primary result was obtained is indicated (Classifier, see section 2 for abbreviations).

4.1. Anatomical Significance

Identification of anatomically significant stenotic lesions in CCTA, i.e., those lesions causing a luminal narrowing of at least 50%, allows a first assessment of the severity of stenosis in patients with symptoms of CAD. While this assessment is often based on visual estimation by a clinician, this is a difficult task (56) with substantial inter-observer variability (57). ML-based automatic approaches could reduce this variability.

Stenosis detection typically requires a local measurement of the lumen diameter and an estimation of the healthy lumen diameter. These estimates can be based on automatically extracted centerlines (section 3.2). Many centerline extraction methods also estimate the luminal radius at each centerline point, assuming a circular coronary artery profile (25, 38). However, circular artery profiles are not a realistic assumption for diseased vessel segments. Automatically extracted centerlines can also be used as an initialization for more detailed lumen segmentation. Huang et al. (44) used centerlines to obtain a reformatted image in which the lumen was segmented using a 3D CNN. Lee et al. (42) use centerlines to obtain a tube-shaped prior that is deformed to segment the coronary lumen.

Lumen segmentation is often considered a preprocessing step for stenosis detection, but it has been shown that stenosis degree can also be directly determined based on image data. Zuluaga et al. (48) detected stenosis and artery bifurcations with an SVM based on features obtained from concentric circles in cross-sectional images. Similarly, Kang et al. (45) used geometrical and plaque features in an SVM to detect obstructive lesions (> 50% narrowing) and non-obstructive lesions (25–50% narrowing). Zreik et al. (20) used a recurrent CNN to detect anatomically significant stenosis along the centerline. Freiman et al. (43) detected stenosis of at least intermediate severity (> 40% narrowing) using deep sparse autoencoders, a variation on CNNs.

Coronary stenoses are located in the arteries, but may restrict blood flow to myocardial segments. Mukhopadhyay et al. (47) used an ML approach to identify myocardial segments (58) affected by coronary stenosis. Hand-crafted feature vectors describing the endocardial surface shape were combined using a bag-of-words approach and classified with an ANN to identify affected segments. Xiong et al. (46) performed analysis of the full myocardium to detect existence of at least one anatomically significant stenosis. Instead of the shape of the endocardial surface, features in this approach described the attenuation and wall thickness of myocardial segments.

4.2. Functional Significance

The sensitivity of CCTA-based anatomical stenosis evaluation for detection of functionally significant stenosis is high when evaluated visually, but its specificity is moderate (41). The current reference standard for determination of functional significance of a stenosis is given by its fractional flow reserve (FFR), i.e., the ratio of flow distal of the stenosis to the flow proximal of the stenosis. FFR is measured invasively by inserting a special catheter in the coronary artery under hyperemic conditions. FFR below 0.80 indicates need for intervention (59). Treatment based on invasive FFR measurements can improve patient outcomes (59), but measurement of FFR is still relatively uncommon, which is due to associated cost and risk, as well as lack of vasodilator drugs (60).

FFR estimation based on CCTA scans (FFRCT) could provide reproducible physical measurement without the drawbacks of invasive procedures. FFRCT has traditionally been based on computational fluid dynamics (CFD) (61, 62), i.e., numerical simulation of blood flow in a coronary tree model extracted from CCTA using lumen segmentation methods (section 4.1). These methods are accurate (63) but computationally expensive due to their iterative nature. This precludes their deployment on local workstations, and instead CFD simulations are typically performed on off-site dedicated systems. ML could be used to significantly speed up estimation of FFRCT.

Itu et al. (55) proposed an ANN model to predict an FFR value for each segment in the coronary artery tree, given local features based on the segment's geometry and global features based on the most severe stenoses. To train this model, a large data set of 12,000 synthetic coronary artery trees was generated and a reference standard was obtained through conventional CFD simulation. By only performing CFD simulations once in a training phase, the time required to perform FFRCT was reduced by two orders of magnitude. The diagnostic value of this method has been demonstrated thoroughly (64–76). Yu et al. (77) further demonstrated additional prognostic value of CT morphological index for the method proposed by Itu et al. (55). Wang et al. (50) proposed to use a recurrent ANN that can model long-range dependencies between segments.

Both conventional CFD-based FFRCT and the methods proposed in Wang et al. (50) and Itu et al. (55) are based only on the geometry of the coronary artery tree model, and are thus susceptible to errors by the segmentation method used to obtain this model (78). Instead, Dey et al. (52) proposed to combine geometric features with semi-automatically obtained plaque and attenuation gradient measurements to identify arteries with functionally significant stenosis. Other methods skip explicit coronary artery centerline extraction and lumen segmentation altogether. Kumamaru et al. (49) trained a CNN to extract a map showing the contrast-enhanced territories in CCTA and used this map in a classifier to predict the minimum FFR value in a patient. Alternatively, analysis can be moved from the cause—the coronary arteries—to the effect, i.e., the myocardium. Han et al. (54) subdivided the separated endocardium and epicardium into the American Heart Association (AHA) 17 segments (58), with 3 features per segment characterizing perfusion and wall thickness. However, the trained boosting classifier showed only moderate accuracy for patientwise prediction of abnormal FFR values. For the same purpose, Zreik et al. (53) trained an SVM based on features from myocardial regions extracted from CCTA. Clinical evaluation of this method yielded improved diagnostic accuracy of FFRCT over visual evaluation of stenosis (79). Hae et al. (51) increased accuracy of FFR-prediction by including the tissue volume subtended to a stenotic lesion in analysis. However, determination of lesion position required additional analysis including artery tree segmentation.

5. Discussion

We have presented a survey of applications of ML for detection, characterization and quantification of atherosclerotic plaque and stenosis in cardiac CT. We found that while ML has been a mainstay of cardiac image analysis for years, the recent emergence of deep learning has accelerated progress in the field. Machine learning has the potential to unburden clinicians from time-consuming tasks and change diagnostic procedures, thereby reducing healthcare costs. Moreover, low-cost ML-based analysis could be added to screening studies as a secondary goal. In this survey, we have focused on ML for CAD analysis. For a broader scope the reader is referred to Al'Aref et al. (80), Litjens et al. (81), Nicol et al. (82), Petersen et al. (83), and Singh et al. (84).

We have reviewed plaque and stenosis analysis methods in separate sections, but formation of plaque and stenosis is naturally related and many papers have proposed simultaneous analysis [e.g., (48, 53)]. Moreover, (semi-)automatic identification of plaque or stenosis is often only an intermediate step for prediction of cardiovascular events. Motwani et al. (85) used stenosis scores and plaque characteristics to develop a model for 5 years all-cause mortality prediction. Similarly, Johnson et al. (86) showed that an ML model taking into account per segment coronary artery characteristics can outperform hand-crafted models for prediction of adverse cardiac events. Van Rosendael et al. (87) developed a model for all-cause mortality prediction in combination with future myocardial infarction based only on hand-crafted features derived from CCTA scans. Furthermore, some methods directly predict presence of CAD from medical images, i.e., chest CT (88) or non-contrast-enhanced cardiac CT (89). While these approaches only require one label per patient and large data sets are thus not expensive to obtain, the interpretability of predictions may be limited. Interpretability might constitute an opportunity, not only to improve reliability but also as it might increase medical knowledge by quantifying the diagnostic relevance of underlying phenomena.

The readiness of automatic analysis methods for clinical implementation depends on the complexity of the task, but also on other factors. ML algorithms require large training sets, and tasks with abundant data may be easier to automate. For example, obtaining a ground truth for e.g., non-calcified plaque characterization is very challenging. Therefore, data sets are generally small and ML algorithms remain at an early developmental stage. In contrast, large data sets are available for the development of ML-based CAC scoring methods, which has led to highly accurate results in both dedicated cardiac CT images (11) and other CT images visualizing the heart (8, 32). Similarly, ML-based FFRCT development is aided by the availability of large data sets with CFD-derived reference values. An important remaining step toward clinical application of FFRCT lies in performance evaluation specifically for subjects around the FFR threshold of 0.8, which were shown to be most challenging (90). Furthermore, a recent study showed that not all CCTA exams are suitable for FFRCT analysis (78).

Many challenges in the adoption of machine learning methods in the clinic are not exclusive to CAD detection in cardiac CT. For example, ML algorithms could show unexpected behavior, motivating research into ML interpretability and explainability (91). Furthermore, it is important to point out that ML algorithms are often trained and evaluated on single center studies with high risk for selective biases, and under exclusion of low quality scans.

Despite these challenges, current rapid development allows for justifiable hope that the importance of ML algorithms in cardiac CT will not cease to increase in near future, with benefits for clinicians and patients alike.

Author Contributions

NH and SV drafted the manuscript, which was critically revised and edited by JW, TL, and II. Authors agree to be accountable for all aspects of the work.

Conflict of Interest

II and TL received institutional research projects by Dutch Technology Foundation cofunded by Pie Medical Imaging and Philips Healthcare (P15-26), the Netherlands Organisation for Health Research and Development with participation of Pie Medical Imaging (104003009). II received institutional research project by Dutch Technology Foundation cofunded by Pie Medical Imaging (12726). II and TL are cofounders and shareholders of Quantib-U BV. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.