Radiomic Features Are Superior to Conventional Quantitative Computed Tomographic Metrics to Identify Coronary Plaques With Napkin-Ring Sign.

BACKGROUND: Napkin-ring sign (NRS) is an independent prognostic imaging marker of major adverse cardiac events. However, identification of NRS is challenging because of its qualitative nature. Radiomics is the process of extracting thousands of quantitative parameters from medical images to create big-data data sets that can identify distinct patterns in radiological images. Therefore, we sought to determine whether radiomic analysis improves the identification of NRS plaques. METHODS AND
RESULTS: From 2674 patients referred to coronary computed tomographic angiography caused by stable chest pain, expert readers identified 30 patients with NRS plaques and matched these with 30 non-NRS plaques with similar degree of calcification, luminal obstruction, localization, and imaging parameters. All plaques were segmented manually, and image data information was analyzed using Radiomics Image Analysis package for the presence of 8 conventional and 4440 radiomic parameters. We used the permutation test of symmetry to assess differences between NRS and non-NRS plaques, whereas we calculated receiver-operating characteristics' area under the curve values to evaluate diagnostic accuracy. Bonferroni-corrected P<0.0012 was considered significant. None of the conventional quantitative parameters but 20.6% (916/4440) of radiomic features were significantly different between NRS and non-NRS plaques. Almost half of these (418/916) reached an area under the curve value >0.80. Short- and long-run low gray-level emphasis and surface ratio of high attenuation voxels to total surface had the highest area under the curve values (0.918; 0.894 and 0.890, respectively).
CONCLUSIONS: A large number of radiomic features are different between NRS and non-NRS plaques and exhibit excellent discriminatory value.

Coronary computed tomographic (CT) angiography is a robust noninvasive imaging modality that can visualize the coronary lumen and the atherosclerotic changes of the vessel wall.[1] Four distinct plaque characteristics have been linked to major adverse cardiovascular events using coronary CT angiography.[2] Out of these 4 characteristics, positive remodeling, low attenuation, and spotty calcification are quantitative high-risk plaque features. The napkin-ring sign (NRS) is defined as a plaque cross-section with a central area of low CT attenuation apparently in contact with the lumen, which is surrounded by a ring-shaped higher attenuation plaque tissue.[3] Because of its qualitative nature, identification of the NRS is affected by clinical experience and inter-reader variability.[4]

See Editorial by

See

Radiological images are multidimensional data sets, where each voxel value represents a specific measurement based on some physical characteristic.[5] Radiomics is the process of obtaining quantitative parameters from these spatial data sets, to create big-data data sets, where each lesion is characterized by hundreds of different parameters.[6] These features aim to quantify morphological characteristics difficult or impossible to comprehend by visual assessment.[7]

Radiomics has proven to be a valuable tool in oncology.[8] Several studies have shown radiomics to improve the diagnostic accuracy,[9,10] staging and grading of cancer,[11] response assessment to treatment,[12-14] and also to predict clinical outcomes.[15,16] However, up until today, there is no data available on radiomics-based analysis of coronary plaques. Coronary atherosclerotic lesions are smaller than tumors and have complex geometric shapes, which might pose a challenge for radiomic feature analysis. Therefore, we sought to assess whether calculation of radiomic features is feasible on coronary lesions. Furthermore, we aimed to evaluate whether radiomic parameters can differentiate between plaques with or without NRS.

Methods

Institutional review board approved the study (SE TUKEB 1/2017) and because of the retrospective study design informed consent was waived. The data and study materials will not be made available to other researchers for purposes of reproducing the results or replicating the procedure because of intellectual property rights and patient confidentiality. However, we made our analysis software open source and freely accessible for other researchers.[17]

Study Design and Population

From 2674 consecutive coronary CT angiography examinations because of stable chest pain, we retrospectively identified 39 patients who had NRS plaques. Two expert readers reevaluated the scans with NRS plaques. To minimize potential variations because of inter-reader variability, the presence of NRS was assessed using consensus read. Readers excluded 7 patients because of insufficient image quality and 2 patients because of the lack of the NRS; therefore, 30 coronary plaques of 30 patients (NRS group; mean age: 63.07 years; interquartile range [IQR], 56.54–68.36; 20% female) were included in our analysis. As a control group, we retrospectively matched 30 plaques of 30 patients (non-NRS group; mean age: 63.96 years; IQR, 54.73–72.13; 33% female) from our clinical database with excellent image quality. To maximize similarity between the NRS and the non-NRS plaques and minimize parameters potentially influencing radiomic features, we matched the non-NRS group based on degree of calcification and stenosis, plaque localization, tube voltage, and image reconstruction. Detailed patient and scan characteristics are summarized in Table 1, whereas detailed description of scan characteristics and image quality measurements are described in Methods 1 section of the Data Supplement.

Table 1.

Patient Characteristics and Scan Parameters

Traditional Plaque Characteristics

All plaques were graded for luminal stenosis (minimal 1% to 24%; mild 25% to 49%; moderate 50% to 69%; severe 70% to 99%) and degree of calcification (calcified; partially calcified; noncalcified). Furthermore, plaques were classified as having low attenuation if the plaque cross-section contained any voxel with <30 Hounsfield unit and having spotty calcification if a <3-mm calcified plaque component was visible. Detailed plaque and imaging information is shown in Table 2.

Table 2.

Plaque and Image Quality Characteristics

Image Segmentation, Conventional Quantitative Metrics, and Data Extraction

Image segmentation and data extraction was performed using a dedicated software tool for automated plaque assessment (QAngioCT Research Edition; Medis Medical Imaging Systems B.V., Leiden, The Netherlands). After automated segmentation of the coronary tree, the proximal and distal ends of each plaque were set manually. Automatic lumen and vessel contours were manually edited by an expert if needed.[18] From the segmented data sets, 8 conventional quantitative metrics (lesion length, area stenosis, mean plaque burden, lesion volume, remodeling index, mean plaque attenuation, and minimal and maximal plaque attenuation) were calculated by the software. The voxels containing the plaque tissue were exported as a DICOM data set using a dedicated software tool (QAngioCT 3D Workbench; Medis Medical Imaging Systems B.V.). Smoothing or interpolation of the original Hounsfield unit values was not performed. Representative examples of volume-rendered and cross-sectional images of NRS and non-NRS plaques are shown in Figure 1.

Figure 1.

Representative images of plaques with or without the napkin-ring sign (NRS). Volume-rendered and cross-sectional images of plaques with NRS in the top (A, C, and E) and their corresponding matched plaques in the bottom (B, D, and E) are shown. Green dashed lines indicate the location of cross-sectional planes. Colors indicate different computed tomographic attenuation values. NCP indicates noncalcified plaque.

Calculation of Radiomic Features

We developed an open-source software package in the R programming environment (Radiomics Image Analysis), which is capable of calculating hundreds of different radiomic parameters on 2- and 3-dimensional data sets.[17] We calculated 4440 radiomic features for each coronary plaque using the Radiomics Image Analysis software tool. Detailed description on how radiomic features were calculated can be found in the Methods 1 section of the Data Supplement, whereas a detailed description of the calculated statistical parameters can be found in the Methods 2 section of the Data Supplement.

Statistical Analysis

Binary variables are presented as frequencies and percentages, whereas ordinal and continuous variables are presented as medians and IQRs because of possible violations of the normality assumption. For robust statistical estimates, parameters between the NRS and the non-NRS groups were compared using the permutation test of symmetry for matched samples using conditional Monte Carlo simulations with 10 000 replicas.[19] For diagnostic performance estimates, we conducted receiver-operating characteristics analysis and calculated area under the curve (AUC) with bootstrapped confidence interval values using 10 000 samples with replacement and calculated sensitivity, specificity, and positive and negative predictive values by maximizing the Youden index.[20] To assess potential clusters among radiomic parameters, we conducted linear regression analysis between all pairs of the calculated 4440 radiomic metrics. The 1−R2 value between each radiomic feature was used as a distance measure for hierarchical clustering. The average silhouette method was used to evaluate the optimal number of different clusters in our data set.[21] Furthermore, to validate our results, we conducted a stratified 5-fold cross-validation using 10 000 repeats of the 3 best radiomic and conventional quantitative parameters. The model was trained on a training set and was evaluated on a separate test set at each fold using receiver-operating characteristics analysis. The derived curves were averaged and plotted to assess the discriminatory power of the parameters. The number of additional cases classified correctly was calculated compared with lesion volume. The McNemar test was used to compare classification accuracy of the given parameters compared with lesion volume.[22]

Because of the large number of comparisons, we used the Bonferroni correction to account for the family-wise error rate. Bonferroni correction assumes that the examined parameters are independent of each other; thus, the question is not how many parameters are being tested but how many independent statistical comparisons will be made. Therefore, based on methods used in genome-wide association studies, we calculated the number of informative parameters accounting for 99.5% of the variance using principal component analysis.[23,24] Overall, 42 principal components were identified; therefore, P values <0.0012 (0.05/42) were considered significant. All calculations were done in the R environment.[25]

Results

Descriptive Results

There was no significant difference between the NRS and non-NRS groups regarding patient characteristics and scan parameters (Table 1). Furthermore, we did not observe any significant difference in qualitative plaque characteristics and image quality parameters (Table 2) implying successful matching of the 2 groups. Median number of voxels contributing to the NRS coronary plaques (1928; IQR, 1413–2560) did not show statistical difference compared with the number of voxels in the non-NRS group (1286; IQR, 1001–1768; P=0.0041).

Statistical Significance and Diagnostic Accuracy of Conventional Quantitative Parameters

Among conventional quantitative imaging parameters, there was no significant difference between NRS and non-NRS plaques (Table 2). Furthermore, none of the conventional parameters had an AUC value >0.8 (Table 3).

Table 3.

Diagnostic Performance of Conventional Quantitative Parameters and Novel Radiomic Parameters to Identify Plaques With the Napkin-Ring Sign

Statistical Significance and Diagnostic Accuracy of Radiomic Parameters

Overall, 4440 radiomic parameters were calculated for each atherosclerotic lesion. Of all calculated radiomic parameters, 20.6% (916/4440) showed a significant difference between plaques with or without NRS (all P<0.0012). Of the 44 calculated first-order statistics, 25.0% (11/44) was significant. Of the 3585 calculated gray-level co-occurrence matrix (GLCM) statistics, 20.7% (742/3585) showed a significant difference between the 2 groups. Among the 55 gray-level run-length matrix (GLRLM) parameters, 54.5% (30/55) were significant, whereas 17.6% (133/756) of the calculated 756 geometry-based parameters had a P<0.0012. A Manhattan plot of the P values of the calculated radiomic parameters is shown in Figure 2. Detailed statistics of the assessed radiomic parameters can be found in Table I in the Data Supplement.

Figure 2.

Manhattan plot of all 4440 calculated P values. The Manhattan plot shows all 4440 calculated P values comparing napkin-ring sign (NRS) vs non-NRS plaques and their distribution among the different classes of radiomic parameters. Radiomic features are lined up on the x axis, whereas the -log2(P) values are plotted on the y axis. The red horizontal line indicates the Bonferroni-corrected P value of 0.0012. Radiomic parameters above the red line were considered statistically significant.

Among all 4440 radiomic parameters, 9.9% (440/4440) had an AUC value >0.80. Of the 44 calculated first-order statistics, 18.2% (8/44) had an AUC value >0.80. Of the 3585 calculated GLCM parameters, 9.7% (348/3585) of the AUC values was >0.80. Among the 55 GLRLM parameters, 54.5% (30/55) had an AUC value >0.80, whereas of the calculated 756 geometry-based parameters, 7.1% (54/756) had an AUC value >0.80. Of all radiomic parameters, short-run low-gray-level emphasis, long-run low-gray-level emphasis, surface ratio of component 2 to total surface, long-run emphasis, and surface ratio of component 7 to total surface had the 5 highest AUC values (0.918; 0.894; 0.890; 0.888, and 0.888, respectively). Detailed diagnostic accuracy statistics of conventional quantitative features and of the 5 best radiomic features for each group are shown in Table 3, whereas detailed diagnostic accuracy results of radiomic parameters can be found in Table I in the Data Supplement.

Cluster Analysis of Radiomic Parameters

Results of the linear regression analysis conducted between all pairs of the calculated 4440 radiomic metrics are summarized using a heatmap (Figure 3). Hierarchical clustering showed several different clusters where parameters are highly correlated with each other (represented by the red areas in Figure 3) but only have minimal relationship with other radiomic features (represented by the black areas in Figure 3). Cluster analysis revealed that the optimal number of clusters among radiomic features in our data set is 44.

Figure 3.

Heatmap and clustering dendrogram of all 4440 calculated radiomic parameters. Each parameter was compared with all other parameters using linear regression analysis. Features were clustered based on R2 values of the corresponding regression models and plotted along both axes. R2 values <0.5 are black, whereas greater values are shown in red with increasing intensity. The 1−R2 values was used as a distance measure between parameters and used for hierarchical clustering. The resulting clustering dendrogram can be seen on the right of the image. Cluster analysis indicated that the optimal number of clusters is 44 based on our radiomics data set.

Cross-Validation Results

Five-fold cross-validation using 10 000 repeats was used to simulate the discriminatory power of the 3 best radiomic and conventional parameter. Average receiver-operating characteristics curves of the cross-validated results are shown in Figure 4. Radiomic parameters had higher AUC values and identified lesions showing the NRS significantly better compared to conventional metrics. Detailed results are shown in Table 4.

Figure 4.

Stratified 5-fold cross-validated receiver-operating characteristic (ROC) curves of the best radiomic and conventional quantitative parameters. Radiomic parameters (blue) have higher discriminatory power to identify plaques with napkin-ring sign compared with conventional quantitative metrics (green). Detailed performance measures can be found in Table 4.

Table 4.

AUC Values of Stratified 5-Fold Cross-Validated ROC Curves of the Best Radiomic and Conventional Quantitative Parameters to Identify Plaques With the Napkin-Ring Sign

Discussion

We demonstrated that coronary plaques consist of sufficient number of voxels to conduct radiomic analysis, and 20.6% of radiomic parameters showed a significant difference between plaques with or without NRS, whereas conventional parameters did not show any difference. Furthermore, several radiomic parameters had a higher diagnostic accuracy in identifying NRS plaques than conventional quantitative measures. Cluster analysis revealed that many of these parameters are correlated with each other; however, there are several distinct clusters, which imply the presence of various features that hold unique information on plaque morphology. Cross-validation simulations indicate that our results are robust when assessing the discriminatory value of radiomic parameters, implying the generalizability of our results.

Radiomics uses voxel values and their relationship to each other to quantify image characteristics. On the basis of our results, it seems not only do radiomic features outperform conventional quantitative imaging markers but also parameters incorporating the spatial distribution of voxels (GLCM, GLRLM, and geometry-based parameters) have a better predictive value than first-order statistics, which describe the statistical distribution of the intensity values. Among GCLM parameters, the interquartile range, the lower notch, the median absolute deviation from the mean of the GLCM probability distribution, Gauss right focus, and sum energy had the 5 highest AUC values. NRS plaques have many low-value voxels next to each other in a group surrounded by higher density voxels. This heterogeneous morphology results in an unbalanced GLCM and therefore higher interquartile rank values, which also means smaller lower notch values and bigger deviations from the mean. Gauss right focus and sum energy both give higher weights to elements in the lower right of the GLCM, which represents the probability of high-density voxels occurring next to each other. Because NRS plaques do not have many high-value voxels next to each other, they received smaller values, whereas non-NRS plaques have higher values, which resulted in excellent diagnostic accuracy.

Among GLRLM statistics, long- and short-run low-gray-level emphasis, long- and short-run emphasis, and run percentage had the best predictive value. Run percentage and long-run emphasis give high values to lesions, where there are many similar value voxels in 1 direction, whereas long-run low-gray-level emphasis adds a weight to the previous parameter by giving higher weights when these voxel runs contain low Hounsfield unit values. NRS plaques’ low-density core has many low CT number voxels next to each other in 1 direction; therefore, NRS plaques have higher values compared with non-NRS plaques, which results in excellent diagnostic accuracy. In case of short-run emphasis and short-run low-gray-level emphasis, the contrary is true, which results in NRS plaques receiving low values, whereas non-NRS plaque have higher values also leading to high AUC values.

Among geometry-based parameters, the first 5 with the best diagnostic accuracy all represent the surface ratio of a specific subcomponent to the whole surface of the plaque. In all cases, the ratio of high-density subcomponents (eg, subcomponent 2 when the plaque was divided into 2 components) to the whole surface had excellent diagnostic accuracy. Because each subcomponent is composed of equal number of voxels because of the equally probable binning, the difference in surfaces is a result of how the high-intensity voxels are situated to each other. In case of NRS plaques, extraction of low attenuation voxels leaves a hollow cylindrical shape of high CT number voxels, which has a relatively large surface. Non-NRS plaques on the contrary do not have such voxel complexes; therefore, the surface of the high attenuation voxels is smaller, and, therefore, the ratio compared with the whole surface is also smaller.

This kind of transition from qualitative to quantitative image assessment was initiated by oncoradiology. Because studies showed that morphological descriptors correlate with later outcomes,[26] reporting guidelines such as the Breast Imaging Reporting and Data System started implementing qualitative morphological characteristics into clinical practice.[27] However, despite all the efforts of standardization, the variability of image assessment based on human interpretation is still substantial.[28] Radiomics, the process of extracting thousands of different morphological descriptors from medical images, has been shown to reach the diagnostic accuracy of clinical experts in identifying malignant lesions.[10] Furthermore, radiomics can not only classify abnormalities to proper clinical categories but also discriminate between responders and nonresponders to clinical therapy and predict long-term outcomes.[12,15] However, there are major concerns on the generalizability of radiomics. Several studies have shown that imaging parameters, reconstruction settings, segmentation algorithms, etc, all affect the radiomic signature of lesions.[29-32] Furthermore, it has been shown that the variability caused by these changeable parameters is in the range or even greater than the variability of radiomic features of tumor lesions.[33] Little is known about cardiovascular radiomics. Several studies will be needed to replicate these results in the cardiovascular domain. The potential of radiomics is extensive; however, the problem of standardized imaging protocols and radiomic analysis need to be solved to achieve robust and generalizable results.

Despite our encouraging results, our study has some limitations that should be acknowledged. All of our examinations were done using the same scanner and reconstruction settings. It is yet unknown how these settings might affect radiomic parameters and therefore influence the applicability of radiomics in daily clinical care. Furthermore, our results are based on a case–control study design. The true prevalence of the NRS is considerably smaller compared with non-NRS plaques in a real population. Therefore, our observed positive predictive values might be higher, whereas our negative predictive values might be smaller than that expected in a real-world setting. Moreover, our limited sample sizes might not allow the accurate assessment of the diagnostic accuracy of the different parameters. However, we performed Monte Carlo simulations and cross-validated our results to achieve robust estimates.

Radiomics is a promising new tool to identify qualitative plaque features such as the NRS. Because the number of CT examinations increases, we are in dire need of new techniques that increase the accuracy of our examinations without increasing the workload of imaging specialists. We demonstrated that radiomics has the potential to identify a qualitative high-risk plaque feature that currently only experts are capable of. Furthermore, our findings indicate that radiomics can quantitatively describe qualitative plaque morphologies and therefore have the potential to decrease intra- and interobserver variability by objectifying plaque assessment. In addition, we observed several different clusters of information present in our data set, implying that radiomics might be able to identify new image markers that are currentlyt unknown. These new radiomic characteristics might provide a more accurate plaque risk stratification than the currently used high-risk plaque features. Radiomics could easily be implemented into currently used standard clinical workstations and become a computer-aided diagnostic tool, which seamlessly integrates into the clinical workflow and could increase the reproducibility and the accuracy of diagnostic image interpretation in the future. Further studies are needed to explore the potential of cardiovascular radiomics.

Sources of Funding

This study was supported by the National Research, Development and Innovation Office of Hungary (NKFIA; NVKP-16-1-2016-0017). Dr Kolossváry received support from the ÚNKP-17-3 New National Excellence Program of the Ministry of Human Capacities, Hungary.

Disclosures

Dr Kolossváry is the creator and developer of Radiomics Image Analysis software package, which was used for radiomic analysis. P. Kitslaar is employed by Medis Medical Imaging Systems B.V. This software was used for the coronary segmentations and image export. The other authors report no conflicts.