Discrimination of pulmonary ground-glass opacity changes in COVID-19 and non-COVID-19 patients using CT radiomics analysis.

PURPOSE: The coronavirus disease 2019 (COVID-19) has evolved into a worldwide pandemic. CT although sensitive in detecting changes suffers from poor specificity in discrimination from other causes of ground glass opacities (GGOs). We aimed to develop and validate a CT-based radiomics model to differentiate COVID-19 from other causes of pulmonary GGOs.
METHODS: We retrospectively included COVID-19 patients between 24/01/2020 and 31/03/2020 as case group and patients with pulmonary GGOs between 04/02/2012 and 31/03/2020 as a control group. Radiomics features were extracted from contoured GGOs by PyRadiomics. The least absolute shrinkage and selection operator method was used to establish the radiomics model. We assessed the performance using the area under the curve of the receiver operating characteristic curve (AUC).
RESULTS: A total of 301 patients (age mean ± SD: 64 ± 15 years; male: 52.8 %) from three hospitals were enrolled, including 33 COVID-19 patients in the case group and 268 patients with malignancies or pneumonia in the control group. Thirteen radiomics features out of 474 were selected to build the model. This model achieved an AUC of 0.905, accuracy of 89.5 %, sensitivity of 83.3 %, specificity of 90.0 % in the testing set.
CONCLUSION: We developed a noninvasive radiomics model based on CT imaging for the diagnosis of COVID-19 based on GGO lesions, which could be a promising supplementary tool for improving specificity for COVID-19 in a population confounded by ground glass opacity changes from other etiologies.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) now officially termed coronavirus disease 2019 (COVID-19) has rapidly spread globally, and on 11th March 2020 was declared a global pandemic by the World Health Organization [1]. As of Aug 15 2020, there have been over 21,000,000 confirmed cases reported in 216 countries [2]. For the diagnosis of COVID-19, next-generation sequencing or real-time reverse transcription polymerase chain reaction methods are used as reference standards [3] but computed tomography (CT) has also been reported to be effective in assisting the early detection of COVID-19 cases [4]. CT imaging appearances of COVID-19 pneumonia have been previously described [5]. The predominant imaging pattern of COVID-19 pneumonia is ground-glass opacity (GGO) in the lung periphery with occasional consolidation on CT [6]. GGOs are frequently observed and found to be present in 77%–100% of the confirmed COVID-19 cases [[7], [8], [9]]. Pure GGO lesions could be seen at the early stage of COVID-19 pneumonia [5] and were reported to be the main finding after symptom onset [10]. Chung et al. [11] found GGOs were observed in 12/21 (57 %) of the patients and one patient had normal CT at the initial scan, followed by a scan after 3 day showing disease progression of a new solitary, rounded peripheral GGO. Although CT has a high sensitivity for detection of COVID-19, the specificity for COVID-19 was low, particularly in regions of low disease prevalence. A recent meta-analysis showed a pooled specificity was 37 % for chest CT [12].

Fleischner Society Glossary of Terms for Thoracic Imaging defined GGO as an area with hazy increased opacity of lung with preserved bronchial structures and vascular markings [13]. The underlying pathogenesis could be partial displacement of the air, thickening of the interstitium, increased blood volumes in the pulmonary capillary system, partial alveoli collapse, or overlaps of these [14]. For patients with pure GGO as primary findings in CT images, the identification of COVID-19 could be confounded by malignancy or other pulmonary infections. Currently, and in the future, it is conceivable that we may see more COVID-19 pneumonia mixed in with other pathologies of similar GGO appearances. There will be a need to be able to discriminate between them. Along with clinical assessment, quantitative imaging may have a role in improving specificity in identifying GGO caused by COVID-19 pneumonia.

Radiomics refer to a data-driven methodology that extracts large amounts of advanced quantitative image-based features [15]. This has been more commonly applied to cancer, but has also been used to non-invasively capture lesion heterogeneity effectively for infectious diseases [16,17]. Peripheral GGO is a common morphological change in CT images. The classification of patients with similar GGO signs could provide valuable clinical and diagnostic information. The purpose of this study is to investigate the use of radiomics analysis in GGOs. We hypothesize that CT radiomics features can be used to improve specificity for detection of COVID-19, and can discriminate GGO of COVID-19 to non-COVID-19 patients.

Methods

Patient recruitment

This study was approved by multiple Institutional Review Boards of Queen Mary hospital (QMH), The University of HK PET CT unit (HKU) and Pamela Youde Nethersole Eastern (PYNEH) hospital. Patients informed consent was waived due to the retrospective nature of this study. The diagram for the flow of participants through the study was summarized in Supplementary Fig. 1. Case group: data were screened between 24/01/2020 to 31/03/2020 2020 from QMH and PYNEH. Patients with laboratory-confirmed COVID-19 by reverse transcription polymerase chain reaction were included, and their initial CT scans were retrieved. Control group: data were screened between 04/02/2012 and 31/03/2020 from HKU. Patients with reported GGOs in the radiological report were included in this study. A board-certified radiologist with fellowship training in cardiothoracic imaging (V.V., with 10 years’ experience) then reviewed cases and included cases that have similar ground glass opacity appearances. For patients in the control group collected after December 2019, underwent strict clinical +/− laboratory assessment prior to entering the unit to exclude potential infection with COVID-19. Clinical details such as history and clinical assessment was obtained as standard for diagnosis, in conjunction with histological and laboratory tests if they were available. Patients with incomplete data were excluded. A total of 301 patients (age mean ± SD: 64 ± 15 years; male: 52.8 %) were enrolled in this study. The images acquisition details are listed in Supplementary Method.

With a balanced distribution of the clinical outcome (COVID-19), recruited patients were separated into the training and testing sets at a proportion of 3:1 by a stratified randomization approach.

Contour of the regions of interest (ROI)

All CT images were obtained and reviewed by two radiologists jointly (C.X. and V.V., of 2 and 10 years’ experience respectively) independently without clinical information to avoid bias. Patients with GGOs fulfilling the definition from the Fleischner Society Glossary of Terms for Thoracic Imaging [13] on lung window (−600 Hounsfield unit [HU] level, 1500 HU width) were included for further analysis. GGOs were manually contoured in one representative slice by the ITK-SNAP software [18]. We excluded the large vessels and arteries when drawing the ROI. GGOs appearing as nodules were also excluded. A test-retest study was conducted in a subset of 30 patients with ROI contoured by two radiologists from the training set for the determination of feature robustness. Features with intraclass correlation coefficients above 0.80 were included for further analysis.

Feature extraction

Radiomics features were extracted using PyRadiomics [19]. The resampled voxel sizes were set to 1 × 1 mm2 pixels for standardization. Defined radiomics features were extracted from original and wavelet filtered images. Wavelet filtration (high pass filter and low pass filter) filtered original images directionally with x and y directions respectively, resulting in 4 combinations of decompositions. Filtered images could present more detailed information of the images from different orientations. Additional details are specified in Supplementary Method.

Feature harmonization

As CT images were collected from different hospitals using different acquisition and reconstruction parameters, radiomics features were first harmonized using ComBat method to reduce the batch effect [20].

Feature selection

Feature selection was conducted in two steps. First, the top 100 features correlated with the outcome analyzed by univariate analysis were selected. Second, regularized multivariate logistic regression with the least absolute shrinkage and selection operator (LASSO) penalty was applied [21]. The LASSO algorithm could select features with a coefficient of larger than zero by the optimal λ.

Model construction

Because of limited access to the COVID-19 data across different medical centers, labelled positive cases are insufficient at the early stage of disease outbreak, which could lead to data imbalance problems. The uneven distribution of positive and negative cases could result in predictions skewed towards the negative class. To solve this problem, we adopted re-sampling techniques during the training process. An open-source solution for automatic calculations of different re-sampling techniques was adopted for the selection of optimal resampling techniques [22].

A radiomics score (Rad-score) was calculated for each patient using the Radial Basis Function kernel support vector machine (SVM) with selected features. According to the optimal prediction threshold of the radiomics model in the training set, each patient was divided into different risk groups (COVID-19 and non COVID-19) by the diagnostic possibility.

Statistical analysis

We used Python version 3.7. for statistical analyses. Categorical variables were compared using Chi-squared test/ Fisher’s exact test, while continuous variables were compared using Kruskal-Wallis test. We used the area under the curve of the receiver operating characteristic curve (AUC), accuracy, sensitivity and specificity to assess the prediction performance and their 95 % confidence intervals were provided. The optimal prediction threshold is defined by Youden index [23]. The calibration performance was shown by calibration plots. Clinical usefulness was evaluated by decision curve analysis. A two-tailed P value of less than 0.05 was considered statistically significant.

Result

Patient baseline characteristics

Fig. 1 depicts the workflow processes. A total of 301 patients were enrolled in this study, including 33 in the case group. The remaining 268 were collected as the control group, including 136 non-COVID19 pneumonia, 48 malignant tumors, and 84 benign lesions of indeterminate nature. No significant differences between the training and testing datasets in terms of COVID-19 events (p = 0.44) were found. The baseline characteristics of the patients are shown in Table 1.

Fig. 1

Analysis workflow. (A) Data collection. (B) Contour of regions of interest. (C) Feature extraction. (D) Feature selection. (E) Model construction and evaluation. ROI = region of interest, 2D = two-dimensional, LASSO = least absolute shrinkage and selection operator, SVM = support vector machine, Rad-score = radiomics model score.

Table 1

Patient baseline characteristics.

Characteristic	Case group	Control group
	(COVID-19)	(non COVID-19)
	33	268
Age (mean ± SD)	50 ± 22	65 ± 13
Sex (n)
Male	18	141
Female	15	127
Disease (n)
Pulmonary infection (COVID19)	33	0
Pulmonary infection (other causes)	0	136
Pulmonary adenocarcinoma	0	48
Benign lesions of indeterminate nature	0	84
Signs (%)
Fever (>37.5 °C)	59 %	25 %
Cough	47 %	13 %
Dyspnea	24 %	18 %
Chest Pain	12 %	5 %
Vomiting	3 %	2 %
Diarrhea	12 %	2 %

Note—COVID-19 = coronavirus disease 2019.

Model evaluation

A total of 474 radiomics features were extracted (102 original and 372 with wavelet filtration). In the test-retest experiment, 417 features demonstrated intraclass correlation coefficients above 0.80 and were included for further analysis (Supplementary Table 1). After LASSO, this reduced the feature number to 13.

The Adaptive Synthetic oversampling technique was used for the data re-sampling process in the training set [22]. The radiomics model using linear regression comprised of 13 selected features achieved an AUC of 0.905, accuracy of 89.5 %, sensitivity of 83.3 %, specificity of 90.0 % in the testing cohort (Table 2, Fig. 2). The details of feature selection are presented in Table 3. Basic metrics first order statistic and high-dimensional textual features (Gray Level Co-occurrence Matrix, Gray Level Run Length Matrix, Gray Level Size Zone Matrix features, and Neighboring Gray Tone Difference Matrix features) contributed to the model construction. The calibration curve presented appropriate agreement between the actual and predicted probabilities of COVID-19 infection and are specified in Supplementary Fig. 2. Decision curve analysis showed clinical usefulness (Supplementary Fig. 3). Patients could be divided into low-risk group (predicted non COVID-19) and high-risk group (predicted COVID-19) by the Rad-scores for the diagnostic possibility (Fig. 3).

Table 2

Prediction performance of radiomics model in the training set and test set.

Patients	TP	TN	FP	FN	AUC	Accuracy %	Sensitivity %	Specificity %
Training Set	27	187	11	0	0.995	95.1	100	94.4
					(0.988, 0.999)	(91.4, 97.5)	(100.0,100.0)	(91.3, 97.6)
Testing Set	5	63	7	1	0.905	89.5	83.3	90.0
					(0.777, 0.999)	(80.3, 95.3)	(53.5, 1.00)	(83.0, 970)

Note—95 % confidence intervals included in parentheses. TP, TN, FP and FN presented as counts. Accuracy, sensitivity and specificity present as percentages.

TP = True positive, TN = true negative, FP = false positive, FN = false negative, AUC = area under the receiver operating characteristic curve.

Fig. 2

Predictive performance of the radiomics models. ROC curves showing the predictive power of the Rad-score model using radiomics feature. ROC = receiver operating characteristic curve, AUC = area under the receiver operating characteristic curve.

Table 3

Description of selected radiomic features in the Rad-score model.

Feature Index	Filter	Feature Class	Feature
1	Original	First order	Kurtosis
2	Original	First order	Total Energy
3	Original	GLCM	Informational Measure of Correlation 2
4	Original	GLCM	Maximal Correlation Coefficient
5	Wavelet (LH)	GLRLM	Long Run Low Gray Level Emphasis
6	Wavelet (LH)	GLSZM	Large Area Emphasis
7	Wavelet (LH)	GLSZM	Large Area Low Gray Level Emphasis
8	Wavelet (LH)	GLSZM	Zone Variance
9	Wavelet (HL)	GLCM	Maximal Correlation Coefficient
10	Wavelet (HL)	NGTDM	Busyness
11	Wavelet (HH)	GLSZM	Small Area Low Gray Level Emphasis
12	Wavelet (LL)	First order	Skewness
13	Wavelet (LL)	NGTDM	Strength

Note—For wavelet filtration, “H” and “L” represent high pass filter and low pass filter on the x and y directions.

GLCM = Gray Level Co-occurrence Matrix, GLRLM = Gray Level Run Length Matrix, GLSZM = Gray Level Size Zone Matrix, NGTDM = Neighboring Gray Tone Difference Matrix.

Fig. 3

Patients risk stratification the Rad-score radiomics model for each patient and some representative examples of CT images showing ground glass opacity as major changes. COVID-19 = coronavirus disease 2019.

Discussion

Our proposed radiomics risk screening method achieved good prediction performance with an AUC of 0.905, accuracy of 89.5 %, sensitivity of 83.3 %, specificity of 90.0 % in the independent testing set. Imaging features have previously been shown to correlate with the pathogenesis of viral infections and could indicate the viral pathogens [24]. Recent COVID-19 studies based on CT images were predominantly of diagnostic [[25], [26], [27], [28]] and prognostic models [29,30]. Diagnostic models mainly focused on detecting COVID-19 pneumonia in suspicious patients with symptoms (body temperature, and signs and symptoms). Most of the previous studies were performed at the image level for the diagnosis of COVID-19. Texture features have been shown to be predictive of COVID-19 infection in early literature using a combination of machine learning and deep learning techniques with quantitative imaging analysis, with a few adopting radiomics analysis. Yue et al. [30] demonstrated that CT radiomics models showed feasibility and accuracy for predicting hospital stay in COVID-19 patients. Barstugan et al. [25] reported a prediction model built based on the textural feature of Grey-Level Size Zone Matrix could achieve a classification accuracy of 99.7 %, sensitivity of 97.6 % and specificity of 99.7 %. This study used a square-shaped patch rather than image segmentation comparing diseased vs normal lungs in same COVID-19 patients. This differed from our study as we focused our comparison on GGO of COVID-19 patients compared to other etiologies with a separate control group. Our study is the first to provide a classification model for COVID-19 diagnosis on GGO lesions, which could be a promising supplementary tool for clinicians in the future. With the aid of automation, for example, automatic lesion detection, segmentation and radiomics analysis, it is conceivable that an alert system could be set up to screen patients undergoing a CT scan for unsuspecting COVID-19 infection in the future.

Using handcrafted radiomics features for model construction for novel emerging infectious diseases is of advantages when the initial case of COVID-19 is limited. The dataset is required to pass through the machine learning training network only once. While maintaining the classification accuracy, we could significantly reduce the computation time and downscale the hardware environment such as GPU. The feature extraction process is based on formulaic calculation [15]. The sample size needed for radiomics studies is less than deep learning-based studies. Radiomics model could be developed fast and be used at the early stage of disease outbreak because of the less computational consumption, lower calculation complexity and smaller sample size required for training. In Shen’s comparison study in non-small cell lung cancer patients [31], radiomics 2D features were reported to have better performance than 3D features, partly because the spatial resolutions of medical imaging is often inconsistent for different hospitals. Considering the multi-center generalizability, labor consumption and calculation costs, we adopted 2D ROI for the feature extraction in this study.

A few limitations are worth noting. First, biopsy or laboratory confirmation in the control groups was not available in most patients. Most of the scans that were performed prior to September 2019, can be deemed almost certain to be negative of COVID-19. Only a few scans were performed after that time that was included in the control group, with none of these had clinical suspicion of COVID-19, with some also having negative RT-PCR tests results. Further work in looking at using radiomics to differentiate from other viral or bacterial pneumonia will be a useful future direction. Second, we did not include clinical history, risk factors, or demographics into the predictive model. This will be important in the clinical scenario, but the purpose of this study was to ascertain discriminability based on radiomics features. Third, this is a retrospective study and therefore has the potential for selection bias. Given the limited access to COVID-19 cases, patients in case and control groups were collected from different institutions, although we used feature harmonization to minimize the batch effect. Confirmation of findings in prospective studies is needed for future research.

Conclusion

In conclusion, we developed a noninvasive radiomics model based on CT imaging for the diagnosis of COVID-19 based on GGO lesions, which could be a promising supplementary tool for improving specificity for COVID-19 in a population confounded by ground glass opacity changes from other etiologies.

Author statement

This original study has not been published elsewhere and has not been submitted simultaneously for publication elsewhere. All the authors have approved the final version of the manuscript for this submission.

Originality

This original study has not been published elsewhere and has not been submitted simultaneously for publication elsewhere.

Ethical approval

This study was approved by multiple Institutional Review Boards in the study. Patients informed consent was waived due to the retrospective nature of this study.

Funding acknowledgement

None.

CRediT authorship contribution statement

Chenyi Xie: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft, Writing - review & editing. Ming-Yen Ng: Investigation, Data curation, Writing - review & editing. Jie Ding: Software, Formal analysis, Writing - review & editing. Siu Ting Leung: Investigation, Data curation, Writing - review & editing. Christine Shing Yen Lo: Investigation, Data curation, Writing - review & editing. Ho Yuen Frank Wong: Investigation, Data curation, Writing - review & editing. Varut Vardhanabhuti: Conceptualization, Methodology, Supervision, Writing - original draft, Writing - review & editing, Project administration.

Declaration of Competing Interest

The authors report no declarations of interest.