Radiomics nomogram for the prediction of 2019 novel coronavirus pneumonia caused by SARS-CoV-2.

OBJECTIVES: To develop and validate a radiomics model for predicting 2019 novel coronavirus (COVID-19) pneumonia.
METHODS: For this retrospective study, a radiomics model was developed on the basis of a training set consisting of 136 patients with COVID-19 pneumonia and 103 patients with other types of viral pneumonia. Radiomics features were extracted from the lung parenchyma window. A radiomics signature was built on the basis of reproducible features, using the least absolute shrinkage and selection operator method (LASSO). Multivariable logistic regression model was adopted to establish a radiomics nomogram. Nomogram performance was determined by its discrimination, calibration, and clinical usefulness. The model was validated in 90 consecutive patients, of which 56 patients had COVID-19 pneumonia and 34 patients had other types of viral pneumonia.
RESULTS: The radiomics signature, consisting of 3 selected features, was significantly associated with COVID-19 pneumonia (p < 0.05) in both training and validation sets. The multivariable logistic regression model included the radiomics signature and distribution; maximum lesion, hilar, and mediastinal lymph node enlargement; and pleural effusion. The individualized prediction nomogram showed good discrimination in the training sample (area under the receiver operating characteristic curve [AUC], 0.959; 95% confidence interval [CI], 0.933-0.985) and in the validation sample (AUC, 0.955; 95% CI, 0.899-0.995) and good calibration. The mixed model achieved better predictive efficacy than the clinical model. Decision curve analysis demonstrated that the radiomics nomogram was clinically useful.
CONCLUSIONS: The radiomics model derived has good performance for predicting COVID-19 pneumonia and may help in clinical decision-making. KEY POINTS: • A radiomics model showed good performance for prediction 2019 novel coronavirus pneumonia and favorable discrimination for other types of pneumonia on CT images. • A central or peripheral distribution, a maximum lesion range > 10 cm, the involvement of all five lobes, hilar and mediastinal lymph node enlargement, and no pleural effusion is associated with an increased risk of 2019 novel coronavirus pneumonia. • A radiomics model was superior to a clinical model in predicting 2019 novel coronavirus pneumonia.

Introduction

Since December 2019, a succession of cases of pneumonia, now known as novel coronavirus–infected pneumonia, has been observed in Wuhan, Hubei Province, China. On January 7, 2020, the 2019 novel coronavirus (COVID-19) was identified as the causative agent, based on virus typing [1, 2]. The disease is now officially named COVID-19 by the World Health Organization. Recent studies revealed that COVID-19 is related to bat-SL-CoV ZC45 and bat-SL-CoV ZXC21 [2], and can spread from human to human, mainly through respiratory droplets, physical contact, and the oral-fecal route [3]. As of February 29, 2020, 79,394 cases had been confirmed, with 2838 deaths in China alone. At present, the infection has spread across China and other countries around the world [4-6].

Definitive diagnosis of COVID-19 pneumonia requires viral nucleic acid detection in throat swabs, sputum, lower respiratory tract secretions, or blood; while the specificity of this test is strong, its sensitivity is poor [7], and most patients show multiple negative results. Furthermore, in most patients, lung imaging findings are observed earlier than clinical symptoms, which makes imaging examinations crucial for screening and an accurate diagnosis [8]. Chest CT, especially high-resolution CT (HRCT), is now a routine procedure in patients with COVID-19 pneumonia at many institutions, because it enables rapid scanning, acquisition of thin sections, and multiplanar reconstruction. However, the reported CT findings of COVID-19 pneumonia were extremely similar with common types of viral pneumonia [8-14]. From our experience, some patients with positive CT findings had negative initial results by viral nucleic acid detection before the confirmation of COVID-19 pneumonia. This led to treatment selection difficulties, delayed treatment, and even misdiagnosis as common types of viral pneumonia, especially in the non-epidemic area.

Machine learning and radiomics provides a non-invasive method for the prediction of COVID-19 pneumonia and can assist radiologists and physicians in performing a quick diagnosis especially when the health system is overloaded. Currently, there are few studies on predicting COVID-19 pneumonia using machine learning or radiomics [15-17]. Hence, the primary objective of this study was to develop and validate a radiomics model for predicting COVID-19 pneumonia in order to help clinicians in quick and accurate diagnosis.

Methods

Patients

This retrospective cross-sectional study was reviewed and approved by the Biomedical Research Ethics Committee of two institutions, and the requirement for patient consent was waived.

The patient enrollment process and excluded criterion for this study are showed in Fig. 1. We retrospectively analyzed 329 patients at two different hospitals in China. A total of 185 consecutive patients with COVID-19 pneumonia, 97 men (mean age, 53.9 years; age range, 26–88 years) and 88 women (mean age, 58.3 years; age range, 22–83 years), were included at Wuhan Huoshenshan hospital between February 11 and 25, 2020. Seven consecutive patients with COVID-19 pneumonia, 5 men (mean age, 47.8 years; age range, 28–75 years) and 2 women (mean age, 59 years; aged 47 and 71), were included at Changhai hospital between January 25 and February 9, 2020. One hundred thirty-seven consecutive patients with other types of viral pneumonia, 72 men (mean age, 54.9 years; age range, 19–92 years) and 65 women (mean age, 52.2 years; age range, 20–95 years), were included at Changhai hospital between April 2011 and January 2020. Viral nucleic acid detection was used to confirm COVID-19 pneumonia and the other types of viral pneumonia. All patients were divided into training set and validation set. The prediction model was developed for a training set that consisted of 136 patients with COVID-19 pneumonia who had positive initial result by viral nucleic acid detection, and 103 patients with the other types of viral pneumonia. Ninety consecutive patients constituted an independent validation sample of 56 patients with COVID-19 pneumonia and 34 patients with other types of viral pneumonia. Fifty-six patients with COVID-19 pneumonia included 49 patients who had initial negative result by viral nucleic acid detection but positive result on CT from Wuhan Huoshenshan hospital and 7 patients who came from Changhai hospital. All clinical results were extracted from the patients’ electronic medical records in the two-hospital information system. All patients with COVID-19 pneumonia were divided into four clinical types including mild, moderate, severe, and critical types based on the clinical classification of COVID-19 pneumonia from the 7th edition of the National Commission of China classification [18].

Fig. 1

The patient enrolment process for this study. COVID-19: 2019 novel coronavirus

CT scanning

Pulmonary CT was performed using 64-, 256-, and 128-slice multidetector row CT scanners (64: Somatom, Siemens Healthcare; 256: Brilliance-16P, Philips Healthcare; 128: uCT 760, United Imaging Healthcare). CT scans were obtained with the following parameters: 120 kV, adaptive tube current, a matrix of 512 × 512, and a beam collimation of 64 × 0.6 mm2, 256 × 0.6 mm2, and 128 × 0.6 mm2. Non-enhanced CT at a slice thickness of 1.0 mm, 1.0 mm, and 0.625 mm was performed, respectively. Images were captured at window settings that allowed viewing of the lung parenchyma (window level and width, − 600 to − 700 HU and 1200–1500 HU, respectively) and the mediastinum (window level and width, 20–40 HU and 400 HU, respectively). The scanning range covered the area from the height level of the superior aperture of the thorax to the diaphragm.

Imaging analysis

We used the original cross-sectional images for analysis. All images were analyzed by two chest radiologists (reader 1 and reader 2, both with 8 years of experience) who were blinded to the clinical details. When their results were not consistent, the final results were determined by consensus.

All lesions were evaluated for the following parameters: (a) location: right, left, or bilateral lungs; (b) distribution: peripheral, central, or diffuse; (c) attenuation: ground glass attenuation including ground glass opacity (GGO) and crazy-paving pattern, consolidation, and mixed patterns of ground glass attenuation and consolidation [19, 20]; (d) maximum lesion range: ≤ 5 cm, 5–10 cm, > 10 cm, only for the biggest one; (e) lobe involvement: the five lung lobes were divided into categories of ≤ 2 lobes, 2–4 lobes, and = 5 lobes; (f) number of lesions: 1, 2, 3, or more; (g) air bronchogram; (h) hilar and mediastinal lymph nodes enlargement: short-axis diameter of a lymph node > 10 mm [21]; and (i) pleural effusion.

Radiomics workflow

The radiomics workflow included (a) image segmentation, (b) feature extraction, (c) feature reduction and selection, and (e) predictive model building.

In this study, we used the artificial intelligence software (uAI-Discover-NCP R001, United Imaging Healthcare, China) to segment images and extract the radiomics features from the lung parenchyma window. A total of 1409 2D and 3D features from primary lesion were extracted. Feature selection comprised three steps: variance analysis, spearman correlation analysis, and least absolute shrinkage and selection operator method (LASSO) logistic regression algorithm. This method has been shown to be effective in prior radiomics studies [22]. A retrospective power analysis was performed. The sequential method of Bonferroni correction was applied to adjust the baseline significance level (α = 0.05) for multiple testing bias [23, 24]. Finally, radiomics scores (Rad-scores) were calculated for each patient via a linear combination of selected features that were weighted by their respective coefficients. More information about radiomics feature extraction methodology is reported in Supplementary digital content 1.

We developed a clinical model using significantly associated CT characteristics including location, distribution, attenuation, maximum lesion range, lobe involvement, number of lesions, air bronchogram, hilar and mediastinal lymph node enlargement, and pleural effusion. Consequently, the mixed model, which combined the Rad-score and significantly associated CT characteristics, was developed and we hoped to improve the accuracy of predicting COVID-19 pneumonia.

Statistical analysis

Normal distribution and variance homogeneity tests were performed on all continuous variables; those with a normal distribution are expressed as the mean and standard deviation while those with non-normal distributions are expressed as medians and ranges. First, we examined group differences in all variables. The Kruskal-Wallis H test (skewed distribution) and chi-square tests (categorical variables) were used to determine statistical differences between the two groups. Second, univariable regression analysis was applied to estimate the effect size of the relationships between all variables and the two groups of viral pneumonia. The group with other types of viral pneumonia was considered as a reference group. Third, multivariable logistic regression analysis was conducted to develop a model for predicting COVID-19 pneumonia in the primary sample, and a nomogram was then constructed. The discrimination performance of established models was quantified by the receiver operating characteristic curve. Area under the curve (AUC) estimates in the prediction models were compared using the Delong non-parametric approach [25]. Calibration curves were plotted via bootstrapping with 500 resamples to assess the calibration of the radiomics model, accompanied by the Hosmer-Lemeshow goodness-of-fit test. The performance of the radiomics model was then tested in an independent validation sample by using the formula derived from the training set. Finally, to estimate the clinical usefulness of the nomogram, decision curve analysis (DCA) was performed by calculating the net benefits for a range of threshold probabilities.

A two-tailed p value less than 0.05 was considered statistically significant. All analyses were performed with R (R version 3.3.3; R Foundation for Statistical Computing; http://www.r-project.org).

Results

Clinical and CT characteristics

One hundred ninety-two (58.36%) and 137 (41.64%) of the patients in the study sample were diagnosed with COVID-19 pneumonia and other viral types of pneumonia, respectively. The 192 patients with COVID-19 pneumonia included 141 moderate cases (73.43%) and 51 severe cases (26.57%) based on the clinical classification of COVID-19 pneumonia from the 7th edition of the National Commission of China classification [18]. The 137 patients with other types of viral pneumonia included 40 cases (29.20%) of Epstein-Barr virus infections, 29 cases (21.17%) of cytomegalovirus infections, three cases (2.19%) of adenovirus infections, and 65 cases (47.44%) of influenza infections. Among the clinical and imaging characteristics that we investigated, we found important differences in smoking history, presence of cough in the validation set, lymphocyte ratios, C-reactive protein (CRP) levels, distribution in the training and total set, maximum lesion range in the training and total set, lobes involvement in the validation and total set, hilar and mediastinal lymph node enlargement in the training and total set, and pleural effusion in the training and total set between patients with COVID-19 pneumonia and those with other types of viral pneumonia (p < 0.05). The patient characteristics are shown in Table 1.

Table 1

Baseline characteristics of patients

Characteristic	Training set			Validation set			Total
	Other type of viral pneumonia (n = 103)	Novel coronavirus pneumonia (n = 136)	p value	Other type of viral pneumonia (n = 34)	Novel coronavirus pneumonia (n = 56)	p value	Other type of viral pneumonia (n = 137)	Novel coronavirus pneumonia (n = 192)	p value
Age, median (range), years	55.00 (19.00–95.00)	60.00 (24.00–83.00)	0.260	49.50 (23.00–83.00)	55.00 (22.00–88.00)	0.431	53.00 (19.00–95.00)	58.00 (22.00–88.00)	0.128
Sex, n (%)			0.768			0.509			0.919
Male	55 (53.40)	70 (51.47)		17 (50.00)	32 (57.14)		72 (52.55)	102 (53.12)
Female	48 (46.60)	66 (48.53)		17 (50.00)	24 (42.86)		65 (47.45)	90 (46.88)
Smoking history, n (%)			0.043			0.027			0.002
No	88 (85.44)	127 (93.38)		25 (73.53)	52 (92.86)		113 (82.48)	179 (93.23)
Yes	15 (14.56)	9 (6.62)		9 (26.47)	4 (7.14)		24 (17.52)	13 (6.77)
Fever, n (%)			< 0.001			0.094			< 0.001
No	30 (29.13)	15 (11.03)		7 (20.59)	4 (7.14)		37 (27.01)	19 (9.90)
Yes	73 (70.87)	121 (88.97)		27 (79.41)	52 (92.86)		100 (72.99)	173 (90.10)
Cough, n (%)			0.885			< 0.001			0.051
No	22 (21.36)	28 (20.59)		12 (35.29)	3 (5.36)		34 (24.82)	31 (16.15)
Yes	81 (78.64)	108 (79.41)		22 (64.71)	53 (94.64)		103 (75.18)	161 (83.85)
WBC, n (%)			0.014			0.071			0.020
Decreased	1 (0.97)	9 (6.62)		3 (8.82)	2 (3.57)		4 (2.92)	11 (5.73)
Normal	95 (92.23)	108 (79.41)		31 (91.18)	48 (85.71)		126 (91.97)	156 (81.25)
Increased	7 (6.80)	19 (13.97)		0	6 (10.71)		7 (5.11)	25 (13.02)
Lymphocyte ratios, n (%)			< 0.001			0.004			< 0.001
Decreased	60 (58.25)	55 (40.44)		24 (70.59)	22 (39.29)		84 (61.31)	77 (40.10)
Normal	33 (32.04)	78 (57.35)		10 (29.41)	34 (60.71)		43 (31.39)	112 (58.33)
Increased	10 (9.71)	3 (2.21)		0	0		10 (7.30)	3 (1.56)
CRP, n (%)			< 0.001			0.030			< 0.001
Normal	21 (20.39)	55 (40.44)		8 (23.53)	26 (46.43)		29 (21.17)	81 (42.19)
Increased	82 (79.61)	81 (59.56)		26 (76.47)	30 (53.57)		108 (78.83)	111 (57.81)
Location, n (%)			1.000			0.180			0.786
Right lung	7 (6.80)	10 (7.35)		6 (17.65)	5 (8.93)		13 (9.49)	15 (7.81)
Left lung	1 (0.97)	2 (1.47)		1 (2.94)	0		2 (1.46)	2 (1.04)
Bilateral lungs	95 (92.23)	124 (91.18)		27 (79.41)	51 (91.07)		122 (89.05)	175 (91.15)
Distribution, n (%)			< 0.001			0.129			< 0.001
Central	0	2 (1.47)		7 (20.59)	20 (35.71)		0	2 (1.04)
Peripheral	11 (10.68)	48 (35.29)		27 (79.41)	36 (64.29)		18 (13.14)	68 (35.42)
Diffuse	92 (89.32)	86 (63.24)					119 (86.86)	122 (63.54)
Attenuation, n (%)			0.150			0.094			0.095
Ground glass shape	30 (29.13)	33 (24.26)		6 (17.65)	19 (33.93)		36 (26.28)	52 (27.08)
Consolidation	10 (9.71)	6 (4.41)		3 (8.82)	1 (1.79)		13 (9.49)	7 (3.65)
Mixed pattern	63 (61.17)	97 (71.32)		25 (73.53)	36 (64.29)		88 (64.23)	133 (69.27)
Maximum lesion range, n (%)			0.044			0.063			0.010
≤ 5 cm	32 (31.07)	30 (22.06)		7 (20.59)	14 (25.00)		39 (28.47)	44 (22.92)
5–10 cm	53 (51.46)	64 (47.06)		24 (70.59)	27 (48.21)		77 (56.20)	91 (47.40)
> 10 cm	18 (17.48)	42 (30.88)		3 (8.82)	15 (26.79)		21 (15.33)	57 (29.69)
Number of lesions, n (%)			1.000			0.131			0.270
1	0	0		2 (5.88)	0		2 (1.46)	0
2	3 (2.91)	4 (2.94)		0	2 (3.57)		3 (2.19)	6 (3.12)
≥ 3	100 (97.09)	132 (97.06)		32 (94.12)	54 (96.43)		132 (96.35)	186 (96.88)
Lobes involvement, n (%)			0.113			0.009			0.009
≤ 2	6 (5.83)	11 (8.09)		7 (20.59)	5 (8.93)		13 (9.49)	16 (8.33)
2–4	28 (27.18)	22 (16.18)		11 (32.35)	7 (12.50)		39 (28.47)	29 (15.10)
5	69 (66.99)	103 (75.74)		16 (47.06)	44 (78.57)		85 (62.04)	147 (76.56)
Air bronchogram, n (%)			0.799			0.428			0.544
No	65 (63.11)	88 (64.71)		19 (55.88)	36 (64.29)		84 (61.31)	124 (64.58)
Yes	38 (36.89)	48 (35.29)		15 (44.12)	20 (35.71)		53 (38.69)	68 (35.42)
Hilar and mediastinal lymph nodes enlargement, n (%)			< 0.001			0.745			< 0.001
No	89 (86.41)	76 (55.88)		23 (67.65)	36 (64.29)		112 (81.75)	112 (58.33)
Yes	14 (13.59)	60 (44.12)		11 (32.35)	20 (35.71)		25 (18.25)	80 (41.67)
Pleural effusion, n (%)			0.004			0.561			0.004
No	74 (71.84)	118 (86.76)		27 (79.41)	48 (85.71)		101 (73.72)	166 (86.46)
Yes	29 (28.16)	18 (13.24)		7 (20.59)	8 (14.29)		36 (26.28)	26 (13.54)

WBC, white blood cell; CRP, C-reactive protein; ground glass shape, included ground glass opacity and crazy-paving pattern

Radiomics analysis

Out of 1409 total extracted features, 1276 were excluded which included low intragroup variance (n = 708) and poor correlation with COVID-19 pneumonia (n = 568). Eventually, 133 radiomics features were obtained. Eventually, 133 radiomics features were obtained. They were further reduced to 3 by using the LASSO regularization method (Fig. 2a, b). Three feathers by LASSO regression were still significantly associated with COVID-19 pneumonia after Bonferroni correction. Finally, the radiomics signature was constructed, and the radiomics score was calculated by using the formula (Formula 1). The Rad-scores of COVID-19 pneumonia were higher than the other types of viral pneumonia (p < 0.001) (Figs. 2c and 3).

Fig. 2

Radiomics feature selection by using a parametric method, the least absolute shrinkage and selection operator (LASSO). a Selection of the tuning parameter (λ) in the LASSO model via 10-fold cross-validation based on minimum criteria. Binomial deviances from the LASSO regression cross-validation procedure were plotted as a function of log (λ). The y-axis indicates binomial deviances. The lower x-axis indicates the log (λ). Numbers along the upper x-axis represent the average number of predictors. Red dots indicate average deviance values for each model with a given λ, and vertical bars through the red dots show the upper and lower values of the deviances. The vertical black lines define the optimal values of λ, where the model provides its best fit to the data. The optimal λ value of 0.1059 with log (λ) = − 2.25 was selected. b LASSO coefficient profiles of the 133 texture features. The dotted vertical line was plotted at the value selected using 10-fold cross-validation in a. The three resulting features with non-zero coefficients are indicated in the plot. c Combo chart of three radiomics features selected by LASSO regression and radiomics score. Combo chart included box plot, density plot, and dot plot. 25th and 75th percentiles connecting lines between groups. COVID-19: 2019 novel coronavirus

Fig. 3

Radiomics feature maps of the three selected features and radiomics scores. a, b 60-year-old woman with 2019 novel coronavirus (COVID-19) pneumonia. Three-dimensional volume-rendered reconstruction and CT image of lung parenchyma automatically label by artificial intelligence software showing multi-focal crazy-paving pattern and consolidation in the both inferior lobes. c, d 37-year-old woman with influenza infection. Three-dimensional volume-rendered reconstruction and CT image of lung parenchyma automatically label by artificial intelligence software showing crazy-paving pattern and consolidation in the right inferior lobe (e) The comparison between 2019-nCoV pneumonia and influenza infection of the above two patients. Wavelet-HHL.firstorder.RootMeanSquared, wavelet-HHL.glcm.Correlation, and wavelet-LLL.firstorder.10. Percentile and radiomics score were − 0.980, 0.434, − 1.243, and 1.722, and 0.415, − 1.001, 2.107, and − 0.711, respectively

Univariable analysis of each parameter

The results of our univariable analysis are shown in Table 2. Clinical characteristics, including smoking history, fever in the training and total set, cough in the validation set, lymphocyte (normal group), and CRP were significantly associated with COVID-19 pneumonia. As for imaging characteristics in the total set, distribution, maximum lesion range > 10 cm, involvement of 5 lobes, presence of hilar and mediastinal lymph node enlargement, and no pleural effusion were significantly associated with COVID-19 pneumonia.

Table 2

The result of univariate analysis

Variables	Training set		Validation set		Total
	OR (95% CI)	p value	OR (95% CI)	p value	OR (95% CI)	p value
Sex, n (%)
Male	1.0 (reference)		1.0 (reference)		1.0 (reference)
Female	1.08 (0.65, 1.80)	0.768	0.75 (0.32, 1.76)	0.510	0.98 (0.63, 1.52)	0.932
Age, median (range), years	1.01 (0.99, 1.02)	0.254	1.01 (0.98, 1.04)	0.434	1.01 (1.00, 1.02)	0.170
Smoking history, n (%)
No	1.0 (reference)		1.0 (reference)		1.0 (reference)
Yes	0.42 (0.17, 0.99)	0.048	0.21 (0.06, 0.76)	0.017	0.33 (0.16, 0.68)	0.003
Fever, n (%)
No	1.0 (reference)		1.0 (reference)		1.0 (reference)
Yes	3.32 (1.67, 6.57)	0.0006	3.37 (0.91, 12.54)	0.070	3.33 (1.81, 6.10)	0.0001
Cough, n (%)
No	1.0 (reference)		1.0 (reference)		1.0 (reference)
Yes	1.05 (0.56, 1.96)	0.885	9.64 (2.47, 37.52)	0.001	1.70 (0.98, 2.93)	0.058
WBC, n (%)
Decreased	1.0 (reference)		1.0 (reference)		1.0 (reference)
Normal	0.13 (0.02, 1.02)	0.052	2.32 (0.37, 14.70)	0.371	0.46 (0.14, 1.47)	0.187
Increased	0.30 (0.03, 2.83)	0.294	NA		1.35 (0.33, 5.58)	0.681
Lymphocyte, n (%)
Decreased	1.0 (reference)		1.0 (reference)		1.0 (reference)
Normal	2.58 (1.49, 4.46)	0.0007	3.71 (1.49, 9.23)	0.005	2.85 (1.78, 4.55)	< 0.0001
Increased	0.33 (0.09, 1.25)	0.103	NA		0.34 (0.09, 1.30)	0.115
CRP, n (%)
Normal	1.0 (reference)		1.0 (reference)		1.0 (reference)
Increased	0.38 (0.21, 0.68)	0.001	0.36 (0.14, 0.92)	0.033	0.37 (0.22, 0.61)	0.0001
Location, n (%)
Right lung	1.0 (reference)		1.0 (reference)		1.0 (reference)
Left lung	1.40 (0.11, 18.62)	0.799	0.00 (0.00, Inf)	0.992	0.90 (0.11, 7.31)	0.918
Bilateral lungs	0.91 (0.34, 2.49)	0.860	2.27 (0.63, 8.11)	0.209	1.28 (0.59, 2.80)	0.533
Distribution, n (%)
Central or peripheral	1.0 (reference)		1.0 (reference)		1.0 (reference)
Diffuse	0.21 (0.10, 0.42)	< 0.0001	0.47 (0.17, 1.26)	0.133	0.27 (0.15, 0.47)	< 0.0001
Attenuation, n (%)
Ground glass shape	1.0 (reference)		1.0 (reference)		1.0 (reference)
Consolidation	1.58 (0.92, 2.72)	0.099	0.65 (0.25, 1.66)	0.365	1.25 (0.79, 2.00)	0.341
Maximum lesion range, n (%)
≤ 5 cm	1.0 (reference)		1.0 (reference)		1.0 (reference)
5–10 cm	1.29 (0.70, 2.39)	0.421	0.56 (0.19, 1.63)	0.288	1.03 (0.61, 1.75)	0.902
> 10 cm	2.49 (1.18, 5.24)	0.016	2.50 (0.54, 11.62)	0.242	2.43 (1.25, 4.70)	0.009
Number of lesions, n (%)
1–2	1.0 (reference)		1.0 (reference)		1.0 (reference)
≥ 3	0.99 (0.22, 4.52)	0.990	1.69 (0.23, 12.57)	0.610	1.20 (0.36, 4.02)	0.768
Lobe involvement, n (%)
≤ 4	1.0 (reference)		1.0 (reference)		1.0 (reference)
5	1.54 (0.87, 2.71)	0.137	4.12 (1.63, 10.43)	0.003	2.03 (1.25, 3.29)	0.004
Air bronchogram, n (%)
No	1.0 (reference)		1.0 (reference)		1.0 (reference)
Yes	0.93 (0.55, 1.59)	0.799	0.70 (0.29, 1.68)	0.429	0.86 (0.55, 1.36)	0.528
Hilar and mediastinal lymph nodes enlargement, n (%)
No	1.0 (reference)		1.0 (reference)		1.0 (reference)
Yes	5.02 (2.60, 9.68)	< 0.0001	1.16 (0.47, 2.87)	0.745	3.19 (1.89, 5.36)	< 0.0001
Pleural effusion, n (%)
No	1.0 (reference)		1.0 (reference)		1.0 (reference)
Yes	0.39 (0.20, 0.75)	0.005	0.64 (0.21, 1.97)	0.439	0.44 (0.25, 0.78)	0.005

WBC, white blood cell; CRP, C-reactive protein; ground glass shape, included ground glass opacity and crazy-paving pattern; OR, odds ratio; CI, confidence interval

Apparent performance of clinical model

The multivariable logistic regression analysis included distribution, maximum lesion, hilar and mediastinal lymph node enlargement, and pleural effusion (Table 3). The final clinical model yielded AUCs of 0.819 (95% confidence interval [CI], 0.765–0.873) and 0.626 (95% CI, 0.502–0.749) in the training and validation samples, respectively (Fig. 4). The sensitivity, specificity, and accuracy of the model for the training sample were 81.6%, 70.9%, and 0.770, respectively, whereas those of the validation sample were 76.8%, 52.9%, and 0.678, respectively.

Table 3

Risk factors for novel coronavirus pneumonia

Variable	Clinical model		Mixed model
	OR (95% CI)	p value	OR (95% CI)	p value
Diffuse distribution	0.10 (0.04, 0.24)	0.00001	0.04 (0.01, 0.22)	0.0002
Maximum lesion range 5–10 cm	2.58 (1.12, 5.90)	0.025	1.84 (0.49, 6.97)	0.368
Maximum lesion range > 10 cm	7.14 (2.58, 19.73)	0.0002	5.16 (1.05, 25.28)	0.043
Hilar and mediastinal lymph nodes enlargement	7.70 (3.57, 16.63)	0.0001	3.91 (1.22, 12.49)	0.022
Presence of pleural effusion	0.26 (0.11, 0.61)	0.002	0.39 (0.10, 1.49)	0.170
Radiomics score	NA	NA	9.06 (5.01, 16.37)	0.00001

OR, odds ratio; CI, confidence interval

Fig. 4

Receiver operating characteristic curves of clinical model in the training and validation sets

Development, performance, and validation of prediction models

Logistic regression analysis identified the Rad-score and 4 characteristics of the clinical model (Table 3). A mixed model was developed and presented as a nomogram (Fig. 5a). The mixed model showed the highest discrimination between COVID-19 pneumonia and other types of viral pneumonia, with an AUC of 0.959 (95% CI 0.933–0.985) in the training set. In the validation sample, the radiomics model yielded the greatest AUC (0.955; 95% CI 0.899–0.995) (Fig. 5b), which confirmed that the mixed model achieved better predictive efficacy than the clinical model (p = 0.0001). The sensitivity, specificity, and accuracy of the model for the training sample were 92.65%, 93.20%, and 0.829, respectively, whereas those of the validation sample were 92.86%, 97.06%, and 0.944, respectively.

Fig. 5

Mixed nomogram developed with receiver operating characteristic and calibration curves. a A mixed nomogram was developed in the training set, incorporating the radiomics signature and CT findings. b Receiver operating characteristic curves in the training and validation sets. Calibration curves of the radiomics nomogram in the training (c) and validation sets (d)

The calibration curve of the mixed nomogram demonstrated good agreement between predicted and observed COVID-19 pneumonia in the training set (Fig. 5c, d). The Hosmer-Lemeshow test yielded a p value of 0.39, suggesting no departure from the good fit. The favorable calibration of the radiomics nomogram was further confirmed in the validation sample, where the Hosmer-Lemeshow test yielded a p value of 0.89, suggesting a perfect fit of the nomogram.

Clinical use

The decision curves in the validation set showed that if the threshold probability is between 0.125 and 0.9, and more than 0.04 in clinical and mixed models, using the radiomics nomogram to predict COVID-19 pneumonia adds more benefit than the treat-all-patients as COVID-19 pneumonia scheme or the treat-all-patients as other types of viral pneumonia scheme (Fig. 6).

Fig. 6

Decision curve analysis (DCA) for the Rad-score. DCA for the clinical and mixed model. The y-axis represents the net benefit. The red line represents the radiomics nomogram. The gray line represents the hypothesis that all patients had 2019 novel coronavirus (2019-nCoV) pneumonia. The black line represents the hypothesis that all patients had other types of viral pneumonia. The x-axis represents the threshold probability, which is where the expected benefit of 2019 COVID pneumonia is equal to the expected benefit of other types of viral pneumonia. The decision curves in the validation set showed that if the threshold probability is between 0.125 and 0.9, and more than 0.04 in clinical model and mixed model, using the radiomics nomogram in the current study to predict 2019 COVID pneumonia adds more benefit than the treat-all-patients as 2019 COVID pneumonia scheme or the treat-all-patients as other types of viral pneumonia scheme

Discussion

In our study, we developed and validated a radiomics-based model that incorporates the radiomics signature and distribution, maximum lesion, hilar and mediastinal lymph node enlargement, and pleural effusion for non-invasive, individualized prediction of COVID-19 pneumonia. The radiomics nomogram demonstrated favorable discrimination in both the training set (AUC, 0.959) and the validation set cohort (AUC, 0.955) and good calibration, which confirms that the mixed model achieved better predictive efficacy than clinical model. DCA indicated the clinical usefulness of the mixed model.

Our study shows the following imaging characteristics of COVID-19 pneumonia: bilateral lungs were the most commonly affected location (91.15%), while the most common type of distribution was diffuse (63.54%). The most common attenuation was a mixed pattern of GGO and consolidation (69.27%). The vast majority of patients with COVID-19 pneumonia developed a large lesion range (5–10 cm, 47.40%; and > 10 cm, 29.69%), ≥ 3 lesions (96.88%), involving 5 lobes (76.56%). Shi et al [26] described the 81 patients with COVID-19 pneumonia and found the imaging features of COVID-19 pneumonia were bilateral, subpleural, they showed GGO with air bronchograms, ill-defined margins, and a slight predominance in the right lower lobe. Pan et al [8] assessed the imaging findings of 63 patients with COVID-19 pneumonia and found that 7 (11.1%) patients had 4 affected lobes, 28 (44.4%) patients had 5 affected lobes, and 54 (85.7%) patients showed patchy/punctate GGO. In addition, some earlier studies assessed CT findings of COVID-19 pneumonia, including GGO, and reported crazy-paving patterns, consolidation, the involvement of multiple lobes, and a diffuse distribution [11–14, 27–29]. The current results are thus consistent with the findings of previous studies.

The present study also provides some new findings compared with other types of viral pneumonia. Firstly, we found that a diffuse distribution was associated with a 73% decrease risk of COVID-19 pneumonia, compared with central or peripheral distribution. Furthermore, a maximum lesion range > 10 cm was associated with a 2.43-fold risk of COVID-19 pneumonia, compared with a maximum lesion range ≤ 5 cm, and the involvement of 5 lobes was associated with a 2.03-fold risk of COVID-19 pneumonia, compared with the involvement of ≤ 4 lobes. These findings show that a larger area and a greater number of lobes were involved in patients with COVID-19 pneumonia than in those with other types of viral pneumonia, but the distribution was central or peripheral, especially peripheral, which was different from the diffuse distribution of other vital pneumonia. Secondly, the presence of pleural effusion was associated with a 0.56 decrease risk of COVID-19 pneumonia, compared with no pleural effusion. Notably, hilar and mediastinal lymph node enlargement was found in a few patients and was observed in 13.54% and 26.28% of patients with COVID-19 pneumonia and other types of viral pneumonia, respectively. However, hilar and mediastinal lymph nodes enlargement was associated with a 3.19-fold risk of COVID-19 pneumonia.

There is much interest in the use of radiomics for assessing radiological image data. Radiomics has been widely used in the assessment of pulmonary nodules or masses, especially lung cancer [30, 31]. However, there are few studies on predicting COVID-19 pneumonia using CT radiomics [15-17]. Li et al [15] developed a deep learning model on chest CT exams, and the model showed good discrimination in COVID-19 (AUC, 0.96) and in community-acquired pneumonia (AUC, 0.95). Wang et al [16] used a deep learning algorithm to screen CT images for COVID-19 pneumonia. The internal 45 validation achieved a total accuracy of 89.5% with specificity of 0.88 and sensitivity of 0.87. The external testing dataset showed a total accuracy of 79.3% with specificity of 0.83 and sensitivity of 0.67. Gozes et al [17] used deep learning CT image analysis and achieved classification results for coronavirus vs non-coronavirus cases per thoracic CT studies of 0.996 AUC (95% CI 0.989–1.00) on Chinese control and infected patients. The above studies all used deep learning CT image analysis. In our study, our nomogram performed well in the training set (AUC, 0.959), where out of 56 patients with COVID-19 pneumonia, 49 patients initially showed a negative result by viral nucleic acid detection, and 7 patients were from another hospital. Our nomogram performed well in the validation set as well (AUC, 0.955), and it also showed good calibration in both the training and validation samples.

To assess the models beyond the purely mathematical perspective provided by performance measures such as the AUC, DCA was used to estimate the predicted net benefit of the model across all possible risk thresholds and to thereby evaluate the effects of various risk thresholds [32, 33]. DCA showed that if the threshold probability was between 0.125 and 0.9, and more than 0.04 in the clinical model and mixed model, using the radiomics nomogram in the current study to predict COVID-19 pneumonia adds more benefit than the treat-all-patients as COVID-19 pneumonia scheme or the treat-all-patients as other types of viral pneumonia scheme.

Our study had several limitations. First, it was retrospective in nature. Second, the group of patients with other types of viral pneumonia only included patients with several common viruses, which might have led to a potential bias. Third, there were only a few patients on whom external validation was performed. Finally, there are some clinical questions of COVID-19 pneumonia not to be resolved, such as classification of severity and exploration of clinical relationship. In the future, we will incorporate the clinical, radiological, and selected radiomics features to develop a deep learning model, and also should focus on multicenter validation with a larger sample size to obtain high-level evidence for the clinical application of the deep learning model. Furthermore, AI rapid productization let radiologists rapidly and conveniently provide the structured reporting for patients.

To summarize, we developed a radiomics prediction model to improve the accuracy that can be achieved using the clinical model and help doctors make more accurate clinical decisions and avoid misdiagnosis.

(DOCX 52.7 kb)