Diagnostic performance of chest CT to differentiate COVID-19 pneumonia in non-high-epidemic area in Japan.

PURPOSE: To evaluate the diagnostic performance of chest CT to differentiate coronavirus disease 2019 (COVID-19) pneumonia in non-high-epidemic area in Japan.
MATERIALS AND METHODS: This retrospective study included 21 patients clinically suspected COVID-19 pneumonia and underwent chest CT more than 3 days after the symptom onset: six patients confirmed COVID-19 pneumonia by real-time reverse-transcription polymerase chain reaction (RT-PCR) and 15 patients proved uninfected. Using a Likert scale and its receiver operating characteristic curve analysis, two radiologists (R1/R2) evaluated the diagnostic performance of the five CT criteria: (1) ground glass opacity (GGO)-predominant lesions, (2) GGO- and peripheral-predominant lesions, (3) bilateral GGO-predominant lesions; (4) bilateral GGO- and peripheral-predominant lesions, and (5) bilateral GGO- and peripheral-predominant lesions without nodules, airway abnormalities, pleural effusion, and mediastinal lymphadenopathy.
RESULTS: All patients confirmed COVID-19 pneumonia had bilateral GGO- and peripheral-predominant lesions without airway abnormalities, mediastinal lymphadenopathy, and pleural effusion. The five CT criteria showed moderate to excellent diagnostic performance with area under the curves (AUCs) ranging 0.77-0.88 for R1 and 0.78-0.92 for R2. The criterion (e) showed the highest AUC.
CONCLUSION: Chest CT would play a supplemental role to differentiate COVID-19 pneumonia from other respiratory diseases presenting with similar symptoms in a clinical setting.

Introduction

Since December 2019, an outbreak of pneumonia caused by a novel coronavirus (SARS-CoV-2) named COVID-19 (coronavirus disease 2019) originating in Wuhan City, Hubei Province of China, has spread around the world. It has become a public health emergency of international concern [1, 2]. There are 109,577 confirmed cases, 28,673 of which are in 104 countries outside of China on March 9th, 2020 [3]. Japan is one of the countries on the brink of an expansion with 457 cases by 12:00 March 10, 2020 [4]. Though the incidence still remains low in comparison with China, the number of cases difficult to identify infection routes is gradually increasing. It is considered that Japan has entered a new phase of the outbreak, switching focus from screening overseas arrivals to catching homegrown cases.

The diagnosis of COVID-19 is laboratory confirmed by a positive real-time reverse-transcription polymerase chain reaction (RT-PCR) result at present [5]. Notable false-negative rates of RT-PCR for COVID-19 have been reported as a limitation [6-8]. Due to its wide availability and fast examination time, the supplemental role of chest computed tomography (CT) to RT-PCR is growing for early detection of patients with COVID-19 pneumonia [6, 7, 9].

Emerging reports revealed that COVID-19 pneumonia have characteristic chest CT features and time-course changes [10-13]. Tao et al. also reported high sensitivity of chest CT (97%) among patients with positive RT-PCR results in Wuhan, the epicenter [7]. However, the reported CT features of COVID-19 pneumonia, such as ground glass opacity (GGO) or bilateral involvement [10], are nonspecific and could be found in other infections and inflammatory conditions [14]. Thus, the diagnostic performance of chest CT in a clinical setting, where the incidence of COVID-19 pneumonia is lower than that of other respiratory diseases, is still unknown. Considering the rapid spread around the world, there is an urgent need to investigate the diagnostic performance of chest CT in the differentiation of COVID-19 pneumonia from other respiratory diseases presenting with similar symptoms.

The purposes of this study were to assess the CT features of COVID-19 pneumonia in Japan and to investigate the diagnostic performance of chest CT in a clinical setting with a low incidence of COVID-19 pneumonia.

Materials and methods

Patients

This retrospective study was approved by the institutional review board and patient consent was waived. Consecutive 32 patients clinically suspected COVID-19 pneumonia by physicians of the Department of Infectious Diseases at our institution between February 14th and March 1st were enrolled in this study. Among them, eight patients with only chest CT obtained within 3 days after the symptom onset were excluded to avoid false negative results at an early stage previously reported [11, 15]. Three patients with no chest CT were also excluded (all excluded cases were proved uninfected later). Thus, the final cohort consisted of 21 patients. The following clinical information was extracted from patient’ electronic medical records: age, gender, exposure history, the result of RT-PCR, the duration between the symptom onset and chest CT or RT-PCR, and the confirmed diagnosis.

The protocols of chest CT

Computed tomography scans of 14 patients were acquired using multidetector CT scanners with 64 or 320 detector rows (Aquillion CX or Aquillion ONE, Canon Medical Systems, Tochigi, Japan) at our institution. Computed tomography scans of remained seven patients were performed using multidetector CT scanners with 16 or 64 detector rows (Aquillion, Aquillion CXL, or Aquillion lightning, Canon Medical Systems, Tochigi, Japan; Bright Speed, GE Medical Systems, Wisconsin, USA) at outside hospitals. All scans were obtained in the supine position during end-inspiration without intravenous contrast material. Acquisition parameters for all CT scans were as follows: reconstructed slice thickness of 5–7 mm; 120 kVp; automatic milliampere setting with range of 60–420 mA. Thin-slice images with 1- or 2-mm thickness were also available in 18 patients.

CT analysis

Blinded to clinical information, two senior radiology residents (R1/R2, both have 3 years experience of general radiology) independently assessed chest CT scans of all cases on the institutional picture archiving and communication systems. When thin-slice CT images were available, those were also reviewed. They assessed the following features: (1) the presence or absence of GGO and/or consolidation, (2) distribution (peripheral predominance and bilaterality), (3) characteristics of opacities (rounded morphology, cavitation, crazy-paving pattern, and reversed halo sign), (4) airway abnormalities (bronchial wall thickening and airway secretions), (5) underlying lung disease (pulmonary emphysema and fibrosis), (6) other findings [pulmonary nodules, pleural effusion, and mediastinal lymphadenopathy (≧ 10 mm in short-axis dimension)], (7) a number of lobes affected, and (8) a total lung severity score. Assessments of GGO, consolidation, cavity, crazy-paving pattern, and reversed halo sign were according to the definitions of Fleischner Society [16]. A total lung severity score was defined as the sum of the degree of involvement of five lobes (0—no involvement; 1—minimal (1–25%); 2—mild (26–50%); 3—moderate (51–75%); 4—severe (76–100%); ranges of possible scores, 0–20) [10]. If the results from the two primary readers were discordant, a third board-certified radiologist with 12 years of experience adjudicated a final determination.

Subsequently, R1 and R2 independently assessed chest CT scans of all cases according to the five criteria using a Likert scale with scores ranging from 1 to 5 (1—definitely no; 2—probably no; 3—equivocal; 4—probably yes; 5—definitely yes) as to (a) GGO-predominant lesions; (b) GGO- and peripheral-predominant lesions; (c) bilateral GGO-predominant lesions; (d) bilateral GGO- and peripheral-predominant lesions; (e) bilateral GGO- and peripheral-predominant lesions without airway abnormalities, nodules, mediastinal lymphadenopathy, and pleural effusion.

Statistics

The prevalence of respective chest CT features was compared using Fisher’s exact test between COVID-19 pneumonia and other respiratory diseases. Welch’s t test was utilized for the continuous variables. p values less than 0.05 were considered statistically significant.

Receiver operating characteristic (ROC) curve analyses were performed for the Likert scales for the five criteria of R1/R2. Area under the curve (AUC) and Youden index were respectively computed. The standard error of the AUC was calculated using the DeLong method [17]. Area under the curve greater than 0.90 was considered excellent diagnostic performance; 0.81–0.90, very good; 0.71–0.80, good; 0.61–0.70, moderate; 0.51–0.60, poor; less than or equal to 0.50, test not useful [18].

Next, all Likert scores were dichotomized with scores of 1–3 indicating negative and scores of 4–5 indicating positive for COVID-19 pneumonia. Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the five criteria were calculated for each reader.

Inter-rater agreement of the Likert scores for the five criteria between R1 and R2 were assessed with the linearly weighted kappa statistic. Values greater than 0.80 were considered excellent; 0.61–0.80, good; 0.41–0.60 moderate; 0.21–0.40 fair; less than or equal to 0.20, poor [19].

Statistical analyses were performed using STATA® version 13 (StataCorp, Texas, USA) or Medcalc® version 18.5 (MedCalc Softwear, Ostend, Belgium).

Results

Patient characteristics are summarized in Table 1. Six patients (29%) were diagnosed COVID-19 pneumonia with positive RT-PCR. The number of days between symptom onset and positive RT-PCR ranged from 1 to 19 days (median 14 days). Fifteen patients were confirmed negative for COVID-19 with negative RT-PCR and/or careful observation more than 2 weeks (71%). The final diagnoses of them were as follows: pneumococcal pneumonia, n = 1; Moraxella pneumonia, n = 1; Legionella pneumonia, n = 1; bacterial pneumonia not specified, n = 3; Pneumocystis pneumonia, n = 1; viral pneumonia not specified, n = 3; viral upper respiratory tract infection, n = 3 (the cases with no abnormalities in chest CT); interstitial pneumonia, n = 2.

Table 1

Patient characteristics

	COVID-19 pneumonia (n = 6)	Others (n = 15)
Age (years), median (range)	58.5 (45–81)	66 (28–87)
Gender
Men	5	7
Women	1	8
Exposure history
Recent travel to China	0	0
Exposure to infected patients	3	0
Unknown exposure	3	15
RT-PCR testing
Performed	6	8
Not performed	0	7
The interval between the onset and chest CT (day), median (range)	9.5 (4–25)	7 (5–26)
The interval between the onset and positive results of RT-PCR (day), median (range)	14 (1–19)	NA

COVID-19 coronavirus disease 2019, RT-PCR real-time reverse-transcription polymerase chain reaction, NA not applicable

CT features

The assessments of CT features are shown in Table 2. In all six cases of COVID-19 pneumonia, GGO with or without consolidation, peripheral-predominant distribution, bilateral lesions were observed on chest CT. Rounded morphology of opacities was statistically associated with COVID-19 pneumonia (p = 0.003), observed in four out of six cases (67%). All other qualitative CT features were not statistically significant. Representative cases of COVID-19 pneumonia and other respiratory diseases are shown in Figs. 1 and 2.

Table 2

The assessments of chest CT features

	COVID-19 pneumonia (n = 6)	Others (n = 15)	p value
GGO and consolidation
No abnormalities	0	3
Presence of GGO without consolidation	4	5
Presence of GGO with consolidation	2	6
Presence of consolidation without GGO	0	1
Distribution
Peripheral predominance	6	11	0.28
Bilateral lung disease	6	10	0.26
Characteristics of opacities
Rounded morphology	4	0	0.003
Cavitation	0	0	NA
Crazy paving pattern	0	3	0.53
Reversed halo sign	0	0	NA
Airway abnormalities
Bronchial wall thickening	0	5	0.26
Airways secretions	0	3	0.53
Underlying lung disease
Pulmonary emphysema	1	1	0.5
Pulmonary fibrosis	0	0	NA
Other findings
Pulmonary nodules	2	5	1
Pleural effusion	0	3	0.53
Mediastinal lymphadenopathy	0	2	1
Number of lobes affected
Median, range	4.5 (3–5)	4.0 (0–5)	0.028
0	0	3
1	0	1
2	0	2
3	1	0
4	2	6
5	3	3
Total lung severity score
Median (range)	5.0 (3–8)	4.0 (0–12)	0.53

COVID-19 coronavirus disease 2019, GGO ground glass opacity, NA not applicable

Fig. 1

Typical CT findings in patients with COVID-19 pneumonia. a A transverse CT image in a 67-year-old man with fever (day 4). CT shows rounded, peripheral ground glass lesion in the right lower lobe. b CT image in a 56-year-old man presented with fever (day 11). CT shows peripheral glass opacities with superimposed fine reticular shadows in the left lower lobe

Fig. 2

CT images from patients with negative RT-PCR test for COVID-19. a A 66-year-old female patient presenting with persistent fever (day 8). CT shows curved linear consolidation with peripheral distribution as well as bronchial wall thickening. This patient was diagnosed as Moraxella Pneumonia. b A 70-year-old female patients with pancreatic cancer presenting with fever (day 7). CT shows extensive ground glass lesion with crazy paving pattern in left upper lobe. Note these shadows are not strongly in peripheral distribution. The patient was diagnosed as pnuemocystis jirovecii pneumonia

About continuous variables, a number of lobes affected in cases of COVID-19 pneumonia was significantly higher than that in cases of other respiratory diseases (p = 0.028). No associations were observed between a total lung severity score and COVID-19 pneumonia.

Diagnostic performance

Figure 3 shows the ROC curves of the five criteria for R1 and R2. The diagnostic performances of the five chest CT criteria were good to very good for R1 (AUCs ranged 0.77–0.88) and good to excellent for R2 (AUCs ranged 0.78–0.92). The criterion of (e) (bilateral GGO- and peripheral-predominant lesions without airway abnormalities, nodules, mediastinal lymphadenopathy, and pleural effusion) showed highest AUC and specificity in both readers, while sensitivity (67%) was lower than others for R1. The sensitivity of other four criteria was same: 100% for R1, and 83% for R2. The details are summarized in Table 3.

Fig. 3

The receiver operating characteristic curves of the five chest CT criteria for R1(a) and R2(b). Respective curves represent the criteria of a GGO-predominant lesions, b GGO- and peripheral-predominant lesions, c bilateral GGO-predominant lesions, d bilateral GGO- and peripheral-predominant lesions, and e bilateral GGO- and peripheral-predominant lesions without airway abnormalities, nodules, mediastinal lymphadenopathy, and pleural effusion

Table 3

Diagnostic performance of the chest CT criteria for the two readers (R1/R2) in the differentiation of COVID-19 pneumonia from other respiratory diseases

		GGO-predominant lesions	GGO- and peripheral-predominant lesions	Bilateral GGO-predominant lesions	Bilateral GGO- and peripheral-predominant lesions	Bilateral GGO- and peripheral-predominant lesions without airway abnormalities, nodules, mLN, and pleural effusion
R1	AUC (SE, 95% CI)	0.77 (0.07, 0.53–0.92)	0.81 (0.09, 0.58–0.95)	0.83 (0.06, 0.61–0.96)	0.81 (0.09, 0.58–0.95)	0.88 (0.07, 0.67–0.98)
	Youden Index	0.53	0.73	0.67	0.73	0.6
	Sensitivity	100%	100%	100%	100%	67%
	Specificity	53%	73%	67%	73%	93%
	PPV	46%	60%	55%	60%	80%
	NPV	100%	100%	100%	100%	88%
	Accuracy	67%	81%	76%	81%	86%
R2	AUC (SE,95% CI)	0.81 (0.09, 0.58–0.95)	0.78 (0.09, 0.55–0.93)	0.86 (0.08, 0.64–0.97)	0.81 (0.09, 0.58–0.95)	0.92 (0.06, 0.71–0.99)
	Youden Index	0.57	0.53	0.63	0.53	0.67
	Sensitivity	83%	83%	83%	83%	83%
	Specificity	60%	60%	67%	60%	80%
	PPV	45%	45%	50%	45%	63%
	NPV	90%	90%	91%	90%	92%
	Accuracy	67%	67%	71%	67%	81%

GGO ground glass opacity, mLN  mediastinal lymphadenopathy, AUC area under the curve, SE standard error, CI confidence interval, PPV positive predictive value, NPV negative predictive value

The inter-rater agreements of the Likert scores for the five criteria between the two readers ranged moderate to excellent (0.58–0.83, listed on Supplemental Table 1).

Discussion

Effective triage and early detection of COVID-19 are essential for disease control and treatment. After the outbreak, the utility of chest CT has been reported in quick succession [6, 7, 10, 11, 13]. Due to the limited number of RT-PCR kits and the possibility of false negative results, chest CT is becoming recognized as a vital component in the diagnostic algorithm for patients with suspected COVID-19 pneumonia [9]. In this study, we analyzed the CT features of COVID-19 pneumonia (n = 6) in Japan, compared with those of other respiratory diseases initially suspected COVID-19 pneumonia. In concordance with previous reports, all six cases showed bilateral GGO predominantly in the peripheries of the lung, without cavitation, airway abnormalities, pleural effusion, and mediastinal lymphadenopathy.

None of any single feature assessed was statistically significant between the two groups, except for rounded morphology of opacities and numbers of affected lobes. On the other hand, our chest CT criteria to differentiate COVID-19 pneumonia from other respiratory diseases showed good to excellent diagnostic performance: AUC ranging 0.77–0.88 for R1 and 0.78–0.92 for R2 (Table 3). While the criteria (a) to (d) showed the highest sensitivity (100% for R1 and 83% for R2), the criterion (e) showed the highest specificity (93% for R1 and 80% for R2) with lower sensitivity for R1 (67%). Our criteria were created with simple CT features easy to understand, so that they can be clinically feasible. The combination of these criteria could be a powerful tool to triage patients based on the estimation of the probability of COVID-19 in a clinical setting. For example, patients who meet the criterion (e) should be highly suspicious for COVID-19 pneumonia, compared with patients who meet all the criteria but the criterion (e).

Specificities of all our CT criteria (53–93% for R1 and 60–80% for R2) were higher than that Tao et al. previously reported (25%) [7]. This might be because we only included chest CT scans performed more than 3 days after the symptom onset. The differences of chest CT images between COVID-19 pneumonia and other respiratory diseases might become clearer after this time interval. Another reason might be the difference of prior probabilities. Bai et al. reported the great variation of specificity (24–94%) among seven readers in the differentiation of COVID-19 pneumonia (cases in China) from viral pneumonia (cases in USA) [13]. In our studies, the specificities for the respective criteria showed less variation as above. The difference might be due to our simple and clear criteria with moderate to excellent inter-reader concordance (weighted kappa, 0.58–0.83, supplemental Table 1), as well as the difference of the control group.

There are some limitations. First, the number of patients confirmed COVID-19 pneumonia was relatively small. This study focused on the diagnostic performance of chest CT to differentiate COVID-19 pneumonia from other respiratory diseases initially suspected COVID-19 pneumonia. The diagnostic performance should be evaluated further in a larger scale study in future. Second, there would be a selection bias due to its retrospective study design. Third, we included patients suspected COVID-19 by physicians of the Department of Infectious Diseases, which might lead another selection bias. The differences of prior probabilities might affect the results.

In conclusion, chest CT would play an important supplemental role to triage and detect patients suspected COVID-19 pneumonia more than other pneumonia, before getting the results of RT-PCR.

Below is the link to the electronic supplementary material.

Supplementary file1 (XLSX 10 kb)