Chest CT for detecting COVID-19: a systematic review and meta-analysis of diagnostic accuracy.

OBJECTIVE: The purpose of this article was to perform a systematic review and meta-analysis regarding the diagnostic test accuracy of chest CT for detecting coronavirus disease 2019 (COVID-19).
METHODS: PubMed, Embase, Web of Science, and CNKI were searched up to March 12, 2020. We included studies providing information regarding diagnostic test accuracy of chest CT for COVID-19 detection. The methodological quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool. Sensitivity and specificity were pooled.
RESULTS: Sixteen studies (n = 3186 patients) were included. The risks of bias in all studies were moderate in general. Pooled sensitivity was 92% (95% CI = 86-96%), and two studies reported specificity (25% [95% CI = 22-30%] and 33% [95% CI = 23-44%], respectively). There was substantial heterogeneity according to Cochran's Q test (p < 0.01) and Higgins I2 heterogeneity index (96% for sensitivity). After dividing the studies into two groups based on the study site, we found that the sensitivity of chest CT was great in Wuhan (the most affected city by the epidemic) and the sensitivity values were very close to each other (97%, 96%, and 99%, respectively). In the regions other than Wuhan, the sensitivity varied from 61 to 98%.
CONCLUSION: Chest CT offers the great sensitivity for detecting COVID-19, especially in a region with severe epidemic situation. However, the specificity is low. In the context of emergency disease control, chest CT provides a fast, convenient, and effective method to early recognize suspicious cases and might contribute to confine epidemic. KEY POINTS: • Chest CT has a high sensitivity for detecting COVID-19, especially in a region with severe epidemic, which is helpful to early recognize suspicious cases and might contribute to confine epidemic.

Introduction

In December 2019, an outbreak of pneumonia associated with a novel coronavirus called severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was reported in Wuhan, Hubei Province, China [1]. Thereafter, the WHO named the disease as coronavirus disease 2019 (COVID-19). With virus spreading globally, the WHO characterizes COVID-19 as a pandemic.

The keys to control COVID-19 are early discovery, early isolation, and early treatment. At present, the real-time reverse transcription polymerase chain reaction (RT-PCR) assay remains the standard of reference, but it was reported that false-negative RT-PCR was not rare and, in these patients, initial chest CT might present abnormal findings indicating COVID-19 [2]. Moreover, shortage of laboratory test kits limited the use of RT-PCR with the spread of the epidemic, especially in regions severely affected by the epidemic. Therefore, some experts suggested that chest CT could be regarded as a diagnosis standard of COVID-19. The guideline of Diagnosis and Treatment of Pneumonitis Caused by 2019-nCoV (trial sixth version) published by the China government recommended chest CT as an effective method to screen suspicious cases [3]. The addition of chest CT for diagnosis of COVID-19 resulted in tens of thousands of clinically diagnosed cases in China which played an important role in controlling epidemic situation in China [4]. Therefore, comprehensive and timely evaluation of the effectiveness of chest CT for COVID-19 diagnosis remains urgent and mandatory. In the present study, we validated the effectiveness of chest CT for COVID-19 diagnosis through a systematical meta-analysis.

Material and methods

Search strategy and eligibility criteria

PubMed, Embase, Web of Science, and CNKI (until March 31, 2020) were searched for articles that focused on the role of chest CT in diagnosis of COVID-19; there were no language restrictions. The keywords were “COVID-19” or “SARS-CoV-2” or “novel coronavirus” or “2019 nCov.” We also checked the reference lists of all key articles for any additional eligible articles. Studies were included if they met the following criteria: (1) reported the performance of chest CT in diagnosing COVID-19, (2) participants were diagnosed as COVID-19 based on the results of multiple RT-PCR, (3) studies directly or indirectly provided enough information to extract 2 × 2 table information of diagnostic test of chest CT for COVID-19, and (4) study sample was larger than 30. We excluded duplicate reports, abstracts from meeting proceedings. The selection of eligible articles was performed by 2 investigators independently. Disparities between investigators were resolved by discussion between them.

Data extraction and quality assessment

The following data were extracted from each study: study site, sample size, characteristics of participants, chest CT findings, the results of multiple RT-PCR, and 2 × 2 table information. The methodological quality was evaluated by using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. QUADAS-2 entries include 4 domains: patient selection, index test, reference standard, and flow and timing. Definitions and judgment criteria for each domain are available in Cochrane Handbook. Data extraction and quality assessment were conducted by two independent authors. Disparities between investigators were resolved by discussion between them.

Statistical analysis

A bivariate random-effects model was used to analyze and pool diagnostic performance (sensitivity and specificity) measurements across studies. The chi square–based Q test was performed to test heterogeneity among studies. And, the I2 value was used to evaluate the percentage of interstudy variation in the total variation. When p < 0.05 and/or I2 > 50%, significant heterogeneity was presumed. The meta-analysis was conducted using the “midas” and “metandi” modules in Stata 12.0 software.

Results

Characteristics of studies and quality assessment

The study selection process is presented in Fig. 1. Briefly, 663 references were identified after searching databases. Eight references were removed due to duplication. After scanning the titles and abstracts, 623 records were excluded and 32 full-text articles were assessed for eligibility. Finally, 16 studies were included in our meta-analysis involving 3186 patients [5-20]. Of these patients, 2689 patients had positive RT-PCR results. Fourteen studies only enrolled patients diagnosed as COVID-19 by RT-PCR [6–14, 16–20]. Therefore, we could only calculate sensitivity from the information provided in these studies. Three studies were conducted in Wuhan [5, 8, 12]; 12 studies were conducted in the other regions, and the remaining one collected data throughout China [6, 7, 9–11, 14–20]. The mean or median age of patients ranged from 37 to 62. Seven studies reported the proportion of severe illness which ranged from 3.8 to 41% [6, 10–12, 14, 17, 19]. The characteristics of eligible studies are summarized in Table 1.

Fig. 1

Flow diagram showing the study selection process for meta-analysis

Table 1

Characteristics of enrolled studies

Author	Sample size	Site	Gender (male, %)	Age	Severity (severe, %)	Median interval between CT and RT-PCR (days)
Ai et al [5]	1014	Wuhan, China	46	Mean age, 51	NA	1
Guan et al [6]	1099a	Throughout China	58.1	Median age, 47	15.7	NA
Xu et al [7]	62	Not Wuhan, China	56	Median age, 47	NA	NA
Li and Xia [8]	51	Wuhan, China	55	Mean age, 58	NA	3
Fang et al [9]	51	Not Wuhan, China	57	Median age, 45	NA	3
Yang et al [10]	149	Not Wuhan, China	54	Mean age, 45	8.7	NA
Wu et al [11]	80	Not Wuhan, China	51	Mean age, 46	3.8	NA
Zhang et al [12]	140a	Wuhan, China	49	Median age, 57	41	NA
Xu et al [13]	90	Not Wuhan, China	43	Mean age, 50	NA	NA
Xu et al [14]	50	Not Wuhan, China	58	Median age, 45	26.0	NA
Zhu et al [15]	116	Not Wuhan, China	46	Median age, 40	NA	< 1
Long et al [16]	36	Not Wuhan, China	56	Mean age, 45	NA	NA
Li et al [17]	78	Not Wuhan, China	49	Mean age, 45	10.3	NA
Wang et al [18]	114	Not Wuhan, China	51	Mean age, 53	NA	NA
Liu et al [19]	73	Not Wuhan, China	56	Mean age, 37	33	< 2 in 88% of patients
Inui et al [20]	112	Japan	53	Mean age, 62	NA	NA

NA not available

aNot all patients received chest CT

The risk of bias and applicability concerns for included studies are shown in Fig. 2. Overall, none of the studies was considered to be seriously flawed according to the QUADAS-2 assessment. At least 4 of 7 items of the QUADAS-2 tool were met in all studies. All studies were considered to have a low risk of bias in the patient selection and reference standard domain. Only two studies reported that the radiologists reading the chest CT images were blinded to the results of RT-PCR which was classified as low risk of bias in the index test domain [5, 20]. The remaining studies were considered to have an unclear risk of bias regarding the index test domain. With regard to the flow and timing domain, we noted that 11 studies had an ambiguous risk of bias due to the absence of mean interval data between chest CT and the RT-PCR assay [6, 7, 10–14]. And, the remaining studies reported a reasonable interval between chest CT and the RT-PCR assay (1–3 days) [5, 8, 9, 15, 19].

Fig. 2

Grouped bar charts of risk of bias (left) and concerns for applicability (right) of 10 included studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool

Diagnostic performance of chest CT for diagnosing COVID-19

Sensitivity was available in all studies ranging from 0.61 to 0.99 [5-20]. However, only two studies reported the specificity of chest CT for diagnosing COVID-19 about 25% (95% CI = 21–30%) and 33% (95% CI = 23–44%), respectively (due to the significant heterogeneity and small number of studies, it was inappropriate to pool the data). We pooled the sensitivity values (92%, 95% CI = 86–96%) (Fig. 3). Cochran’s Q test revealed a significant heterogeneity (Q = 419, p < 0.01; I2 = 96.4). Given that the severity of illness, experience of radiologists, and severity of epidemic might contribute to the heterogeneity, we classified the studies into two categories according to whether or not the study site was located in Wuhan where the epidemic was the most severe, patients suffered more severe illness and the radiologists might have more experience. Three studies were conducted in Wuhan which reported sensitivity values about 97% (95% CI = 95–98%) [5], 96% (0.87–100%) [8], and 99% (95% CI = 96–100%) [12], respectively. Due to the small number of enrolled studies, it was inappropriate to pool sensitivity among these three studies, but the sensitivity values were very close to each other. After excluding these three studies, we pooled the sensitivity values of the remaining studies. Nevertheless, the heterogeneity was still so significant that it was inappropriate to pool the sensitivity (Q = 212, p < 0.01; I2 = 94.5). The sensitivity value of the individual study ranged from 0.61 to 0.98 and is summarized in Fig. 3.

Fig. 3

Coupled forest plots of pooled sensitivity and specificity. Numbers are pooled estimates (dots within squares) with 95% CIs (horizontal lines). Corresponding heterogeneity statistics are provided at the bottom

Due to the specificity only reported in two studies [5, 15], it was inappropriate to plot hierarchical summary receiver operating characteristic (HSROC) curves.

In addition to the significant heterogeneity of the value of chest CT in diagnosing COVID-19, the CT findings in COVID-19 were also varied across enrolled studies. The characteristics of chest CT findings in COVID-19 patients with abnormal CT images are summarized in Table 2. Although the reported proportion might vary across different studies, ground-glass opacity (GGO) and consolidative opacities were some of the most common CT findings, with reporting rates about 49–94% and 11–73%, respectively. Besides, interlobular septal thickening, pleural thickening, and bronchiectasis were also reported with various rates across the studies. With regard to lesion distribution, almost all studies reported that involvement of multiple lobes of both lungs was most common (65–97%). Peripheral zone and lower lobes were more predisposed to be affected.

Table 2

CT findings in COVID-19 patients with positive chest CT from included studies

Author	No. of COVID-19 patients with abnormal chest CT	Chest CT finding	Distribution or location	Chest CT manifestation in patients with initial false-negative RT-PCR
Ai et al [5]	580	GGO (49%); consolidation (55%)	94% of patients had bilateral chest findings	15 COVID-19 patients had initial negative RT-PCR. 10/15 patients have positive initial CT findings suggesting COVID-19, and 14/15 patients had positive CT findings prior or parallel to the initial positive results
Guan et al [6]	975	GGO (56%); local patchy shadowing (42%); bilateral patchy shadowing (52%); interstitial abnormalities (15%)	NA	NA
Xu et al [7]	61	Most patients showed bilateral or multiple lobular or subsegmental areas of consolidation or bilateral GGO	84% of patients had bilateral involvement on chest CT	NA
Li and Xia [8]	51	GGO (90%); consolidation (62%); vascular enlargement (82%); thickened interlobular septa (71%); air bronchogram sign (69%); bronchus deformation due to fibrosis and strip-like lesions (20%); pleural effusion (2%)	75% of patients had all five lobes affected; 16% of patients had both lower lobes affected; 6% of patients had the right lobe affected; 2% of patients had the left upper lobe and right lower lobe affected; 2% of patients had the left upper lobe and right middle lobe affected;	NA
Fang et al [9]	50	Most patients had GGO	72% of patients had peripheral and subpleural lesions. Often in the lower lobes.	15 patients had negative initial RT-PCR results. All these patients had positive chest CT prior initial positive RT-PCR
Yang et al [10]	132	GGO and consolidation were the most common presentation	By median, each patient had 3 involved lobes; 35.9% lesions located in the periphery	NA
Wu et al [11]	55	GGO was the most common presentation	65% of patients had bilateral chest findings	NA
Zhang et al [12]	134	Most patients had bilateral multiple GGOs and consolidation	90% of patients had bilateral chest findings	NA
Xu et al [13]	69	GGO (94%); consolidation (17%); crazy-paving pattern (12%); interlobular thickening (48%); combined linear opacities (80%); air bronchogram (10%); adjacent pleura thickening (72%)	77% of patients had more than 2 lobes involved, more than half of the patients presented bilateral, multifocal lung lesions, with peripheral distribution. Lesions were inclined to distribute in the lower lobes.	NA
Xu et al [14]	41	GGO (73%); consolidation (37%); thickened intralocular septa (73%); thickened interlobular septa (80%); air bronchogram (54%)	Right upper lobe (73%); right middle lobe (54%); right lower lobe (95%); left upper lobe (80%); left lower lobe (88%). 95% of patients have more than 2 lobes involved	NA
Zhu et al [15]	30	GGO (50%); consolidation (13%); spider web sign (13%); crazy-paving pattern (3%); pleural effusion (7%)	97% of patients had bilateral chest findings	NA
Long et al [16]	35	GGO (86%); consolidation (71%); pleural effusion (6%); lymphadenopathy (3%)	Right upper lobe (53%); right middle lobe (56%); right lower lobe (72%); left upper lobe (56%); left lower lobe (67%). 72% of patients had peripheral distribution pattern of lesions.	6 patients had initial negative RT-PCR, and all these patients had initial positive CT findings
Li et al [17]	56	GGO (80%); consolidation (21%); thickened interlobular septa (45%); fibrotic lesion (54%); pleural effusion (9%)	88% of patients had peripheral distribution and 83% of patients had both lungs involved; right upper lobe (57%); right middle lobe (54%); right lower lobe (86%); left upper lobe (75%); left lower lobe (86%)	NA
Wang et al [18]	110	GGO (73%); consolidation (73%)	44% of patients had lesions only located in peripheral zone, and 56% of patients had patients located in both peripheral and central zone. 73% of patients had lesions distributing multiple lobes of both lungs	NA
Liu et al [19]	70	GGO (89%); paving stone sign (40%); consolidation (11%); thickened interlobular septa (93%); pleural effusion (4%)	74% of patients had bilateral chest findings	NA
Inui et al [20]	68	GGO and/or consolidation (100%); airway abnormalities (44%)	82% of patients had bilateral lung involvement; 56% of patients had a peripheral distribution pattern of lesions; right upper lobe (49%); right middle lobe (38%); right lower lobe (49%); left upper lobe (56%); left lower lobe (69%)	NA

GGO ground-glass opacity, NA not available

In the present review, we also summarized the performance of chest CT in patients with initial false-negative RT-PCR (Table 2). Three studies reported related information [5, 9, 16]. After combining data among these studies, 36 patients had initial false-negative RT-PCR, but 31/36 patients had positive initial chest CT.

Discussion

The main results of this meta-analysis are as follows: (1) the sensitivity of chest CT for COVID-19 was great in Wuhan but varied among other regions, (2) few studies reported the specificity of chest CT which was about 25–33%, (3) typical chest CT findings of COVID-19 were GGO and consolidative opacities which involved multiple lobes of both lungs, and (4) chest CT had a high sensitivity in patients with initial false-negative RT-PCR.

Chest CT plays an important role in detection of COVID-19, especially in the initial and peak periods of epidemic, in China. Although the RT-PCR assay remains the standard of reference, it has been reported that false-negative result after the initial test was not rare and shortage of laboratory kit in the early stage of the outbreak restricted the early detection of COVID-19. As our findings, chest CT had great sensitivity for early detection of COVID-19, especially in regions more affected by epidemic such as Wuhan. Therefore, a clinical diagnosis criterion based on typical CT imaging features was temporarily adopted in the guideline of diagnosis and treatment, which was only applicable in Hubei Province, China [3]. This move allowed to early detect a large number of clinical diagnoses of COVID-19 under the background of shortage of RT-PCR assay which contributed to effective control of epidemic situation in China. Nevertheless, as mentioned above, among regions other than Wuhan, the reported sensitivity of chest CT varied and was generally lower than that in Wuhan. Several reasons might underlie this phenomenon, such as heterogeneity of experience of radiologists, severity of illness, and epidemic. However, unfortunately, most included studies did not provide related data to further confirm this hypothesis. Wu et al [11] reported a relatively low sensitivity of chest CT (69%), and in their study, the proportion of severe patients was only about 3.8% far less than the average level of China which was about 15% [6]. The reported sensitivity of chest CT by Inui et al [20] was particularly low among enrolled studies, about 61%. Although the authors did not report the severity of enrolled patients directly, 73% of patients were asymptomatic and the remaining patients only had mild symptoms [20]. Li et al [17] and Liu et al [19] analyzed the CT results according to the severity of illness and found that almost all mild patients had normal CT images. In addition to severity of illness, Bai et al [21] investigated the performance of radiologists in reading chest CT images of COVID-19 which found the experience of radiologists had a great impact on the diagnosis accuracy of chest CT. Overall, chest CT has a great sensitivity for detecting COVID-19, especially in regions with severe epidemic situation, and is helpful to early detect suspicious cases, which is vital to control epidemic.

In contrast to the great sensitivity of chest CT, the specificity was relatively low with reporting about 25–33%. According to our review, the typical chest CT findings of COVID-19 included GGOs, consolidations, and interlobular thickening, which were usually multifocal and involved bilateral lungs, which was consistent with the results by Salehi et al [22]. In mild patients or early period of COVID-19, chest CT could be negative or pure GGO lesions. The CT imaging features might overlap between COVID-19 and other viral pneumonias, which could reduce the specificity of chest COVID-19. Due to the low specificity of chest CT which might overwhelm available recourses, especially during an influenza epidemic, the American College of Radiology discourages systemic use for diagnosing COVID-19 [23]. Nevertheless, with further investigation and more experienced radiologist, more and more features have been found to be helpful in distinguishing COVID-19 from other pneumonias, such as multifocal or bilateral involvement. So, the specificity of chest CT might improve in the future. In addition, considering the rapidly spreading epidemic of COVID-19, it was a priority to identify any suspicious case in order to isolate the patients and avoid cross infection. Therefore, in the context of emergency disease control, sensitivity was more important than specificity. On the other hand, as mentioned above, although RT-PCR was still regarded as standard reference, false-negative results were not rare. In our review, more than 5% of patients had initial false-negative RT-PCR results and turned positive after multiple tests. However, 86% of these patients presented positive chest CT before the initial negative RT-PCR results. And, more and more cases with initial false-negative RT-PCR but initial positive chest CT have been reported [24]. Last but not the least, chest CT alone could not diagnose COVID-19. According to the guideline of Diagnosis and Treatment of Pneumonitis Caused by 2019-nCoV (trial sixth version) published by the China government, typical chest CT findings could be a diagnostic criterion to screen suspected cases when combined with epidemiology history, clinical manifestations, and laboratory results [3]. Given the concerns over false-negative result and limited availability of RT-PCR and a continuing increase in global cases, the British Society of Thoracic Imaging also underscores the importance of radiographic assessment, especially when there is diagnostic uncertainty [25].

In addition to detection, chest CT also plays an important role in the management of COVID-19. As other pneumonias, the severity of COVID-19 is also positively related to chest CT findings. Intensive care unit (ICU) patients on admission often presented with bilateral multiple lobular and subsegmental consolidations, while non-ICU patients presented with bilateral GGOs and subsegmental consolidation [26]. Moreover, chest CT is helpful to monitor disease progression of COVID-19. Pan et al [27] investigated 21 confirmed patients and summarized four stages of COVID-19: early, progressive, peak, and absorption. Growth of GGOs and expansion of consolidation are indicators of disease progression and, otherwise, might indicate the improvement [28-31].

In conclusion, on the basis of limited and heterogeneous data, chest CT offers the great sensitivity for detecting COVID-19, especially in a region with severe epidemic situation. The specificity is low. In the context of emergency disease control, chest CT provides a fast, convenient, and effective method to early recognize suspicious cases and contributes to reduction of cross infection.