Value of chest computed tomography scan in diagnosis of COVID-19; a systematic review and meta-analysis.

PURPOSE: Real-time polymerase chain reaction (RT-PCR) and chest computed tomography (CT) scan are main diagnostic modalities of coronavirus disease 2019 (COVID-19). However, there is still no consensus on which of these methods is superior to the other. Therefore, the present meta-analysis was designed to answer to the question whether CT scan can be used in diagnosis of COVID-19 or not.
METHODS: Searches were performed in Medline, Embase, Scopus, and Web of Science databases until the end of April 2020. Two researchers gathered the data of diagnostic accuracy studies that had attempted to evaluate sensitivity and specificity of CT scan in diagnosis of COVID-19.
RESULTS: Data of 9 studies were included. Area under the curve of ground glass opacity (GGO), consolidation, pleural effusion, other CT features, and simultaneous observation of GGO with other CT features was 0.64 (95% CI 0.60-0.69), 0.30 (95% CI 0.26-0.34), 0.60 (95% CI 0.56-0.64), 0.61 (95% CI 0.56-0.65), and 0.90 (95% CI 0.87-0.92), respectively. Sensitivity and specificity of simultaneous observation of GGO with other CT scan features was higher than all of the other signs. Sensitivity, specificity, and diagnostic odds ratio of this sign was calculated to be 0.90, 0.89, and 20, respectively.
CONCLUSION: Simultaneous observation of GGO and other features of viral pneumonia in CT scan had optimum performance in detection of COVID-19. However, it is suggested to make the final diagnosis based on both CT scan and RT-PCR, as none of the two diagnostic modalities are reliable alone. © Italian Association of Nuclear Medicine and Molecular Imaging 2020.

Introduction

Coronavirus disease 2019 (COVID-19), which is caused by infection with severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), is now known as a global pandemic that has affected millions of people. The speed at which this disease spread was so high that within 4 months of the initial reports of the disease in Wuhan, China, it was dispersed all around the world [1]. Statistics are indicative of the death of about 15% of those affected, most of which are individuals with old age. However, mortality among young people is also considerable [2].

One of the ways to control the disease is identifying contagious people. In the literature, real-time polymerase chain reaction (RT-PCR) has been introduced as the best diagnostic method [3]. However, this test is costly and time-consuming and is also associated with a large number of false negative results. In addition, the skill of the person obtaining the sample and the time interval between manifestation of the symptoms and performing the test affect RT-PCR findings [4, 5]. Therefore, it cannot yet be said that RT-PCR is the gold standard for detection of COVID-19. These limitations have inspired some studies to suggest performing chest computed tomography (CT) scan. CT scan results are prepared faster that RT-PCR and this is an advantage of CT scan [5].

However, CT scan also has limitations, among which are its dependence on the skill of the radiologist [6] as well as its need for sterilizing the device after each time of using it for patients suspected to COVID-19. On the other hand, still no consensus has been reached on the diagnostic accuracy of CT scan in diagnosis of COVID-19. Therefore, the present systematic review was designed with the aim of gathering all the existing evidence on the diagnostic accuracy of chest CT scan in COVID-19 to find out if CT scan could be used for diagnosis of COVID-19 or not.

Methods

Study design and search strategy

PICO in the present systematic review is as follows: problem (P): COVID-19 disease; index test (I): chest CT scan; comparison (C): RT-PCR findings as reference test, and objective (O): accuracy of chest CT. To reach the aims of the study, searches were performed in Medline (via PubMed), Embase, Scopus and Web of Science databases until the end of April 2020. Keywords including words related to COVID-19 and chest CT scan were selected. For finding keywords, Mesh and Emtree sections of Medline and Embase databases were searched. In the next step, the titles and abstracts of related articles were studied and finally, with the guidance of a specialist in the field of respiratory illnesses, the keywords were finalized. The search strategy is presented in appendix 1.

In addition, a manual search was performed in pre-print databases as well as searches via Google and Google Scholar search engines so that unpublished articles were also included in the study. In addition, by evaluating citation tracking and reference tracking, we attempted to find studies that could have been missed in the systematic search.

Inclusion criteria

In the present systematic review, diagnostic accuracy studies performed on chest CT scan in identification of COVID-19 cases were included. Both retrospective and prospective studies were evaluated. Exclusion criteria consisted of lack of a group without COVID-19, case-series and case-report studies performed on confirmed COVID-19 patients, not reporting sensitivity and specificity or true positive (TP), false positive (FP), true negative (TN), and false negative (FN) cases, and review studies.

Data gathering and quality control

Initially, 2 independent researchers evaluated the records found. These 2 researchers assessed titles and abstracts of the studies and evaluated the full texts of potentially relevant studies to determine if they met the selection criteria. Finally, they summarized the studies in a checklist, individually. The checklist consisted of the following information: name of the first author, year of publication, study design, sample size, number of COVID-19 and non-COVID-19 patients, mean age, sex distribution, chest CT scan features for diagnosis of COVID-19, reference standard, CT findings interpreter, and TP, TN, FP, and FN. Any disagreement was resolved through discussion with a third researcher.

Quality assessment of the studies was performed based on the instructions suggested by quality assessment of diagnostic accuracy studies (QUADAS-2) [7].

Statistical analyses

All analyses were performed using STATA 14.0 statistical software. The analyses were performed using “midas” command. Data were recorded as TP, TN, FP, and FN and the output of the statistical software was reported as summary of receiver operating characteristics curve (SROC), sensitivity, specificity, diagnostic likelihood ratio and diagnostic odds ratio with 95% confidence interval (CI). Heterogeneity between the studies was evaluated using I2 test; and due to presence of significant heterogeneity, random effect model was used. Deeks’ funnel plot asymmetry test was applied for evaluating publication bias. The majority of studies had assessed the value of CT scan in diagnosis of COVID-19 based on CT scan features. Therefore, analyses were stratified for this variable. In all analyses, p < 0.05 was considered statistically significant.

Results

Characteristics

The performed search yielded 339 non-duplicate articles. By evaluating the full texts of these articles, finally, 9 articles were included in the present systematic review [5, 6, 8–14] (Fig. 1).

Fig. 1

Flow diagram of the present meta-analysis

6 studies were performed on the Chinese population, 1 study was on data from Italy, 1 study was performed in Japan and 1 study was done in the United States. 7 studies were retrospective cohorts. These studies included the data of 1942 patients suspected to COVID-19 (999 patients with COVID-19 and 943 Non-COVID-19 individuals). The most important CT features evaluated in the studies were ground-glass opacity (GGO), mixed GGO, consolidation, pleural effusion, air bronchogram, centrilubular nodules, and thickened interlobar septa. The gold standard was RT-PCR in 8 studies and follow-up of patients until the time of definite diagnosis in 1 study. Evaluation of CT scan findings was done by radiologists in all the studies. Table 1 shows a summary of the included studies.

Table 1

Summary of included studies

Author; year; country	Study type	Sample size	Number of COVID19	Number of non-COVID-19	Mean age	Number of males	CT features for diagnosis	Gold standard	CT interpreter
Ai T; 2020; China [8]	RCS	1014	601	413	51	467	GGO, Consolidation, reticulation/thickened interlobar septa, nodules	RT-PCR	Radiologist
Bai H; 2020; China / USA [6]	RCS	424	219	205	54.42	222	GGO, Mixed GGO, Consolidation, reticulation/thickened interlobar septa, nodules	RT-PCR	Seven separate radiologists
Caruso D; 2020; Italy [9]	PCS	158	62	96	57	83	Peripherally GGO with multilobe and posterior involvement, bilateral distribution, and subsegmental vessel enlargement (> 3 mm)	RT-PCR	Radiologist
Cheng Z; 2020; China [10]	RCS	33	11	22	45.84	15	GGO, Mixed GGO, Consolidation, Air bronchogram, Centrilubular nodules, Reticular pattern	RT-PCR	Radiologist
Himoto Y; 2020; Japan [11]	RCS	21	6	15	63.8	12	GGO-predominant lesion, Bilateral GGO-predominant lesion, GGO- and peripheral predominant lesion	RT-PCR	Two separate radiologists
Long C; 2020; China [5]	RCS	87	36	51	46.1	46	GGO, Consolidation, GGO with consolidation, Pleural effusion, presence of GGO or Consolidation	Repeated RT-PCR	Radiologist
Yang H; 2020; China [12]	RCS	55	13	42	29.8	0	GGO, Patch-like shadow, Fiber Shadow, Pleural effusion	Final diagnosis based on follow up	NR
Zhao D; 2020; China [13]	CCS	34	19	15	42.2	17	Multiple mottling and GGO	RT-PCR	Radiologist
Zhu W; 2020; China [14]	RCS	116	32	84	40	56	GGO, Consolidation, Pleural effusion	RT-PCR	Radiologist

CCS case–control study, CT computed tomography, GGO ground glass opacity, NR not reported, PCS prospective cohort study, RCS retrospective cohort study, RT-PCR real-time polymerase chain reaction

One study had assessed the performance of 7 radiologists, separately, and another had compared the performance of 2 radiologists in identifying COVID-19 based on the findings of CT scan. Data of these articles were reported separately, based on the evaluation results of each radiologist. In Table 2, TP, TN, FP, and FN results described by the studies have been reported based on the interpreting radiologist as well as CT scan feature.

Table 2

Summary of CT scan findings in patients with suspected COVID-19

Author; year; country	CT features for diagnosis	CT interpreter	TP	FP	FN	TN
Ai T; [8]	GGO, Consolidation, reticulation/thickened interlobar septa, nodules	Radiologist	580	308	21	105
Bai H; [6]	GGO, Mixed GGO, Consolidation, reticulation/thickened interlobar septa, nodules	Radiologist 1	158	13	62	192
		Radiologist 2	157	24	62	181
		Radiologist 3	206	156	13	49
		Radiologist 4	28	0	2	28
		Radiologist 5	25	2	5	26
		Radiologist 6	22	3	8	26
		Radiologist 7	21	0	9	28
Caruso D; [9]	Peripherally GGO with multilobe and posterior involvement, bilateral distribution, and subsegmental vessel enlargement (> 3 mm)	Radiologist	60	45	2	54
Cheng Z; [10]	GGO	Radiologist	11	20	0	2
	Mixed GGO		7	16	4	6
	Consolidation		6	17	5	5
	Air bronchogram		8	6	3	16
	Centrilubular nodules		3	17	8	5
	Reticular pattern		9	5	2	17
Himoto Y; [11]	GGO-predominant lesion	Radiologist 1	6	7	0	8
	GGO- and peripheral predominant lesion	Radiologist 1	6	4	0	11
	Bilateral GGO- predominant lesion	Radiologist 1	6	5	0	10
	Bilateral GGO- and peripheral predominant lesion	Radiologist 1	6	4	0	11
	Bilateral GGO- and peripheral predominant lesion without other signs	Radiologist 1	4	1	2	14
	GGO-predominant lesion	Radiologist 2	5	6	1	9
	GGO- and peripheral predominant lesion	Radiologist 2	5	6	1	9
	Bilateral GGO- predominant lesion	Radiologist 2	5	5	1	10
	Bilateral GGO- and peripheral predominant lesion	Radiologist 2	5	6	1	9
	Bilateral GGO- and peripheral predominant lesion without other signs	Radiologist 2	5	3	1	12
Long C; [5]	GGO	Radiologist	11	8	25	43
	Consolidation		6	22	30	29
	GGO with consolidation		19	21	17	39
	Pleural effusion		2	7	34	44
	Presence of GGO, Consolidation, or both		35	51	1	0
Yang H; [12]	GGO	NR	6	26	7	16
	Patch-like shadow		5	15	8	27
	Fiber Shadow		3	3	10	39
	Pleural effusion		5	11	6	31
Zhao D; [13]	Multiple mottling and GGO	Radiologist	17	1	2	14
Zhu W; [14]	GGO	Radiologist	15	10	17	74
	Consolidation		4	7	28	77
	Pleural effusion		2	2	30	82

CT computed tomography, GGO ground glass opacity, NR not reported, TP true positive, FP false positive, TN true negative, FN false negative

Risk of bias assessment

Since the design of 8 studies was retrospective cohort/case–control, the risk of bias was considered to be high regarding patient selection. In addition, the flow and timing status was unclear in 3 studies. Finally, risk of bias and applicability of reference standard was unclear in 1 study as the final diagnosis of COVID-19 was said to be done based on follow-up (Fig. 2 and Table 3).

Fig. 2

Risk of bias assessment of included studies based on QUADAS-2 guideline

Table 3

Risk of bias assessment among included studies

Meta-analysis

Analyses were stratified according to type of CT feature. On this basis, AUCs of GGO alone, consolidation, pleural effusion, other CT features, and GGO with other CT features were found to be 0.64 (95% CI 0.60–0.69), 0.30 (95% CI 0.26–0.34), 0.60 (95% CI 0.56–0.64), 0.61 (95% CI 0.56–0.65), and 0.90 (95% CI 0.87–0.92), respectively (Fig. 3 and Table 4).

Fig. 3

Summary receiver operating characteristics (SROC) of computed tomography (CT) findings in detection of SARS-CoV2 infection. AUC: Area under the curve; GGO: Ground glass opacity; SENS: Sensitivity; SPEC: Specificity; GGO with other features: GGO + Consolidation, GGO + pleural effusion, or GGO + other signs

Table 4

Performance of CT scan in diagnosis of SARS-CoV2 infection based on CT features

CT features	Number of experiments*	AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PLR (95% CI)	NLR (95% CI)	DOR (95% CI)
GGO	7	0.64 (0.60–0.69)	0.61 (0.39–0.79)	0.62 (0.32–0.85)	1.6 (0.8–3.1)	0.63 (0.39–1.02)	3 (1—7)
Consolidation	3	0.30 (0.26–0.34)	0.22 (0.09–0.45)	0.61 (0.22–0.90)	0.6 (0.3–1.0)	1.27 (0.77–2.09)	0 (0–1)
Pleural effusion	3	0.60 (0.56–0.64)	0.12 (0.03–0.38)	0.89 (0.73–0.96)	1.1 (0.4–2.8)	0.99 (0.87–1.12)	1 (0–3)
Other sign	5	0.61 (0.56–0.65)	0.48 (0.26–0.71)	0.70 (0.44–0.87)	1.6 (0.6–4.1)	0.74 (0.41–1.35)	2 (0–2)
GGO with other features#	20	0.90 (0.87–0.92)	0.89 (0.82–0.93)	0.72 (0.53–0.85)	3.2 (1.8–5.5)	0.16 (0.10–0.24)	20 (10–41)

*There are different stratifications in the 6 included studies such as comparing the performance of different radiologists in detection of COVID-19 patients and assessment of different CT features in diagnosis of SARS-CoV2 infection. Therefore, number of experiments is higher than the total number of included studies

#GGO with other features: GGO + Consolidation, GGO + pleural effusion, or GGO + other

AUC area under the receiver operating characteristics (ROC) curve, CI confidence interval, CT computed tomography, GGO ground glass opacity, NLR negative likelihood ratio, PLR positive likelihood ratio, DOR diagnostic odds ratio

Diagnostic performance of CT features in diagnosis of COVID-19 has been reported in Table 4. Overall, sensitivity and specificity of GGO with other CT features was higher than all of the other signs. Sensitivity and specificity of this sign were 0.90 and 0.89, respectively. It should be noted that diagnostic odds ratio of GGO with other features was calculated to be 20, which was significantly higher than all of the other signs (Figs. 4 and 5).

Fig. 4

Sensitivity, specificity, and positive and negative diagnostic likelihood ratios (DLR) of ground glass opacity (GGO) with other features in computed tomography findings in detection of SARS-CoV2 infection. #, GGO with other features: GGO + Consolidation, GGO + pleural effusion, or GGO + other signs

Fig. 5

Diagnostic score and diagnostic odds ratio of ground glass opacity (GGO) with other features in computed tomography findings in detection of SARS-CoV2 infection. GGO with other features: GGO + Consolidation, GGO + pleural effusion, or GGO + other signs

Publication bias

In Fig. 6, publication bias was assessed based on CT features. The findings shown in this figure confirm that there is no publication bias in evaluation of the diagnostic value of CT features in diagnosis of COVID-19.

Fig. 6

Publication bias assessment based on computed tomography findings. GGO with other features: GGO + Consolidation, GGO + pleural effusion, or GGO + other signs

Discussion

The present meta-analysis attempted to evaluate the diagnostic value of CT scan in detection of patients with COVID-19 for the first time. Analyses showed that the only reliable sign for diagnosis of COVID-19 via CT scan is the simultaneous presence of GGO and other features of viral pneumonia. However, the presence of each of these signs alone does not have acceptable diagnostic value for screening positive cases. Sensitivity and specificity of GGO with other features of viral pneumonia in diagnosis of COVID-19 were calculated to be 90% and 89%, respectively.

Although the overall analysis of the present study is indicative of the acceptable value of CT scan in detection of COVID-19, we should not forget about the differences in skills and abilities of radiologists. For instance, the study by Bai et al. evaluated the performance of 7 radiologists in diagnosis of COVID-19 based on CT scan findings. The results of the study showed that the sensitivity calculated for interpretation by these radiologists ranged from 70 to 97%. In addition, the specificities reported for CT scan in the study ranged from 24 to 100% [6]. Therefore, the diagnostic value of CT scan in diagnosis of COVID-19 is largely dependent on the experience and skill of the interpreter.

In the majority of the studies, the gold standard for definitive diagnosis of COVID-19 was RT-PCR. However, the diagnostic value of RT-PCR has been questioned in some studies. For example, Li et al. retrospectively evaluated 610 patients with suspected COVID-19 and showed that there was a considerable number of false negative cases in RT-PCR evaluation. In addition, the performance of this test depends on the number of days passing from the onset of symptoms and in a considerable portion of people there is a need for serial evaluations to confirm the presence of SARS-CoV-2. These researchers believe that the reason for these false negative results is the small number of viruses in some obtained samples, errors in measurement, and the limitations of sample transportation [4]. Also, in another study, Long et al. showed that the first RT-PCR performed has cases of false negative. Therefore, it is best that patients with abnormal findings in their CT scan be considered as probable COVID-19 cases and undergo serial RT-PCR evaluations [5].

Another imaging modality that can be used in diagnosis of COVID-19 is magnetic resonance imaging (MRI). The value of MRI in diagnosis and detection of lung diseases has been proven in previous studies [15-17]. Use of MRI for identifying COVID-19 patients has been suggested in existing studies. These studies believe that MRI is able to detect COVID-19-related cardiovascular complications. For example, in a study on patients recovering from COVID-19, Puntmann et al. found that 78% of patients had some degree of cardiac involvement, while 60% had myocardial infarction [18]. Another study found that sensitivity and specificity of MRI in identifying nodules after COVID-19 were 92.7% and 100%, respectively. Therefore, these researchers concluded that MRI could be used as an alternative to CT scan [19]. However, MRI has its limitations. First, MRI equipment is not available in most health care facilities in developing countries, and second, it costs much more than a CT scan. Therefore, before making a recommendation to use MRI in suspected COVID-19 patients, its cost-effectiveness should be considered.

One of the weak points of the included studies was that the time of performing CT scan had not been specified. It is not known how much time had passed from the onset of the symptoms when the CT scans were obtained. In 1 study they could be obtained on the first day of hospital admission and in another study, it could be performed on the next days. Therefore, the day on which CT scan was performed and its value was calculated is unknown in these studies, while its performance could change with time. This is also a limitation of the findings of the present study.

Another limitation observed among the studies was the retrospective nature of the majority of these studies. That is why in risk of bias assessment section, the risk of bias in patient selection was reported to be high in 8 studies. Therefore, prospective studies should better be carried out for evaluation of the diagnostic value of CT scan in diagnosis of COVID-19.

Obtaining results rapidly, determining the severity of the disease, and helping in choosing a treatment plan for the patients are among the strong points of CT scan, which cannot be achieved using PCR. In addition, PCR result is positive in many asymptomatic patients without any evidence of the disease on imaging. To sum all up, CT scan and PCR both have their own weak and strong points and it seems that they can complement each other for diagnostic evaluation of patients with suspected COVID-19, especially in those who are hospitalized. Of course, both of these modalities are tools that help the physician make a diagnosis and, in the end, it is the physician who will choose the best diagnostic modality based on clinical evidence, epidemiological status of the region he is managing, and other factors.

Conclusion

The findings of the present study showed that presence of GGO with other features of viral pneumonia in CT scan had a sensitivity and specificity of 90% and 89%, respectively, in detection of cases with COVID-19. However, due to limitations such as not reporting the time interval between onset of symptoms and CT scan evaluation and using RT-PCR as the gold standard in these studies, we should be cautious in interpreting the findings. In addition, COVID-19 diagnosis based on CT scan is dependent on the skill and experience of the radiologist, which is a serious limitation for using CT scan for diagnosing the disease. On the other hand, existing evidence indicate the presence of numerous false negative cases in RT-PCR, requiring serial RT-PCR in some cases to reach a definitive diagnosis.

Therefore, it seems that both CT scan and RT-PCR have serious limitations in diagnosis of COVID-19 and relying on either of these 2 modalities alone for diagnosis of COVID-19 could lead to misdiagnosis. Therefore, it is generally suggested to make the final diagnosis based on both CT scan and RT-PCR, as none of the 2 diagnostic modalities are reliable alone. Finally, serial evaluation of patients using these tests may increase their diagnostic accuracy.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 14 kb)