Diagnostic performance between CT and initial real-time RT-PCR for clinically suspected 2019 coronavirus disease (COVID-19) patients outside Wuhan, China.

INTRODUCTION: Chest CT is thought to be sensitive but less specific in diagnosing the 2019 coronavirus disease (COVID-19). The diagnostic value of CT is unclear. We aimed to compare the performance of CT and initial RT-PCR for clinically suspected COVID-19 patients outside the epicentre-Wuhan, China.
MATERIALS AND METHODS: Patients clinically suspected of COVID-19 infection who underwent initial RT-PCR and chest CT at the same time were retrospectively enrolled. Two radiologists with specific training reviewed the CT images independently and final diagnoses of the presence or absence of COVID-19 was reached by consensus. With serial RT-PCR as reference standard, the performance of initial RT-PCR and chest CT was analysed. A strategy of combining initial RT-PCR and chest CT was analysed to study the additional benefit.
RESULTS: 82 patients admitted to hospital between Jan 10, 2020 to Feb 28, 2020 were enrolled. 34 COVID-19 and 48 non-COVID-19 patients were identified by serial RT-PCR. The sensitivity, specificity was 79% (27/34) and 100% (48/48) for initial RT-PCR and 77% (26/34) and 96% (46/48) for chest CT. The image readers had a good interobserver agreement with Cohen's kappa of 0.69. No statistical difference was found in the diagnostic performance between initial RT-PCR and chest CT. The comprehensive strategy had a higher sensitivity of 94% (32/34).
CONCLUSIONS: Initial RT-PCR and chest CT had comparable diagnostic performance in identification of suspected COVID-19 patients outside the epidemic center. To compensate potential risk of false-negative PCR, chest CT should be applied for clinically suspected patients with negative initial RT-PCR.

2019 novel coronavirus disease

Reverse transcription-polymerase chain-reaction

Middle East respiratory syndrome

Severe acute respiratory syndrome

Ground glass opacities

Introduction

On March 13, 2020, the World Health Organization officially announced that the new coronavirus disease (COVID-19) has become a global pandemic [1]. By the time of this announcement, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused more than 2,000,000 infections and 100,000 deaths worldwide, with a rapidly increasing number especially in the United States and Europe. The outbreak continues to advance, with cases doubling every 3–4 days that threatens medical systems to become overburdened.

An early and rapid detection method of COVID-19 is crucial for breaking trains of community transmission, to initiate contact tracing, and provide timely treatment [2]. Viral nucleic acid test by reverse transcription–polymerase chain reaction (RT-PCR) is the first line screening method of choice [3]. However, the sensitivity of RT-PCR is insufficient, ranging from 50 to 62% according to previous large-scale reports [4,5]. Some patients may be highly suspicious, based on established close contact with confirmed cases, typical clinical manifestations and CT image appearance, but may still develop a false negative initial RT-PCR, which increases the risk of community transmission and delay in treatment. Nonetheless, RT-PCR remains the gold standard tool for confirming COVID-19 infection, incorporating multiple RT-PCR tests to make up for its shortcomings.

CT has been an important imaging modality in the diagnosis and management of patients with viral pneumonia, as demonstrated in the large-scale outbreaks in severe acute respiratory syndrome (SARS-CoV) [6] and Middle East respiratory syndrome (MERS-CoV) [7]. Recent studies showed CT of COVID had typical appearance of viral lung infection, with a sensitivity of 60–98% [8,9]. Notably, CT changes may be identified before patients become symptomatic and RT-PCR positive [[10], [11], [12]]. In Hubei, there was a temporary period where diagnoses of COVID-19 were based on CT changes even without positive PT-PCR result to ensure timely treatment and isolation measures. On the contrary, no CT abnormality was found in part of the confirmed cases when pneumonia was absent. Other kinds of viral pneumonia can also mimic COVID-19 pneumonia, which makes it difficult to differentiate. Few studies include non COVID-19 patients as control in the analysis. Moreover, most of the study results are from Wuhan or Hubei patients, which may potentially cause over estimation by enrolling more severe cases. Thus, the diagnostic performance of CT in COVID-19 is less clarified.

We aimed to compare the diagnostic performance of chest CT and initial real-time RT-PCR for COVID-19 outside Wuhan, with serial RT-PCR tests and observation results as reference standard.

Materials and methods

Study design and participants

This study was approved by our institutional review board and informed consent was waived. We retrospectively reviewed the medical database in the University of Hong Kong-Shenzhen hospital for patients suspected for COVID-19 who underwent high resolution chest CT and real-time RT-PCR from January to March 2020, and those who followed the national standard for diagnosis, treatment and observation protocol were enrolled. Exclusion criteria were incomplete clinical or laboratory information, and images with excessive motion artifacts. Clinical information was collected including travel and exposure history (when available), clinical symptoms, and blood test. The RT-PCR of respiratory secretion specimens from nasopharyngeal swab, oropharyngeal swab, endotracheal aspirate, or bronchoalveolar lavage was performed using real-time RT-PCR kits form BGI Genomics (Shenzhen, China) with National Medical Products Administration emergency approval.

The diagnosis and exclusion of COVID was based on national mandatory management guideline as shown in Fig. 1 . An initial RT-PCR in accordance with a previously published protocol [13] was provided to clinically suspected COVID-19 patients when they were admitted. Those with positive results underwent repeat RT-PCR test for confirmation. The cases of negative initial or 2nd RT-PCR results were quarantined for an observation period, which was defined as 14 days from the first day of exposure or onset of symptoms (for patients with unknown exposure history). Multiple repeat RT-PCR tests were performed during observation if necessary, any positive results would be repeated to ensure a diagnosis. Suspicious patients were categorised as non-COVID-19 cases if no positive RT-PCR result was found when the observation ended. The local government publishes anonymised detailed reports of newly confirmed cases every day, which severed as a cross-reference to find missed diagnoses.

Fig. 1

Flowchart of this study

RT-PCR: reverse transcription polymerase chain reaction

COVID-19: novel coronavirus disease 2019.

CT image acquisition and analysis

Right after the swab sampling, non-contrast high resolution CT thorax images were acquired with 1-mm slice thickness following acquisition parameters of usual protocol and reformatted with soft tissue and lung windows. All images were transferred to a stand-alone workstation for analysis. Typical chest CT findings were extracted from previously published reports [14,15] and served as diagnostic reference including: ground-glass opacification with or without consolidation, crazy paving patten, peripheral and diffuse distribution, and bilateral/multilobular involvement. Two experienced general radiologists (LJX and LZD, 17 and 14 years of experience), who had specific training of COVID-19 presentation on chest CT by online courses and real-life cases, independently reviewed the images to determine whether CT findings were positive. Image readers were aware of the patients’ epidemiological history and clinical characteristic, but blind to personal information and RT-PCR results, which is the same as clinical setting. At first, their own results were recorded for inter-observer reliability test. Then the final CT results were determined by their consensus discussion for diagnostic performance analysis.

To compensate potential false negative of initial RT-PCR, a screening strategy by combing initial RT-PCR and CT was once advocated in Hubei province (Wuhan is the provincial capital). Patients were confirmed as COVID-19 when initial RT-PCR was positive, or initial RT-PCR negative but CT positive. When initial RT-PCR and CT were negative, follow up procedure was provided to the patient. We also analysed our data to explore the diagnostic value of this strategy.

Statistical analysis

Continuous variables were presented as medians and ranges, categorical variables were presented as counts and percentages. Interobserver reliability on CT diagnosis of COVID-19 was evaluated using Cohen's kappa calculation (κ < 0.40, poor agreement; 0.40 ≤ κ < 0.60, moderate agreement; 0.60 ≤ κ < 0.80, good agreement; and κ ≥ 0.80, excellent agreement). The diagnostic performance of initial RT-PCR and CT were evaluated by calculating sensitivity, specificity, accuracy, then, were compared by McNemar Chi-squared test for paired proportions. Statistical analysis was performed using IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp. A P value of less than 0.05 indicated a significant difference.

Results

Clinical characteristics and laboratory findings

82 patients admitted to the University of Hong Kong-Shenzhen hospital between Jan 10, 2020 to Feb 28, 2020 were retrospectively enrolled in our study based on our criteria. The demographic and clinical characteristics of patients by group are summarised in Table 1 .

Table 1

Demographic and clinical characteristics of 82 included patients in the study.

Characteristic	COVID-19 (n = 34)			Non COVID-19 (n = 48)		p-value
Sex, no. (%) of patients
Male	17	(50)		32	(67)	0.129
Female	17	(50)		16	(33)
Age, years
Median (range)	52	(8-74)		37	(1-76)
Epidemiologic Contact
Wuhan/COVID-19	29	(85)		9	(19)	<0.001
Indirect contact or unknown	5	(15)		39	(81)
Symptoms, no. (%) of patients
Fever	25	(74)		30	(63)	0.295
Cough	17	(50)		34	(71)	0.097
Muscle soreness	7	(21)		6	(13)	0.323
Sputum production	6	(18)		13	(27)	0.318
Fatigue	2	(6)		4	(19)	0.675
Sore throat	2	(6)		9	(19)	0.092
Headache	3	(9)		5	(10)	0.811
Blood test, no. (%) of patients
Leukocytosis	2	(6)		11	(23)	0.037
Lymphopenia	8	(24)		2	(4)	0.008

With multiple RT-PCR results as reference, 34 COVID-19 and 48 non-COVID-19 patients were confirmed following our management protocol. 85% of COVID-19 cases (29/34, p < 0.05) had direct contact with COVID-19 patients or Wuhan city. The most common onset symptoms were fever and dry cough, and there were no significant differences in patient symptoms between the two groups. The incidence of lymphopenia was significantly higher in COVID-19 patients, where as leukocytosis was more common in non-COVID-19 patients, which is consistent with previous reports.

Diagnostic performance comparison

The diagnostic performance of initial RT-PCR and CT comparison is demonstrated in Table 2 .

Table 2

Diagnostic performance of CT and initial RT-PCR in COVID-19.

	Initial RT-PCR only	CT only	Initial RT-PCR plus CT
Results (n)
TP	27	26	32
TN	48	46	48
FP	0	8	2
FN	7	2	0
Diagnostic Performance (%)
Sensitivity	79	77	88
95 CI	66–93	62–91	86–100
Specificity	100	96	100
95 CI	100	90–100	90–100
Accuracy	92	88	98
95 CI	91–92	88	95

RT-PCR: reverse transcription polymerase chain reaction.

TP: true positive; TN: true negative; FP: false positive; FN: false negative.

CI: confidence interval.

Of 34 COVID-19 patients, 27 were positive at initial RT-PCR, and 7 had positive results in the observation period. All 48 non-COVID-19 patients had negative result at initial RT-PCR, none was found positive during observation with 1–8 RT-PCR tests (median 2).

Chest CT were reported by the two radiologists as positive for COVID-19 in 41% (34/82) and 34% (28/82) of all patients. A good agreement between them was observed with a Cohen's kappa of 0.69. At final agreement of CT, 26/34 COVID-19 were correctly diagnosed. 6/34 (18%) COVID-19 were completely normal in CT.

The sensitivity to identify COVID-19 was 79% (95% confidence interval [CI], 66-93%) in initial RT-PCR and 77% (95% CI 62-91%) in CT. The specificity was 100% (95% CI 100%) in initial RT-PCR and 96% (95% CI 90-100%) in CT. The accuracy was 92% (95% CI 91-92%) in initial RT-PCR and 88% (95% CI 88%) in CT. There is no statistical difference in the abovementioned indicators. Typical false negative CT images are shown in Fig. 2 .

Fig. 2

False negative CT images

A. Axial CT image of a 64-year-old woman shows bronchiectasis (arrow head) without any evidence of GGO. Initial RT-PCR was negative. She was confirmed COVID-19 at the third RT-PCR during quarantine observation.

B. Axial CT image of a 32-year-old woman. The patchy sub-pleural GGO (red frame) was the only positive finding. Her initial RT-PCR was positive.

With the initial RT-PCR plus CT strategy, 32/34 COVID-19 patients were correctly diagnosed, with an improved sensitivity of 94% (95% CI 86-100%). The specificity, accuracy was 100%, and 98% (95% CI, 98%), respectively. Statistical analysis between this strategy and initial RT-PCR, CT was unavailable since it was related to those two methods.

Discussion

As some countries are struggling with COVID-19 outbreak, Wuhan, China has been reporting zero new cases of coronavirus for days. China, South Korea and Singapore seemed to have the transmission under control by taking early detection and isolation as the most important measures. The successful experience of massive drive-through testing in South Korea is now accepted by many other countries.

Indeed, RT-PCR test of throat swab or nasal swab specimens is currently the most convenient and efficient method of rapid COVID-19 screening. The reported sensitivity of RT-PCR for COVID-19 is 50–62%, which is acceptable but still comes with a moderate proportion of missed diagnoses. Our study shows that initial RT-PCR yield a sensitivity of 79% and specificity of 100%, which is a little bit higher than previous reports. The RT-PCR testing accuracy may be affected by a number of factors including viral load in the respiratory tract, specimens source, sampling procedures and timing, quality control of the test, and inherent performance of the testing kits [16]. Therefore, the RT-PCR is unlikely to be a reliable and independent tool for COVID-19 screening. Furthermore, in the case of coronavirus pandemic, PCR testing kits are in limited supply in many countries, even in developed countries, and there may not be enough staff to sample and process the test.

On the other hand, the ease of access to CT and relatively low cost in China makes it an efficient modality in the management of respiratory transmitted diseases like SARS and H1N1 flu outbreak [17]. Chest CT have played an important role in early detection, evaluation, and treatment response monitoring of COVID-19 infection. However, chest CT manifestation of COVID-19 pneumonia overlaps with other types of viral pneumonia, bringing potential impact on its specificity [18]. Although it is just three months from initial outbreak of the disease, our result showed that experienced radiologists with special training in the diagnosis of COVID-19 had excellent diagnostic accuracy of 88%, and good interobserver agreement. No statistical difference was found between chest CT and initial RT-PCR in terms of sensitivity, specificity and accuracy. This result is similar with the study of Harrison X. Bai et al. [19], which demonstrated that the accuracy of three Chinese radiologists to differentiate COVID-19 from non-COVID-19 pneumonia was 83%, 80% and 60%. Our study is significant by demonstrating that radiologists in low prevalence area are also capable of diagnosing COVID-19 in clinical setting with high specificity but moderate sensitivity, even when patients without pneumonia were enrolled.

The reported sensitivity of CT in COVID-19 is as high as 98% [20]. A large group study of Wuhan patients reported a sensitivity of 97% for chest CT, with 308/601 patients showing typical CT manifestations before RT-PCR test was positive [21]. In our study, the sensitivity of chest CT was 77%, which was close to the earliest large group study by W. Guan et al. [21] (86.2%, 840/975 CT scans). They also reported that more normal CT cases were found in non-severe than severe patients. Unlike SARS-CoV or MERS-CoV, the SARS-CoV-2 infection is more likely to cause mild respiratory symptoms, fewer severe cases, but more contagious [22]. Pneumonia may be absence in mild patient, resulting in normal chest CT [23]. Thus, a possible explanation for the lower sensitivity in chest CT is that patients in our study were far from the epidemic center, consequently, more mild patients without pneumonia were admitted in our hospital. In the 8 cases of false negative CT in our study, 6 were completely normal, one presented with a very small patchy subpleural GGO, and a case only presented with bronchiectasis. Patients with normal chest CT contributed most of the decrease in sensitivity. A multi-center study by Yang, W. et al. [24] also found that most of the cases outside Wuhan were mild, 17/149 cases had normal CT on admission.

Consistent with most of the recent literatures, we also found 5 patients who had characteristic chest CT features with negative initial RT-PCR and confirmed by serial RT-PCR test. As mentioned before, COVID-19 patient may have false negative initial RT-PCR result caused by different factors. Our result demonstrated that the combined strategy of initial RT-PCR following with CT yield a remarkably improved sensitivity from 79% to 94%. The top priority of transmission control is to screen any clinically suspected case to identify and quarantine the infected patients, provide timely treatment and find out all close contacts for further examination. Disease control authorities may pursue measures with higher sensitivities while sacrificing a bit of specificity. Given the extremely high sensitivity of CT in the epidemic center and relatively low sensitivity of RT-PCR, it was once argued that CT should had equal importance with RT-PCR for confirming COVID-19 in Hubei province, which was later temporarily adopted in the 5th trial edition of the national guideline of diagnosis and treatment. In fact, initial RT-PCR and chest CT are both provided in the Guangdong province of China when symptomatic patients are first admitted. The comprehensive screening strategy has been working effectively in China since the beginning of outbreak.

There may be concern about the future cancer risk of ionizing radiation from CT. Low-dose chest CT has been widely applied in lung cancer screening, with a typical radiation dose of 1–2 mSv per scan. A recent study [25] found that in the 25 studies with higher quality methods, 21 out of 25 did not support cancer induction by low-dose radiation with a cumulative dose up to 100 mSv (approximately 50–100 low-dose chest CT scans). However, we should still be cautious about the use of CT while assessing children and infants.

Our study had several limitations. First, the cohort size was relatively small due to the clinical setting in single center of a low incidence area. It is also possible that the diagnostic accuracy of chest CT may be improved with Chinese radiologists working near the epidemic center of the disease who have more experience in the diagnosis of COVID-19 with other lung infection. Future study may include artificial intelligence for images reading to avoid subjective bias and improve diagnostic performance. Noteworthy, most of the COVID-19 patients in the study had mild to moderate presentation in chest CT, making it easier for radiologist to make differentiate diagnosis, and consequently skew the specificity. Lastly, most of the RT-PCR kits currently in use were developed in short period and may not be adequately tested prior to application in the context of urgent needs. Better reference standard needs to be set up for comparing diagnostic method of COVID-19.

In conclusion, initial RT-PCR and chest CT had similar and good diagnostic performance in rapid screening of clinically suspected COVID-19 patients outside the epidemic center. A radiologist with proper training can differentiate COVID-19 with other common pathogen induced pneumonia in clinical setting. Normal chest CT can be found in RT-PCR positive COVID-19 cases, and typical CT manifestations can be found in RT-PCR negative cases. To compensate potential risk of false-negative in initial screening RT-PCR, chest CT should be applied for clinically suspected COVID-19 patients with negative initial RT-PCR.

CRediT authorship contribution statement

Jian-Long He: Conceptualization, Methodology, Formal analysis, Writing - original draft, Writing - review & editing. Lin Luo: Formal analysis, Investigation. Zhen-Dong Luo: Investigation, Data curation. Jian-Xun Lyu: Investigation, Data curation. Ming-Yen Ng: Writing - review & editing. Xin-Ping Shen: Methodology, Supervision. Zhibo Wen: Conceptualization, Methodology, Supervision, Writing - review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.