Chest CT in the Emergency Department for Diagnosis of COVID-19 Pneumonia: Dutch Experience.

Background Clinicians need to rapidly and reliably diagnose coronavirus disease 2019 (COVID-19) for proper risk stratification, isolation strategies, and treatment decisions. Purpose To assess the real-life performance of radiologist emergency department chest CT interpretation for diagnosing COVID-19 during the acute phase of the pandemic, using the COVID-19 Reporting and Data System (CO-RADS). Materials and Methods This retrospective multicenter study included consecutive patients who presented to emergency departments in six medical centers between March and April 2020 with moderate to severe upper respiratory symptoms suspicious for COVID-19. As part of clinical practice, chest CT scans were obtained for primary work-up and scored using the five-point CO-RADS scheme for suspicion of COVID-19. CT was compared with severe acute respiratory syndrome coronavirus 2 reverse-transcription polymerase chain reaction (RT-PCR) assay and a clinical reference standard established by a multidisciplinary group of clinicians based on RT-PCR, COVID-19 contact history, oxygen therapy, timing of RT-PCR testing, and likely alternative diagnosis. Performance of CT was estimated using area under the receiver operating characteristic curve (AUC) analysis and diagnostic odds ratios against both reference standards. Subgroup analysis was performed on the basis of symptom duration grouped presentations of less than 48 hours, 48 hours through 7 days, and more than 7 days. Results A total of 1070 patients (median age, 66 years; interquartile range, 54-75 years; 626 men) were included, of whom 536 (50%) had a positive RT-PCR result and 137 (13%) of whom were considered to have a possible or probable COVID-19 diagnosis based on the clinical reference standard. Chest CT yielded an AUC of 0.87 (95% CI: 0.84, 0.89) compared with RT-PCR and 0.87 (95% CI: 0.85, 0.89) compared with the clinical reference standard. A CO-RADS score of 4 or greater yielded an odds ratio of 25.9 (95% CI: 18.7, 35.9) for a COVID-19 diagnosis with RT-PCR and an odds ratio of 30.6 (95% CI: 21.1, 44.4) with the clinical reference standard. For symptom duration of less than 48 hours, the AUC fell to 0.71 (95% CI: 0.62, 0.80; P < .001). Conclusion Chest CT analysis using the coronavirus disease 2019 (COVID-19) Reporting and Data System enables rapid and reliable diagnosis of COVID-19, particularly when symptom duration is greater than 48 hours. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Elicker in this issue.

Chest CT analysis using the COVID-19 reporting and data system (CO-RADS) is fast and achieves a high performance for diagnosing COVID-19, particularly when symptom duration is greater than 48 hours.

■ Radiologist interpretation of emergency department chest CT exams from 1070 patients in six medical centers yielded an area under the receiver operating characteristics curve (AUC) of 0.87 for a diagnosis of coronavirus disease 2019 (COVID-19) with real-time reversetranscription polymerase chain reaction as reference standard.

■ A positive chest CT interpretation showed high performance for the diagnosis of COVID-19 pneumonia with odds-ratios of 25.9.

■ For symptom duration less than 48 hours, the AUC of chest CT for the diagnosis of COVID-19 fell to 0.71 (P<.001).

Introduction

The ongoing coronavirus disease (COVID-19) pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has brought about a need for timely and high diagnostic performance tests for detecting COVID-19. The reference standard for diagnosing COVID-19 is a SARS-CoV-2 realtime reverse-transcription polymerase chain reaction (RT-PCR) in respiratory tract specimens.

Unfortunately, RT-PCR has limited sensitivity, and clinical test performance is dependent on test sample quality, viral load kinetics, and duration of symptoms (1-5). Moreover, the time required for laboratory testing and reporting of RT-PCR results can be substantial, which is undesirable in crowded emergency departments. Hence, in hospitals there is a need for rapid and reliable diagnostics of COVID-19 for appropriate isolation in patient groups with high suspicion of disease. CT is widely available and offers the potential of fast triage and robust rapid diagnosis with limited burden to patients. However, the use of CT scanning for diagnosing COVID-19 has been strongly debated with mixed recommendations (6, 7).

The Dutch Radiological Society has developed a standardized reporting scheme for chest CT in patients presenting with moderate to severe symptoms of COVID-19(8). This ‘COVID-19 reporting and data system’ (CO-RADS) is a likelihood classification for the presence of pulmonary involvement of COVID- 19, with scores varying from 1 (very low suspicion) to 5 (very high suspicion), dependent on the type and distribution of the pulmonary abnormalities (Table 1). This CT classification has moderate to substantial interobserver agreement(8). Yet, the performance of CO-RADS and its clinical applicability have not been validated in a real-life setting.

Table 1:

Chest CT CO-RADS Classification for the Diagnosis of COVID-19

This multicenter study aimed to assess the performance of the CO-RADS classification for diagnosing COVID-19 in patients presenting to the emergency department with moderate to severe symptoms suspicious for COVID-19, both for the overall study group and stratified by duration of symptoms. Chest CT was compared with two reference standards: SARS-CoV-2 RT-PCR, as well as a clinical diagnostic reference standard.

Materials and Methods

Ethics

This study was approved by the institutional review boards of all participating centers. Informed consent was waived by the local IRBs prior to the study.

Patients

This retrospective, multicenter study in 4 university medical centers and 2 large teaching hospitals evaluated consecutive adult patients presenting to the emergency department between March 20th and April 3rd 2020* (April 10th for center F) with moderate to severe symptoms suspicious of COVID- 19 who received a non-contrast enhanced CT-scan at presentation. Suspected COVID-19 was defined as (a) cough and clinically relevant dyspnea requiring hospital admission with or without fever >38 °C, (b) fever without a known cause or (c) fever with anosmia. As standard practice in all these hospitals, patients received a chest CT scan if there was a potential indication for hospital admission.

Patients were excluded from analysis if RT-PCR was not performed or if they were transferred from other hospitals with a known, RT-PCR proven COVID-19 diagnosis. Patients who only had a chest CT with intravenous contrast were also excluded. Patients without reported CO-RADS were excluded from further analysis. Demographic and clinical information, including duration of symptoms, was retrieved from electronic patient records.

Imaging and CO-RADS reporting

Non-contrast enhanced CT scans were obtained with various CT scanners (Canon Aquilion Vision, Canon Aquilion One Genesis, Canon Medical Systems, Otawara Japan; Somatom Force, Somatom Definition Flash, Somatom Definition AS+, Siemens Healthineers, Erlangen, Germany; Lightspeed 16, GE Healthcare, Chicago, IL; Ingenuity 128, Philips Healthcare, Amsterdam, the Netherlands) according to existing local imaging protocols, preferably a low dose protocol (Table E1). All scans were prospectively evaluated by local radiologists with varying levels of experience as part of regular care, without knowledge of RT-PCR results. The current study exclusively used the CO-RADS classification as adjudicated in the official radiological report.

Reference Standard

CT was compared with two reference standards. The first reference standard was SARS-CoV-2 RT-PCR of a clinical specimen. COVID-19 infection was considered ‘proven’ if at least one RT-PCR for SARS-CoV- 2 in a throat, nasal, sputum, bronchoalveolar lavage fluid and/or fecal sample was positive. If initial RTPCR was negative, subsequent RT-PCR testing was generally performed, depending on the clinical likelihood of disease.

A reference standard for COVID-19 diagnosis has yet to be established. While widely used, a large proportion of patients with negative RT-PCR remain clinically highly suspect for COVID-19. In daily routine, this subgroup is isolated and remains in isolation until COVID-19 is ruled out clinically and/or by repeated RT-PCR in order to avoid nosocomial COVID-19 transmission to non-infected patients. To address the limited sensitivity of PCR and the need to avoid missing a diagnosis in patients who have COVID-19 in the inpatient setting, the study established a clinical reference standard that was designed to be highly sensitive (Figure 1).

Figure 1:

Flowchart clinical reference standard. Abbreviations: COVID-19 = coronavirus disease 2019; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2; PCR = polymerase chain reaction; ICU = intensive unit; BAL = bronchoalveolar lavage.

In this clinical reference standard, RT-PCR positive patients were designated as ‘proven’ COVID-19. RTPCR negative patients were classified into either ‘probable’ COVID-19, ‘possible’ COVID-19 or ‘no’ COVID-19 based on clinical data. These patients were classified by local teams of clinical physician assessors blinded to CT-scan and laboratory results. First, assessors determined if an alternative diagnosis explained the presenting symptoms, in which case the patient was classified as ‘no’ COVID- 19. If no alternative diagnosis was established, patients were classified as a ‘probable’ diagnosis of COVID-19 if they had contact with persons with suspected or proven COVID-19, required high oxygen therapy (5L 02 for ≥24 hours or 3L 02 for ≥48 hours), required ICU admission due to respiratory failure, or in case of unexplained death during admission. The remaining patients were classified as ‘possible’ COVID-19 if their nasopharyngeal RT-PCR had been collected less than 2 or greater 7 days after onset of symptoms. Classification in the ‘proven’, ‘probable’ and ‘possible’ COVID-19 categories were considered positive for the clinical reference standard.

Statistical Analysis

Data from all participating centers were collected in line with GDPR standards. Statistical analyses were performed using SPSS statistics version 25 (IBM, Armonk, New York). Continuous data are presented as mean ± standard deviation or median and interquartile range (IQR). Categorical data are presented as proportions. Performance estimates are reported as proportions along with confidence intervals. Receiver operating characteristic (ROC) analysis was performed calculating the area under the curve (AUC) for CO-RADS against both reference standards. Sensitivity, specificity, and diagnostic odds ratios at various cut-off points of the CO-RADS classification were calculated. Results are displayed per center and for all centers combined. Since pulmonary involvement may not be immediately visible, and because RT-PCR loses sensitivity at a later stage after beginning of symptoms, subgroup analysis was performed based on the duration of symptoms, grouping presentations of less than 48 hours, 48 hours through 7 days, and more than 7 days. Significance testing between subgroups of ROC analysis was performed with MedCalc Statistical Software version 19.3.1 (MedCalc Software Ltd, Ostend, Belgium; ; 2020). P values <.05 were considered significant.

Results

Patient demographics

Of 1833 total patients with suspected COVID-19, 763 were excluded from the study group. Eighty-eight patients were excluded because they were not diagnosed in an emergency department, 403 were excluded because they had no or only mild symptoms, 129 had no RT-PCR test, 53 other had already a RT-PCR proven COVID-19, 56 had no CO-RADS in the original report, and 34 had a contrast enhanced CT, leaving a total of 1070 patients were included in this study (Figure 2). In the study group 626/1070 (59%) were men. Median age was 66 (IQR 54-75). Median duration of symptoms at admission was 7 days (IQR 3-10). Baseline patient characteristics for each center are shown in Table 2.

Figure 2:

Inclusion flowchart. Patient were excluded when they were not diagnosed in the emergency department, had no or only mild symptoms, did not have a RT-PCR, and if patient was RT-PCR proven at time of CT scan or no CO-RADS reporting system was used in the original report. * 10th of April for center F. Abbreviations: COVID-19 = coronavirus disease 2019; RT-PCR = real-time reversetranscription polymerase chain reaction; CO-RADS = COVID-19 reporting and data system.

Table 2:

Baseline Patient Characteristics per Center

536/1070 (50%) patients had ‘proven’ COVID-19 based on a positive RT-PCR; in 497/536 (93%) of these, the initial RT-PCR was positive. According to the clinical reference standard, there were an additional 70/1070 (7%) RT-PCR negative patients with ‘probable’ COVID-19, 67/1070 (6%) with ‘possible’ COVID- 19 and 397/1070 (37%) patient with ‘no’ COVID-19. Of the 1070 CT scans, 235/1070 (22%) were scored as CO-RADS 1, 140/1070 (13%) as CO-RADS 2, 134/1070 (13%) as CO-RADS 3, 120/1070 (11%) as CORADS 4 and 441/1070 (41%) as CO-RADS 5 (Table 3).

Table 3:

CO-RADS CT Score per Reference Standard Category

Validation results CO-RADS

Using RT-PCR as a reference standard, AUC was 0.87 (95%CI 0.84-0.89; range across hospitals 0.82- 0.90; Table 4). At a CO-RADS positivity threshold of ≥4, sensitivity was 86% (95%CI 83%-89%), and specificity was 81% (95%CI 78%-84%), and the odds ratio for a COVID-19 diagnosis was 25.9 (95%CI 18.7-35.9).

Table 4:

Performance of CO-RADS

Compared with the clinical reference standard, AUC was 0.87 (95%CI 0.85-0.89; range across hospitals 0.85-0.89). At a CO-RADS positivity threshold of ≥4, sensitivity was 77% (95%CI 74%-81%) and specificity was 90% (95%CI 87%-93%), and the odds ratio for a COVID-19 diagnosis was 30.6 (95%CI 21.1-44.4). Results per CO-RADS category are visualized in Figure 3, and results at different CO-RADS cut-offs are displayed in Table 5.

Figure 3:

Performance per CO-RADS category. Bar chart of percentage ‘proven’ COVID-19, ‘probable’, ‘possible’, and ‘no’ COVID-19 per CO-RADS CT score. Abbreviations: COVID-19 = coronavirus disease 2019; CO-RADS = COVID-19 reporting and data system.

Table 5:

Sensitivity, Specificity and Diagnostic Odds Ratios of the COVID-19 Reporting and Data System (CO-RADS) according to Different Cut-offs in the CO-RADS Classification

Duration of symptoms

Pulmonary manifestations of COVID-19 on CT (CO-RADS ≥3) were seen in 67% of the patients with a symptom duration of less than 48 hours and in 95% of the patients with a symptom duration of more than 48 hours. The performance of CT was worse in the subgroup of patients with symptom duration of less than 48 hours (n=220/1070) compared with patients with a symptom duration greater than 48 hours for the diagnosis of COVID-19 with an AUC of 0.71 (95%CI 0.62-0.80; p<.001) against RT-PCR, and 0.68 (95%CI 0.60-0.76; p<.001) against the clinical reference standard. In the subgroups of patients with symptom duration between 48 hours and 7 days (n=430/1070) AUC was 0.86 (95%CI 0.83-0.90) against RT-PCR, and 0.89 (95%CI 0.86-0.93) against the clinical reference standard. For patients with a symptom duration of more than 7 days (n=376/1070) AUC was 0.86 (95%CI 0.82-0.90) against RT-PCR, and 0.89 (95%CI 0.85-0.93) against the clinical reference standard (Figure 4).

Figure 4:

ROC analysis based on durations of symptoms. (a) ROC analysis based on durations of symptoms for CO-RADS against RT-PCR. (b) ROC analysis based on durations of symptoms for CO-RADS against clinical reference standard. Blue line: <48 hours complaints (n=52/220 RT-PCR positive); red line: 48 hours – 7 days complaints (n=239/430 RT-PCR positive); grey line: >7 days complaints (n=232/376 RT-PCR positive). Abbreviations: ROC = receiver operating characteristics; RT-PCR = realtime reverse-transcription polymerase chain reaction; AUC = area under the curve.

Discussion

Large numbers of patients suspected of coronavirus disease 2019 (COVID-19) have flooded the emergency departments during the first peak of COVID-19, creating the need for rapid and reliable diagnosis to guide clinicians in risk stratification, isolation strategies and treatment decisions. During this pandemic we demonstrated high performance of chest CT using the COVID-19 reporting and data system (CO-RADS) for the diagnosis of COVID-19 in clinical practice. This high level of performance suggests that chest CT can be used to optimize/expedite emergency care for patients suspected of having COVID-19 pneumonia.

The AUC of CO-RADS for the diagnosis of COVID-19 was 0.87 (95%CI 0.84-0.89) when compared with RT-PCR and reached a sensitivity of 86% at a specificity of 81% at a CO-RADS positivity threshold of ≥4. When compared to our clinical reference standard, we also found good performance of CT, reaching an AUC of 0.87 (95%CI 0.85-0.89), and sensitivity of 77% at a specificity of 90% and an OR of >30. Our subgroup analysis based on duration of symptoms showed lower performance of chest CT when performed within the first 48 hours of symptoms with an AUC of 0.71 (95%CI 0.62 – 0.80) against RTPCR and 0.68 (95%CI 0.60-0.76) against the clinical reference standard (p<.001).

Previous studies have reported higher sensitivities for CT diagnosis of COVID-19(3, 9, 10), but this may be exaggerated due to biased samples and cohorts(11). Reports on CT specificity are scarce and thus far disappointingly indicating values often below 50%(3, 6, 12-17). Previous studies did not use a well circumscribed imaging classification system (3, 10). Our study may indicate that employing CO-RADS improves CT performance in diagnosis of COVID-19 in clinical practice. Our observation that CT had lower performance within the first 48 hours of symptoms, is in line with a recent observational study(18). Since sensitivity of RT-PCR declines after 7 days of symptoms(1, 19), CT may aid diagnosing COVID-19 in patients presenting with a longer duration of symptoms.

Beyond diagnostic challenges, the first wave of COVID-19 also introduced patient management issues related to workflow, isolation, personal protective equipment, and treatment decisions. During initial risk estimation in the emergency department, RT-PCR results are usually not immediately available and even when they come available, negative RT-PCR does not exclude COVID-19, especially when the pretest probability of COVID-19 is high (20). Our study showed that CT can be a useful risk stratification tool for COVID-19: which may be advantageous in to counteract emergency department crowding(21).

However, 41/375 (11%) of CO-RADS 1 and 2 patients had RT-PCR proven COVID-19. Retrospective analysis of corresponding CT scans did not reveal misclassification errors in the original reporting. An explanation may be that these patients had no pulmonary manifestations of COVID-19. The proportion of patients with only extrapulmonary symptoms is not well documented but may be in the order of 3- 26%(22-24). The lack of pulmonary findings on CT for this subset of RT-PCR positive patients underscores that CO-RADS 1-2 alone should not be used to rule-out COVID-19. Lack of pulmonary involvement on CT, however, may allow for earlier de-isolation when initial RT-PCR is negative.

41/120 (34%) of CO-RADS 4 and 60/441 (14%) of CO-RADS 5 patients did not have a positive RT-PCR. But a substantial number of these RT-PCR negative patients, 21/41 (51%) for CO-RADS 4 category and 40/60 (67%) for CO-RADS 5 category, were considered as a probable or possible COVID-19 according to our clinical reference standard, and therefore would not qualify for removal from isolation. An alternative diagnosis, like an alternative pulmonary infection or congestive heart failure was established in 21/40 (53%) of the ‘no’ COVID-19 patients with a CO-RADS of 4 and 5 (Figure 5 and Figure 6; Table E2). We recommend patients with CO-RADS 4 or 5 and a negative RT-PCR should remain isolated in a single bedroom until repeat RT-PCR is negative or an alternative diagnosis is found that explains the complaints.

Figure 5:

Example chest CT scans of patients with a true positive (case 1) and false positive (case 2) CO-RADS 4 score. (case 1). True positive CO-RADS 4 chest CT: (a,b) two axial slices and (c) a coronal slice of a 79-year-old female with 9 days of symptoms. CT shows diffuse ground-glass opacities close to visceral pleural surfaces but superimposed on emphysematous changes. Also note the widened esophagus. Coronavirus disease 2019 was RT-PCR confirmed. (case 2) False positive CO-RADS 4 chest CT: (d,e) two axial slices and (f) a coronal slice of a 51- year-old female with 2 days of symptoms. CT shows bilateral multifocal areas of consolidation with halo and subtle areas of ground glass without contact to visceral pleural surfaces. RT-PCR for SARS-CoV2 was repeatedly negative and an alternative diagnosis was established with a blood culture confirmed line sepsis.

Figure 6:

Example chest CT scans of patients with a true positive (case 3) and false positive (case 4) CO-RADS 5 score. (case 3) True positive CO-RADS 5 chest CT: two axial slices (a,b) and a coronal slice (c) of a 69-year-old male with 7 days of symptoms. CT shows bilateral multifocal areas of groundglass and consolidation in vicinity of the visceral pleural surface. Also, few thickened vessels in areas of groundglass are observed. Coronavirus disease 2019 was RT-PCR confirmed. (case 4) False positive CO-RADS 5 chest CT: two axial slices (d,e) and a coronal slice (f) of a 42-year-old male with more than 7 days of symptoms. CT shows diffuse groundglass opacities in the close vicinity of visceral pleural surfaces. Also, a crazy paving pattern is observed. RT-PCR for SARS-CoV2 was negative, and a diagnosis of pneumocystis jirovecii pneumonia was made based on bronchoalveolar lavage fluid.

In patients with an uncertain CT diagnosis (i.e. CO-RADS 3), 35/134 (26%) had a positive RT-PCR and 76/134 (57%) were classified as ‘no’ COVID-19 in the clinical reference standard. The added value of CT in this group was limited. Fortunately, only 134/1070 (13%) of all patients had a CO-RADS 3 classification. This proportion of uncertain diagnosis is still relatively high compared with other reporting and data systems such as BI-RADS (1.2-14%)(25) and LUNG-RADS (6%)(26), but much lower than PI-RADS (40%)(27).

Our study has limitations. The CO-RADS classification was introduced in the early phase of the first COVID-19 peak in the Netherlands. Radiologists may not have been optimally trained, which could have negatively influenced performance. In addition, we focused on patients presenting to the emergency department and incidence of COVID-19 was high. Our findings may not be reproducible to lower-incidence settings. Our clinical reference standard was designed to be highly sensitive but was not validated in a control group and may be false positive, especially in the ‘possible’ COVID-19 category. Furthermore, before implementation of this CT strategy, good infection control processes need to be in place. Specifically, cleaning the CT scanner room and safe room turnover for the safe scanning of new patients must be considered.

The implications of our results are of potential importance. Chest CT exams interpreted using the CORADS system allow for a rapid test result in the emergency department of patient with suspected COVID-19 pneumonia. This suggests a potential role for chest CT in helping to optimize risk stratification and isolation strategies of patients urgently presenting for hospital care during the first and second wave of this pandemic.

In conclusion, using the CO-RADS chest CT reporting system for emergency department subjects, pulmonary manifestations of COVID-19 were detected in more than 95% of patients with moderate to severe upper respiratory symptoms 48 hours after symptom onset. CO-RADS score greater than or equal to 4 provided odds ratios above 25 for the diagnosis of COVID-19.