Diagnostic performance and inter-observer variability of CO-RADS in the triage of patients with suspected COVID-19 infection: initial experience in Zagazig University Hospital.

Purpose: In many healthcare settings in developing nations, multislice computed tomography (MSCT) imaging may be the only available diagnostic modality for patients with suspected COVID-19 infection, due to a shortage of laboratory kits. This study aimed to evaluate the diagnostic performance and interobserver variability of CO-RADS (COVID-19 Reporting and Data System) in the triage of patients with suspected COVID-19 infection in Zagazig University Hospital. Material and methods: This study included 2500 patients with suspected COVID-19 infection, mean age 60.61 years ± 13.89. 61.4% were male. Unstable patients requiring urgent invasive ventilation, acute coronary syndrome patients, pregnant females, and patients with RT-PCR results available prior to MSCT were excluded from this study. RT-PCR was performed in all patients included in the study.
Results: Fever and dry cough were the most common clinical symptoms, detected in 80.16% and 52.00%, respectively. The most common comorbidities were cardiovascular diseases, followed by chronic lung disease and diabetes, found in 27.36%, 22.80%, and 18.00%, respectively. Of the 1500 RT-PCR-positive patients, 40% had CO-RADS score 5, while 3.4% had CO-RADS score 1. Of the 1000 RT-PCR-negative patients, 36% had CO-RADS score 2 and 1% were scored as CO-RADS 5. There was excellent agreement in the studied patients as the weighted κ value was 0.846, which was more pronounced at CO-RADS 5 (24.40%). The sensitivity of CO-RADS was higher in the 2nd scenario (83.27% vs. 55.27%) while the specificity was higher in the 1st scenario (95% vs. 65%).
Conclusion: The CO-RADS scoring system is a sensitive and specific method that can help in the diagnosis of COVID-19 during the peak of the COVID-19 pandemic. CO-RADS is a triage test in resource-constrained environments, assisting in the optimization of RT-PCR tests, isolation beds, and intensive care units.

Introduction

In Wuhan, Hubei Province, China, a huge outbreak of coronavirus disease 2019 (COVID-19) happened in December 2019 [1]. The WHO announced the COVID-19 outbreak a pandemic on 30 January 2020, with more than 174,502,686 cases and 3,770,361 deaths by June 2021 [2].

The International Committee on Taxonomy of Viruses named COVID-19 as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Coronavirus is among the group of viruses causing the common cold as well as more severe respiratory infections: severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS), which have mortality rates of 10% and 37%, respectively [3].

Early diagnosis of COVID-19 is vital for prompt patient isolation and treatment [4].

Definitive diagnosis of COVID-19 is performed using a reverse transcription-polymerase chain reaction (RT-PCR) assay; however, reported sensitivities range from 42% to 83%. RT-PCR depends on viral load, test sample quality, and symptom duration [5,6].

In patients with typical symptoms and imaging findings of COVID-19, a positive RT-PCR result may occur after numerous negative results. Also, RT-PCR test results take hours or even days. Increasingly, RT-PCR tests are insufficient and cannot be used for every patient [7,8].

Multi-slice computed tomography (MSCT) imaging plays a significant role in the diagnosis of COVID-19 [9]. Its sensitivity has been reported to be 97% in a study on 1014 patients [10]. The diagnostic value of MSCT has been proven in PCR-confirmed cases, clinically suspicious cases with inconclusive laboratory test results, and asymptomatic individuals with known exposure [11]. In many healthcare settings in developing nations, even in industrialized countries, MSCT imaging may be the only available diagnostic modality due to a shortage of diagnostic laboratory kits [12].

CO-RADS (COVID-19 Reporting and Data System) is a CT-based system that assesses the suspicion of pulmonary involvement in COVID-19; it must be interpreted together with the clinical and laboratory findings. The authors chose the term CO-RADS because it depends on other standardized reporting systems, such as LI-RADS (Lung Imaging Reporting and Data System), PI-RADS (Prostate Imaging Reporting and Data System), and BI-RADS (Breast Imaging Reporting and Data System) [13,14].

The level of suspicion increases from very low (CO-RADS category 1) to very high (CO-RADS category 5); 2 additional categories encode a technically insufficient examination (CO-RADS category 0) and RT-PCR-proven SARS-CoV-2 infection (CORADS category 6) [15].

The aim of the present study was to determine the diagnostic performance and inter-observer variability of CO-RADS in the triage of 2500 patients with suspected COVID-19 infection at Zagazig University Hospital.

Material and methods

Patients

This retrospective study was performed on 2500 patients (1535 men and 965 women; age range, 18-95 years; mean age 60.61 ± 13.89 years) with suspected COVID-19 infection. They were referred from the emergency triage unit to the MSCT unit in the Department of Radiodiagnosis of Zagazig University Hospital during the period from November 2020 to May 2021. RT-PCR was performed in all patients included in the study.

Inclusion criteria: respiratory symptoms persisting for ≤ 2 weeks and present during the last 24 hours prior to MSCT, peripheral capillary oxygen saturation < 95% and/or respiration rate > 20/min, and individuals exposed to patients with known or suspected COVID-19. Exclusion criteria: unstable patients requiring urgent invasive ventilation, acute coronary syndrome patients, pregnant females, and patients with RT-PCR results available prior to MSCT.

The physician took the medical history and performed a physical examination. Symptoms related to possible COVID-19 infection were assessed and vital signs were registered. Thereafter, arterial blood gas, blood samples, and nasopharyngeal swabs were obtained from all patients.

RT-PCR was performed from nucleic acid testing of respiratory secretions. Sample collection was done using a swab to collect respiratory material found in the nasopharynx. After collection, the swab was sealed in a tube and then sent to the laboratory. All virus tests were done in the laboratory of Zagazig University Hospital.

Approval for the retrospective analysis of the patients with suspected COVID-19 infection was obtained from the Ethics Commission of our Hospital. Medical Ethics Committee approval was obtained prior to the study. Informed consent was waived, and data collection and storage were performed in accordance with local guidelines.

Multi-slice computed tomography technique

MSCT was done on a 128slice CT scanner (Philips Ingenuity 128) using a standard CT protocol.

Preparation

Technologists were required to wear protective garments. Patients had to be quiet and co-operative, to prevent motion artifacts distorting the image. Also, dense materials should were not allowed within the scanning field.

Image acquisition

Non-enhanced and contrast-enhanced scans were performed in 2300 and 200 patients, respectively, using the following parameters: 120 kV, 200 mA, 5 mm beam collimation, matrix 512 × 512, 1.25 pitch, 0 gantry tilt, and 370 mm FOV. Patients were scanned in a supine position in suspended deep inspiration with arms extended overhead to reduce beam hardening artifacts. Scanning extended from the thoracic inlet to the upper abdomen. Contrast-enhanced MSCTs were requested in 200 patients when pulmonary embolism was considered a relevant differential diagnosis. It was performed after weight-adapted IV injection of 64-100 ml of contrast material (Omni-paque 350, Schering, Germany). Every chest MSCT examination was read by 2 teams of radiologists with 10-15 years’ experience in interpreting chest CT. In patients with MSCT findings suggestive of COVID-19 pneumonia, the radiologists informed the clinician immediately. The clinician would then order immediate isolation of the patient for clinical monitoring and treatment.

CO-RADS score

All MSCTs were reported shortly after MSCT images were obtained and before RT-PCR results were available. Two teams of radiologists read the MSCT images and classified them according to the CO-RADS classification, which was recently created by the Dutch Association of Radiologists (NVVR) [16].

The following MSCT findings were recorded: the presence of ground glass opacity (GGO), consolidation, vascular enlargement, crazy paving, reticulations, and reverse halo. Furthermore, the location and distribution of opacities were recorded: (1) peripheral (i.e. close to visceral pleural surfaces including the fissures) versus central (i.e. peribronchovascular), (2) unifocal versus multifocal (i.e. more than 3 lesions) predominance, (3) anterior versus posterior lung predominance, and (4) whether lesions occurred bilaterally. Pleural effusion was noted as well.

The structured reports concluded in a 5-point scale indicating the probability of COVID-19 pneumonia. It was defined as follows:

CO-RADS 1 (as seen in implies a very low level of suspicion for lung involvement by COVID-19 based on normal CT or non-infectious CT findings.

CO-RADS 1. Level of suspicious of COVID-19 infection is very low. The multislice computed tomography (MSCT) images were of a male patient, 65-year-old, with proximal interruption of the right pulmonary artery; he complained of dyspnoea for 5 days. RT-PCR was negative. A) Scout of the lungs shows right-sided volume loss and overinflation of the left lung. B) Axial contrast-enhanced MSCT of the chest (mediastinal window) shows absence of the right pulmonary artery, associated with mediastinal shift to the right side. C) Coronal MSCT of the chest (lung window) shows volume loss of the right lung, right-sided honeycombing cysts, and reticular septal thickening with subpleural and basal predominance; mild traction bronchiectasis can also be observed; picture of right idiopathic pulmonary fibrosis

CO-RADS 2 (as seen in implies a low level of suspicion for lung involvement by COVID-19 based on infectious CT features not compatible with COVID-19, such as bronchitis, infectious bronchiolitis, bronchopneumonia, lobar pneumonia, and pulmonary abscess. Centrilobular nodules, tree-in-bud, segmental, or lobar consolidation, and lung cavitation are classified as CO-RADS 2.

CO-RADS 2. Level of suspicious of COVID-19 infection is low. The multislice computed tomography (MSCT) images of a 48-year-old female who had fever and coughing for 3 days. The RT-PCR test was negative. Axial MSCT images of chest (lung window) show mild centrilobular emphysema, multiple centrilobular nodules with a linear branching pattern (tree in bud opacities) (yellow arrow), and mild bronchiectasis; picture of bronchiolitis

CO-RADS 3 (indeterminate appearance) (as seen in implies equivocal findings for lung involvement by COVID-19 based on CT features of other viral pneumonias or non-infectious findings such as perihilar GGO, homogenous extensive GGO, GGO with smooth interlobular septal thickening, and organizing pneumonia without other typical findings of COVID-19. CO-RADS 3 also includes small GGOs that are not centrilobular (otherwise they would be CO-RADS 2) or not located close to the visceral pleura (otherwise they would be CO-RADS 4).

CO-RADS 3. COVID-19 unsure or indeterminate. The multislice computed tomography (MSCT) image of a 78-year-old male with fever, coughing, and dyspnoea for 2 days, who tested negative for COVID-19. Axial MSCT image of the chest (lung window) reveals a unifocal ground glass opacity (yellow arrow)

CO-RADS 4 (as seen in implies a high level of suspicion for lung involvement by COVID-19 based on CT findings similar to those for CO-RADS 5 but a lack of contact with the visceral pleura, located unilaterally, in a peri-bronchovascular distribution or when the findings are superimposed on pre-existing lung abnormalities.

CO-RADS 4. The level of suspicion is high. The multislice computed tomography (MSCT) image of a 50-year-old female who had fever for 1 week with abdominal pain and diarrhoea. On the day of admission she had a dry cough and dyspnoea. The O2-saturation was low. She was positive for COVID-19. Axial MSCT images of the chest (lung window) reveal unilateral multiple areas of ground glass opacity peripherally located in the left lung (yellow arrows)

CO-RADS-4. The level of suspicion is high. 54-year-old female with fever, dyspnoea, tiredness, and cough starting 3 days before the multislice computed tomography (MSCT) scan. She was positive for COVID-19. Axial MSCT images of the chest (lung window) reveal multiple rounded shape ground glass opacities in peripheral location sparing the sub-pleural space in the right lung (yellow arrows)

CO-RADS 5 (as presented in implies a very high level of suspicion for lung involvement by COVID-19 based on typical CT findings (Table 1), which are ground-glass opacities with or without consolidations in lung regions close to visceral pleural surfaces and a multifocal bilateral distribution.

Table 1

Typical features for pulmonary involvement of COVID-19 [16]

Obligatory features:

• Ground-glass opacities, with or without consolidations, in lung regions close to visceral pleural surfaces, including the fissures (subpleural sparing is allowed)

• Multifocal bilateral distribution

Confirmatory patterns:

• Ground-glass regions:

- unsharp demarcation, (half) rounded shape

- sharp demarcation, outlining the shape of multiple adjacent secondary pulmonary lobules

• Crazy paving

• Patterns compatible with organizing pneumonia

• Thickened vessels within parenchymal abnormalities found in all confirmatory patterns

CO-RADS 5. Typical appearance for COVID. The multislice computed tomography (MSCT) image of a 42-year-old male admitted with fever (38.5oC) and dyspnoea. He was tachypnic (30 bpm), with low oxygen saturation (SpO2 75%). He was positive for COVID-19. Axial MSCT images (lung window) reveal bilateral multifocal areas of ground glass opacity and consolidation in central and peripheral locations; vascular thickening ( yellow arrow) and crazy paving appearance (red arrow) are detected

CO-RADS 5. Typical appearance for COVID. The multislice computed tomography (MSCT) image of a young female, 22-year-old, who had fever for 3 days with progressive coughing, dyspnoea, and vomiting. The PCR test was positive for COVID-19. After 2 days, oxygen saturation was 66%, the patient was admitted to intensive care and required mechanical ventilation. The patient died 24 hours later due to poor evolution to respiratory distress syndrome and to multi-organs failure. Axial MSCT images of the chest (lung window) reveal bilateral ground glass opacities with posterior predominance. Thickened vessels within ground-glass opacities are detected

CO-RADS 5 requires the presence of at least 1 confirmatory pattern that coincides with the temporal evolution of the disease. Multiple GGOs that show half rounded and unsharp demarcation but can be accompanied by sharply delineated ground glass areas that outline the shape of multiple adjacent secondary pulmonary lobules have been described early in the course of COVID-19. Crazy paving pattern and opacities that resemble organizing pneumonia such as reverse halo signs have been described later in the course of the disease. Subpleural curvilinear bands in an arching tethered pattern are also considered typical. Thickened vessels within parenchymal abnormalities are frequently found in all confirmatory patterns [17].

Statistical analysis

Statistical analysis was done using SPSS software version 27 (IBM, 2020). The data are presented in tables. Quantitative data are presented as mean, standard deviation, and range. Qualitative data are presented as frequencies and proportions. The Shapiro-Wilk test was used to determine the distribution characteristics of variables and variance homogeneity. Pearson’s c2 test and Fisher’s exact test were used to analyse qualitative variables as appropriate. The Mann-Whitney U test (MW) and Students’ t-test were used to analyse quantitative variables. Weighted k was used to measure inter-observer agreement (Landis and Koch, 1977). A p-value < 0.05 was accepted as statistically significant. Diagnostic accuracy, sensitivity, specificity, PPV, and NPV were calculated for 2 different scenarios. The RT-PCR test was considered as the gold standard.

Results

Baseline characteristics of the patient study group (Table 2)

This study included a total of 2500 patients (1535 males [61.4%] and 965 females [38.6%]; 18-95 years, mean age 60.61 ± 13.89 years) with suspected COVID-19 infection. They were referred from the emergency triage unit to the MSCT unit in the Department of Radiodiagnosis during the period from November 2020 to May 2021.

RT-PCR was performed as a reference standard and RT-PCR results were available after a mean of 19 hours (maximum 5 days).

All patients were classified into 2 groups: (positive RT-PCR): in 1500 patients (60%) with at least 1 positive RT-PCR result for COVID-19 within 5 days of MSCT, and (negative RT-PCR): in 1000 patients (40%) with 1 or more negative RT-PCR results.

Repeated RT-PCR was performed because of high clinical suspicion for COVID-19 infection but with a negative initial RT-PCR in 450 patients. In this study, fever and dry cough were the most common clinical symptoms, detected in 2004 (80.16%) and 1300 (52%) patients, respectively.

In this study, the most common comorbidities were cardiovascular diseases, followed by chronic lung disease and diabetes, found in 27.36%, 22.8%, and 18%, respectively.

Multislice computed tomography findings in 2500 patients (Table 3)

In this study, bilateral, peripheral, and posterior distribution and multifocal (more than three lesions) involvement as well as ground glass opacities, vascular enlargement, and crazy paving appearance were significantly more frequent in RT-PCR-positive patients (p < 0.05). Conversely, consolidation was more frequently seen in RT-PCR-negative patients (p < 0.05). All other morphological features did not differ significantly between the 2 groups.

Association between RT-PCR and CO-RADS (Table 4)

Of the 1500 RT-PCR-positive patients, 600 (40%) had CO-RADS score 5 and 51 (3.4%) had CO-RADS score 1. Of the 1000 RT-PCR-negative patients, 360 (36%) had CO-RADS score 2 and 10 (1%) were scored as CO-RADS 5. 420 patients (28%) with CO-RADS score 3 (indeterminate) were RT-PCR positive while 300 patients (30%) were RT-PCR negative.

Interobserver variability of CO-RADS (Table 5A-C)

Two teams of observers assessed 2500 MSCT scans using CO-RADS. There was absolute agreement in the assigned CO-RADS category in 2231 (89.24%) observations.

There was excellent agreement in the studied patients as the weighted κ value was 0.846, which was more pronounced in CORADS 5 (24.4%).

Diagnostic performance of CO-RADS (Table 6)

A confirmed case of COVID-19 was defined as a positive result on real-time RT-PCR assay of nasopharyngeal swab specimens.

In this study, 2 scenarios were done. In the first scenario, considering only CORADS 4 and CORADS 5 as COVID positive, the sensitivity, specificity, PPV, NPV, and accuracy were 55.27%, 95.00%, 94.31%, 58.61%, and 71.16%, respectively (Table 6A-B). In the second scenario, considering CO-RADS 3, CO-RADS 4, and CO-RADS 5 as COVID positive, the sensitivity, specificity, PPV, NPV, and accuracy were 83.27%, 65.00%, 78.11%, 72.14%, and 75.96%, respectively (Table 6C-D).

Table 6

Diagnostic performance of CO-RADS in 2500 patients

First scenario:

COVID-19 (CO-RADS 4 + CO-RADS 5) vs. no COVID-19 (CO-RADS 1 + CO-RADS 2 + CO-RADS 3)

A

PCR diagnosis

Total

COVID-19

No COVID-19

First scenario

COVID-19

829

50

879

No COVID-19

671

950

1621

Total

1500

1000

2500

B

Value

95% CI

Sensitivity

55.27%

52.71% to 57.80%

Specificity

95.00%

93.46% to 96.27%

PPV

94.31%

92.65% to 95.62%

NPV

58.61%

57.19% to 60.01%

Accuracy

71.16%

69.34% to 72.93%

Second scenario:

COVID-19 (CO-RADS 3 + CO-RADS 4 + CO-RADS 5) vs. no COVID-19 (CO-RADS 1 + CO-RADS 2)

C

PCR diagnosis

Total

COVID-19

No COVID-19

Second scenario

COVID-19

1249

350

1599

No COVID-19

251

650

901

Total

1500

1000

2500

D

Value

95% CI

Sensitivity

83.27%

81.28% to 85.12%

Specificity

65.00%

61.95% to 67.96%

PPV

78.11%

76.58% to 79.57%

NPV

72.14%

69.63% to 74.52%

Accuracy

75.96%

74.24 % to 77.62%

There were 350 false positive cases. The final diagnosis of the false positive cases were community-acquired pneumonia (n = 70), interstitial pneumonia (n = 45), drug-induced pneumonia (n = 60), eosinophilic pneumonia (n = 50), pulmonary oedema (n = 25), and pleuritis (n = 100).

The sensitivity of CO-RADS was higher in the second scenario (83.27% vs. 55.27%) while the specificity was higher in the first scenario (95% vs. 65%).

Discussion

The RT-PCR test is the gold standard test for diagnosing COVID-19. It has a high specificity, but its sensitivity has been reported to be as low as 60-70% [18]. Therefore, excluding a diagnosis of COVID-19 needs multiple negative tests, with insufficient test kits in some regions [19].

In response to MSCT reports of pulmonary abnormalities predating conversion to positive RT-PCR results, Chinese authorities broadened the official definition of COVID-19 infection to include patients with typical MSCT findings, even with a first negative RT-PCR result. This definition has resulted in a large number of presumptive cases of COVID-19 and an increasing diagnostic role for MSCT. However, the presence of mild or no MSCT abnormalities in many early cases of infection highlights the difficulties of early diagnosis [20,21].

CO-RADS was developed as a categorical system to assess suspicion of pulmonary involvement by COVID-19 on MSCT scans and to enable a standardized method for easy communication with clinicians, as well as workflow optimization, especially with increasing numbers of cases and logistic constraints [22].

Each CO-RADS grade corresponds to a very low, low, indeterminate, high, or very high level of suspicion for pulmonary COVID-19 involvement. The levels of suspicion are based on the similarity between the findings of a given imaging modality and the typical MSCT manifestations of COVID-19 that are reported in the literature [23].

Accordingly, MSCT findings must be interpreted together with the duration and type of symptoms, as well as clinical and laboratory findings, when it comes to building a clinical diagnosis of COVID-19 before RT-PCR tests are available [24].

Our study was based on retrospective evaluation of 2500 patients with suspected COVID-19 infection. About 1535 were male (61.4%) and 965 were female (38.6%). They presented clinically with different manifestations, but fever and dry cough were the most frequent symptoms, accounting for 80.16% and 52%, respectively.

In a study on 136 MSCT scans performed due to clinical suspicion of COVID-19, Barbosaa et al. [24] reported that most patients were male (60%) and the mean age was 58.2 years (median 61 years, range 19-88 years) including 51.6% with more than 60 years. The most common symptoms were cough, reported in 60.4%, fever in 37.0%, and dyspnoea in 39.6%. The mean of referred symptom duration was 6 days (median 5 days; IQR 1-7 days).

Our patients had some co-morbidities, such as cardiovascular diseases in 27.4%, chronic lung diseases in 22.8%, and diabetes mellitus in 18.0%, as well as some less frequent diseases. A study done by Prokop et al. [16] mentioned that cardiovascular diseases represent the highest association to COVID infection accounts for 44% followed by lung diseases 39%, cancer 21%, immune deficiency 16%, and diabetes 14%.

In our study, the patients were classified into 1500 positive RT-PCR cases and 1000 negative RT-PCR cases. Bilateral, peripheral, and posterior, distribution and multifocal (more than three lesions) involvement as well as ground glass opacities, vascular enlargement, and crazy paving appearance were significantly more frequent in RT-PCR-positive patients (p < 0.05). Conversely, consolidation was more frequently seen in RT-PCR-negative patients (p < 0.05). All other morphological features did not differ significantly between the 2 groups.

Xu et al. [25] stated that the MSCT manifestations of viral pneumonia are related to the pathogenesis of pulmonary viral infection, so viral pneumonias caused by similar viruses will show similar disease patterns and MSCT manifestations. GGO and consolidation are CT manifestations of both COVID-19 pneumonia and influenza virus pneumonia. However, to our knowledge, the differences between COVID-19 pneumonia and influenza virus pneumonia have not yet been studied. Similar results were obtained by Chung et al. [26].

Zhao et al. [27] reported that in patients with COVID-19 pneumonia, GGO was seen in 44 patients (84.6%), consolidation in 28 patients (53.8%), and mosaic attenuation in 10 patients (19.2%). Six patients (11.5%) had bronchial wall thickening, 12 (23.1%) had centrilobular nodules, 14 (26.9%) had interlobular septal thickening, 7 (13.5%) had crazy paving pattern, 18 (34.6%) had air bronchogram, 1 (1.9%) had mucoid impaction, and no patient had obvious pleural effusion. With regard to lesion distribution, lesions were distributed bilaterally in 38 patients (73.1%). For axial distribution, lesions were located in the outer zone in 35 patients (67.3%). For longitudinal distribution, lesions were located in the lower zone in 32 patients (61.5%).

Of the 1500 RT-PCR-positive patients, 600 (40%) had CO-RADS score 5, while 51 (3.4%) had CO-RADS score 1. Of the 1000 RT-PCR-negative patients, 360 (36%) had CO-RADS score 2, and 10 (1%) were scored as CO-RADS 5. 420 patients (28%) with CO-RADS score 3 (indeterminate) were RT-PCR positive while 300 patients (30%) were RT-PCR negative.

Özel et al. [28] reported that 111 patients (39.6%) had positive RT-PCR results. CO-RADS 5 patients had statistically significantly higher positive RT-PCR than the other groups (p < 0.001). All CO-RADS 2 patients had negative RT-PCR results. Also, CO-RADS group 4 and 5 had 109 patients with negative RT-PCR results.

Two teams of observers assessed 2500 MSCT scans using CO-RADS. There was absolute agreement in the assigned CO-RADS category in 2231 (89.24%) observations. There was excellent agreement in the studied patients as the weighted kappa value was 0.846, which was more pronounced at grade 5 (24.4%).

Results from a study done by Prokop et al. [16] on randomly selected patients clinically suggested to be COVID-19 positive showed moderate to substantial interobserver agreement, even though all observers were from different hospitals and had different experiences. Observer agreement was highest in CO-RADS categories 1 and 5, with κ values of 0.58 and 0.68, respectively. In 68.2% of observations, there was absolute agreement of scores, and in 15.2% of observations, scores varied between CO-RADS categories 1 and 2 or between CO-RADS categories 4 and 5, indicating that in more than 80% of the cases, the observers agreed on the suspicion for pulmonary involvement of COVID-19 being either low to very low or high to very high.

In this study, repeated RT-PCR was performed in 450 patients because of high clinical suspicion for COVID-19 infection but with a negative initial RT-PCR.

A study was done by Li and Xia [29] on 5 patients with COVID-19 infection, who had initial negative RT-PCR results. By reporting the chest MSCT findings in all 5 patients, they showed typical imaging findings, including ground glass opacity and/or mixed ground glass opacity and consolidation. After isolation for presumed COVID-19 pneumonia, all patients were confirmed to have COVID-19 infection by repeated swab tests, which means that a combination of repeated swab tests and MSCT scanning may be helpful for individuals with a high clinical suspicion of COVID-19 infection but negative initial RT-PCR results.

In this study, 2 scenarios were done; in the first scenario, considering only CO-RADS 4 and CO-RADS 5 as positive COVID-19, the sensitivity, specificity, PPV, NPV, and accuracy were 55.27%, 95.00%, 94.31%, 58.61%, and 71.16%, respectively. In the second scenario, considering CO-RADS 3, CO-RADS 4 and CO-RADS 5 as positive COVID, the sensitivity, specificity, PPV, NPV, and accuracy were 83.27%, 65.00%, 78.11%, 72.14%, and 75.96%, respectively. The sensitivity of CO-RADS was higher in the second scenario (83.27% vs. 55.27%) while the specificity was higher in the first scenario (95% vs. 65%).

A study done by Bellini et al. [30] to assess the dia-gnostic performance of CO-RADS on 572 patients (142 COVID-19 and 430 non COVID-19 patients) found that when a threshold of CO-RADS ≥ 4 was used, readers with different levels of expertise were able to discriminate between patients with positive and negative RT-PCR testing with a sensitivity of 61% and a specificity of 81%.

Barbosaa et al. [24] reported that considering only CO-RADS 4 and CO-RADS 5 as COVID-positive cases showed high specificity, which gives confidence for the selection of patients with positive results as those who should benefit from investigative or intervention measures, because there will be few false positive results. However, considering CO-RADS 3, CO-RADS 4, and CO-RADS 5 as COVID-positive cases showed high sensitivity, which provides confidence for the selection of patients with negative CT results as those in whom investigative or intervention measures can be avoided, such as RT-PCR tests, isolation, and hospitalization, because false negative CT results are rare.

Conclusions

We can conclude that the CO-RADS scoring system is a sensitive and specific method that can aid in the diagnosis of COVID-19 during the peak of an outbreak. CO-RADS should be integrated as a triage test in resource-constrained environments during the COVID-19 pandemic to assist in the optimization of RT-PCR tests, isolation beds, and intensive care units.

Table 2

Baseline characteristics of 2500 included patients

Parameters	Data
Age (years); mean ± SD	18-95; 60.61 ± 13.89
Sex, n(%)
Males	1535 (61.4)
Females	965 (38.6)
Symptoms at presentation, n(%)
Fever	2004 (80.2)
Tiredness	90 (3.6)
Dry cough	1300 (52.0)
Dyspnoea	1080 (43.2)
Abdominal symptoms	500 (20.0)
IQR (interquartile range), days	2-10
Number of RT-PCR assays, n(%)
1	2050 (82)
2	200 (8)
3	250 (10)
Comorbidities, n(%)
Hypertension	385 (15.4)
Obesity	230 (9.2)
Diabetes	450 (18.0)
Cardiovascular disease	684 (27.4)
Chronic lung disease	570 (22.8)
Immune deficiency	60 (2.4)
Cancer	96 (3.8)
Chronic kidney disease	120 (4.8)

Table 3

Patterns of pulmonary involvement on multislice computed tomography (MSCT) images in 2500 patients

MSCT findings	+ve RT-PCR	–ve RT-PCR	p-value
Bilateral involvement	1000	200	< 0.001*
Peripheral distribution	1370	210	< 0.001*
Posterior involvement	985	100	< 0.001*
Multifocal involvement	1060	115	< 0.001*
Consolidation	102	710	< 0.001*
Pleural effusion	95	100	0.796
Reversed halo	60	67	0.712
Ground glass opacity	1249	350	< 0.001*
Vascular enlargement	800	50	< 0.001*
Crazy paving	775	20	< 0.001*
Reticulations	240	249	0.788

Statistically significant.

Table 4

Association between RT-PCR and CO-RADS

CO-RADS	Positive RT-PCR	%	Negative RT-PCR	%
1	51	3.4	290	29.0
2	200	13.3	360	36.0
3	420	28.0	300	30.0
4	229	15.3	40	4.0
5	600	40.0	10	1.0
Total	1500	100.0	1000	100.0

Table 5

Inter-observer variability of CO-RADS

A				Team 2					Total
				CO-RADS 1	CO-RADS 2	CO-RADS 3	CO-RADS 4	CO-RADS 5
	Team 1	CO-RADS 1	Count	341.0	0	0	0	0	341.0
			% of total	13.6	0	0	0	0	13.6
		CO-RADS 2	Count	20.0	540.0	0	0	0	560.0
			% of total	0.8	21.6	0	0	0	22.4
		CO-RADS 3	Count	0	59.0	485.0	176.0	0	720.0
			% of total	0	2.4	19.4	7.0	0	28.8
		CO-RADS 4	Count	0	0	0	255.0	14.0	269.0
			% of total	0	0	0	10.2	0.6	10.8
		CO-RADS 5	Count	0	0	0	0	610.0	610.0
			% of total	0	0	0	0	24.4	24.4
	Total		Count	361.0	599.0	485.0	431.0	624.0	2500.0
			% of total	14.4	23.9	19.4	17.2	24.9	100.0

Green cells represent concordance; purple cells represent discordance

B	χ2 tests
		Value	df	Asymptotic significance (2-sided)
	McNemar-Bowker test	269.000	4	< 0.001
	Number of valid cases	2500

C	Symmetric measures	Value	Asymptotic standard errora	Approximately Tb	Asymptotic significance
	Measure of agreement, κ	0.846	0.009	74.600	0.000
	Number of valid cases	2500

Not assuming the null hypothesis. bUsing the asymptotic standard error assuming the null hypothesis