Diagnosing COVID-19 pneumonia in a pandemic setting: Lung Ultrasound versus CT (LUVCT) - a multicentre, prospective, observational study.

BACKGROUND: In this coronavirus disease 2019 (COVID-19) pandemic, fast and accurate testing is needed to profile patients at the emergency department (ED) and efficiently allocate resources. Chest imaging has been considered in COVID-19 workup, but evidence on lung ultrasound (LUS) is sparse. We therefore aimed to assess and compare the diagnostic accuracy of LUS and computed tomography (CT) in suspected COVID-19 patients.
METHODS: This multicentre, prospective, observational study included adult patients with suspected COVID-19 referred to internal medicine at the ED. We calculated diagnostic accuracy measures for LUS and CT using both PCR and multidisciplinary team (MDT) diagnosis as reference. We also assessed agreement between LUS and CT, and between sonographers.
RESULTS: One hundred and eighty-seven patients were recruited between March 19 and May 4, 2020. Area under the receiver operating characteristic (AUROC) was 0.81 (95% CI 0.75-0.88) for LUS and 0.89 (95% CI 0.84-0.94) for CT. Sensitivity and specificity for LUS were 91.9% (95% CI 84.0-96.7) and 71.0% (95% CI 61.1-79.6), respectively, versus 88.4% (95% CI 79.7-94.3) and 82.0% (95% CI 73.1-89.0) for CT. Negative likelihood ratio was 0.1 (95% CI 0.06-0.24) for LUS and 0.14 (95% CI 0.08-0.3) for CT. No patient with a false negative LUS required supplemental oxygen or admission. LUS specificity increased to 80% (95% CI 69.9-87.9) compared to MDT diagnosis, with an AUROC of 0.85 (95% CI 0.79-0.91). Agreement between LUS and CT was 0.65. Interobserver agreement for LUS was good: 0.89 (95% CI 0.83-0.93).
CONCLUSION: LUS and CT have comparable diagnostic accuracy for COVID-19 pneumonia. LUS can safely exclude clinically relevant COVID-19 pneumonia and may aid COVID-19 diagnosis in high prevalence situations.

Introduction

Coronavirus disease 2019 (COVID-19) is caused by the novel and rapidly spreading severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Cases continue to rise worldwide [1, 2]. With the pandemic hotspot moving to middle- and low-income regions such as Russia, India, Pakistan, Latin America and Africa, the consequences are potentially catastrophic [3]. Since there is no effective treatment yet, early detection is crucial in halting COVID-19.

The SARS-CoV-2 reverse transcriptase polymerase chain reaction (PCR) assay is the gold standard for diagnosing COVID-19. Although it is highly specific, it has limited sensitivity, long turnaround times and there is a worldwide shortage of test capacity. Serological tests are not useful in acute cases, and reliable rapid antigen tests are unfortunately not available [4, 5]. This hampers immediate triage and decision-making. Moreover, microbiological tests do not give insight into lung involvement, which is the main cause of morbidity and mortality [6, 7]. Correct assessment of lung involvement is thus crucial for appropriate triage, clinical management and efficient allocation of scarce medical resources.

The World Health Organization (WHO) recently advocated chest imaging, especially when PCR results are not readily available or the initial PCR is negative but clinical suspicion of COVID-19 remains high [8]. However, chest radiography (CXR) sensitivity is low [9, 10], and there are differences of opinion on the role of computed tomography (CT) in COVID-19 [9-12].

In view of the unmet clinical need for a fast and reliable test to diagnose or rule out COVID-19 pneumonia, bedside lung ultrasound (LUS) has attracted attention [13-23]. Although many clinicians might be unfamiliar with LUS, its diagnostic properties are better than CXR and physical examination combined, and equivalent to chest CT in diagnosing acute respiratory pathologies including pneumonia [24-27]. In addition, it has the advantage over CT of being portable, quick, radiation free, easy to disinfect and low cost. Moreover, it integrates real-time imaging into clinical decision-making at the bedside, reducing time to diagnosis and treatment [28-30].

Unfortunately, the literature on the use of LUS in COVID-19 is limited [8]. Therefore, our objective was to investigate the role of LUS in suspected COVID-19 patients at the emergency department (ED). To our knowledge, this is the first prospective study analysing and comparing diagnostic accuracy of LUS and CT in diagnosing pneumonia in patients with suspected COVID-19.

Methods

Study design and participants

This is a pragmatic, multicentre, prospective, observational study. Patients were recruited from three academic hospitals in the Netherlands (Radboud University Medical Centre, Nijmegen and both locations of the Amsterdam University Medical Centers) between March 19 and May 4, 2020. On May 4 the percentage of positive PCRs in patients with suspected COVID-19 fell below 10%. This is used as the epidemic threshold by the Dutch Institute for Research and Healthcare (NIVEL) and National Institute for Public Health and the Environment (RIVM) [31]. The study was registered with the Netherlands Trial Register and approved by the Medical Ethical Committee (CMO region Arnhem-Nijmegen).

All patients 18 years and older who were referred to the ED for internal medicine with suspected COVID-19 were eligible. In our hospitals we adhered to the case definition of the RIVM and WHO. A suspected case of COVID-19 was initially defined as having either fever, malaise, myalgia and respiratory symptoms, but later also included gastrointestinal symptoms, loss of smell or taste and unexplained delirium in the elderly.

Patients were included when CT, LUS and PCR were all performed. Exclusion criteria were age under 18, no verbal consent and uninterpretable CT or LUS. The study protocol has been published and is available online.

Medical workup, admission and decision process

All patients received regular medical workup (history, physical examination and routine laboratory tests). In addition, they received a chest CT and/or PCR as indicated by local protocol. CXR was not routinely performed in the workup of suspected COVID-19 in our hospitals. Clinical criteria for admission were: saturation <94% and/or respiratory rate >20/min. It was standard practice in participating hospitals that all admitted patients with suspected COVID-19 were discussed daily by a multidisciplinary team (MDT), consisting at least of consultants in infectious disease, respiratory disease and microbiology. In patients with a high clinical suspicion but negative PCR a diagnosis of COVID-19 could be made by this expert panel on the basis of clinical, laboratory, microbiological and CT data, after excluding alternative diagnoses. No clinical decisions were made based on LUS findings; the MDT was blinded for LUS results.

LUS

LUS was performed or supervised by internists (mostly registrars) who were certified in point-of-care ultrasound and had performed at least 20 supervised LUS. Both LUS and CT were done at presentation to the ED or within 24 h of admission. The majority of LUS were performed by the treating physician as an extension of the physical examination. As such, they were not blinded for the patient history or clinical picture. They were blinded for the PCR and CT result.

Handheld ultrasound systems were used with settings amenable to the detection of B-line artefacts (e.g. lung preset, or if this preset was not available, abdominal preset with tissue harmonic imaging switched off and dynamic range put at the lowest level) [16]. See supplementary figure S1 for the scan protocol.

In keeping with pre-specified criteria in recent Chinese, Italian and British literature, LUS was deemed positive if there were three or more B-lines and/or consolidation in two or more zones unilaterally or in one or more zones bilaterally. When COVID-19 features were not found or just in one zone unilaterally, the scan was deemed negative (see supplementary table S1) [15–17, 21, 22]. See supplementary figure S2 and videos for examples of COVID-19 sonographic features.

CT

The chest CTs were assessed by the local radiologists. The radiologists were blinded for LUS and PCR results, but not for clinical information. The likelihood of COVID-19 pneumonia was reported via the COVID-19 Reporting and Data System (CO-RADS), which uses a scale from 1 (very low) to 5 (very high), using pre-specified criteria (see supplementary table S2) [32]. A CO-RADS score of 1 or 2 is regarded as negative, a score of 3 is equivocal, and a score of 4 or 5 is deemed positive. For the purpose of this study equivocal CTs were considered negative.

PCR

A PCR test on swab samples obtained from the oropharynx or nasopharynx (or if available, from sputum, faeces, tracheal aspirate or bronchoalveolar lavage) was performed in all patients according to WHO standards. The same PCR assay was used in all participating hospitals. In case of a negative or indeterminate test result but a high clinical suspicion, PCR was repeated. If a patient had an indeterminate test and no PCR was repeated, the PCR was considered negative.

Outcomes

Following WHO recommendations we used both (serial) PCR (gold standard) and clinical follow-up (MDT diagnosis) to assess the sensitivity, specificity, diagnostic accuracy, area under the receiver operating characteristic (AUROC) curve, predictive values, positive likelihood ratio (PLR) and negative likelihood ratio (NLR) of LUS and CT. Agreement was calculated between LUS and CT. Interobserver agreement was calculated between sonographers.

Statistical analysis

No sample size calculations were performed for this study. Normally distributed continuous variables are summarised by the mean and standard deviation (sd). Not normally distributed continuous variables are summarised by the median and interquartile range (IQR). Differences between groups were tested using the independent t-test for normally distributed outcomes. Difference in means and 95% confidence interval were calculated. The widths of the intervals have not been adjusted for multiplicity; inferences drawn may therefore not be reproducible. A non-parametric Mann–Whitney U test was used to compare continuous outcomes that were not normally distributed between groups. Sensitivity and specificity of LUS and CT were compared using the McNemar test.

Discriminatory power of LUS and CT were determined by a receiver operating characteristic (ROC) curve and the corresponding AUROC with 95% confidence intervals for each cut-off.

Agreement between LUS and chest CT was quantified with the Cohen's kappa statistic. Interobserver agreement between sonographers was measured with a weighted kappa via the intraclass correlation coefficient with a two-way mixed effects model. Three sonographers independently assessed ultrasound recordings of 60 patients. This number was chosen as it allowed rejection of the null hypothesis that agreement was moderate (κ 0.5) in case true agreement was good (κ 0.8) with 80% power. The power calculation was performed assuming proportions of negative and positive ratings of 40% and 60%. A two-sided significance level of 5% was used for all analyses. Data were analysed by AL, BK, FS and KA using SPSS version 26 (IBM, Chicago, IL, USA).

Results

From March 19 until May 4, 2020, 187 patients were recruited for the study (figure 1). There were no missing data or adverse events. PCR positive patients on average had longer duration of symptoms and more often required oxygen therapy and intensive care unit admission than PCR negative patients. See table 1 for patient demographics and clinical characteristics.

FIGURE 1

Study population flowchart. ED: emergency department; CT: computed tomography.

TABLE 1

Patient characteristics

	All patients	PCR positive	PCR negative	p-value
Subjects n	187#	86	100
Age years	63.7±15.7	63.4±14.8	64.1±16.5	0.668
Male	108 (57.8)	50 (58.1)	58 (58.0)	0.97
Admission	140 (74.9)	65 (75.6)	75 (75.0)	0.855
Admission ED to IC	9 (4.8)	8 (9.3)	1 (1.0)	<0.001
In hospital mortality	9 (4.8)	5 (5.8)	4 (4.0)	0.001
Obesity (based on clinical assessment)	52 (27.8)	27 (31.4)	25 (25)	0.676
Duration of symptoms days	6.5±5.1	7.3±4.1	5.8±5.8	0.007
Comorbidities
Asthma	10 (5.3)	3 (3.5)	7 (7.0)	0.033
Chronic cardiovascular disease	44 (23.5)	18 (20.9)	26 (26.0)	0.104
COPD (GOLD stage >2)	30 (5.3)	12 (14.0)	18 (18.0)	0.134
Current malignancy	35 (18.7)	9 (10.5)	26 (26.0)	<0.001
Diabetes mellitus	36 (19.3)	21 (24.4)	15 (15.0)	0.001
Laboratory analysis on admission
CRP mg·L−1 median (interquartile range)	59 (81.5)	68 (83.8)	49.0 (80.8)	0.73
Procalcitonin ng·mL−1 median (interquartile range)	0.1 (0.25)	0.12 (0.21)	0.1 (0.37)	0.004
Positive blood culture	12 (6.4)	1 (1.2)	11 (11.0)	0.204
Positive influenza A/B	0	0	0	0.593
Observations
Modified early warning score (MEWS)	2.6±1.8	2.9±1.6	2.3±1.8	0.605
Temperature °C	37.5±1.2	37.8±1.1	37.2±1.3	0.174
Respiratory rate min-1	21.2±6.9	21.9±7.2	20.7±6.7	0.888
Oxygen saturation %	95.8±3.2	94.9(3.5	96.4±2.8	0.165
Oxygen therapy	58 (0.31)	34 (39.5)	24 (24.0)	0.004
Intubation	9 (4.8)	8 (9.3)	1 (1.0)	<0.001

Data are presented as mean±sd or n (%), unless otherwise stated. Statistically significant differences between the PCR positive and PCR negative groups are emphasised in bold text. ED: emergency department; IC: intensive care; GOLD: Global Initiative for Chronic Obstructive Lung Disease; CRP: C-reactive protein. #: One patient had an indeterminate PCR and no repeat PCR, therefore the total amount of patients (187) differs from the sum of the PCR positives (86) and PCR negatives (100).

Study population

Seventy-five patients had an initial positive PCR result, while 11 patients turned out positive after repeated testing. Fifteen patients with negative PCR results were eventually diagnosed with COVID-19 (table 2 and supplementary figures S3 and S4). The most common alternative diagnoses in the PCR negative group were bacterial/aspiration pneumonia (14%), upper respiratory tract infection (11%), progression of malignant disease (9%), exacerbation asthma/COPD (7%) and decompensated heart failure (7%) (supplementary table S3).

TABLE 2

Diagnostic results

	PCR positive	PCR negative	Final diagnosis
Lung ultrasound: positive	79	29	91
Lung ultrasound: negative	7	71	10
CT: positive	76	18
CT: negative	10	82

Data are presented as n. CT: computed tomography.

Diagnostic performance

For LUS we found a sensitivity of 91.9% (95% CI 84.0–96.7) and a specificity of 71.0% (95% CI 61.1–79.6) compared to PCR as a reference standard. AUROC was 0.81 (95% CI 0.75–0.88). The PLR and NLR were 3.2 (95% CI 2.3–4.3) and 0.1 (95% CI 0.06–0.24), respectively.

Comparing CT to PCR as a reference standard, we found a sensitivity of 88.4% (95% CI 79.7–94.3) and specificity of 82.0% (95% CI 73.1–89.0). AUROC was 0.89 (95% CI 0.84–0.94). The PLR and NLR were 4.9 (95% CI 3.2–7.5) and 0.14 (95% CI 0.08–0.26), respectively. See tables 2 and 3 for full results, supplementary figures S5–S7 for ROC curves and supplementary table S4 for details on false negative LUS and CT.

TABLE 3

Diagnostic accuracy for COVID-19

	Lung ultrasound versus PCR	CT versus PCR	Lung ultrasound versus COVID-19 diagnosis#
Sensitivity % (95% CI)	91.9 (84.0–96.7)	88.4 (79.7–94.3)	90.1 (82.5–95.2)
Specificity % (95% CI)	71.0 (61.1–79.6)	82.0 (73.1–89.0)	80.0 (69.9–87.9)
Positive likelihood ratio	3.2 (2.3–4.3)	4.9 (3.2–7.5)	4.5 (2.9–6.9)
Negative likelihood ratio	0.1 (0.06–0.24)	0.14 (0.08–0.26)	0.12 (0.07–0.2)
PPV % (95% CI)	73.2 (66.6–78.8)	80.9 (73.4–86.6)	84.2 (77.7–89.2)
NPV % (95% CI)	91.0 (83.1–95.4)	89.1 (82.0–93.7)	90.8 (81.7–95.6)
Accuracy % (95% CI)	80.7 (74.2–86.1)	85.0 (79.0–89.8)	82.8 (76.6–87.9)

CT: computed tomography; COVID-19: coronavirus disease 2019; PPV: positive predictive value; NPV: negative predictive value; CI: confidence interval. #: According to multidisciplinary team, COVID-19 is highly likely.

When we compared LUS to the MDT diagnosis, the AUROC increased to 0.85 (95% CI 0.79–0.91). Specificity increased to 80.0% (95% CI 69.9–87.9) and PLR to 4.5 (95% CI 2.9–6.9), while the NLR remained similar at 0.12 (95% CI 0.07–0.2).

Using both PCR and the MDT diagnosis as a reference, the McNemar test found no significant difference in sensitivity or specificity between LUS and CT (supplementary table S5). The agreement between LUS and chest CT was substantial: 0.65. The interobserver agreement between scanning physicians was good: 0.89 (95% CI 0.83–0.93).

Discussion

To our knowledge this is the first study prospectively analysing and comparing the diagnostic accuracy of LUS and chest CT in COVID-19 pneumonia. We found that LUS and CT have similarly good diagnostic accuracy for COVID-19 pneumonia, with AUROC between 0.8 and 0.9. In addition, we found substantial agreement between imaging modalities and good agreement between scanning physicians.

Screening test

The sensitivity, negative predictive value and negative likelihood ratio found for LUS are promising (table 3). Only seven patients in our cohort had a false negative LUS compared to PCR (supplementary table S4). Three of those patients also did not have a positive CT, indicating they did have COVID-19, but without pulmonary involvement. The reason for the lack of abnormalities could have been the short duration of symptoms (2 days in these patients). Indeed, imaging studies have been shown to be negative in PCR positive patients in the first days of the disease [32, 33]. The four other patients had very mild abnormalities on CT, predominantly in the posterior and inferior zones, that hardly reached the pleura and therefore were (almost) undetectable via LUS. In keeping with what was observed in previous studies, it seems important that the inferior and posterior regions are examined. This can be challenging in obese or immobile patients as illustrated by the fact that one of the false negative ultrasounds occurred in a patient in which the posterior fields could not be examined.

Interestingly, five of the seven “false negative” patients actually did have sonographic COVID-19 features, but only in one unilateral zone, so we regarded those as negative. This might have been a sign of mild or early stage COVID-19 pneumonia. If we had regarded these patients as positive, the AUROC would have increased to 0.87 (95% CI 0.82–0.93) (supplementary figure S6).

Importantly, none of the patients that were missed on LUS required supplemental oxygen at any stage, nor did they need admission due to COVID-19-related symptoms. Our results demonstrate that LUS can be a safe screening tool for clinically relevant pulmonary involvement in COVID-19 patients who present at the ED.

However, when an alternative diagnosis is lacking, a high post-test probability of COVID-19 without pulmonary involvement may remain, especially when patients present early in the disease course. In those cases, patients should be instructed to home-quarantine and to seek medical help when symptoms worsen. With LUS as an extension of the physical examination, these decisions can be made within minutes of the patient presenting to the ED without any further imaging.

Diagnostic test

Specificity for LUS was lower than CT: 71.0% (95% CI 61.1–79.6) and 82.0% (95% CI 73.1–89.0), respectively. The difference in specificity between CT and LUS of ∼10% is consistent with what has previously been reported in the literature [24, 26, 27].

One explanation for the low specificity of both modalities could be the lack of sensitivity of the PCR. This automatically generates more “false positives” [32, 33]. We therefore also compared LUS with the MDT diagnosis. In our cohort 29 patients had a positive LUS and negative PCR. Twelve of them were eventually diagnosed with COVID-19 by the MDT. This led to a marked increase in specificity and PLR. Due to incorporation bias (incorporation of the CT results in the MDT diagnosis), CT should not be compared to the MDT diagnosis as this would overestimate accuracy.

Another reason for the lower specificity is that the sonographic features of COVID-19 are not exclusive to COVID-19. They are also observed in other aetiologies like (viral or atypical) pneumonia, interstitial lung disease (ILD), acute respiratory distress syndrome (ARDS), heart failure and atelectasis. However, this is also the case for CT, albeit to a lesser extent [9, 12, 32, 33]. The most common diagnoses in patients with a false positive LUS were: pneumonia of another source (6), atelectasis in bedridden patients (5) and drug-induced pneumonitis (3). There were no patients with underlying ILD in our cohort. When the prevalence of COVID-19 decreases and/or the prevalence of diseases which produce similar findings (e.g. ILD or influenza) increases, the positive predictive value of LUS will probably decrease concomitantly. More research has to be done to assess whether LUS can distinguish COVID-19 from similar disease processes.

Strengths

The results of this study are in line with the literature on diagnostic accuracy of LUS in acute respiratory failure caused by different aetiologies ranging from pneumonia to ARDS [24, 26, 27, 29, 30, 34, 35]. Almost all LUS were performed by acute internal medicine registrars with modest LUS experience, which underscores the applicability of LUS in a real-life setting.

Advantages of LUS over CT include: ease of use, affordability, repeatability and avoidance of radiation [13, 14, 21–23]. LUS can be used in (unstable) patients without the need to transport them. This reduces exposure in healthcare personnel and equipment, which minimises wasting of scarce resources and potentially prevents nosocomial spread of infection [9, 12, 18, 23]. In addition, by integrating imaging into clinical decision-making at the bedside, LUS can reduce time to diagnosis and treatment at the ED [28, 30]. One can also reduce diagnostic uncertainty by scanning other structures, such as the inferior vena cava, heart and deep venous system of the lower extremities [24, 28–30]. If uncertainty still remains about the cause of respiratory symptoms or hypoxia, the treating physician can always employ additional conventional tests like CT.

COVID-19 has laid bare health disparities along socioeconomic, racial, cultural and ethnic lines across and within nations [2, 3]. CT is costly and might not be readily available, even in high-income countries [8, 20]. LUS is an affordable alternative, especially when handheld devices are used. It may be used in any care setting, further reducing barriers to adequate care [17, 19]. It has even been shown that it can be accurately performed by non-physicians who are guided remotely, so patients could be screened extramurally [36]. LUS has a steep learning curve, for physicians and other (para)medical personnel, and the basics can be learned in under 2 h [37-39].

Limitations/bias

Our study has some important limitations. First, this was an observational study; no blinding, randomisation or power calculation was performed. Second, our cohort was a convenience sample. We have tried to enrol every adult patient with suspected COVID-19 referred to internal medicine when a certified sonographer was present. We therefore feel that our cohort is random and representative, and the chance of selection bias is minimal. Our cohort seems consistent with what is found in the literature in terms of mean duration of symptoms, comorbidity and mortality. We had a high number of patients with malignancies because all three hospitals are tertiary oncology centres. Third, the radiologists and scanning physicians were aware of the patients' history and clinical pictures. While this may have led to some bias, clinical tests should preferably be guided by thorough clinical assessment. The integration of imaging with clinical assessment in real-time is one of the main advantages of bedside ultrasound. Blinding sonographers for research purposes is therefore not desirable as the results would not be generalisable to daily practice.

Conclusion

Our study shows that the diagnostic accuracy of LUS is comparable to CT. We demonstrated that LUS can help in triage by excluding clinically relevant COVID-19 pneumonia at the ED and may aid in diagnosis of COVID-19 in a high-prevalence setting. It may prove especially useful in situations where CT or PCR results are not readily available. We advocate the use of LUS as an extension of the physical examination and encourage setting up training programmes worldwide so this tool can be used during and subsequent to the pandemic. We also suggest that further studies be conducted in different settings to validate our findings.

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material 00539-2020.SUPPLEMENT