Comparison of admission chest computed tomography and lung ultrasound performance for diagnosis of COVID-19 pneumonia in populations with different disease prevalence.

PURPOSE: Chest computed tomography (CT) is considered a reliable imaging tool for COVID-19 pneumonia diagnosis, while lung ultrasound (LUS) has emerged as a potential alternative to characterize lung involvement. The aim of the study was to compare diagnostic performance of admission chest CT and LUS for the diagnosis of COVID-19.
METHODS: We included patients admitted to emergency department between February 21-March 6, 2020 (high prevalence group, HP) and between March 30-April 13, 2020 (moderate prevalence group, MP) undergoing LUS and chest CT within 12 h. Chest CT was considered positive in case of "indeterminate"/"typical" pattern for COVID-19 by RSNA classification system. At LUS, thickened pleural line with ≥ three B-lines at least in one zone of the 12 explored was considered positive. Sensitivity, specificity, PPV, NPV, and AUC were calculated for CT and LUS against real-time reverse transcriptase polymerase chain reaction (RT-PCR) and serology as reference standard.
RESULTS: The study included 486 patients (males 61 %; median age, 70 years): 247 patients in HP (COVID-19 prevalence 94 %) and 239 patients in MP (COVID-19 prevalence 45 %). In HP and MP respectively, sensitivity, specificity, PPV, and NPV were 90-95 %, 43-69 %, 96-72 %, 20-95 % for CT and 94-93 %, 7-31 %, 94-52 %, 7-83 % for LUS. CT demonstrated better performance than LUS in diagnosis of COVID-19, both in HP (AUC 0.75 vs 0.51; P < 0.001) and MP (AUC 0.85 vs 0.62; P < 0.001).
CONCLUSIONS: Admission chest CT shows better performance than LUS for COVID-19 diagnosis, at varying disease prevalence. LUS is highly sensitive, but not specific for COVID-19.

Introduction

Lung ultrasound (LUS) has emerged as a potential imaging technique for first-line (screening) modality with low costs and widespread availability in Coronavirus disease 2019 (COVID-19) [1]. LUS findings described for COVID-19 pneumonia are single or confluent interstitial artifactual signs, small hyperechoic lung regions, thickened pleural lines, and consolidations. Several studies describing LUS findings in COVID-19 were recently published, with a report that suggest pivotal role of LUS, reserving the use of CT when LUS is not sufficient to answer clinical question [[2], [3], [4], [5]]. However, it is not known how accuracy of LUS might vary across disease prevalence, namely different phases of severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) epidemic [6]. Additionally, the risk of disease transmission represents an issue for sonographers and other patients [7,8].

Unenhanced chest computed tomography (CT) was shown as a rapid tool to suggest diagnosis of COVID-19 pneumonia in patients with moderate-severe respiratory symptoms, with high sensitivity and potential for stratification of disease severity [9,10]. However, high workload of CT and scanner cleaning procedure are main issues for the widespread use of CT in diagnosis of COVID-19 [[11], [12], [13]]. Furthermore, the diagnostic performance of CT for COVID-19 remains not fully clear. In particular, variations in disease prevalence could influence either negative or positive predictive positive values (NPV and PPV), with potential concerns about its utility also in patients with moderate to severe symptoms

In the present study, we aimed to compare LUS and chest CT performance for COVID-19 diagnosis in two cohorts with different prevalence, at the beginning of the outbreak and after three weeks of lockdown.

Materials and methods

Study population

The present retrospective study was approved by the Local Ethics Committee (institutional review board -IRB- approval number 481/2020/OSS*/AUSLPC). The informed consent was waived by the IRB. The study included consecutive patients presenting at emergency department and undergoing admission chest CT and LUS in two periods: 1. during the outbreak of COVID-19 epidemic, from February 21 to March 6, 2020 (high prevalence group, HP); 2. After three weeks of lockdown, from March 30 to April 13, 2020 (moderate prevalence group, MP). During these two periods the indication to test subjects with real-time reverse transcriptase polymerase chain reaction (RT-PCR) test for SARS-CoV-2 by nasal-pharyngeal swabs was based on the guidelines drafted by local authorities: cases with symptoms (body temperature>37.5 °C, cough, and dyspnea) or asymptomatic subjects exposed to ascertained positive patients during 48 h before symptoms onset [14]. Differences were applied between the HP and MP groups, first of all the capacity of RT-PCR was insufficient during HP period and was appropriate during MP; for this reason, priority was given to most severe cases in HP (for prompt isolation and clinical management). Second, clinical presentation was overall more severe in HP compared to MP, which could have contributed in further clinical disparity between the two groups (see results).

In the emergency department, LUS was performed to complete clinical evaluation in patients with fever or respiratory symptoms. Indication for CT was based on current guidelines described in the Consensus statement of the Fleischner Society [7]. Considering the environment of high community disease burden and critical resource limitations, patients underwent CT in case of significant pulmonary dysfunction or damage (eg, hypoxemia, moderate-to-severe dyspnea) [7]. Exclusion criteria were: 1. patients without clinical and epidemiological suspicion of COVID-19 who did not perform RT-PCR test for SARS-CoV-2 by nasal-pharyngeal swabs; 2. unavailable clinical data. The diagram showed in Fig. 1 summarizes the patients enrollment process. Two hundred and thirty-six patients of this population were analyzed in a previous study [10].

Fig. 1

Diagram showing the patient selection process.

Abbreviations: COVID-19, coronavirus disease 2019; CT, computed tomography; HP, high prevalence; LUS, lung ultrasound; MP, moderate prevalence; RT-PCR, reverse-transcription polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Epidemiological history, clinical features, and laboratory findings were recorded at admission. CT was performed within 12 h after clinical evaluation and LUS; the time elapsed between LUS and CT was recorded in all cases. CT was performed within 12 h after clinical evaluation and LUS. All patients were categorized in four clinical types: mild, moderate, severe, or critical [15]. The patient admission to intensive care unit (ICU) or death was recorded in all cases.

LUS protocol and interpretation

Bedside LUS was performed with Esaote MyLab X7 (Esaote Group, Genova, Italy) or Philips Affiniti 70 (Philips, Amsterdam, Holland) with both convex array probe (1−8 MHz) and linear array probe (3−17 MHz) by an emergency physician. The probe was protected by single-use plastic cover; after the examination of each patient the cover was changed and the probe was disinfected by a 70 % ethanol solution [16]. Each patient was investigated in supine, lateral decubitus, and prone positions. The anterior and posterior axillary lines divided each hemithorax in three zones: anterior, lateral and posterior. Each zone was divided in upper and lower by a horizontal circumferential line passing through the nipples. As a consequence the two lungs were divided in a total of 12 zones (Fig. 2 ), which were explored with both the convex and linear probes, using longitudinal and transverse plane as follows:

Fig. 2

The 12 zone investigated with lung ultrasound. In the antero-lateral view (a) each hemithorax was divided by the anterior and posterior axillary line in anterior and lateral zone divided by an horizontal zone passing through nipples, identifying four zones: anterior upper zone, anterior lower zone, lateral upper zone, and lateral lower zone. In the postero-lateral view (b) the spinal line, the posterior axillary line and the horizontal nipples lines depicted two additional zone for each hemithorax: posterior upper zone and posterior lower zone.

Abbreviations: ALZ, anterior lower zone; AUZ, anterior upper zone; DL, diaphragmatic line; LLZ, lateral lower zone; LUZ, lateral upper zone; PLZ, posterior lower zone; PUZ, posterior upper zone.

pleura was assessed for smooth, continuous or interrupted pleural line [2];

interstitial syndrome was defined by B-lines (Fig. 3 ) defined as vertical artifactual hyperechoic lines arising from the pleura and continuing in the depth of the image [17]. Three or more B-lines between two adjacent ribs in longitudinal plane or close together in transverse image was considered pathological [17];

Fig. 3

A 67 years old male of the early outbreak group admitted to the emergency department for persistent fever and cough from three days. (a) Lung ultrasound performed with convex probe showed in the right lateral lower zone, three B-lines (arrows) and thickened pleural line (arrowheads) considered positive for COVID-19 interstitial pneumonia. (b) Axial unenhanced CT image at the level of inferior pulmonary veins confluence in left atrium, showed pure ground-glass opacities in the middle lobe and right inferior lobe (arrows) while in the left lower lobe depicted a ground-glass opacity with visible intralobular lines, known as “crazy-paving” appearance (arrowheads). Chest CT findings were considered “typical” for COVID-19 pneumonia [20].

Abbreviations: COVID-19, coronavirus disease 2019; CT, computed tomography.

“white lung”, when no horizontal reverberation (A-lines) or separated B-lines are visible, and the density of peripheral lung increases without reaching true consolidation [1];

consolidation, in case of complete filling of the alveoli with pus or inflammatory changes, the echo-structure of the lung itself become visible as hypoechoic lesion (“liver-like”) with air bronchogram [1,18];

pleural effusion.

Type and distribution of LUS findings were obtained by emergency physician report and recorded by zone, in each lung. When no clear distribution predominance was observed, the pattern was considered diffuse. The overall number and the years of experience of emergency physician performing LUS in the two periods was also registered.

As previously described, a thickened pleural line with pathological B-line pattern associated or not with consolidations or white lung appearance in at least one zone was considered positive for COVID-19 pneumonia [1]. In addition, to quantify COVID-19 pneumonia extent, a semiquantitative score was calculated for each patients (minimum 0, maximum 4), dividing each lung in two zones (upper and lower) and giving 1-point if LUS abnormalities occurred in at least two quadrant of the upper or lower zone explored (e.g LUS abnormalities in anterior and posterior lower zone = 1-point).

When consolidations were not accompanied by pathological B-lines, was considered the alternative diagnosis of pneumonia other than COVID-19. The coexistence of bilateral pathological B-lines in ≥1 zone without pleural abnormalities, associated with pleural effusion was in accordance with pulmonary edema [19].

Chest CT protocol and interpretation

Chest CT protocol was described in a previous study [10]. Unenhanced chest CT was performed in the supine position during an inspiratory breath hold, moving from the apex to the lung bases, with a 16-slice scanner (Emotion 16; Siemens, Forchheim, Germany). Low-dose CT acquisition was performed with the following parameters: tube voltage, 110 kV if body weight was 80 kg or less and 130 kV if patients weighed more than 80 kg; tube current, 40 mAs; pitch, 1; and collimation, 0.625 mm. After each examination the room was decontaminated by a solution at 62–71 % of ethanol or 0.1 % of sodium hypochlorite [16]. The scanner cleaning procedure required around 15−20 minutes for each patient. Image data sets were reconstructed with 1–2 mm slice thickness using both sharp kernels (B70f) with standard lung window settings (window width, 1500 HU; window center, -500 HU) and medium-soft kernels (B40f) with soft-tissue window settings (window width, 300 HU; window center, 40 HU).

Chest CT interpretation was independently performed by two radiologists (D.C. and G.D.V.) blinded to clinical data, respectively with 5 and 14 years of experience. Patients with “indeterminate” or “typical” (Fig. 3) features at CT as described by the Radiological Society of North America (RSNA) classification system for reporting COVID-19 pneumonia (e-Table 1) were considered positive [20]. The total extent of COVID-19 pneumonia (visual CT score) was expressed as percentage of total lung volume and estimated to the nearest 5% in three lung zones showed in a previous report and averaged to produce a global percentage of abnormalities extent [10,21,22]. For each lung, the prevalence of CT abnormalities in the upper-middle zone or in the middle-lower zone was recorded; when a clear predominant cranio-caudal distribution pattern was absent, abnormalities were classified as diffuse. Consensus formulation for the visual scores was obtained as reported in the study by Cottin et al. [23]. The 5 % most divergent observations for CT abnormalities global extent and discordance over the categorical CT assessment were resolved by consensus. Chest CT interpreted as alternative diagnosis other than COVID-19 pneumonia were also recorded.

Reference standard

Nasal and pharyngeal swabs were sampled within 12 h after CT and LUS, and tested with RT-PCR for SARS-CoV-2. In cases of negative RT-PCR results and persistent clinical suspicion of COVID-19, the patients were retested. Positive RT-PCR defined a patient “positive” for COVID-19. Conversely, double negative RT-PCR swabs defined a patient “negative” for COVID-19. In patients with only a single negative RT-PCR swab, the presence of serum IgG anti- SARS-CoV-2 (Abbott SARS-CoV-2 IgG; Abbott Diagnostics, IL, USA) based on the chemiluminescence enzyme immunoassays (CLIA) performed >14 days after symptoms onset were used to reclassify the case (“negative” in absence of significant serum IgG title or “positive” when present) [24].

Statistical analysis

Categorical variables were expressed as counts and percentage, with corresponding 95 % confidence interval (95 %CI) using Wilson method. Continuous variables were showed as median and 95 %CI for the median. The difference between patients of the two different epidemic periods were assessed by Chi-square test or Fisher's exact test for categorical variables and Mann-Whitney U test for continuous variables, as appropriate.

The agreement between CT and LUS was tested using the weighted Cohen’s kappa (K) [25]. The agreement using K value was interpreted as follows: <0.20, poor; 0.21−0.40, fair; 0.41–0.60, moderate; 0.61–0.8, substantial; 0.81–1, excellent [26].

Primary outcome analysis: sensitivity, specificity, PPV, NPV, and accuracy (and their 95 %CI) were calculated for chest CT and LUS against RT-PCR and serology standard of reference. Receiver operating characteristic (ROC) curve analysis was performed for CT and LUS. The area under the ROC (AUC) was used to assess the performance of the two techniques for the diagnosis of COVID-19 pneumonia. The ROC curves of CT and LUS were compared by the methodology of DeLong et al. [27]. An additional analysis calculated the same metrics for LUS using as standard of reference the combination of RT-PCR, serology, and the presence of an “indeterminate” or “typical” pattern at CT [20].

Secondary outcome analysis: in COVID-19 patients (using as standard reference RT-PCR and serology) ICU admission and death (related to COVID-19) were merged into binary composite “clinical outcome”. The ROC curve were calculated for age, visual CT score, and LUS score in relation to “clinical outcome”. The highest value of the Youden Index was obtained to determine an appropriate cutoff in relation to the “clinical outcome” in order to transform continuous variables to categorical variables. Visual CT score and LUS score were correlated with “clinical outcome” using the Kaplan-Meier method (product-limit). The “clinical outcome” functions were compared between independent groups of patients by means of the log-rank test. Multivariable Cox proportional hazards regression analysis was used to examine the association between age, gender, comorbidities, visual CT score, and LUS score with “clinical outcome” to estimate hazard ratios (HRs) and 95 % confidence intervals (CIs).

A P value <0.05 was considered statistically significant. Statistical analysis was performed using MedCalc software (version 14.8.1, MedCalc Software Ltd, Ostend, Belgium).

Results

Study population characteristics

In the referring local population (about 287,000 residents) the positivity rate of SARS-CoV-2 during HP period was 42.5 % (positive swabs 607/1429 performed; 95 %CI 40–45 %) while was 18 % (positive swabs 599/3262 performed; 95 %CI 17–20 %) in the MP period. Table 1 summarizes patients demographics, clinical, and laboratory findings. A total number of 486 patients (males 298/486, 61 %, 95 %CI 57−65%; median age 70 years old, 95 %CI 68−72 years) were included in the study. The HP group comprised 247/486 (51 %; 95 %CI 46–55 %) patients, while the MP cohort 239/486 (49 %; 95 %CI, 45–54 %) patients. Patients diagnosed with COVID-19 were 233/247 (94 %; 95 %CI 91−97%) in the HP group and 108/239 (45 %; 95 %CI 39−52%) in the MP group. In the HP group, median age (68 years vs 73 years, P = 0.003) and female proportion (30 % vs 48 %, P < 0.001) were lower as compared to the MP group. Conversely, the frequency of subjects with known exposure to COVID-19 patients was higher in HP group (36 %, 95 %CI 31−43%) compared to MP (18 %, 95 %CI 14−24%; P < 0.001) as well as the frequency of subjects with temperature above 37.5°C (54 %, 95 %CI 48−60%; 16 %, 95 %CI 12−22%; P < 0.001) and cough (63 %, 95 %CI 57−69%; 30 %, 95 %CI 25−36%; P < 0.001). Percentage of peripheral oxygen saturation (%SpO2) was lower in HP group (median value 92 %, 95 %CI 91−92%) compared to MP group (median value 97 %, 95 %CI 96−97%; P < 0.001) as well as the white blood cell count (WBC) (6.3 vs 8.6  ×  103/μL, P<0.001). Conversely, C-reactive protein (CRP, 8.2 vs 3.8 mg/dl; P < 0.001) and lactate dehydrogenase (LDH, 355 vs 286 U/L; P < 0.001) levels were higher in HP group. Considering patients diagnosed with COVID-19, higher frequency of a severe or critical disease was observed in HP group (160/233, 69 %, 95 %CI 62–74 %) compared with MP group (34/108, 31 %, 95 %CI 23–41 %; P < 0.001).

Table 1

Patients demographics, clinical, and laboratory findings at admission.

Variable	Total (n = 486)	High prevalence period (n = 247)	Moderate prevalence period (n = 239)	P value
Disease prevalence	341 (70 %, 66-74 %)	233 (94 %, 91–97%)	108 (45 %, 39–52 %)	<0.001
Age	70 (68–72; 57–80)	68 (65–70; 57–76)	73 (69–76; 57–82)	0.003
Gender
• Males	298 (61 %, 57–65 %)	173 (70 %, 64-75 %)	125 (52 %, 46–59 %)	<0.001
• females	188 (39 %, 34–43 %)	74 (30 %, 25-36%)	114 (48 %, 41–54 %)
Smoking history
• never	110 (23 %, 19-27 %)	61 (25 %, 20–30%)	49 (20 %, 16-26 %)	0.05
• former	52 (11 %, 8–14 %)	34 (14 %, 10–19%)	18 (8 %, 5-12%)
• current	23 (4 %, 3–7%)	12 (5 %, 3–8 %)	11 (5 %, 3–8 %)
• unknown	301 (62 %, 58–66 %)	140 (56 %, 50–63 %)	161 (67 %, 61-73 %)
Exposure to subject with known COVID-19 infection	134 (28 %, 24–32 %)	90 (36 %, 31–43 %)	44 (18 %, 14-24 %)	<0.001
Comorbidity
• cardiovascular	281 (58 %, 54–62 %)	137 (55 %, 49–62 %)	144 (60 %, 54-66 %)	0.33
• pulmonary	67 (14 %, 11–17 %)	37 (15 %, 11–20 %)	30 (13 %, 9-17 %)	0.51
• oncological	73 (15 %, 12–18 %)	33 (13 %, 10–18 %)	40 (17 %, 13-22 %)	0.36
• neurological	68 (14 %, 11–17 %)	35 (14 %, 10–19 %)	33 (14 %, 10-19 %)	0.99
• hepatic failure	8 (2 %, 1–3 %)	4 (1 %, 0.6–4 %)	4 (2 %, 0.6-4 %)	0.59
• chronic kidney failure	24 (5 %, 3–7 %)	11 (4 %, 3–8 %)	13 (5 %, 3-9 %)	0.52
• diabetes	85 (17 %, 14–21 %)	46 (19 %, 14–24 %)	39 (16 %, 12-22 %)	0.48
Symptom
• fever	388 (79 %, 76-83 %)	235 (95 %, 92-97 %)	153 (64 %, 58-70 %)	<0.001
• cough	228 (47 %, 43-51 %)	156 (63 %, 57-69 %)	72 (30 %, 25-36 %)	<0.001
• dyspnea	230 (47 %, 43-52 %)	115 (47 %, 40-53 %)	115 (48 %, 42-54 %)	0.94
• asthenia	55 (11 %, 9-14 %)	31 (13 %, 9-17 %)	24 (10 %, 7-14 %)	0.68
• other	162 (33 %, 29-38 %)	58 (23 %, 19-29 %)	104 (44 %, 37-50 %)	<0.001
Symptoms onset (days)	7 (6-7; 3-10)	7 (5-8; 3-8)	7 (6-7; 2-10)	0.27
Respiratory rate (acts/minute)	20 (20-20; 18-24)	20 (18-20; 18-24)	20 (20-20; 18-24)	0.54
SpO2 (%)	95 (94-95; 90-97)	92 (91-92; 87-95)	97 (96-97; 94-98)	<0.001
Body temperature at admission (°C)	37 (36.9-37.2; 36-38)	37.7 (37.5-38; 37-38.4)	36.4 (36.2-36.5; 36-37.2)	<0.001
Body temperature > 37.5 °C	172 (35 %, 31-40 %)	133 (54 %, 48-60 %)	39 (16 %, 12-22 %)	<0.001
Red blood cell count (x 106/μL)	4.7 (4.6-4.8; 4.3-5.1)	4.8 (4.6-4.9; 4.3-5.2)	4.7 (4.6-4.7; 4.2-5)	0.11
Hemoglobin level (g/dl)	13.6 (13.5-13.9; 12.3-14.8)	13.8 (13.5-14.2; 12.6-14.9)	13.4 (13.3-13.7; 12.1-14.7)	0.02
Hematocrit (%)	42.5 (41.9-43; 38.6-45.7)	42.7 (41.9-43.5; 38.8-45.8)	42 (41.5-43; 38.3-45.3)	0.19
White blood cell count (x 103/μL)	7.4 (7.1-7.9; 5.4-10.4)	6.3 (5.9-7; 4.7-9.3)	8.6 (8-9.3; 6.2-11.5)	<0.001
Lymphocytes count (x 103/μL)	1 (1-1.1; 0.7-1.4)	1 (0.9–1; 0.7–1.3)	1.1 (1-1.2; 0.7–1.6)	0.005
Platelet count (x 103/μL)	202 (196–216; 160–267)	183 (176–192; 145-233)	235 (224-253; 179–313)	<0.001
CRP (mg/dl)	6.1 (5.4–7.1; 1.6–13.6)	8.2 (6.6-9.2; 3-14.8)	3.8 (2.6–5.9; 0.7–11.1)	<0.001
Lactate dehydrogenase (U/L)	324 (308–341; 242–444)	355 (332-385; 273–480)	286 (257–311; 223-407)	<0.001
eGFR (ml/min/1.73 m2)	76 (74–79; 56-93)	76 (71–79; 57–92)	78 (73–85; 54-96)	0.37
Blood Urea level (mg/dl)	40 (38–44; 31-61)	40 (38–45; 31-60)	41 (38–46; 30-63)	0.89
Blood Sodium level (mEq/l)	138 (137–138; 135–140)	137 (136–137; 134-138)	139 (138–139; 136–141)	<0.001
Blood Glucose level (mg/dl)	122 (119–125; 106–149)	121 (117-125; 107-149)	123 (119-130; 105–149)	0.94
GOT (U/L)	37 (36–39; 26–60)	42 (38–46; 31–64)	31 (28-36; 22–50)	<0.001
GPT (U/L)	27 (25–30; 18–45)	30 (27–34; 20–49)	24 (21–27; 16–42)	0.002

Categorical variables are expressed as counts and percentage, with corresponding 95 % confidence interval (95 %CI) using Wilson method in parenthesis. Continuous variables are showed as median with 95 %CI for the median and interquartile range in parenthesis. Significant P values (<0.05) obtained are showed in bold type.

Abbreviations: COVID-19, coronavirus disease 2019; CRP, C-reactive protein; eGFR, estimated glomerular filtrate rate; GOT, glutamic oxaloacetic transaminase; GPT, glutamic pyruvic transaminase; SpO2, peripheral oxygen saturation.

LUS and chest CT findings

Lung ultrasound – In HP, LUS identified pathological B-lines in 179/247 (72 %; 95 %CI 67–78 %) patients, consolidation and pathological B-lines in 44/247 (18 %; 95 %CI 13–23 %) cases, while white lung was observed in 9/247 (4%; 95 %CI 2–7 %) patients; consolidations without pathological B-lines was found in 8/247 (3%; 95 %CI 2–6 %) patients. No significant differences in the rate of LUS findings was demonstrated in MP: pathological B-lines were identified in 150/239 (63 %; 95 %CI 56–68 %; P=0.22) patients, consolidations and pathological B-lines in 32/239 (14 %; 95 %CI 10−18%; P=0.19) cases, white lung in 9/239 (4%; 95 %CI 2–7 %; P=0.86) observations, while consolidations without pathological B-lines in 4/239 (2%; 95 %CI 1–4 %; P=0.41) LUS. Similar rate of pleural effusion was demonstrated between HP (33/247, 13 %; 95 %CI 10−18%) and MP (32/239, 13 %; 95 %CI 10−18%; P=0.9). LUS was considered positive for COVID-19 pneumonia more frequently in HP group (232/247, 93 %, 95 %CI 90–96 %) compared with MP group (191/239, 80 %, 95 %CI 74–84 %; P < 0.001). In COVID-19 patients, the median LUS score was similar in the two groups (median 4, 95 %CI 3–4 for both groups; P = 0.53). A similar rate of pneumonia other than COVID-19 (HP 3 % vs MP 2 %, P = 0.38) was observed in both groups. No difference in the rate of pulmonary edema was found between HP and MP (respectively 2 % vs 3 %, P = 0.37). The number of reporting emergency physician was higher in the MP as compared to the HP (45 vs 38 physicians); in addition the median years of experience was lower in the MP as compared to HP (11 years vs 15 years, P = 0.001).

Chest CT - Time elapsed between LUS and CT was not significantly different in the two periods (HP 1 h and 30 min vs MP 1 h and 35 min; P = 0.53). Frequency of CT categories for COVID-19 pneumonia are detailed in e-Table 2. CT positive (“typical” or “indeterminate”) for COVID-19 were more frequent in the HP group as compared to MP group (217/247, 88 %, 95 %CI 83−91 %; 143/239, 60 %, 95 %CI 53–66 %; P < 0.001). In COVID-19 patients the median extent of CT pulmonary involvement was similar in the two groups (HP 30 %, 95 %CI, 25–30 %; MP 30 %, 95 %CI, 25–35 %; P=0.63). A significant difference in CT alternative diagnosis was demonstrated between HP and MP (0.4 % vs 9%, P < 0.001). During HP only 1/247 (0.4 %; 95 % CI 0.1−2 %) patient manifested pneumonia other than COVID-19, while in the MP were identified 6/239 (3 %; 95 % CI 1–5 %) patients with pneumonia other than COVID-19, 4/239 (2%; 95 % CI 1–4 %) CT scans with cancer, 8/239 (3%; 95 % CI 2–6 %) cases with pulmonary edema, and 3/239 (1%; 95 % CI 0.4–4 %) patients with interstitial lung disease.

The agreement between chest CT and LUS (Table 2 ) for the detection and localization of parenchyma abnormalities considering all patients was only fair (K 0.29, 95 % CI 0.24−0.34); this finding was confirmed both in HP and MP groups (HP K 0.23, 95 %CI 0.15−0.3; MP K 0.27, 95 % CI 0.2−0.34). For both CT and LUS, the main distribution pattern was diffuse (CT 60 %, 95 %CI 57−63%; LUS 52 %, 95 %CI 49−55%), followed by middle-lower zone predominance (CT 17 %, 95 %CI 15−20%; LUS 28 %, 95 %CI 25−31%). The agreement between CT and LUS for the diagnosis of COVID-19 was poor in both periods (HP K 0.05, 95 %CI 0.08−0.19; MP K 0.16, 95 % CI 0.04−0.28). In particular, LUS considered positive patients with CT negative for COVID-19 in 27/30 (90 %; 95 %CI 74–96 %) cases during HP and in 68/96 (71 %; 95 %CI 61–79 %) cases of MP.

Table 2

Agreement between chest computed tomography and lung ultrasound for the diagnosis of COVID-19 interstitial pneumonia in 486 patients.

Lung ultrasound	Chest computed tomography
	No alterations	Upper-middle zone	Middle-lower zone	Diffuse	Total
No alterations	74	15	27	57	173 (18 %, 16−20 %)
Upper-middle zone	3	1	4	16	24 (2 %, 2−4 %)
Middle-lower zone	56	14	58	141	269 (28 %, 25−31 %)
Diffuse	35	23	77	371	506 (52 %, 49−55 %)
Total	168 (17 %, 15−20 %)	53 (6%, 4−7%)	166 (17 %, 15−20 %)	585 (60 %, 57−63 %)

Values in parenthesis correspond to 95 % confidence interval (95 %CI). The agreement between computed tomography and lung ultrasound is fair, with a weighted K Cohen value of 0.29 (95 % CI, 0.24−0.34).

Abbreviations: COVID-19, coronavirus disease 2019.

Table 3 shows the performance of both LUS and chest CT for diagnosis of COVID-19. In HP the sensitivity, specificity, PPV, and NPV were 94 % (95 %CI, 90−97%), 7% (95 %CI, 0.2−34%), 94 % (95 %CI, 91−97%), and 7% (95 %CI, 0.2−32%) for LUS while 90 % (95 %CI, 85−93%), 43 % (95 %CI, 18−71%), 96 % (95 %CI, 93−98%), and 20 % (95 %CI, 8−38%) for CT. In MP the sensitivity, specificity, PPV, and NPV were 93 % (95 %CI, 86−97%), 31 % (95 %CI, 23−39%), 52 % (95 %CI, 45−60%), and 83 % (95 %CI, 70−92%) for LUS while 95 % (95 %CI, 89−98%), 69 % (95 %CI, 61−77%), 72 % (95 %CI, 64−79%), and 95 % (95 %CI, 88−98%) for CT. CT performance for diagnosis of COVID-19 was significantly better than LUS both in HP (AUC 0.75 vs 0.51; P < 0.001) and MP (AUC 0.85 vs 0.62; P < 0.001) (Fig. 4 , Table 3). The performance for the diagnosis of COVID-19 of CT was similar in the two periods (P = 0.13), while was significantly better for LUS in the MP as compared to HP (P = 0.01).

Table 3

Chest computed tomography and lung ultrasound performance for diagnosis of COVID-19 infection.

Metrics	High prevalence group (prevalence 94 %)		Moderate prevalence group (prevalence 45 %)
	CT	LUS	CT	LUS
Sensitivity (%)	90 (85−93)	94 (90−97)	95 (89−98)	93 (86−97)
Specificity (%)	43 (18−71)	7 (0.2−34)	69 (61−77)	31 (23−39)
PPV (%)	96 (93−98)	94 (91−97)	72 (64−79)	52 (45−60)
NPV (%)	20 (8−38)	7 (0.2−32)	95 (88−98)	83 (70−92)
AUC P value*	0.75 (0.61−0.88)	0.51 (0.43−0.58)	0.85 (0.81−0.9)	0.62 (0.57−0.66)
	<0.001		<0.001

Abbreviations: AUC, are under the ROC curve; COVID-19, coronavirus disease 2019; CT, computed tomography; LUS, lung ultrasound; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operating characteristic curve.

Sensitivity, specificity, positive predictive values, negative predictive values, and area under the ROC curve are expressed as percentage and values, with corresponding 95 % confidence interval (95 %CI) in parenthesis. Significant P values (<0.05) obtained are showed in bold type.

ROC curves were compared by the methodology of DeLong et al. [27].

Fig. 4

Comparison of ROC curves for diagnosis of COVID-19 in both high prevalence and moderate prevalence groups. Chest CT showed a significant (P < 0.001) better performance in comparison to lung ultrasound in both periods (AUC in the HP 0.75 vs 0.51; AUC in the MP 0.85 vs 0.62).

Abbreviations: AUC, area under the ROC curve; COVID-19, coronavirus disease 2019; CT, computed tomography; HP, high prevalence group; MP, moderate prevalence group; ROC, receiver operating characteristic.

Considering as reference standard RT-PCR, serology, and “typical”/”indeterminate” CT pattern, the LUS performance showed similar results both in the HP group (sensitivity 94 %, specificity 8%, PPV 85 %, NPV 20 %, AUC 0.51) and MP group (sensitivity 93 %, specificity 30 %, PPV 50 %, NPV 85 %, AUC 0.62), as showed in e-Table 3.

Clinical outcome analysis

In both groups, the rate of COVID-19 patients that reached the clinical outcome was similar (HP 106/233, 45 %, 95 % CI 39−52%; MP 39/108, 36 %, 95 % CI 28–45 %; P = 0.14). E-table 4 shows the best cutoff obtained by the ROC curves analysis in relation to clinical outcome: LUS score>2 and visual CT pneumonia extent>30 % in HP; LUS score>3 and visual CT pneumonia extent>35 % in MP. Significant shorter mean time of ICU admission or death (Fig. 5 ) was demonstrated for patients with visual CT pneumonia extent>30 % (35 vs 91 days, P < 0.001) in HP group, and >35 % (36 vs 74 days, P < 0.001) in the MP group. LUS score>2 in the HP group was associated with shorter time of ICU admission or death (56 vs 85 days, P < 0.001), while a LUS score>3 in the MP group failed to identify patients with better prognosis (55 vs 68 days, P = 0.07). The multivariable Cox analysis (Table 4 ) confirmed the association between worse clinical outcome and both LUS score>2 (HR 2.2, 95 %CI 1.4−3.3; P<0.001), and visual CT pneumonia extent>30 % (HR 4, 95 %CI 2.7−6; P<0.001) in the HP group; in the MP group visual CT pneumonia extent >35 % (HR 3.7, 95 % CI 1.9−7.3; P<0.001) showed a significant association with worse clinical outcome while LUS score>3 was not significantly related with worse prognosis.

Fig. 5

Kaplan-Meier estimates (with 95 % confidence interval) of ICU/death admission for COVID-19 pneumonia extent assessed by LUS and CT. In HP a LUS score >2 (a) and a visual CT score >30 % (b) were significantly associated with shorter ICU admission or death occurrence (both P < 0.001). In MP, a CT visual score>35 % was significantly associated with shorter ICU admission or death occurrence (P < 0.001), while a LUS score >3 (d) failed to identify patients with better prognosis (P = 0.07).

Abbreviations: COVID-19, coronavirus disease 2019; CT, computed tomography; HP, high prevalence group; ICU, intensive care unit; LUS, lung ultrasound; MP, moderate prevalence group.

Table 4

Multivariable Cox proportional hazard results for age, gender, comorbidities, pneumonia extent assessed by CT and LUS to predict ICU admission/death (“clinical outcome”) in COVID-19 patients.

Variables	High prevalence group (94 %)
	Hazard ratio (95 % CI)	P value
Age >70 years old	1.8 (1.2−2.7)	0.003
Oncological comorbidities	1.7 (1.1−2.8)	0.03
CT pneumonia extent >30 %	4 (2.7−6)	<0.001
LUS pneumonia extent score >2	2.2 (1.4−3.3)	<0.001

Values in parenthesis correspond to 95 % confidence interval (95 %CI). Significant P values (<0.05) obtained are showed in bold type.

Abbreviations: COVID-19, coronavirus disease 2019; CT, computed tomography; ICU, intensive care unit; LUS, lung ultrasound.

Discussion

In the present study we aimed to compare chest CT and LUS performance in two periods of COVID-19 epidemic with disease prevalence varying from 94 % in early phase to 45 % in late phase. In both periods, chest CT was significantly more accurate than LUS for the diagnosis of COVID-19. Furthermore, chest CT granted better risk stratification of clinical outcome compared with LUS in both early and late phase.

LUS has been proposed and implemented in the diagnostic work-up of COVID-19 [2,4,5]. This is a rapid, repeatable, bedside test that can help risk stratification and management of patients with respiratory symptoms [28]. LUS was reported with a characteristics pattern in COVID-19, with thickened pleural line and a large number of B-lines, representing interstitial pneumonia, distributed mainly in the lower and dorsal zone of both lungs [2,17]. LUS is an highly sensitive technique, for instance the sensitivity of LUS for the diagnosis of pulmonary edema reached 88 % [19]. Even in COVID-19 LUS was demonstrated in preliminary study very sensitive [29]. We found high sensitivity for COVID-19 diagnosis, ranging from 93 % to 94 %, similar to chest CT. However, pattern for COVID-19 at LUS are characteristic but non-specific [17]. The presence of artifactual B-lines is reported in several other pathologies, such as pulmonary edema, interstitial fibrosis, and asthma [19,30,31]. In addition B-lines can occur even in healthy individuals and should be interpreted in relation with the age of the observed patient [32,33]. Thus, specificity was low in both populations (7% and 31 % in HP and MP, respectively). PPV was very high (94 %) in a disease prevalence population of around 90 %, but the reduction of disease prevalence (in our sample around 50 %) determines a decrease to 52 %. NPV is very low (7%) in the HP but increases to 83 % in MP. In addition the use of LUS is limited in several conditions, particularly in obese patients. Moreover, LUS is highly dependent on operator experience, without clear evidence-based guidelines about the training required to achieve adequate skills [28,34]. Our results support this statement, demonstrating that despite significant lower experience of the readers in MP (11 years vs 15 years), LUS performance in diagnosis of COVID-19 was significantly better in MP as compared with HP, proving the relevance of specific training to assess pulmonary abnormalities related to COVID-19.

Furthermore, to reduce the risk of virus transmission to other patients and to operators, rigorous and standardized procedures are required to protect operators and for decontamination during and at the end of examination [8].

We showed fair agreement in abnormalities localization between CT and LUS. Lu et al. demonstrated higher agreement (K Cohen of 0.529) between CT and LUS, however considering only patients positive for COVID-19 [35]. The fair agreement could be explained by the difficulty of LUS to detect interstitial pneumonia distant from pleura or in the apical regions [2].

According with previous studies, in both cohorts chest CT sensitivity was high, included between 90 % and 95 % depending on disease prevalence [9,11,36]. The specificity (HP group 43 %, MP group 69 %) was similar to the reported by Caruso et al. (56 %), however higher than showed by Ai et al. (25 %) [9,36]. One reason for the lower specificity of this study is that radiologists used low threshold for diagnosing COVID-19 at CT, considering positive patients with patterns seen in COVID-19 but less typical; conversely, we considered positive patients with typical or indeterminate chest CT pattern, as defined by the RSNA classification system [9,20,36]. As expected by the huge difference in disease prevalence, PPV and NPV of CT were different in the two groups [6]. The PPV in the HP cohort was very high (96 %), while it declined (72 %) after three weeks lockdown. The opposite was observed for NPV, which raised from 20 % in the initial outbreak to 95 % in the late outbreak.

As previously reported, CT stratified COVID-19 patients with worse outcome in both periods explored in the present study, particularly a pneumonia extent >30 % of whole lung volume was a predictor of ICU admission or death [10]. LUS quantification of pneumonia extent showed prognostic value in the HP group, but failed to identify patients with better prognosis in the MP group. This finding reflects the LUS peculiarity to assess only peripheral lung zones, which represent around only 1/16 of the total lung volume [37]. Bonadia et al. have recently demonstrated an higher proportion of involved area by COVID-19 pneumonia assessed by LUS in patients who subsequently died [38]. However, this report was conducted on only 36 patients without considering other potential predictors of death (e.g. age, sex, and comorbidities) additional to LUS evaluation of COVID-19 pneumonia extent [38].

The present study has several limitations. First, it is based on a retrospective design, in a single hospital. Second, the interobserver agreement of LUS was not evaluated; however our study reflects a daily practice scenario, and it is well known that ultrasound depends on observer skills and experience. Third, in several patients more than two RT-PCR tests for SARS-Cov-2 in nasal-pharyngeal swab are needed to confirm the diagnosis of COVID-19; thus the definition of a patient “negative” on the basis of only two negative nasal-pharyngeal swabs could affect imaging performance, by increasing the number of false positive CT or LUS which actullay were true positive[39]. However, the rate of patients that require three or more RT-PCR tests in nasal-pharyngeal swab is quite low, around 5% of the patients [39].

In conclusion, admission chest computed tomography shows a better performance than LUS for diagnosis of COVID-19, both in population with very high (94 %) and lower (45 %) disease prevalence. Lung ultrasound is an highly sensitive technique, however it requires specific training and is limited by extremely low specificity and unreliable positive predictive value with disease prevalence below 50 %. Therefore, LUS for COVID-19 diagnosis should not be considered as a screening tool and represent an expanded clinical evaluation to rule out major pulmonary involvement; yet it seems safe to integrate positive LUS with CT for more accurate risk stratification.

Funding sources

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Transparency document

CRediT authorship contribution statement

Davide Colombi: Conceptualization, Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Supervision. Marcello Petrini: Conceptualization, Methodology, Formal analysis. Gabriele Maffi: Conceptualization, Methodology, Formal analysis. Gabriele D. Villani: Conceptualization, Methodology, Formal analysis. Flavio C. Bodini: Conceptualization, Methodology, Formal analysis. Nicola Morelli: Conceptualization, Methodology, Formal analysis. Gianluca Milanese: Conceptualization, Methodology, Formal analysis, Writing - review & editing. Mario Silva: Conceptualization, Methodology, Formal analysis, Writing - review & editing. Nicola Sverzellati: Conceptualization, Methodology, Formal analysis, Writing - review & editing. Emanuele Michieletti: Conceptualization, Methodology, Formal analysis, Writing - review & editing.