Validation of a composed COVID-19 chest radiography score: the CARE project.

OBJECTIVES: The aim of this study was to validate a composed coronavirus disease 2019 (COVID-19) chest radiography score (CARE) based on the extension of ground-glass opacity (GG) and consolidations (Co), separately assessed, and to investigate its prognostic performance.
METHODS: COVID-19-positive patients referring to our tertiary centre during the first month of the outbreak in our area and with a known outcome were retrospectively evaluated. Each lung was subdivided into three areas and a three-grade score assessing the extension of GG and Co was used. The CARE was derived from the sum of the subscores. A mixed-model ANOVA with post hoc Bonferroni correction was used to evaluate whether differences related to the referring unit (emergency room, COVID-19 wards and intensive care unit (ICU)) occurred. Logistic regression analyses were used to investigate the impact of CARE, patients' age and sex on the outcome. To evaluate the prognostic performance of CARE, receiver operating characteristic curves were computed for the entire stay and at admission only.
RESULTS: A total of 1203 chest radiographs of 175 patients (120 males; mean age 67.81±15.5 years old) were examined. On average, each patient underwent 6.8±10.3 radiographs. Patients in ICU as well as deceased patients showed higher CARE scores (p<0.05, each). Age, Co and CARE significantly influenced the outcome (p<0.05 each). The CARE demonstrated good accuracy (area under the curve (AUC)=0.736) using longitudinal data as well as at admission only (AUC=0.740). A CARE score of 17.5 during hospitalisation showed 75% sensitivity and 69.9% specificity.
CONCLUSIONS: The CARE was demonstrated to be a reliable tool to assess the severity of pulmonary involvement at chest radiography with a good prognostic performance.

Introduction

The significant role of diagnostic imaging became clear since the beginning of the current coronavirus disease 2019 (COVID-19) pandemic [1]. In fact, it has been even suggested that computed tomography (CT) findings might be more sensitive than reverse transcription PCR (RT–PCR) in detecting the infection [2, 3]. Surely, during the course of the outbreak it has been demonstrated that features at imaging can contribute to the assessment of patients affected by this chameleonic disease at diagnosis and during follow-up [4, 5]. A great body of literature, not only describing the most common features (e.g. ground-glass (GG) and consolidations (Co)) at CT but also proposing scores of disease severity which correlate the findings at imaging to the outcome, has been published [6, 7]. Nevertheless, as already shown for previous infectious disease outbreaks like Severe Acute Respiratory Syndrome (SARS), Middle East Acute Respiratory Syndrome (MERS), and H1N1, also for COVID-19 it has become progressively evident that chest radiography can be applied as a main tool to diagnose and monitor such patients [8-10]. In fact, radiographs guarantee an adequate evaluation of the infection simultaneously optimising organisational, safety and radiation exposure issues. Regarding COVID-19, for instance, Fichera et al. [4] showed the necessity of a long-term monitoring of COVID-19 patients in intensive care unit (ICU) despite the prompt amelioration that usually occurs after intubation. Moreover, to further contribute to the management of COVID-19 patients, chest radiography-based scores, partially recalling scores proposed for previous Coronavirus epidemics such as SARS and MERS, have been proposed [8, 11–16]. In particular, Borghesi et al. [15] successfully validated a score demonstrating that men over 50 and women over 80 years are at high risk for severe COVID-19 lung disease. Toussie et al. [14] using a similar score and considering clinical information such as comorbidities and body mass index showed a good prognostic performance. Furthermore, Wong et al. [13] demonstrated the good diagnostic value of a score addressing GG and Co at chest radiography although they did not investigate their contribution separately or the prognostic value of the score.

Thus, the aim of our study to validate a composed COVID-19 chest radiography score (CARE), based on the extension of GG and Co, considering both subscores separately, and evaluating its role in predicting patients’ outcome analysing data collected during the entire hospitalisation of each patient. Moreover, we assessed the prognostic role of the first score at hospital admission taking into account clinical and laboratory findings.

Materials and methods

Study design

Institution Review Board approval was obtained for this retrospective study. An electronic search of the database of our hospital was performed to identify adult COVID-19-positive patients (i.e. with at least one RT–PCR positive test) admitted to our tertiary centre during the first month of the pandemic in our area (i.e. February 21, 2020), who underwent at least one chest radiography and with a known outcome (i.e. deceased, recovered). For each patient, all chest radiography performed since the access to the emergency room (ER) and during the entire hospitalisation were collected and analysed. Moreover, demographic information, clinical symptoms (i.e. fever >37.5°, cough, dyspnoea, gastrointestinal symptoms separately and any other symptom, such as neurological and cutaneous, grouped together) and laboratory findings (i.e. red and white blood cell count, haemoglobin, lymphocyte count, C-reactive protein (CRP) and procalcitonin) at hospital admission were recorded. The referring unit (i.e. for each examined radiography) was documented.

One radiologist with 11 years of experience in thoracic imaging performed the assessment of all chest radiographs blind to patients’ information including outcome and referring unit. To assess the repeatability of the method, half of the population was also independently analysed by a radiologist with 5 years of experience in chest imaging.

CARE score

At chest radiography, each lung was subdivided in three areas (i.e. upper area, from the apices to the superior margin of the hilum; middle area, from the upper to the lower margin of the hilum; and lower area, from the lower margin of the hilum to the costophrenic angle) and a four-grade score describing, separately, the extension of GG (i.e. hazy opacity not obliterating bronchi and vessels) and Co (i.e. area of attenuation obscuring airways and vessels) was used [17] (figure 1). In particular, in each area, the GG was graded from 0 to 3 (i.e. 0=normal parenchyma; 1=less than one-third of each area was affected; 2=more than one-third but less than two-thirds of each area were affected; and 3=more than two-thirds of each area were affected). Given the clinical impact of pulmonary Co [18, 19], these were scored from 4 to 6 using the same grade of extension applied for the GG (i.e. 0=normal parenchyma; 4=less than one-third of each area was affected; 5=more than one-third but less than two-thirds of each area were affected; 6=more than two-thirds of each area were affected). Then, the global score (i.e. CARE), deriving from the sum of the GG and Co subscores, was computed (figure 1). The maximum score per patient that could be reached was 36.

FIGURE 1

Graphic representation of the COVID-19 chest radiography score (CARE) separately addressing ground-glass and consolidations.

The occurrence of additional findings such as pleural effusion, pneumothorax, pneumomediastinum and s.c. emphysema was also recorded.

Statistical analyses

The mixed-model ANOVA with post hoc Bonferroni correction was used to evaluate if differences related to the referring unit (i.e. ER, COVID-19 wards and ICU) occurred in the maximum CARE score as well as in the maximum GG and Co subscores. Logistic regression analyses for repeated measurements, using the general estimation equations framework, were applied to evaluate the impact of the CARE (including a separate assessment of GG and Co subscores) and patients’ age, sex on the outcome.

A t-test was applied to compare the highest CARE score as well as the highest GG and Co subscores reached during the hospitalisation in COVID-19 wards by patients who were treated only in these wards and by patients who had to be transferred to ICU.

The repeatability of the CARE and of the subscores was calculated by the intra-class correlation coefficient (ICC) applying the two-way mixed model with consistency and average measures. Values >0.750 were considered excellent [20].

Moreover, using data collected at admission, the relationship between the CARE, and its subscores and clinical and laboratory findings was assessed using the Spearman correlation coefficient.

To evaluate if any difference occurred between alive and deceased regarding the CARE, the subscores, clinical and laboratory findings, at admission, the Mann–Whitney U-test was applied.

To evaluate the prognostic performance of the CARE and the subscores, receiver operating characteristic (ROC) curves were computed for longitudinal data, using the highest score, as well as for information at admission only.

All statistical analyses were performed using SPSS (IBM SPSS Statistics version 26, IBM Armonk, NY, USA) and applying p<0.05 as the level of significance.

Results

Overall performance of the CARE

A total amount of 1203 chest radiographs of 175 patients (120 males, 55 females; mean age 67.81±15.5 years old) were examined. On average, each patient underwent 6.8±10.3 chest radiographs; in particular, patients in ICU at least for 1 day underwent 15.64±14.3 (median 11, minimum 2, and maximum 61) examinations, while patients treated only in COVID-19 wards had on average 2.5±1.8 (median 2, minimum 1 and maximum 9) chest radiographs. Thirty-six patients were hospitalised in ICU, whereas 95 were treated only in COVID-19 wards and 44 were discharged after access to the ER.

The average length of hospitalisation was 10.5±14.1 days (range 0–80 days) and the average stay in ICU was 15.1±14.7 days.

Overall, 32 patients died (22 males and 10 females; mean age 81.4±10.1 years old).

The maximum CARE score, as well as the maximum GG and Co subscores were higher in ICU than at ER admission or during admission/stay in COVID-19 wards (p<0.05, each) (table 1). Pleural effusion occurred in 19 patients, 5 of them deceased. No other additional findings were detected.

TABLE 1

Differences in mean CARE and subscores according to the referring units

	ER		COVID-19 wards		ICU		Post hoc Bonferroni# p-value
							ER versus COVID-19 wards	ER versus ICU	COVID-19 wards versus ICU
	mean	95% CI	mean	95% CI	mean	95% CI
Ground-glass subscore	1.2	1–1.4	1.3	1.1–1.5	1.7	1.5–1.9	0.431	0.000	0.000
Consolidations subscore	1	0.7–1.3	1.4	1.2–1.7	1.8	1.5–2.1	0.005	0.000	0.009
CARE	2.2	1.8–2.7	2.8	2.4–3.1	3.5	3.2–3.9	0.022	0.000	0.000

CARE: COVID-19 chest radiography score; ER: emergency room; ICU: intensive care unit. #: Mixed-model ANOVA; applied level of significance p<0.05.

The logistic regression analysis showed that age and CARE significantly influenced the outcome (p<0.001, each), whereas sex did not play a significant role (p=0.602). Moreover, the Co subscore influenced the outcome (p=0.004) whereas the GG did not significantly contribute (p=0.239).

The CARE demonstrated good accuracy (area under the curve (AUC)=0.736). In particular, a CARE score of 17.5 showed 75% sensitivity and 69.9% specificity. The Co subscore showed moderate accuracy (AUC=0.685); a Co subscore of 4.50 demonstrated 71.9% sensitivity and 51.7% specificity. The GG subscore showed good accuracy (AUC=0.714); a GG subscore of 5.5 showed 75% sensitivity and 53.8% specificity (figure 2).

FIGURE 2

ROC curves demonstrating the diagnostic performance of the COVID-19 chest radiography score (CARE) and both subscores (i.e. ground-glass (GG) and consolidation (Co) subscores). The CARE showed higher accuracy, sensitivity and specificity than the subscores.

Patients who were in COVID-19 wards and then had to be moved to ICU showed significantly higher CARE scores than patients hospitalised in COVID-19 wards only (mean±sd CARE score 21.97±9.17 versus 9.91±9, respectively; p<0.001) (table 2).

TABLE 2

Maximum CARE, consolidation and ground-glass subscores recorded during the stay in COVID-19 wards of positive patients treated in COVID-19 wards only and patients transferred to ICUs

	Subscores				CARE
	Ground-glass		Consolidations
	mean±sd	p-value	mean±sd	p-value	mean±sd	p-value
COVID-19 wards only (n=95)	4.72±4.3	<0.001	5.8±6.6	<0.001	9.9±9	<0.001
Patients moved from COVID-19 wards to ICU (n=36)	10.6±4.5		14.7±8.9		22±9.1

CARE: COVID-19 chest radiography score; ICU: intensive care unit; t-test; applied level of significance, p<0.05.

The most severe CARE scores occurred in lower areas of survivors and deceased; in each investigated area, significantly higher CARE scores were found in patients who deceased (table 3; p<0.05, in each area). The highest Co subscores occurred in the lower areas and significantly higher scores occurred in all areas, except the upper, on both sides, of the deceased (figure 3). The most severe GG subscores occurred in the left lung of patients who deceased (1.8±1.2 in the upper left area, 1.9±1.1. in the middle left area, and 1.9±1.2 in the left lower area) and were significantly higher than in survivors (0.8±0.9 in the upper left area, 0.8±1.1. in the middle left area, and 1.4±1.1 in the left lower area; p<0.05 each) (figure 4).

TABLE 3

Differences in CARE scores, ground-glass and consolidations subscores, in each investigated pulmonary area, according to the outcome

	Subscores						CARE
	Ground-glass			Consolidation
	Survivors (n=143)	Deceased (n=32)	p-value	Survivors (n=143)	Deceased (n=32)	p-value	Survivors (n=143)	Deceased (n=32)	p-value
Upper right area	0.9±1	1.6±1.3	0.006	0.9±1.8	1.5±2.1	0.223	1.7±2.2	2.8±2.4	0.026
Middle right area	0.9±1.1	1.6±1.1	0.006	1.2±2.1	2.5±2.4	0.008	1.9±2.4	3.7±2.5	0.001
Lower right area	1.5±1	1.6±0.9	0.441	2.2±2.3	3.5±2.3	0.006	3.1±2.4	4.6±2.1	0.002
Upper left area	0.8±0.9	1.8±1.2	0.000	0.9±1.7	0.9±1.8	0.888	1.5±2	2.6±2.1	0.010
Middle left area	0.8±1.1	1.9±1.1	0.000	2.4±2.4	3.3±2.4	0.006	1.6±2.3	3.8±2.4	0.000
Lower left area	1.4±1	1.9±1.2	0.029	2.4±.24	3.8±2.2	0.003	3.3±2.4	5.1±2.7	0.001
All areas	5.7±4.8	9.9±5.7	0.000	7.7±8.2	13.2±9.3	0.003	12.1±10	21.3±10.3	0.000

CARE: COVID-19 chest radiography score; t-test, applied level of significance p<0.05.

FIGURE 3

Chest radiograph of a deceased 87-year-old patient affected by COVID-19 well demonstrating extensive consolidations in the lower and middle areas on both sides (COVID-19 chest radiography score (CARE) of 36; white dotted lines representing the examined areas).

FIGURE 4

Chest radiographs of (a) an 85-year-old male affected by COVID-19 discharged after 11 days of hospitalisation and (b) of an 81-year-old male who died after 20 days of hospitalisation showing the more extensive ground-glass on the left side in the latter (left side ground-glass subscore of 3 and 8 in (a) and (b), respectively).

Using 80 randomly selected patients out of the 175 (i.e. 619 chest radiographs), the CARE, including the two separate subscores, showed excellent repeatability (CARE, ICC=0.973, 95% CI (0.968–0.977); GG, ICC=0.892, 95% CI (0.875–0.907); Co, ICC=0.965, 95% CI (0.959–0.970)).

Performance of the CARE at hospital admission

Overall, 154 patients had fever (>37.5°C) (88%), 97 had cough (55.4%), 62 had dyspnoea (35.4%), 17 had gastrointestinal symptoms (10%) and 32 were referred with other symptoms (18.2%) (e.g. neurological symptoms). The mean duration (±sd) of symptoms before hospital admission was of 5.1±4.4 days (range 0–21 days).

A statistically significant difference between survivors and deceased occurred for red blood cell count (4.6±0.6 versus 4.2±0.7×1012·L−1 for survivors and deceased, respectively; p=0.026), CRP (72±73 versus 135±82 mg·L−1 for survivors and deceased, respectively; p=0.001), procalcitonin (0.2±0.4 versus 0.8±1 ng·mL−1 for survivors and deceased, respectively; p=0.001) and the length of symptoms before hospital admission (5.5±4.4 versus 2.7±3.1 days for survivors and deceased, respectively; p=0.001). No other statistically significant differences occurred for the other examined variables including clinical symptoms (p>0.05, each).

The CARE showed a moderate statistically significant correlation with CRP (r=0.503, p=0.000) and procalcitonin (r=0.514, p=0.00). A low positive correlation occurred between the CARE and white blood cell count (r=0.238, p=0.002), whereas a negative correlation emerged with lymphocytes (r=−0.378, p=0.00). Regarding symptoms, GG and the CARE showed a low correlation only with dyspnoea (r=0.225 and r=0.191, respectively; p<0.05, each) (figure 5).

FIGURE 5

Scatter plots (a–f) and box plots (g–i) demonstrating the relationship between the COVID-19 chest radiography score (CARE) and its subscores and laboratory and clinical findings at hospital admission. Co: consolidation; GG: ground-glass.

The CARE at admission showed a good prognostic performance (AUC=0.740). In particular, a CARE score of 3.5 demonstrated 85% sensitivity and 53% specificity. The GG and Co subscores demonstrated moderate accuracy (AUC=0.695 and AUC=0.669, respectively). A GG subscore of 2.5 showed 74% sensitivity and 50% specificity while a Co a score of 2 showed 67% sensitivity and 37% specificity.

Discussion

The CARE demonstrated to be a good predictor of COVID-19 patients’ outcome. Our results are in line with those of previous studies evaluating the performance of other chest radiography-based scores [14-16]. Nevertheless, it should be highlighted that previous scores did not separately assess the contribution of GG and Co. Our results demonstrated that investigating both features new insights into the disease can be provided. In fact, although both subscores showed moderate and good accuracy, according to the logistic regression analyses, the GG score seemed not to influence the outcome. In addition, overall, the CARE showed the best prognostic performance in terms of accuracy, sensitivity and specificity also at admission. This evidence suggests the importance of considering both subscores to provide an accurate evaluation of COVID-19 infection severity and confirms that particular attention should be given to Co.

Wong et al., [13] similarly to us, investigated GG and Co but their study included a smaller population (i.e. 64 patients), did not evaluate the role of such features independently, and did not explore the prognostic value of their score. Our results are also in line with those of Toussie et al. [14] at admission although their score was not composed of separately assessed subscores.

Regarding the demographic variables, while Borghesi et al. [15] identified higher scores in men in the age range of 50–79 years, in our population, age but not sex played a significant role influencing the outcome. Moreover, analysing longitudinal data, we have been able to investigate the behaviour of the disease in patients initially hospitalised in COVID-19 wards because of mild clinical conditions, who then had to be transferred to ICU. This assessment has demonstrated that patients who then had to be moved to the ICU had higher CARE scores than patients who did not require ICU support during the hospitalisation in COVID-19 wards. Therefore, the application of the CARE in clinical practice may provide crucial information to clinicians treating patients in COVID-19 wards even suggesting the optimal timing for patients’ transfer to ICU. Prospective studies on a larger sample size are necessary to evaluate whether the CARE may even improve patient management, allowing an early transfer to ICU based on the score itself.

In terms of anatomical distributions, as in most of the literature, it emerged that lower areas are mostly affected. Nevertheless, Toussie et al. [14] did not find any correlation between opacifications in the left lower lung zone and intubation. The authors attributed this difference to the difficulties related to a suboptimal evaluation of lower zones in radiography acquired by portable devices. To clarify such differences, future studies may include a quality assessment of the analysed radiographs.

Considering data at admission, the negative correlation with lymphocytes confirmed the already well-known pattern of lymphocytopenia occurring in COVID-19 patients and suggests its significant impact on the pulmonary manifestation of the disease [21, 22]. Similarly, the correlation with indexes such as CRP and procalcitonin highlights the importance of the inflammatory cascade, which is an aspect of main interest for the therapeutic management of such patients [23].

This study is affected by several limits. Firstly, we did not correlate the CARE with autoptic findings of deceased patients as it would have exceeded the aim of the study. Moreover, it cannot be overlooked, that especially during the first phase of the pandemic, the possibility to perform autopsies on COVID-19 was highly debated [24, 25]. Certainly, further research in this direction is necessary to assess the role of the score in a more holistic approach, aiming also to improve our knowledge regarding the pathological process caused by the virus.

Moreover, we did not perform any correlation with CT findings but most of the patients treated in our tertiary centre were mainly assessed by chest radiography, which guaranteed, as mentioned above, a constant monitoring preserving healthcare providers’ safety and optimising the organisation of the diagnostic procedures. Further studies testing the same score or adapted versions suitable for CT examinations are encouraged.

In conclusion, the CARE demonstrated to be a reliable tool to assess the severity of pulmonary involvement at chest radiography with a good prognostic role at admission and during the hospitalisation. Its application in larger prospective studies is expected to provide further insights into its contribution in the clinical management of COVID-19 patients.