A Novel Prediction Model of COVID-19 Progression: A Retrospective Cohort Study.

INTRODUCTION: Estimating the risk of disease progression is of utmost importance for planning appropriate setting of care and treatment for patients with coronavirus disease 2019 (COVID-19). This study aimed to develop and validate a novel prediction model of COVID-19 progression.
METHODS: In total, 814 patients in the training set were included to develop a novel scoring system; and 420 patients in the validation set were included to validate the model.
RESULTS: A prediction score, called ACCCDL, was developed on the basis of six risk factors associated with COVID-19 progression: age, comorbidity, CD4+ T cell count, C-reactive protein (CRP), D-dimer, and lactate dehydrogenase (LDH). For predicting COVID-19 progression, the ACCCDL score yielded a significantly higher area under the receiver operating characteristic curve (AUROC) compared with the CALL score, CoLACD score, PH-COVID-19 score, neutrophil-lymphocyte ratio, and lymphocyte-monocyte ratio both in the training set (0.92, 0.84, 0.83, 0.83, 0.76, and 0.65, respectively) and in the validation set (0.97, 0.83, 0.83, 0.78, 0.74, and 0.60, respectively). Over 99% of patients with the ACCCDL score < 12 points will not progress to severe cases, and over 30% of patients with the ACCCDL score > 20 points will progress to severe cases.
CONCLUSION: The ACCCDL score could stratify patients with at risk of COVID-19 progression, and was useful in regulating the large flow of patients with COVID-19 between primary health care and tertiary centers.

Key Summary Points

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.14602290.

Introduction

Since November 2019, the outbreak of the novel coronavirus disease (COVID-19), which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has influenced almost all countries and regions. Evidence of human-to-human transmission has been reported among close contacts of patients with COVID-19 [1]. Although considerable efforts have been made to reduce transmission, the overall upward trend of COVID-19 is continuing around the world. As of 26 January 2021, the outbreak of COVID-19 brings the cumulative numbers to over 98 million reported cases and over 2.1 million deaths globally [2].

The COVID-19 pandemic not only affected the economic development throughout the world but also forced people to strongly modify their daily routines, putting a significant strain on the social aspects of daily life. Diminished volumes of habitual physical activity and increased sedentary levels have been observed as a result of COVID-19 home confinement [3]. A worldwide multicenter study elucidates the risk of psychosocial strain during the early COVID-19 home confinement period in 2020 [4]. The results of the ECLB-COVID19 international online survey also show that isolation alters physical activity and eating behaviors in a health-compromising direction [5]. Consequences of inactivity, including a higher mortality rate and poorer general health and fitness, have been reported. In order to mitigate the negative psychosocial effects of home confinement, implementation of national strategies focused on promoting social inclusion through a technology-based solution is strongly suggested [4]. The insight from the ECLB-COVID19 multicenter study provides a clear remit for the urgent implementation of technology-based intervention to foster an Active and Healthy Confinement Lifestyle [6]. Souissi et al. also suggest combining video feedback with pedagogical activity during the pandemic-induced online coaching or physical education to improve movement learning in school-aged children [7].

The prevalence of severe cases has been reported in patients with COVID-19 [8, 9]. Richardson et al. included 5700 patients hospitalized with COVID-19 in New York, and reported that 373 patients (14.2%) were severe cases that required treatment in the intensive care unit (ICU) [8]. In a report of 72,314 cases in China, 14% were severe cases that required ventilation in the ICU [9]. The risk factors of COVID-19 progression have been reported, and broadly include older age [10], comorbidity [11], C-reactive protein (CRP) [12], CD4+ T cell count [13], D-dimer, lactate dehydrogenase (LDH) [14], leukocyte count, lymphocyte count, procalcitonin, and so on [15]. Prediction models, which combine multiple risk factors to estimate the risk of COVID-19 progression, could help physicians to decide which patients can be treated at primary health care centers and who needs transfer to tertiary centers. Some prediction models for early identification of patients with risk of COVID-19 progression have been reported in previous studies [16-21]. However, most of the existing prediction models lack validation, are inadequately reported, or are at high risk of bias. Thus, it is not a surprise to see different results from different models. Few existing prediction models have the potential to be extensively used by healthcare providers.

Before performing this study, we proposed some hypotheses. First, besides previously reported predictors, there are some other novel predictors which are associated with COVID-19 progression. Second, including novel predictors might further improve the sensitivity and specificity of previously reported prediction models. Third, it is possible to develop a novel prediction model of COVID-19 progression, which is superior to previously reported prediction models. In this cohort of 1234 patients with COVID-19, we aimed to (1) develop and validate a novel prediction model of COVID-19 progression; (2) validate previously reported prediction models, including the CALL score [16], CoLACD score [17], PH-COVID-19 score [18], neutrophil–lymphocyte ratio [19], and lymphocyte–monocyte ratio [20, 21].

Methods

Study Population

In this retrospective study, 1332 consecutive patients with COVID-19 admitted to Shanghai Public Health Clinical Center between January 20, 2020 and November 30, 2020 were analyzed. Shanghai Public Health Clinical Center is a tertiary teaching hospital in Shanghai, China. Exclusion criteria: (1) patients requiring immediate ICU admission at hospital admission (n = 15); (2) patients who did not yet have a definite outcome, i.e., who were still being treated at the time of analysis (n = 83). Finally, 1234 non-severe patients with COVID-19 at hospital admission were enrolled. The 814 patients admitted to hospital between January 20, 2020 and August 31, 2020 constituted the training set, and 420 patients admitted to hospital between September 1, 2020 and November 30, 2020 constituted the validation set.

The study was approved by the Clinical Research Ethics Committee of Shanghai Public Health Clinical Center (No. 2021-S013-01). In our hospital, patients with COVID-19 were insulated in negative pressure isolation rooms. As researchers, we could not obtain the chance of physical contact with patients with COVID-19. Therefore, at hospital admission, patients provided verbal consent for their clinical data to be used for medical study using voice or video calls. The procedures were in accordance with the ethical standards of the Helsinki Declaration (1964, amended most recently in 2008) of the World Medical Association.

Diagnostic Criteria

Patients with COVID-19 were confirmed by the positive results of SARS-CoV-2 nucleotide tests in the nasopharyngeal or throat swab specimens using the real-time reverse transcription polymerase chain reaction (RT-PCR) method. Severe COVID-19 cases were diagnosed using at least one of the following standards: (1) respiratory frequency ≥ 30 breath/min; (2) resting oxygen saturation ≤ 93%; (3) oxygenation index ≤ 300 mmHg; (4) mechanical ventilation; (5) shock; (6) other organ failures and ICU admission.

Data Collection

Demographic and clinical data including age, sex, body mass index (BMI), comorbidity, epidemiological history, clinical manifestation, vital signs, laboratory parameters, and clinical outcomes were collected from the electronic medical records of Shanghai Public Health Clinical Center. In our hospital, the laboratory parameters of patients with COVID-19 were centralized in the intensive testing laboratory which is specialized for patients with COVID-19.

Calculation of Previously Reported Prediction Models of COVID-19 Progression

The CALL score was calculated using four clinical factors [16]: comorbidity (without, 1 point; with, 4 points); age (≤ 60 years, 1 point; > 60 years, 3 points); lymphocyte count (> 1.0 × 109/L, 1 point; ≤ 1.0 × 109/L, 3 points); LDH (≤ 250 U/L, 1 point; 250–500 U/L, 2 points; > 500 U/L, 3 points).

The CoLACD score was calculated using four clinical factors [17]: COVID-19 lymphocyte ratio (≥ 17.6%, 0 points; < 17.6%, 1 point); age (< 50, 0 points; 50–65, 1 point; ≥ 65, 2 points); CCI score (< 3, 0 points; ≥ 3, 1 point); dyspnea (without, 0 points; with, 1 point).

The PH-COVID-19 score was calculated using eight predictors [18]: age (< 20, − 2 points; 20–29, 0 points; 30–39, 2 points; 40–49, 4 points; 50–59, 6 points; 60–69, 8 points; 70–79, 10 points; 80–89, 12 points; 90–99, 14 points; > 99, 15 points); sex (female, 0 points; male, 2 points); diabetes (without, 0 points; with, 2 points); chronic obstructive pulmonary disease (without, 0 points; with, 1 point); immunosuppression (without, 0 points; with, 1 point); hypertension (without, 0 points; with, 1 point); obesity (without, 0 points; with, 1 point); chronic kidney disease (without, 0 points; with, 2 points).

The neutrophil–lymphocyte ratio (NLR) = neutrophil count/lymphocyte count [19].

The lymphocyte–monocyte ratio (LMR) = lymphocyte count/monocyte count [20].

Statistical Analysis

Normally distributed data, non-normal distribution data, and categorical data are presented as mean ± standard deviation, median (interquartile range, IQR), and frequency, respectively. The statistical differences were compared using the Student’s t test for normally distributed data, the non-parametric Mann–Whitney test for non-normal distribution data, and the chi-square test for categorical data. The significance of each variable was assessed using univariate and multivariate Cox proportional hazards model for identifying the independent risk factors of COVID-19 progression. All the variables at a statistically significant level after multivariate Cox analysis were candidates for formulation of a novel scoring model, and their relevant points were determined by the hazard ratio (HR) values to reflect their weights of impact on COVID-19 progression. The predictive performances of the scoring models were assessed using receiver operating characteristic (ROC) curves and areas under the ROC curves (AUROCs). The Delong test was used to compare the areas under two or more correlated ROC curves [22]. Two sets of cutoffs were calculated: (1) maximizing the Youden index (sensitivity + specificity − 1); (2) obtaining a specificity of at least 95%. All significance tests were two-tailed, and p < 0.05 was considered statistically significant. The statistical analyses were performed using the SPSS software, version 15.0 (SPSS Inc. Chicago, Illinois, USA), the MedCalc software, version 16.1 (MedCalc Software bvba, Ostend, Belgium), and the R software, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Clinical Characteristics of Patients on Hospital Admission in the Training Set

Clinical characteristics of patients on hospital admission in the training set are shown in Table 1. The median age was 36 years (IQR 25–51), 110 (13.5%) were older than 60 years, 468 (57.5%) were male, 225 (27.6%) had obesity, and 146 (17.9%) had at least one comorbidity. The median white blood count (WBC), lymphocyte count, CD4+ T cell count, LDH, CRP, D-dimer, procalcitonin, and erythrocyte sedimentation rate (ESR) levels were 5.5 × 109/L (IQR 4.3–6.8), 1.5 × 109/L (IQR 1.1–1.9), 577 cells/µL (IQR 408–771), 201 U/L (IQR 178–234), 0.5 mg/L (IQR 0.5–7.0), 0.3 ng/mL (IQR 0.2–0.5), 0.03 µg/L (IQR 0.02–0.10), and 30 mm/h (IQR 10–74), respectively. The 814 patients in the training set were divided into progressive group (n = 18, 2.2%) and stable group (n = 796, 97.8%) based on whether they progressed to severe cases during the hospitalization. Age, obesity, comorbidity, lymphocyte count, CD4+ T cell count, LDH, CRP, D-dimer, procalcitonin, and ESR were significantly different between progressive group and stable group (p < 0.05) (Table 1).

Table 1

Clinical characteristics of patients on hospital admission in the training set

	Overall (n = 814)	Stable group (n = 796)	Progressive group (n = 18)	p value
Age (years)	36 (25–51)	36 (25–51)	65 (59–72)	< 0.001
≤ 60	704 (86.5%)	700 (87.9%)	4 (22.2%)	< 0.001
> 60	110 (13.5%)	96 (12.1%)	14 (77.8%)	< 0.001
Male sex, n (%)	468 (57.5%)	456 (57.3%)	12 (66.7%)	0.426
Obesity, n (%)	225 (27.6%)	216 (27.1%)	9 (50%)	0.032
Comorbidity, n (%)	146 (17.9%)	134 (16.8%)	12 (66.7%)	< 0.001
Hypertension	98 (12.04%)	88 (11.06%)	10 (55.6%)	< 0.001
Diabetes mellitus	43 (5.28%)	38 (4.77%)	5 (27.8%)	< 0.001
Heart disease	18 (2.21%)	15 (1.88%)	3 (16.7%)	< 0.001
WBC count (109/L)	5.5 (4.3–6.8)	5.5 (4.4–6.8)	5.4 (3.6–7.4)	0.609
Lymphocyte (109/L)	1.5 (1.1–1.9)	1.5 (1.1–1.9)	0.9 (0.5–1.2)	< 0.001
≤ 1.0	189 (23.2%)	177 (22.2%)	12 (66.7%)	< 0.001
> 1.0	625 (76.8%)	619 (77.8%)	6 (33.3%)	< 0.001
CD4+ T cell (cells/µL)	577 (408–771)	578 (414–775)	252 (119–554)	< 0.001
≤ 410	205 (25.2%)	193 (24.2%)	12 (66.7%)	< 0.001
> 410	609 (74.8%)	603 (75.8%)	6 (33.3%)	< 0.001
LDH (U/L)	201 (178–234)	200 (177–232)	314 (253–368)	< 0.001
≤ 250	659 (81.0%)	655 (82.3%)	4 (22.2%)	< 0.001
> 250	155 (19.0%)	141 (17.7%)	14 (77.8%)	< 0.001
CRP (mg/L)	0.5 (0.5–7.0)	0.5 (0.5–6.3)	53.2 (21.2–83.1)	< 0.001
≤ 10	654 (80.3%)	650 (81.7%)	4 (22.2%)	< 0.001
> 10	160 (19.7%)	146 (18.3%)	14 (77.8%)	< 0.001
D-dimer (ng/mL)	0.3 (0.2–0.5)	0.3 (0.2–0.5)	0.7 (0.4–1.1)	< 0.001
≤ 0.5	654 (80.3%)	646 (81.2%)	8 (44.4%)	< 0.001
> 0.5	160 (19.7%)	150 (18.8%)	10 (55.6%)	< 0.001
PCT (µg/L)	0.03 (0.02–0.10)	0.03 (0.02–0.10)	0.08 (0.04–0.25)	0.002
≤ 0.05	517 (63.5%)	511 (64.2%)	6 (33.3%)	0.007
> 0.05	297 (36.5)	285 (35.8%)	12 (66.7%)	0.007
ESR (mm/h)	30 (10–74)	29 (10–73)	58 (41–92)	< 0.001
≤ 15	279 (34.3%)	278 (34.9%)	1 (5.6%)	< 0.001
> 15	535 (65.7%)	518 (65.1%)	17 (94.4%)	< 0.001

WBC white blood count, LDH lactate dehydrogenase, CRP C-reactive protein, PCT procalcitonin, ESR erythrocyte sedimentation rate

Independent Risk Factors Associated with COVID-19 Progression

The independent risk factors associated with COVID-19 progression are shown in Table 2. Multivariate Cox proportional hazards analysis showed that age > 60 years (HR 6.3, 95% CI 2.1–29.4, p < 0.001), comorbidity (1 comorbidity, HR 4.0, 95% CI 1.4–16.9, p = 0.002; ≥ 2 comorbidities, HR 6.2, 95% CI 1.9–26.8, p < 0.001), CD4+ T cell count ≤ 410 cells/µL (HR 3.2, 95% CI 1.5–11.8, p = 0.025), LDH > 250 U/L (HR 4.0, 95% CI 1.7–19.8, p = 0.007), CRP > 10 mg/L (HR 3.1, 95% CI 1.2–10.5, p = 0.034), and D-dimer (HR 2.9, 95% CI 1.0–9.7, p = 0.042) were the independent risk factors associated with COVID-19 progression.

Table 2

Independent risk factors associated with COVID-19 progression

	Univariate Cox analysis		Multivariate Cox analysis
	HR (95% CI)	p value	HR (95% CI)	p value
Age (years)
≤ 60	1	–	1	–
> 60	22.4 (8.2–79.1)	< 0.001	6.3 (2.1–29.4)	< 0.001
Obesity
Without	1	–	1	–
With	4.2 (1.5–12.4)	0.028	2.5 (0.8–7.7)	0.332
Comorbidity
Without	1	–	1	–
1 comorbidity	9.2 (3.6–26.8)	< 0.001	4.0 (1.4–16.9)	0.002
≥ 2 comorbidities	20.5 (6.8–66.5)	< 0.001	6.2 (1.9–26.8)	< 0.001
Lymphocyte (× 109/L)
> 1.0	1	–	1	–
≤ 1.0	6.6 (2.6–18.9)	< 0.001	3.4 (0.8–15.3)	0.083
CD4+ T cell (cells/µL)
> 410	1	–	1	–
≤ 410	5.9 (2.3–16.9)	< 0.001	3.2 (1.5–11.8)	0.025
LDH (U/L)
≤ 250	1	–	1	–
> 250	15.3 (5.5–52.4)	< 0.001	4.0 (1.7–19.8)	0.007
CRP (mg/L)
≤ 10	1	–	1	–
> 10	14.3 (5.1–48.0)	< 0.001	3.1 (1.2–10.5)	0.034
D-dimer (mg/L)
≤ 0.5	1	–	1	–
> 0.5	5.1 (2.1–13.9)	< 0.001	2.9 (1.0–9.7)	0.042
PCT (µg/L)
≤ 0.05	1	–	1	–
> 0.05	3.5 (1.3–9.7)	0.012	2.2 (0.7–6.9)	0.166
ESR (mm/h)
≤ 15	1	–	1	–
> 15	8.9 (1.2–68.9)	0.032	2.8 (0.3–25.2)	0.354

LDH lactate dehydrogenase, CRP C-reactive protein, PCT procalcitonin, ESR erythrocyte sedimentation rate, HR hazard ratio, CI confidence interval

Construction of a Novel Prediction Model of COVID-19 Progression

On the basis of the HR values of multivariate Cox analysis, we assign 6 points to age (≤ 60 years, 1 point; > 60 years, 6 points), 6 points to comorbidity (absence of comorbidity, 1 point; 1 comorbidity, 4 points; ≥ 2 comorbidities, 6 points), 4 points to LDH (≤ 250 U/L, 1 point; > 250 U/L, 4 points), and 3 points to CD4+ T cell (> 410 cells/µL, 1 point; ≤ 410 cells/µL, 3 points), CRP (≤ 10 mg/L, 1 point; > 10 mg/L, 3 points), and D-dimer (≤ 0.5 ng/mL, 1 point; > 0.5 ng/mL, 3 points). In order to facilitate clinical use and further assessment, a novel prediction model of COVID-19 prediction was established, called the ACCCDL score, which scores from 6 to 25 points. The calculation of the ACCCDL score is shown in Table 3.

Table 3

Calculation of the ACCCDL score

	Points
Age (years)
≤ 60	1
> 60	6
Comorbidity
Without	1
≥ 1 comorbidity	4
≥ 2 comorbidities	6
CD4+ T cell (cells/µL)
> 410	1
≤ 410	3
CRP (mg/L)
≤ 10	1
> 10	3
D-dimer (ng/mL)
≤ 0.5	1
> 0.5	3
LDH (U/L)
≤ 250	1
> 250	4

CRP C-reactive protein, LDH lactate dehydrogenase

Clinical Characteristics of Patients on Hospital Admission in the Validation Set

Clinical characteristics of patients on hospital admission in the validation set are shown in Table 4. The median age was 35 years (IQR 28–49), 20 (4.8%) were older than 60 years, 291 (69.3%) were male, and 68 (16.2%) had at least one comorbidity. The 420 patients in the validation set were divided into progressive group (n = 5, 1.2%) and stable group (n = 415, 98.8%) based on whether they progressed to severe cases during the hospitalization. Compared with stable group, progressive group had higher proportions of age > 60 years (40% vs. 4.3%, p < 0.001), comorbidity (80% vs. 15.4%, p < 0.001), CD4+ T cell count ≤ 410 cells/µL (60% vs. 13.0%, p = 0.002), CRP > 10 mg/L (40% vs. 3.9%, p < 0.001), D-dimer > 0.5 ng/mL (60% vs. 9.9%, p < 0.001), and LDH > 250 U/L (60% vs. 4.3%, p < 0.001) (Table 4).

Table 4

Clinical characteristics of patients on hospital admission in the validation set

	Overall (n = 420)	Stable group (n = 415)	Progressive group (n = 5)	p value
Age (years)	35 (28–49)	35 (28–48)	49 (30–65)	< 0.001
≤ 60	400 (95.2%)	397 (95.7%)	3 (60%)	< 0.001
> 60	20 (4.8%)	18 (4.3%)	2 (40%)	< 0.001
Male sex, n (%)	291 (69.3%)	287 (69.2%)	4 (80%)	0.601
Comorbidity, n (%)	68 (16.2%)	64 (15.4%)	4 (80%)	< 0.001
Hypertension	37 (8.80%)	34 (8.2%)	3 (60%)	< 0.001
Diabetes mellitus	17 (4.05%)	15 (3.6%)	2 (40%)	< 0.001
Heart disease	2 (0.48%)	2 (0.48%)	0	0.876
CD4+ T cell (cells/µL)	691 (524–884)	692 (525–887)	222 (212–487)	0.001
≤ 410	57 (13.6%)	54 (13.0%)	3 (60%)	0.002
> 410	363 (86.4%)	361 (87.0%)	2 (40%)	0.002
CRP (mg/L)	0.5 (0.5–0.5)	0.5 (0.5–0.5)	5.6 (0.7–25.9)	< 0.001
≤ 10	402 (95.7%)	399 (96.1%)	3 (60%)	< 0.001
> 10	18 (4.3%)	16 (3.9%)	2 (40%)	< 0.001
D-dimer (ng/mL)	0.2 (0.2–0.3)	0.2 (0.2–0.3)	0.5 (0.5–1.1)	< 0.001
≤ 0.5	376 (89.5%)	374 (90.1%)	2 (40%)	< 0.001
> 0.5	44 (10.5%)	41 (9.9%)	3 (60%)	< 0.001
LDH (U/L)	175 (157–204)	174 (156–203)	253 (209–292)	0.001
≤ 250	399 (95%)	397 (95.7%)	2 (40%)	< 0.001
> 250	21 (5%)	18 (4.3%)	3 (60%)	< 0.001

LDH lactate dehydrogenase, CRP C-reactive protein;

Comparison of AUROCs of Prediction Models

Pairwise comparison of AUROCs of prediction models is shown in Table 5 and Fig. 1. In the training set, the ACCCDL score yielded a significantly higher AUROC compared with the CALL score (0.92 vs 0.84, p = 0.047), the CoLACD score (0.92 vs 0.83, p = 0.043), the PH-COVID-19 score (0.92 vs 0.83, p = 0.029), the NLR model (0.92 vs 0.76, p < 0.001), and the LMR model (0.92 vs 0.65, p < 0.001). In the validation set, the ACCCDL score also yielded a significantly higher AUROC compared with the CALL score (0.97 vs 0.83, p = 0.018), the CoLACD score (0.97 vs 0.83, p = 0.014), the PH-COVID-19 score (0.97 vs 0.78, p = 0.012), the NLR model (0.97 vs 0.74, p < 0.001), and the LMR model (0.97 vs 0.60, p < 0.001).

Table 5

Pairwise comparison of AUROCs of prediction models

	Training set		Validation set
	AUROC	(95% CI)	AUROC	(95% CI)
ACCCDL score	0.92	(0.90–0.94)	0.97	(0.95–0.99)
CALL score	0.84	(0.81–0.87)	0.83	(0.79–0.87)
CoLACD score	0.83	(0.80–0.86)	0.83	(0.79–0.87)
PH-COVID-19 score	0.83	(0.80–0.85)	0.78	(0.74–0.82)
NLR	0.76	(0.72–0.79)	0.74	(0.69–0.79)
LMR	0.65	(0.62–0.69)	0.60	(0.55–0.64)
Comparison of AUROCs
ACCCDL vs CALL	p = 0.047		p = 0.018
ACCCDL vs CoLACD	p = 0.043		p = 0.014
ACCCDL vs PH-COVID-19	p = 0.029		p = 0.012
ACCCDL vs NLR	p = 0.002		p < 0.001
ACCCDL vs LMR	p < 0.001		p < 0.001

CALL score, a scoring model based on four predictors (comorbidity, age, lymphocyte and LDH); CoLACD score, a scoring model based on four predictors (CoVID-19 lymphocyte ratio, age, CCI score, dyspnea); PH-COVID-19 score, a scoring model based on eight predictors (age, sex, diabetes, chronic obstructive pulmonary disease, immunosuppression, hypertension, obesity, and chronic kidney disease)

NLR neutrophil–lymphocyte ratio, LMR lymphocyte–monocyte ratio

Fig. 1

Comparison of AUROCs of prediction models. For predicting COVID-19 progression, the ACCCDL score yielded a significantly higher AUROC compared with the CALL score, the CoLACD score, the PH-COVID-19 score, the NLR model, and the LMR model both in the training set (0.92, 0.84, 0.83, 0.83, 0.76, and 0.65, respectively) and in the validation set (0.97, 0.83, 0.83, 0.78, 0.74, and 0.60, respectively)

Cutoff Value and Accuracy of the ACCCDL Score

Cutoff value and accuracy of the ACCCDL score is shown in Table 6. The cutoff value is 12 points by maximizing the Youden index, and the corresponding sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) was 88.9%, 80.3%, 9.2%, and 99.7%, respectively, in the training set; and 80.0%, 94.7%, 15.4%, and 99.7%, respectively, in the validation set. The cutoff value is 20 points by using specificity ≥ 95%, and the corresponding sensitivity, specificity, PPV, and NPV was 55.6%, 97.6%, 34.5%, and 99%, respectively, in the training set; and 20.0%, 99.8%, 50%, and 99.0%, respectively, in the validation set.

Table 6

Cutoff value and accuracy of the ACCCDL score

Training set (n = 814)		Validation set (n = 420)
Cutoff value > 12
Sensitivity	88.9%	Sensitivity	80.0%
Specificity	80.3%	Specificity	94.7%
PPV	9.2%	PPV	15.4%
NPV	99.7%	NPV	99.7%
PLR	4.51	PLR	15.09
NLR	0.14	NLR	0.21
Cutoff value > 20
Sensitivity	55.6%	Sensitivity	20.0%
Specificity	97.6%	Specificity	99.8%
PPV	34.5%	PPV	50%
NPV	99%	NPV	99.0%
PLR	23.27	PLR	83.00
NLR	0.46	NLR	0.80

PPV positive predictive value, NPV negative predictive value, PLR positive likelihood ratio, NLR negative likelihood ratio

Discussion

Defining the prognosis of patients with COVID-19 is of the utmost importance for planning the most appropriate setting of care and treatment. By using the clinical data of 1234 patients with COVID-19, we developed and validated a novel prediction score, called the ACCCDL score, based on six variables (age, comorbidity, CD4+ T cell count, CRP, D-dimer, and LDH), aimed at predicting progression toward severe COVID-19. The ACCCDL score ranges from 6 points (age ≤ 60 years, absence of comorbidity, CD4+ T cell > 410 cells/µL, CRP ≤ 10 mg/L, D-dimer ≤ 0.5 ng/mL, LDH ≤ 250 U/L) to 25 points (age > 60 years, ≥ 2 comorbidities, CD4+ T cell ≤ 410 cells/µL, CRP > 10 mg/L, D-dimer > 0.5 ng/mL, LDH > 250 U/L). The power of the ACCCDL score for predicting COVID-19 progression was excellent with AUROC of 0.92 (95% CI 0.90–0.94) in the training set and 0.97 (95% CI 0.95–0.99) in the validation set.

In this study, two cutoff points were reported for the ACCCDL score. A high cutoff (20 points) with high specificity and PPV is used to rule in patients who had risk of COVID-19 progression. A low cutoff (12 points) with high sensitivity and NPV is used to rule out the risk of COVID-19 progression. A total of 641 patients (78.7%) in the training set and 394 patients (93.8%) in the validation set had the ACCCDL score ≤ 12 points, and these patients could be managed at primary health care centers or district hospitals because over 99% of patients with the ACCCDL score ≤ 12 points will not progress to severe cases. About 34.5% (10/29) of patients with the ACCCDL score > 20 points in the training set, and 50% (1/2) of patients with the ACCCDL score > 20 points in the validation set will progress to severe COVID-19, and can benefit from early transfer to tertiary centers.

The ACCCDL score has several attractive features. First, it is inexpensive and very easily calculated, which makes it easier to use. Second, compared with previously reported prediction models, the ACCCDL score provides higher prediction performance and more accurate evaluation for the risk of COVID-19 progression. Third, the ACCCDL score provided a clinically available method for the large flow of patients between primary health care and tertiary centers.

Ji et al. enrolled 208 patients with COVID-19 and derived a scoring system (the CALL score) predicting progression to severe COVID-19 based on patients’ age, comorbidity, lymphocyte count, and LDH [16]. In the study by Ji et al., the CALL score could identify patients who had low risk of COVID-19 progression with an AUROC of 0.91 (95% CI 0.86–0.94) [16]. However, in another study of 210 patients with COVID-19, Grifoni et al. reported the opposite conclusion that the power of the CALL score for predicting progression to severe COVID-19 was low with an AUROC of 0.62 (95% CI 0.53–0.69) [23]. The two studies had relatively small samples and come to the opposite conclusion. Other external validations are needed to further validate the prediction performance of the CALL score. In this large sample study, we found that the power of the CALL score for predicting progression to severe COVID-19 was high with an AUROC of 0.84 (95% CI 0.81–0.87) in the training set and an AUROC of 0.83 (95% CI 0.79–0.87) in the validation set. There are several possible explanations for the differences. First, the statistic difference might be caused by the size of the sample. Second, there are some differences in the study populations regarding age, comorbidity, and the prevalence of severe cases, known as the spectrum bias [24, 25].

The probable reasons why the ACCCDL score is superior to previous ones is worthy of being discussed. First, compared with the CALL score, the ACCCDL score includes CD4+ T cell count, CRP, and D-dimer levels and this might improve the sensitivity and the specificity. Previous studies also had reported that CD4+ T cell count, CRP, and D-dimer levels could help clinicians to identify patients with COVID-19 progression [12-14]. Second, compared with the CALL score which just considers whether patients have comorbidity (without, 1 point; with, 4 point), the ACCCDL score includes the number of comorbidities (absence of comorbidity, 1 point; 1 comorbidity, 4 points; ≥ 2 comorbidities, 6 points) and this also might improve the sensitivity and the specificity. Guan et al. also reported that patients with any comorbidity yielded poorer clinical outcomes than those without, and the number of comorbidities was one of the risk factors of reaching the composite endpoints (admission to an ICU, invasive ventilation, or death). The hazard ratio (95% CI) was 1.79 (1.16–2.77) among patients with at least one comorbidity and 2.59 (1.61–4.17) among patients with two or more comorbidities [26].

This study has some limitations. First, this study is a single-center study. Second, the retrospective design might have caused selection bias [27]. Therefore, the ACCCDL score needs to be externally validated in a prospective study to confirm its prediction reliability. Third, in our cohort, the majority of patients with COVID-19 (more than 70%) were imported cases, which mainly included overseas students and business people. Generally, the overseas students and business people are young and had low risk of comorbidity. Therefore, in this study, both the training set and validation set included < 15% of 60 years old patients, < 20% had more than one comorbidity. The diagnostic performance of the ACCCDL score deserves to be validated in a cohort with more patients who have older age and more common comorbidity, which might strengthen the results of this study. Fourth, the ACCCDL score was not evaluated using patients with negative RT-PCR and a high clinical suspicion of COVID-19 because in Shanghai, China, our hospital is the only designated hospital for confirmed adult patients with COVID-19. All patients were confirmed by the positive results of SARS-CoV-2 nucleotide tests before they were transferred to our hospital.

Conclusions

The ACCCDL score is a simple and accurate model for the prediction of COVID-19 progression. Over 99% of patients with the ACCCDL score ≤ 12 points will not progress to severe cases, and over 30% of patients with the ACCCDL score > 20 points will progress to severe cases. On the basis of the results, we suggest that patients with the ACCCDL score ≤ 12 points can be managed at primary health care centers or district hospitals, and patients with the ACCCDL score > 20 points should be transferred early to tertiary centers for more intensive care and management. The ACCCDL score was useful in regulating the large flow of patients with COVID-19 between primary health care and tertiary centers.

We developed a novel prediction model of COVID-19 progression, called the ACCCDL score, based on six variables (age, comorbidity, CD4+ T cell count, CRP, D-dimer, and LDH).

The ACCCDL score yielded a higher predictive performance compared with previous reported prediction models including the CALL score, CoLACD score, PH-COVID-19 score, neutrophil–lymphocyte ratio, and lymphocyte–monocyte ratio.

Over 99% of patients with the ACCCDL score ≤ 12 points will not progress to severe cases, and can be managed at primary health care centers. Over 30% of patients with the ACCCDL score > 20 points will progress to severe cases, and can benefit from early transfer to tertiary centers.