A biomarker based severity progression indicator for COVID-19: the Kuwait prognosis indicator score.

BACKGROUND: COVID-19 is a worldwide pandemic that is mild in most patients but can result in a pneumonia like illness with progression to acute respiratory distress syndrome and death. Predicting the disease severity at time of diagnosis can be helpful in prioritizing hospital admission and resources.
METHODS: We prospectively recruited 1096 consecutive patients of whom 643 met the inclusion criterion with COVID-19 from Jaber Hospital, a COVID-19 facility in Kuwait, between 24 February and 20 April 2020. The primary endpoint of interest was disease severity defined algorithmically. Predefined risk variables were collected at the time of PCR based diagnosis of the infection. Prognostic model development used 5-fold cross-validated regularized logit regression. The model was externally validated against data from Wuhan, China.
RESULTS: There were 643 patients with clinical course data of whom 94 developed severe COVID-19. In the final model, age, CRP, procalcitonin, lymphocyte percentage, monocyte percentages and serum albumin were independent predictors of a more severe illness course. The final prognostic model demonstrated good discrimination, and both discrimination and calibration were confirmed with an external dataset.
CONCLUSION: We developed and validated a simple score calculated at time of diagnosis that can predict patients with severe COVID-19 disease reliably and that has been validated externally. The KPI score calculator is now available online at covidkscore.com.

Introduction

The outbreak of pneumonia in the Hubei province of the People’s Republic of China in December 2019 was identified to be due to a novel corona virus, namely severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The disease termed COVID-19 became a pandemic affecting more than 3,419,000 people worldwide with around 243,000 deaths (up to 2 May 2020) (Dong et al. 2020). The majority of patients with COVID-19 recover, but a subset of patients develop severe disease characterized by a cytokine storm that increases the risk of mortality (Mehta et al. 2020). The main cause of mortality in those patients is acute respiratory distress syndrome (ARDS) or septic shock which occurs in 15–20% of patients (Zhu et al. 2020).

A study on the New York experience on hospitalized patients with COVID-19 reported that out of 5700 patients, 373 (14.2%) were treated in the ICU, 320 (12.2%) received invasive mechanical ventilation, 81 (3.2%) were treated with kidney replacement therapy, and 553 (21%) died (Richardson et al. 2020).

In many countries around the world, admission to hospitals is reserved for those with severe symptoms which usually do not develop from the onset of symptoms or from the time of diagnosis with the PCR test. Patients with severe symptoms present usually after a mild first phase of the disease (Chen et al. 2020).

To date there has not been an effective therapeutic modality in the form of an antiviral medication or vaccine against the disease, but many regimens have been tested and some experts suggested suppressing the immune system to avoid the cytokine storm leading to ARDS (Mehta et al. 2020). All these proposed treatments carry their own risks, and immunosuppression might increase the risks for other viral and bacterial infections thus hardly justifying their use in mildly symptomatic patients.

The challenge today is determining and stratifying which patient is likely to progress to severe disease at time of diagnosis. Answering this question might justify early treatment and admission to hospitals. In this study we examine the initial cohort of patients in Kuwait to determine what risk factors at time of positivity of a test can predict a worse outcome, as all patients with a positive test even if asymptomatic are admitted to a single centre in the State of Kuwait.

Clinical significance

We created and externally validated a simple score that can predict, which patients will have a severe progression of COVID-19 from the time of diagnosis. This score allows better allocation of medical resources at time of diagnosis.

Methods

Study design

We obtained the ethical approval from the Kuwait Ministry of Health, ethical review committee (Ref 1402/2020). This study was a prospective cohort study aimed at evaluating predictors of COVID-19 severity amongst COVID-19 in-patients admitted to the Infectious Diseases Hospital in Kuwait between 24 February 2020 and 20 April 2020.

Definition of cases and inclusion criteria

All consecutive patients meeting the case definition (upper respiratory symptoms of any degree of severity with or without a travel history) and who had tested positive by PCR for SARS-CoV-2 are diagnosed to have COVID-19 and admitted for quarantine and observation. Testing for COVID-19 was undertaken via real-time reverse transcriptase polymerase chain reaction (qRT-PCR) assay of nasal swab specimens by taqpath COVID-19 CE-IVD_PCR kit (Cat no. A48067, Thermo Fisher Scientific, Waltham, MA, USA) according to manufacturer’s protocol and performed on Quant Studio 7 Flex PCR system (Thermo Fisher Scientific, Waltham, MA, USA). All diagnostic tests (Supplementary material A and B) were performed at the Jaber Hospital laboratory. All positive patients remain on admission till they have had resolution of symptoms (afebrile for more than 72 hours plus saturation ≥94%) and additional requirements for discharge are being more than 7 days since symptoms onset, completing 14 days since testing positive and presence of improvement of any documented consolidation on chest X-ray. Discharge also requires two consecutive negative tests >24 h apart, refer patient flow diagram mentioned in Figure 1. A standardized form was completed prospectively for data collection, including demographic data, clinical data, and radiographic/laboratory results.

Figure 1.

Patient flow diagram.

Severity grouping (main outcome)

We pre-specified the main outcome to be moderate-severe COVID-19 defined based on need for hospital support, while mild cases will have a mild clinical course needing only symptomatic management (the majority). The severity grouping algorithm was determined prospectively as follows:

Assign missing status to everyone on the severity variable

Assign moderate to severe status to those with hospital course that led to:

Consolidation on chest x-ray or shortness of breath on admission

ICU admission

Hospital stay >14 days AND in hospital receiving active treatment (see severity grouping 3. below)

Death

Assign moderate to severe status additionally to those receiving any of the following treatments regardless of hospital stay or discharge status:

Systemic glucocorticoids

Intravenous immunoglobulin (IVIg) derived from plasma

Oxygen therapy

Non-invasive ventilation OR mechanical ventilation

Extracorporeal membrane oxygenation

Continuous renal replacement therapy

Assign mild status to those meeting the following:

Not on treatment, not discharged but duration in hospital >10 days

Discharged and no treatment received while in hospital

Discharged and duration in hospital >14 days and severity score not already assigned to severe status above

Apart from the severity group outcome, ICU admission and death were defined as hard outcomes for the purposes of model evaluation but not in model building as they were subsumed within the main outcome.

External validation cohort

The medical information of patients previously reported (Yan et al. 2020) formed the validation cohort. This data had been collected between 10 January and 18 February 2020 with clinical outcomes followed up to 24 February 2020. The data was reported in a time series format with multiple rows per patient each dated and with different laboratory results. The earliest recorded report of the five laboratory values of interest were extracted for each patient as well as their age and their discharge status (dead or alive). The data custodians reported that they had extracted data by querying medical records using standard forms that included epidemiological, demographic, clinical, laboratory and mortality outcome information (Yan et al. 2020). The data custodians also reported that exclusions had been made for pregnant and breast-feeding women, patients younger than 18 years and records that were less than 80% complete. The data was collected at the Tongji Hospital, Wuhan, China. The Wuhan cohort was from a centre that prioritized admission of ill patients and that was why the Wuhan cohort had more severe cases. In Kuwait we admitted everyone.

Statistical analysis

Categorical variables were summarized using percentages while continuous variables were summarized using medians with interquartile ranges. Variables that may potentially predict the occurrence of severe COVID-19 were analyzed for inclusion via a regularized logistic regression procedure (a machine learning algorithm).

All promising predictor variables (demographics, laboratory test results, comorbidities and selected symptoms; see details in Supplementary material with details of tests used and the variables considered) based on physician judgement for inclusion were assessed at admission prior to the outcome being known. Variables used in the outcome definition were excluded from consideration as a predictor. All predictors included in the analysis were converted to variable scores (if a continuous variable) prior to entry into the regularized regression procedure. The scores for these transformed continuous variables were 0 or 1 representing values below or above the median while for untransformed binary variables was also 0 or 1 (absent vs present respectively). We decided to categorize continuous variables for the often-criticized goal of aiding clinical interpretation and maintaining simplicity. While this may have introduced loss of information, we did not plan to reconsider this approach unless there was a problem with predictive performance since using continuous variables would make the model less applicable to rapid implementation during the pandemic. Of note cut-points for continuous variables were predefined (at the median) and decided upon prior to data analysis.

The regularized regression procedure used was a 5-fold cross validated lasso logit regression and we identified the best model using the largest value of the tuning parameter that was within one standard deviation of the optimum value (i.e. the value that minimizes the mean squared error of prediction), which leads to a more parsimonious model. Then an unrestricted logit model was fitted to the selected set of predictors from the lasso model and these coefficients used to determine a preliminary severity risk score for COVID-19. This was done by rounding the beta coefficient from the unrestricted logit model on the selected variables to create integer weights for each variable. The variable weights multiplied by the variable score for each variable were summed into a severity score, called the Kuwait Progression Indicator (KPI) score for each patient. Regularized regression was run using lassopack in Stata (Ahrens et al. 2020). Time to event analyses were not considered even though this was a dynamic cohort because duration of stay varied based on physician discretion and there was no real risk of over-representing those with the moderate to severe disease outcome for COVID-19 since mild cases represent the bulk of clinical phenotypes seen.

To assess discrimination of the model, we computed the area under the receiver operating characteristic curve (also known as the C statistic) with 95% confidence intervals. To test model internal validity, a straightforward and fairly popular approach was used which was to randomly split the data in two parts: one to develop the model and another to measure its performance. We used randomtag in Stata to tag the dataset (N = 700 for training and the rest for internal validation and later excluded those with missing severity outcomes – see results) and the KPI score was developed on the training data-set. We also assessed discrimination and calibration on an external data set. Operating characteristics of the KPI score were then assessed using application to the full Kuwait cohort with known outcome status. Calibration of the model was assessed using pmcalplot in Stata (Ensor et al. 2018). All analyses were performed using Stata MP version 15 (College Station, TX, USA) and the confidence level was set at 95%.

Results

Baseline characteristics

From 24 February 2020 till 20 April 2020, 1096 consecutive patients with COVID-19 were recruited. Of the 1096 patients, clinical course could be defined for 643 patients (the rest had not yet reached the outcome) and these met the inclusion criteria to form the cohort for this study. Details of the basic characteristics of these patients are given in Table 1. The clinical course was a mild COVID-19 course for 549 patients, a moderate – severe COVID-19 course for 94 patients leading to 15 events per variable for the selected model. Of the 94 severe cases, 42 were admitted to the intensive care unit because of a worsening respiratory status and 19 died.

Table 1.

Baseline characteristics of patients with COVID-19.

Characteristic	Median (IQR) or N (%) with clinical course data (N = 643)	Median (IQR) or N (%) validation cohort (N = 375)
Age	39 (29–54)	62 (46–79)
Males	466 (72.5%)	224 (59.7%)
Ethnicity
Asian*	269 (40.4%)	–
Kuwaiti	257 (40.0%)	–
Others	126 (19.6%)	–
Wuhan residents	–	142 (37.9%)
Duration of stay (days)**	17 (8–23)	10 (5–16)
Diabetes mellitus	111 (17.3%)	–
Hypertension	134 (20.8%)	–
Asthma	37 (5.7%)	–
CAD/IHD**	37 (5.7%)	–
ICU admission	42 (6.5%)	–
Death	19 (2.9%)	174 (46.4%)

*India/Bangladesh/Philippines.

**Coronary/ischaemic heart disease.

Of these 643 patients, 581 had data on the KPI score parameters and of these 363 had been randomly allocated to a training set and 218 to a validation set (using the randomtag in Stata). The prediction model was built on the 363 subjects in the training data-set and this model was validated on the 218 patients in the internal validation data-set.

The model was also validated on the external validation data set which consisted of 375 subjects from Wuhan, China of whom 309 (82%) had data available for computation of the KPI score A brief overview of the data is given in Table 1. This study reported high sensitivity C-reactive protein (hs-CRP).

KPI score of COVID-19

Variables selected through lasso logit regression included age, CRP, procalcitonin, lymphocyte percentage, monocyte percentage and serum albumin and the KPI scoring system is also depicted in Table 2. The range of scores seen were from minus 32 to plus 22 for both the Kuwait cohort and the Wuhan cohort. The area under the curve (AUC) for the KPI score on the training sample (N = 363) was equal to 0.834 (95% CI, 0.779–0.889), which indicates very good model discrimination (Figure 2).

Table 2.

The final prediction model based on the training cohort.

Kuwait progression indicator score (KPI score) for COVID-19Please give your patient zero points if criterion not met
Criterion	Points	Your patient
Age > =41 years	4
CRP > =7 mg/L	2
Procalcitonin> =0.05 ng/ml	16
Lymphocyte percent > =31.5%	−9
Monocyte percent > =9.2%	−8
Albumin > =39.5 g/L	−15
TOTAL

Low progression risk total< =−7.

Uncertain progression risk total −6 to 15.

High progression risk total > =16.

Figure 2.

Area under the receiver operating characteristic curve for the severity score in the training (top left), internal validation (top right) and external validation (bottom) cohorts.

Validation

The internal validation cohort (N = 218) demonstrated equally good discrimination with AUC 0.794 (95% CI, 0.710–0.879) and this is depicted in Figure 2. A calibration plot of observed against expected probabilities for assessment of prediction model performance on the validation cohort demonstrated quite good model calibration (Figure 3). Similarly, the external validation cohort (N = 309) demonstrated very good discrimination with AUC 0.888 (95% CI, 0.854–0.922) and this is depicted in Figure 2. A calibration plot of observed against expected probabilities for assessment of prediction model performance on the external validation cohort also demonstrated very good model calibration (Figure 3).

Figure 3.

Calibration plots for the severity score in the internal (left) and external (right) validation cohorts.

Performance measures

A cut-off for low, intermediate and high risk was chosen according to the score’s performance (thresholds at 90% sensitivity and 90% specificity) in the training cohort. Patients with a score −7 or less were at low risk and those with a score 16 or above were at high risk of more severe illness requiring hospital management. The risk groups also demonstrated good discrimination of the various outcomes (Table 3) and the interval likelihood ratios by risk level (based on the KPI score) are presented in Table 4. From the interval likelihood ratios and the baseline prevalence of a severe clinical course (14.6%) we could compute posterior probabilities. The interval likelihood ratio for the severe risk level was 5.33. This means that for a patient at this risk level, the posterior probability of a severe clinical course is 48%. For a patient at the low risk level (with a score of −7 or less), the posterior probability of a severe clinical course is down to only 4%. Given the way the thresholds were created, at the low risk threshold the score has a sensitivity of 90% and similarly at the high risk threshold it has a specificity of 90%. In the external validation cohort from Wuhan, an individual with a high risk level KPI score (> =16) had a 21-fold (OR 21.2; 95% CI 11.5 − 38.9; p < 0.001) increased odds of in-hospital death compared to those with low to intermediate risk scores.

Table 3.

Odds ratio of outcomes by category of severity score**.

Outcome	Severity score	Risk level	Odds ratio	95% CI
Moderate to severe course in hospital	< =−7	Low	1 (Reference)
	−6 to 15	Intermediate	4.27	2.07–8.82
	> =16	High	23.66	11.10–50.43
ICU admission	< =−7	Low	1 (Reference)
	−6 to 15	Intermediate	14.37	1.86–110.75
	> =16	High	109.31	14.57–820.03
Death	< =−7	Low	1 (Reference)
	−6 to 15	Intermediate	4.25	0.47–38.34
	> =16	High	42.93	5.53–333.53

**Kuwait cohorts with score and severity outcome data.

Table 4.

Interval likelihood ratios by risk level**.

Risk Level	Mod-Severe	Mild	Likelihood ratio	95% CI
Low	10	242	0.225	0.125–0.407
Intermediate	36	204	0.963	0.732–1.266
High	44	45	5.334	3.761–7.566
Total	90	491

**Kuwait cohorts with score and severity outcome data.

Discussion

In this single centre prospective study using a machine learning algorithm we report a prognostic score that stratifies patients with COVID-19 according to the risks of severe illness (clinically and radiologically), ICU admission or death. The latter is based on age and laboratory tests at presentation. We have found that higher age, higher CRP, higher procalcitonin, lower lymphocyte percentage, lower monocyte percentage and lower serum albumin were the most significant predictors of progression of disease, the need for medical support and treatment and the need for ICU admission or death if the score put the patient in the high risk category. The AUC of the model was equal to 0.83 (95% CI, 0.78–0.89), which indicates good discrimination between the groups. It is important to note that this cohort of patients include all the patients who tested PCR positive for COVID-19 in Kuwait, and included asymptomatic contacts of a positive case, and all these patients underwent the laboratory investigations tested in this model at time of diagnosis.

Bacterial infections trigger extrathyroidal synthesis of PCT, which is actively maintained by elevated values of IL-6, IL-1β, and TNF-α, while it has been thought that viral infections will hinder PCT production due to interferon-γ (Schuetz et al. 2011). That is why PCT has been pursued much later after the start of the pandemic than other biomarkers. Today, it seems that PCT is elevated early in those destined to develop severe disease (Hu et al. 2020) and one hypothesis we have formed is that, interferon-γ responses are likely delayed in early COVID19 disease, especially in those destined to progress to severe disease and this is in keeping with our findings. Lymphocyte and monocyte percentages being low could indicate that the virus might directly infect these cells resulting in their death since they express the coronavirus receptor ACE2.Alternatively, the inflammatory cytokines mentioned previously are disordered, perhaps leading to lymphocyte or monocyte apoptosis (Tan et al. 2020). Serum albumin has also been demonstrated by others to be associated with severity in COVID19. Postulated mechanisms include its antioxidant and anticoagulant properties but more relevant may simply be that the same process that signals a defective immune response to COVID19 might lead to an epiphenomenon that triggers a decrease in serum albumin early on in the disease (Violi et al. 2020). Finally, CRP is an acute phase reactant and it is thus expected to be elevated in cases that start off with greater subclinical inflammation and that thus have a tendency to progress (Wang 2020).

Gong et al. (2020), created a COVID-19 severity model based on lab tests of 372 non severe patients who were admitted to three clinical centres in Wuhan. They found that old age, and higher serum lactate dehydrogenase, C-reactive protein, the red blood cell distribution width, blood urea nitrogen, direct bilirubin and lower albumin, are associated with severe COVID-19. Our initial analysis using standard regression techniques also picked up these same variables (except lactate dehydrogenase) but depending on the randomly selected training samples the selection process remained very unstable and selection of the final model was improved by resorting to 5-fold cross-validated lasso logit regression that has the capacity to improve out-of-sample predictions. Another study from Wuhan (Yan et al. 2020) input data from 375 patients in a machine learning algorithm, and found that lactate dehydrogenase, lymphocyte count and CRP as the most significant predictors of severity, which was not confirmed in this study as lymphocyte count per se did not have any predictive value. Another study from China examined 487 patients in Zhejiang Province to establish a score distinguishing high risk patients (Shi et al. 2020). They identified older age, male gender, and presence of hypertension as predictors of severe disease at time of admission. We examined the effects of several comorbidities in our model including hypertension, ischaemic heart disease, chronic respiratory disease, renal insufficiency and diabetes mellitus but found no predictive value over and above age which is consistent with recent data regarding the age component of fatality (Dudel et al. 2020).

A systematic review by Wynants et al identified 31 prediction models related to COVID-19 (Wynants et al. 2020). The review included 10 prognostic models, all using data from China for predicting mortality risk, progression to severe disease, or length of hospital stay. The predictors included in more than one prognostic model from these studies were age (n = 5), sex (n = 2), features derived from CT scoring (n = 5), C reactive protein (n = 3), lactate dehydrogenase (n = 3), and lymphocyte count. We again picked up several of these variables (except lactate dehydrogenase and lymphocyte count) when we ran standard regression modelling but, as explained previously, the model selection process was unstable (different variables selected) when different randomly selected training samples were used. None of these studies based their model on consecutive patients or patients with a diagnosis of COVID-19 not requiring hospital admission. One of the advantages of our study is that and because of regulations in Kuwait, all patients with PCR positivity confirming COVID-19 are admitted to a hospital designated for COVID-19. Kuwait enforced tough measures to identify and isolate patients with COVID-19 since February 24th when the first case was reported in the country from travellers returning from Iran. All patients coming from countries with COVID-19 were quarantined for 2 weeks in quarantine institutions. There is currently a full border lockdown with partial curfew daily for 16 hours as well as closure of schools, Universities, government offices and businesses. These measures along with lockdown and aggressive testing in hot spots in Kuwait, allowed us to capture most patients with the disease in Kuwait which makes our data reported in this study truly representative of patients with COVID-19 at time of diagnosis.

Our predictive model is simple and most importantly determined at time of diagnosis/presentation (Kuwait cohort) which allows for distribution of resources and prioritization. With increase in cases with COVID-19, the health system will not be able to afford to admit all patients even the asymptomatic ones to hospitals, and having such score that is reliable with regards to out-of sample predictions will allow stratification of patients for admission. Also, when more data is available regarding treatment for COVID-19, these prediction models can be used to identify patients at high risk to start treatment early. Therefore, prediction models based on all patients with COVID-19 at the time of diagnosis, will serve the clinical purpose of utilizing rapidly diminishing resources better.

Our moderate to severe group of patients included those who died, were admitted to ICU, received ECMO, received supportive respiratory and renal treatment and stayed for more than 14 days in the hospital while either receiving treatment or demonstrating radiological signs or shortness of breath. This definition allowed us to capture the spectrum patients who truly required hospital support and treatment to allow prioritization of patients and their treatment at time of diagnosis

Although we internally validated our score, a limitation of this study is that it lacks external validation which we will be looking forward to performing with external institutions. Another limitation is the exclusion of patients not achieving the clinical course outcome of the study, including those with recent admission. Strengths include good discrimination and calibration results and use of a machine learning algorithm to improve out-of-sample predictions.

In conclusion, this simple prognostic score provides over-burdened health care systems during the pandemic with a much-needed tool that can stratify patients at diagnosis. This should facilitate the decision making around admission versus home quarantine and will be of importance to the health care needs of the current pandemic. The KPI score calculator is now available online at covidkscore.com

Click here for additional data file.