General Severity of Illness Scoring Systems and COVID-19 Mortality Predictions: Is "Old Still Gold?"

How to cite this article: Siddiqui SS, Patnaik R, Kulkarni AP. General Severity of Illness Scoring Systems and COVID-19 Mortality Predictions: Is "Old Still Gold?" Indian J Crit Care Med 2022;26(4):416-418.

Severity of illness scoring systems in intensive care unit (ICU) are used to assess the disease severity, prognostication and mortality prediction, and with some systems prediction of length of stay and better resource allocation. An ideal scoring system should have easy to record variables, should be well calibrated, have high level of discrimination, and be applicable to all critically ill patients.[1] Widely used general severity of illness scoring systems for critically ill patients include various iterations of mortality prediction model (MPM), acute physiology and chronic health evaluation (APACHE), and simplified acute physiology score (SAPS). Coronavirus disease-2019 (COVID-19)pandemic spreads globally in a short time. About 5–10% of COVID-19 patients with severe pneumonia require ICU admission for ventilatory support.[2] The heterogeneous nature of COVID-19 with pulmonary and extrapulmonary manifestations makes it an enigma for the medical fraternity. Furthermore, long-COVID or post-COVID syndrome is another distinct entity occurring in a minority of patients. These factors make it difficult to predict outcomes in critically ill COVID-19 patients. Any scoring system used to predict mortality in COVID-19 should incorporate parameters of all relevant organ systems.

The study by Asmarawati et al. published in this issue of the journal compared the utility of sequential organ failure assessment (SOFA), quick sequential organ failure assessment (qSOFA), acute physiology and chronic health evaluation (APACHE II), and new early warning signs (NEWS-2) scores estimate mortality of COVID-19 patients requiring intensive care unit (ICU).[3] We would like to underscore a few important observations about this study. Firstly, the 4 scoring systems used in the study are of diverse genre, some of them being used primarily as triaging/screening tools in high workload areas, in-patient wards (qSOFA and NEWS-2) to define the level of care, while SOFA is used to define the various organ dysfunction/failure, while APACHE II takes other factors also into account, like age, comorbid illnesses apart from physiological disturbances (Table 1).

Table 1

Comparison of scoring systems used in the study

	qSOFA	NEWS-2	SOFA	APACHE II
Year of publication	2016	2017	1994	1985
Origin (patient population)	USA	UK	Europe	USA
Abbreviation expansion	Quick sequential organ failure assessment	National early warning score	Sequential organ failure assessment	Acute physiology, age, and chronic health evaluation
Total variables	3	8	6	17
Range of score	0–3	0–23	0–24	0–71
Prime utility locations	ED/Ward/HDU	ED/Ward/HDU	ICU	ICU
Variables/parameters	Clinical	Clinical	Clinical and investigational	History (age, comorbidities), clinical and investigational
Time taken for calculation	Fast	Fast	Medium	Prolonged time
Serial monitoring	Yes	Yes	Yes	Not much used
Remarks	Identify patients with suspected infection who are at greater risk for a poor outcome outside the intensive care unit (ICU)	Early warning system for identifying acutely ill patientsUsed mainly in the UK	Worst values over 24 hours are usedIncludes a treatment-related variable (dose of vasopressor agent)	Worst values over 24 hours are usedNot validated for children (aged <16 years), patients with an ICU stay of less than 8 hours, or those who are readmitted to the unit within the same hospital admissionSeveral patient populations like burn and CABG patients were not included in the original study of designing the score

ED, emergency department; HDU, high-dependency unit; CABG, coronary artery bypass grafting; ICU, intensive care unit

Notwithstanding the differences in the scoring systems, the authors found similar discrimination by all scoring systems with similar area under the receiver operating characteristics (AUROC) curves on day 0 and day 5. The ability of a prognostic index to predict an outcome (e.g. inhospital mortality) is evaluated based on its calibration and discrimination.[4] Calibration refers to the relation between predicted mortality and observed mortality. The calibration of a prognostic model generally deteriorates over time due to changes in ICU admission and discharge criteria, the evolution of intensive care support, and medical advancement. In contrast, discrimination refers to the ability of a prognostic index to differentiate between patients who will and will not survive. This metric is based on the AUROC curve, with a larger area indicative of greater accuracy.

Secondly, while serial use of qSOFA, NEWS-2, and SOFA scores have been well documented, the application of daily APACHE II scores to predict patient survival rate is not used often.[5] A recent study assessing the utility of dynamic APACHE II score to predict outcome of ICU patients concluded that APACHE II score on day 3 of ICU admission is an optimal predictor of outcomes in ICU patients.[6] However, in the current study, APACHE II score on days 0 and 5 was used for mortality prediction; the basis of using day 5 score has not been given by the authors. Though, both the studies are different in terms of their patient population, geographic region, it remains largely speculative which is the optimal day to calculate APACHE II for the best prediction of mortality in COVID-19 patients.

Thirdly, although the study incorporated consecutive sampling, majority were critically ill COVID-19 patients (54.7%) with preponderance of acute respiratory distress syndrome (ARDS) patients (50.9 %) and consequent high mortality (43.4 %).

Lastly, all these 4 scoring systems used in this study involve parameters of multiple organs; however, none of them incorporate any parameter representing immune system. COVID-19 manifestations result from the release of various cytokines, including interleukin (IL)-1 IL-6, IL-12, IL-18, IL-33, interferon (IFN)-gamma, IFN-alpha, tumor necrosis factor (TNF)-alpha, and transforming growth factor (TGF)-beta. The elevation in the neutrophil-to-lymphocyte ratio (NLR) is related to immune disorders in patients with COVID-19. In fact, secondary hemophagocytic lymphohistiocytosis after COVID-19 infection has also been reported. Thus, the key role of the immune system cannot be overemphasized in this context. This is especially important considering the fact that most of the treatments involve immunomodulation. This includes steroids that are the sine qua non for treating severe COVID-19 infections. Not involving the immune system as part of the prediction model for COVID-19 by the authors is probably a missed opportunity.

With the continued rage of COVID-19 pandemic, several bespoke scores for COVID-19 prognostication have also been developed, like Coronavirus Clinical Characterization Consortium (4C) mortality score, COVID-GRAM Critical Illness Risk Score, and CANPT score.[7-9] These COVID-specific prognostic scores are different from other prognostic scores as they also incorporate one aspect of the immune system: for example, 4C mortality score has C-reactive protein; COVID-GRAM and CANPT scores have NLR. Whether incorporation of these parameters will improve their mortality prediction requires larger validation studies and also comparison with the currently available general scoring systems.

Mortality prediction with general scoring systems in a specific disease like COVID-19 may have certain other pitfalls also. Prognostication using the general scoring systems on being applied to a new group of patient population, affected by a new disease, whose pathophysiology largely remains unknown and no definite therapy, is difficult. Additionally, the presence of different strains, likely to need varied approach of management in different parts of the world would confound the ability to predict outcomes. In other words, if a model fits adequately in the first wave of COVID-19, it may not be the appropriate mortality prediction model later as the treatment and disease both evolve.[10]

In the present era of electronic medical record, machine-learning-based methods to predict inhospital mortality can reduce the predicament of varied clinical phenotypes. Linden et al. have shown that this remains to be the future endeavor.[11]

In conclusion, the study findings demonstrate the utility of general severity of illness scoring systems for prediction of mortality in moderate-to-severe COVID-19. Finding unity in diverse manifestations of COVID-19 with general mortality prediction models seems to have a role. Thus, the idiom “old is gold” in this context still holds true, till we get a better scoring system.

Orcid

Suhail S Siddiqui https://orcid.org/0000-0001-7109-0566

Rohit Patnaik https://orcid.org/0000-0002-2321-0743

Atul P Kulkarni https://orcid.org/0000-0002-5172-7619