Illness severity and organ dysfunction scoring in Pediatric Intensive Care Unit.

The illness severity scoring systems provide objective measures for inter- and intra-unit comparisons with time and also provide useful information for comparing the severity of illness of patients, at the time of enrollment into clinical trials. These scores are an essential part of the improvement in clinical decisions and in stratifying patients with poor outcomes. Appropriate application of these models helps in decision-making at the right time and in decreasing mortality. However, it is also important to note that the choice of illness scores should accurately match the setting in which they are designed. In Indian setting, there is no Pediatric Intensive Care Unit illness severity score is designed until now as per our patient profile and resources. The purpose of this review article is to provide an idea regarding the evolution of illness severity scores in developed countries till date along with their utility. This review emphasizes the need for the development of pediatric illness severity score as per the local resources.

Introduction

Critically ill children are characterized by large variations in the normal body homeostasis. These variations can be estimated by the drift of the physiological variables from the normal range. Scores can be constructed from deviations of these drifted variables. Broadly, these scores can be divided into two categories. The first category belongs to the prognostic scores which predict the risk of death at the time of entry into Intensive Care Unit (ICU). The other category is of the descriptive or outcome scores which describe the course of illness after the admission into the ICU. The scoring systems provide objective measures for inter- and intra-unit comparisons with time and also provide useful information for comparing the severity of illness of patients, at the time of enrollment into clinical trials.[1] In this review, two most frequently used predictive scores in Pediatric ICU (PICU) - pediatric risk of mortality (PRISM) and the pediatric index of mortality (PIM) scores and one descriptive score to assess the multiorgan dysfunction, pediatric logistic organ dysfunction score (PELODS) are discussed [Table 1]. Text material was collected by a systematic search in PubMed, Google (1984–2013) for original articles. Few paid articles were obtained from National Medical Library New Delhi, India.

Table 1

Commonly used scoring systems in Pediatric Intensive Care Unit

Prognostic Scores

Pediatric risk of mortality score

PRISM scores are generally used in sick neonates, infants, children, or adolescents. Three versions of PRISM have been published till date. The first version was named as Physiologic Stability Index (PSI) which was a subjective score developed by a panel of intensivists, containing 34 physiological variables from seven physiologic systems of the body.[2] PSI was developed from the Therapeutic Intervention Scoring System which reflects the severity of illness by assessing therapeutic needs. Each variable was assigned a score of 1 (abnormality worth concern but not to change therapy), 3 (need to change therapy), and 5 (life-threatening). As the PSI score contained a large number of physiological variables and it was a subjective score, Pollack published PRISM score (an improved version of the PSI) in 1988.[1] Data were collected from nine PICUs during 1984 and 1985 in North America. The number of physiologic variables had been decreased from 34 to 14, and the number of ranges had been decreased from 75 to 23 compared to PSI. It has been shown that PRISM II score was easier to calculate and is a better reflection of the severity of illness than PSI.[3] One of the major limitation of the PRISM score was its underestimation of deaths after cardiac surgery.

PRISM III, a third-generation was developed in 1996 based on a sample size of 11,165 patients from 32 pediatric ICUs all over North America.[4] Physiologic variables reflective of mortality risk were re-evaluated to update. Age groups were defined as follows: Neonates (0 to <1 month), infants (1-12 months), child (>12-144 months), and adolescent (>144 months). Subscores used were: (1) Cardiovascular and neurologic vital signs: Five parameters (2) acid-base and blood gas: Five parameters (3) biochemistry tests: Four parameters (4) hematology tests: Three parameters (prothrombin time and activated partial thromboplastin time counted as one). PRISM III contains 17 variables and the predictive power of the physiologic variables were objectively assessed and their ranges, eliminating some ranges that did not contribute significantly to mortality risk (e.g., high systolic blood pressure [SBP]), and revising the ranges of the retained physiologic variables. Variables such as temperature, pH, arterial oxygen pressure (PaO2 ), creatinine, blood urea nitrogen, white blood cell count, and platelet count have been added [Table 2]. Although these are important changes, the variables of the greatest importance in outcome prediction are the same in both PRISM and PRISM III such as low SBP, altered mental status, and abnormal pupillary reflexes were retained. The most abnormal value of the variable is to be noted while entering the data during the first 12 h (PRISM III-12) or during the first 24 h (PRISM III-24) after entry into PICU. PRISM III-24 was very well validated with a large sample size involving a lot of different PICUs. Its discrimination capacity to differentiate between the critically ill children who die and those who survive was 0.944 ± 0.021 (area under receiver operating characteristic [AU-ROC] ± standard error of the mean [SEM]) and calibration was excellent (P = 0.5504). Pollack also estimated the value of the PRISM III-12 score. The discrimination capacity and calibration of PRISM III-12 were 0.941 ± 0.021 (AU-ROC ± SEM) and 0.4168, respectively.[4] However, there are several limitations with PRISM. First, many PICUs do not calculate due to its time-consuming process. Second, the units which participated in the validation of this score had over 40% of the deaths in the first 24 h, so there is a danger that the score may diagnose death rather than predicting it. Third, the worst-24-h scores blur the differences between units: A child managed in a well-equipped and high manpower tertiary level center who rapidly recovers will have a score that suggests a mild illness, while the same child who is inadequately managed in a less well equipped and low manpower tertiary level will have a score that suggests severe illness - the less equipped tertiary level ICU's high mortality will be incorrectly attributed to its having sicker patients than the well-equipped unit. Fourth, users have to pay money to get this score resulted in underutilization many countries, outside North America.[56] Validation of PRISM score outside North America had shown mixed results. A study from Pakistan by Qureshi et al.[7] had shown good discrimination and calibration of PRISM III (AUC 0.78 [0.67–0.89]; x2 = 7.49, P = 0.49) in their PICU. A study from china by Choi et al.[8] had shown PRISM III accurately predicted mortality in PICU (AUC 0.79 [0.65–0.98]; P = 0.395). Another study from India by Taori et al.[9] showed good discriminatory performance and calibration with PRISM score. A study by Thukral et al. from India[10] had shown that PRSM underpredicted mortality in their PICU. The likely reasons for underprediction of mortality in their study were attributed to differences in their patient clinical profile, lesser resources, and differences in the quality of care when compared to those ICUs where the score was developed.

Table 2

Pediatric risk of mortality III score

In 1997 Pollack et al. developed a physiology based measure of physiologic instability that has an expanded scale compared with the PRISM III score and called it as the PRISM III-acute physiology score (PRISM III-APS).[11] PRISM III-APS consists 59 ranges of 21 physiologic variables. Data were collected from 32 PICU's (11,165 admissions, 543 deaths). Patients who had PRISM III-APS score of >80 had mortality >97%. However, this score should not be used routinely for quality assessments or calculating risk of individual patients because it is highly sensitive to small changes in physiological status.

Pediatric index of mortality

To overcome problems faced with PRISM III, PIM model was designed. The first version (PIM) was published in 1997[6] and the score was updated in 2003 (PIM2) and 2013 (PIM3). PIM uses eight physiological variables within 1 h of PICU admission. Data were collected from seven PICUs in Australia and one ICU in the UK. The variables used by PIM that are not used by PRISM are the presence of a specified diagnosis; use of mechanical ventilation and the plasma base excess [Table 3]. The score was well calibrated (P = 0.37) and well discriminated (AU-ROC = 0.90).[6] The advantages of PIM score are: It is easy to use and available in the public domain at free of cost. A major limitation of PIM is the effect of treatment given prior to admission to the PICU, and it is represented by a problem called lead time bias, i.e., patients with a given severity-of-illness score may have a higher mortality rate if they have been extensively treated before they are admitted to ICU.[12] However, it was found that the time spent in hospital before admission to intensive care was not statistically significant when added to the PIM model.[6] In developing countries like India, where preadmission management is not well organized as compared to developed countries, it may not affect the assessment of severity of illness by PIM model.

Table 3

Pediatric index of mortality score

PIM score was updated in 2003 which was validated in 20,787 critically ill children from 14 ICUs in Australia, New Zealand, and the UK.[13] PIM2 has 10 variables with the discrimination value of 0.90 (95% confidence interval, 0.89–0.91) and good calibration (P = 0.17). Changes made in PIM2 as compared to PIM are: First, three variables, which provide the main reason for ICU admission, are being added to PIM2: (a) Admission for recovery from surgery or procedure, (b) following cardiac bypass (c) for low-risk diagnosis. Secondly, a variable named “Specific Diagnosis” was replaced by two new variables: “High-Risk Diagnosis” and “Low-Risk Diagnosis.” Third, in “High-Risk Diagnosis,” the criteria for cardiac arrest had been changed, and liver failure was included along with the removal of Intelligent Quotient below 35 [Table 4].

Table 4

Pediatric index of mortality 2 score

The advantage of PIM2 score is that it avoids problems of early treatment bias as it includes only data at entry into the PICU. Its main weakness is that it has not been tested in many countries around the world. A study from Argentenia showed that PIM2 has an adequate discrimination between death and survival but has poor calibration with a reasonable prediction of outcome.[14] Ng et al. from china, showed that the discrimination of PIM1 and PIM2 were satisfactory, but calibration was not possible due to insufficient deaths.[15] Imamura et al. from Japan, found that PIM2 has excellent discriminatory power and good calibration, although it over-predicted deaths.[16] Sankar et al. from India also validated PIM and PIM2 scores in their setup and showed that both PIM and PIM2 scores had good calibration but only acceptable discrimination.[17]

Pediatric index of mortality-3

To ensure the continued applicability of the models, re-calibration using new data should be performed regularly. Hence, PIM3 was developed using data of 53,112 admissions from various PICUs in Australia, New Zealand, UK, and Ireland[18] [Table 5]. The final model well discrimination power (AUC, 0.88, 0.88–0.89); however, in the combined dataset, the model performed better in Australasia than in the UK/Ireland (AUC, 0.92, 0.91–0.93 and 0.87, 0.86–0.88, respectively). Changes made from PIM2 to PIM3 are as follows: (1) Diagnoses influencing the risk of mortality were divided into three categories: Very high-, high-, and low-risk groups. Diagnoses which had odds ratios >5 in the interim multivariable model are classified as very high-risk diagnoses. High-risk diagnoses groups and low-risk diagnoses groups had odds ratios between 1 and 5 and below 1, respectively. In contrast to PIM2, these diagnoses groups were assigned using a categorical variable and patients with multiple weighted diagnoses were assigned to only one group, which have high risk. For example, a patient with hypoplastic left heart syndrome (a high-risk diagnosis) who is admitted with acute bronchiolitis (a low-risk diagnosis) would be coded only as having a high-risk diagnosis. (2) SBP is known to have a nonlinear relationship with the risk of mortality; both very high and very low SBP are indicative of poor health status. SBP 120 was included as a predictor, and where SBP was missing, a value of 120 was used. (3) Two transformations for the value of base excess was considered: The absolute value of base excess and base excess as a quadratic function. Where base excess was missing, a value of zero was used. (4) Four approaches for incorporating PaO2 and FiO2 in the model. (4a) ([FiO2 × 100]/PaO2 ) was calculated in the same manner as PIM2 replacing the ratio with zero if PaO2 or FiO2 was missing; (4b) Replacing the ratio with 0.23 if PaO2 or FiO2 missing, derived from the normal value of PaO2 in air ([0.21 × 100]/90); (4c) The natural logarithm of ([PaO2 /FiO2] × 100) replacing the ratio with 430 if PaO2 or FiO2 missing; (4d) The absolute value of the difference between the calculated ratio ([FiO2 × 100]/PaO2 ) and the normal value (0.23). PIM2 over predicted the risk of mortality in children admitted to ICU in 2010 and 2011.[18] Even though, recalibrating the coefficients improved the performance, cardiac bypass no longer predicted mortality, and the prediction was poor among low-risk patients.

Table 5

Pediatric index of mortality 3 score

Outcome scores or descriptive scores

Descriptive or Outcome scores which describe the course of illness after the admission into PICU. Multiorgan dysfunction syndrome (MODS) is well described by outcome score. Seven organs have been considered in organ dysfunction namely, respiratory, cardiovascular, neurologic, hematologic, renal, hepatic, and gastrointestinal. Wilkinson et al.[19] and Proulx et al.[20] defined the diagnostic criteria of these organ dysfunctions. The diagnostic accuracy of the variables used in these definitions has never been validated, in spite of that; these diagnostic criteria of pediatric MODS are extensively used by practitioners and investigators. In critically ill adults, three quantitative scoring systems estimating the severity of cases of MODS have been developed and validated: The multiple organ dysfunction score,[21] the logistic organ dysfunction score,[22] and the Sepsis Organ Failure Assessment score.[23] There is a direct relationship between the number of organ dysfunctions and the mortality rate in children.[24] However, mortality in the ICU is a not only related to the number of failing systems but also the degree of dysfunction of each system. In fact, the predictive weight of the different organ systems is not similar. For example, the cardiovascular and neurologic systems are more predictive of death than hepatic or renal dysfunction. The relative weight and the severity of the organ dysfunction are not taken into account in the MODS score which may cast doubt on its reliability and its usefulness.

Pediatric logistic organ dysfunction score

Two scores were developed for the assessment of MODS in children in a cohort of 594 patients admitted in three French and Canadian PICUs between January and May 1997.[24] Pediatric multiple organ dysfunction (PEMOD) system and PELOD system included one and several variables, respectively. Severity level score of organ dysfunction was graded from 1 to 4 for the PEMOD system and three levels with scores of 1, 10, and 20 for PELOD system. For both systems, calibrations were good (P = 0.23 and P = 0.44, respectively). The PELOD system was more discriminant than the PEMOD system (AU-ROC curves 0.98 and 0.92, respectively). PELODS was validated by a prospective, observational, multicenter cohort study in seven multidisciplinary, tertiary care PICUs of university-affiliated hospitals (two French, three Canadian, and two Swiss)[25] which included 1806 consecutive patients. PELODS included six organ dysfunctions and 12 variables and was recorded daily for each variable, the most abnormal value each day was used to calculate daily PELOD for first 5 days of stay (dPELOD) and during the whole stay was used to calculate the PELODSs [Table 6]. The discrimination of the PELODS was 0.91 ± 0.01, and the calibration was good (P = 0.54). The discrimination value of the dPELODS was quite good with the AU-ROC curve ranged from 0.79 to 0.85 during first 5 days. PELODS can be used as an outcome measure of clinical trials, the severity of illness of patients treated, a marker of severity of illness in quality assurance and costing studies in PICUs.

Table 6

Pediatric logistic organ dysfunction score

PELODS also has its own limitation like treatment bias may be a problem because the PELODS includes data that can be modulated by the care provided during PICU stay. Thus, the PELODS cannot differentiate between the therapy and severity of disease, but this bias is unavoidable unless one is ready to give no treatment to critically ill children for the ideal score which is unethical. PELODS has not been tested in countries other than Canada, France, and Switzerland. PELODS is not validated to predict post-ICU morbidity, and mortality and further studies are required before the PELODS can be used as a surrogate outcome of post-ICU morbidity and mortality.

Pediatric logistic organ dysfunction score II

PELOD II was designed to update and improve the PELODS, using a larger and more recent dataset of 3671 consecutive patients. Discrimination (AU-ROC 0.934) and calibration (Chi-square test for goodness-of-fit = 9.31, P = 0.317) score were good.[26] The changes made compared to PELODS was the addition of mean arterial pressure and lactatemia in the cardiovascular dysfunction and removal of hepatic dysfunction [Table 7].

Table 7

Pediatric logistic organ dysfunction score 2

PELOD-2 has its own limitations. Data were collected using the set of 8 days (days 1, 2, 5, 8, 12, 16, and 18, plus the PICU discharge) in PICU that were previously identified as the optimal time points for measurement of dPELOD. Hence, an abnormal value of a variable measured on a day outside this predetermined set of days could be missed. PELOD-2 was developed and validated with a dataset that originated from only two countries (France and Belgium) which are different from other parts of the world population.[2728] Thus, the extrapolation to other countries has to be verified. Interobserver variability was not studied and should be evaluated in future studies on new populations.

Conclusion

PRISM, PIM, PELOD were very well validated with respect to short-term outcome (death in PICUs). However, these scores (PRISM and PIM) may not be applicable to developing nations like India as it is different from those nations, where these scores were validated. The reasons were resource limitation, different patient characteristics, and inadequate training of the staff. Validation of these scores in developing countries had shown mixed results. Moreover, clinical profile of our patient population includes infections and malnutrition while genetic disorders, trauma constitute major clinical profile in those nations where the scores are developed. Hence, there is high need to design composite scores for developing nations like India, which include variables like malnutrition, resources, etc., No score discussed in this article was validated to predict or to describe long-term outcomes, like mortality or morbidity observed after PICU stay. It is also needed for us to know, the predictors of mortality and morbidity that can be attributed to ICU-related events in children so as to improve the quality of care for sick children.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.