Evaluating the Ability of PRISM4 and PIM3 to Predict Mortality in Patients Admitted to Pediatric Intensive Care Unit; a Diagnostic Accuracy Study.

Introduction: Limited resources and the large number of children in need of services in the pediatric intensive care unit (PICU) emphasize the need for effective allocation of resources for improving the outcome of at-risk patients. This study aimed to evaluate and compare the accuracy of PRISM4 and PIM3 systems in prediction of in-hospital mortality of patients admitted to PICU.
Methods: The present retrospective cross-sectional study was a diagnostic accuracy study performed on patients admitted to PICU of Qods Hospital, Qazvin, Iran, during one year. Scores of PRISM4 and PIM3 scales were calculated for each patient using the available calculators, and the outcome of patients regarding in-hospital mortality was recorded. Finally, screening performance characteristics of the mentioned scales in prediction of patients' mortality were calculated and reported.
Results: 218 patients with the mean age of 40.68 ± 37.92 (2-160) months were studied (57.8% female). There was a significant direct correlation between PIM3 score and duration of stay in PICU (p < 0.0001; r = 0.259), need for inotropic drug administration (p = 0.001), and mortality rate (p = 0.001). In addition, area under the receiver operating characteristic (ROC) curve of PIM3 and PRISM4 in prediction of mortality among patients admitted to the PICU was 0.939 (95%CI: 0.880 - 0.998) and 0.660 (95%CI: 0.371 - 0.950), respectively (p = 0.001). Based on the findings, the best cut-off point for PIM3 scale in prediction of mortality was the score of 4 and it was estimated to be the core of 8 for PRISM4 scale. Sensitivity and specificity of PIM3 scale in prediction of mortality in the cut-off of 4 points were 100.00 (95% CI: 56.09- 100.00) and 81.51 (95% CI: 75.47- 86.38), respectively. These measures were 42.85 (95%CI: 11.80- 79.76) and 98.10 (95%CI: 94.89- 99.39) for PRISM4 model, which indicates the higher sensitivity of PIM3 system in this regard.
Conclusion: based on the results of the present study, the accuracy of PIM3 is significantly higher than PRISM4 in prediction of in-hospital mortality among patients admitted to the PICU. It seems that considering the 100% sensitivity of PIM3 in prediction of outcome, this model is a better tool for screening patients who are at risk for in-hospital mortality in order to pay more attention and allocate more resources to improve their outcome.

1. Introduction :

Among the most important aims in the pediatric intensive care unit (PICU) are providing quality care, providing relief for the disease without causing any permanent disabilities and side effects, and reducing the mortality of critically ill children. Since the foundation of the first PICU in 1955 in Sweden, considerable advances have been made in the quality of care provided for the admitted patients, which has led to decrease in mortality rate from 8-18% in previous years to 3-5% (average 2.7%) in recent years (1, 2). This considerable achievement is the result of employing the best human resources and efficient use of the most recent diagnostic and therapeutic technologies. Evaluating the quality and efficacy of the measures and assessing the cost-effectiveness of intensive care is very important in efficient ressource allocation among patients (3).

A reliable index of the efficiency of the mentioned unit is the standardized mortality ratio (SMR). SMR is the ratio of mortalities observed in the studied group to the expected mortality in the general public (2). Prediction of the risk of mortality and morbidity in patients admitted to the ICU is very effective in improving the quality of measures taken and decreasing probable errors and can be of great help in allocating the available resources.

In order to estimate the risk of mortality and the prognosis of the patient for allocating resources and evaluating the quality of service, many scoring systems have been introduced to date, among which PRISM (pediatric risk of mortality) and PIM (pediatric index of mortality) are two of the major models used for predicting mortality in the pediatric population (4, 5).

PRISM uses the physiological and laboratory parameters in the first 24 hours of admission and the variables used in PIM include clinical and laboratory indices, underlying illnesses or surgical conditions leading to hospital admission. In the studies performed in various population, the results of evaluating the two models for determining which is superior in determining prognosis have been different (5). The study performed by EA Ozer et al. in 2004 on 105 babies less than 1-year-old in PICUs of hospitals in Turkey showed that PIM was a better scale than PRISM for predicting mortality in countries with a higher rate of mortality and underlying disease (6). Meanwhile, in the study by Rajia et al. on 50 children in Egypt in 2019 showed that both systems had good predictions but PRISM was a better predictor (7).

Therefore, the aim of this study was evaluating and comparing PRISM4 and PIM3 systems in predicting in-hospital mortality of patients admitted to PICU.

2. Methods:

The present retrospective cross-sectional study is a diagnostic accuracy study performed on patients admitted to the PICU of Qods Hospital, Qazvin, Iran, from May 2019 until May 2020. The scores of PIM3 and PRISM4 scales were calculated for each patient using the available calculators and the outcome of patients regarding in-hospital mortality was recorded. Finally, the screening performance characteristics of the mentioned scales in prediction of patient outcomes was calculated. The protocol of the study was approved by the ethics committee of Qazvin University of Medical Sciences (ethics code: IR.QUMS.REC.1399.182). The researchers adhered to the principles of ethics in biomedical research and observed confidentiality of patients’ data throughout the study.

2.2. Participants

All the patients over 1 month and less than 12 years old, admitted to PICU of Qods Children Hospital in Qazvin, Iran, were included in the study during the mentioned time using census method for sampling. Patients with the ICU stay of less than 12 hours or mortality within less than 12 hours were excluded from the study. In addition, profiles with missing or incomplete data regarding the required variables were also excluded from the study. It should be noted that the PICU of the hospital is managed with a closed system under the supervision of a fellowship of intensive care.

2.3. Data gathering

Demographic and baseline characteristics of patients including age, sex, initial diagnosis, underlying illnesses, duration of hospitalization in the ICU, and all clinical and laboratory variables required for calculation of PIM3 and PRISM4 scores, based on their latest versions (8, 9), were extracted from their clinical records and recorded in a designed checklist. In addition, outcomes such as need for mechanical ventilation, need for inotropic drug administration, duration of PICU stay, and in-hospital mortality after 12 hours were recorded for all patients. Then PIM3 and PRISM4 scores were calculated and recorded for all patients by a senior resident under the supervision of a pediatrician using online calculators and entering the required variables based on the patients’ condition upon admission. PIM and PRISM are two clinical decision rules or scoring systems that have been designed for predicting mortality among patients admitted to PICU and have changed during the recent years, the last versions of which are available to researchers and practitioners under the names PRISM4 and PIM3. In a multi-center study published in 2019, which was performed on 10078 patients admitted to PICU, PRISM3 was revised and after undergoing changes for improving the accuracy of predictions and having less percent error it was reintroduced under the name PRISM4.

2.4. Outcomes

The main outcome evaluated in the present study was in-hospital mortality of patients admitted to PICU. In addition, the duration of PICU stay and need for inotropic drugs were also studied as secondary outcomes.

After filling out the checklists, the data were entered to SPSS software version 23 and analyzed. Data are reported as mean ± standard deviation (SD), frequency (%), or median (interquartile range (IQR)). Based on the distribution of samples in the population, correlation coefficient, chi square, t-test, or their non-parametric counterparts were used and, in all comparisons, level of significance was considered to be less than 0.05. To calculate the accuracy of the two mentioned systems in evaluation of patients’ outcomes, area under the receiver operating characteristic (ROC) curve was used and the screening performance characteristics of the two studied scoring systems in the best cut-off points extracted from the ROC curve were calculated with 95% confidence intervals (CI) using VassarStats online calculator and reported.

3. Results:

3.1. Baseline characteristics of the studied patients

218 patients with the mean age of 40.68 ± 37.92 (range: 2-160) months were studied (57.8% female). Table 1 shows the baseline characteristics of the studied patients. The most common chief complaints of patients on admission were respiratory problems (29.8%), and eating disorders (17.9%). The most common underlying illnesses among the studied cases were endocrine diseases (13.8%) and musculoskeletal diseases (13.3%). Mean duration of PICU stay was 5.94 ± 4.97 (Range: 1-45) days. Median and IQR of PRISM4 and PIM3 scores in the studied patients were 1.6 (0.7 – 3.1) and 1.0 (1.0 -1.0), respectively. The rate of mortality among patients admitted to PICU in the present series was estimated to be 3.2% (7 cases).

Table1

Characteristics of the studied population

Variable	Value
Age (months)	40.68 ± 37.92
Gender
Female	126 (57.8)
Male	92 (42.2)
Diagnosis
Medical	199 (91.3)
Surgical	19 (8.7)
Underlying disease
Immunodeficiency	3 (1.4)
Malignancy	8 (3.7)
Cardiovascular disease	16 (7.3)
Hematologic disease	10 (4.6)
Endocrine disorders	30 (13.8)
Renal disease	10 (4.6)
Musculoskeletal disease	29 (13.3)
Vital signs
Temperature (Celsius)	37 (36.6 – 37.7)
Systolic blood pressure(mmHg)	107 (97 – 116)
Diastolic blood pressure(mmHg)	64 (56 – 75)
Heart rate (in a minute)	132 (116 – 154)
Glasgow coma scale	15 (15 – 15 )
Outcomes
Recovered and discharged	203 (93.7)
Transferred to other hospitals	8 (3.7)
Dead	7 (3.2)
In need of mechanical ventilation	21 (9.6)
In need of inotropic drugs	62 (28.4)
Score
PRISM4	1.6 (0.7 – 3.1)
PIM3	1.0 (1.0 -1.0)

Findings are presented as mean ± standard deviation, number (percentage) and median (interquartile range).

3.2. Evaluating the predictive value of the studied models

There was a significant direct correlation between PIM3 score and duration of stay in PICU (p < 0.0001; r = 0.259), need for inotropic drug administration (p = 0.001), and mortality rate (p = 0.001). However, the correlation between PRISM4 score and duration of stay in PICU (p = 0.697; r = 0.27), need for inotropic drug administration (p = 0.139), and mortality rate (p = 0.107) was not significant.

Area under the ROC curve of PIM3 and PRISM4 in predicting the need for inotropic drugs among patients admitted to PICU were 0.659 (95%CI: 0.575 – 0.743) and 0.558 (95%CI: 0.469 – 0.646), respectively (figure 1). Based on the findings, the best cut-off point for PIM3 scale in prediction of need for inotropic drugs was the score of 4 and for PRISM4 scale, it was estimated to be the score of 8.

Figure 1

The area under the rock curve of PIM3 and PRISM4 criteria in predicting mortality (A) and the need for inotropic drugs (B) in patients admitted to the pediatric intensive care unit

In addition, the area under the ROC curve of PIM3 and PRISM4 in prediction of the mortality of patients admitted to PICU were 0.939 (95%CI: 0.880 – 0.998) and 0.660 (95%CI: 0.371 – 0.950), respectively (figure 1). Based on these findings, the best cut-off point for PIM3 scale in prediction of mortality was the score of 4 and for PRISM4 scale, it was estimated to be the score of 8.

Table 2 shows the screening performance characteristics of the two studied scales in prediction of mortality at their best cut-off points. Sensitivity and specificity of PIM3 model in prediction of mortality at the cut-off point of 4 were 100.00 (95% CI: 56.09- 100.00) and 81.51 (95% CI: 75.47- 86.38), respectively. These measures were 42.85 (95%CI: 11.80- 79.76) and 98.10 (95%CI: 94.89- 99.39) for PRISM4, respectively, which indicates the higher sensitivity of PIM3 system for the mentioned purpose.

Table 2

Screening performance characteristics of PIM3 and PRISM4 scoring systems in predicting the mortality of patients admitted in pediatric intensive care unit

Character	PIM3	PRISM4
TP	7	3
TN	172	207
FP	39	4
FN	0	4
Sensitivity	100.00 (56.09- 100.00)	42.85 (11.80- 79.76)
Specificity	81.51 (75.47- 86.38)	98.10 (94.89- 99.39)
Positive predictive value	15.21 (6.83- 29.48)	42.85 (11.80- 79.76)
Negative predictive value	100.00 (97.27- 100.00)	98.10 (94.89- 99.39)
Positive likelihood ratio	0.179 (0.089 – 0.358)	0.75 (0.25 – 2.18)
Negative likelihood ratio	0 (0 – NaN)	0.019 (0.007 – 0.051)

Data are presented with 95% confidence interval. NaN: the calculation cannot be performed because the entered values include one or more instances of zero.

4. Discussion:

Based on the results of the present study, the frequency of in-hospital mortality in the studied PICU was 3.2% (7 cases out of the 218 cases admitted during one year). The accuracy of PIM3 system was significantly higher than PRISM4 system in the prediction of in-hospital mortality.

It seems that PIM3 system is a better screening tool for patients at risk of in-hospital mortality considering its 100% sensitivity. Unlike PRISM4, PIM3 system has a significant and direct correlation with duration of PICU stay as well as the need for inotropic drugs as patients with longer durations of PICU stay and those in need of inotropic drugs had higher PIM3 scores.

In accordance with the present results, a study by Lopez (2018) demonstrated that PIM can adequately predict the mortality rate in ICU patients (10). Lee et al. in a study in 2017 showed that PIM3 score is a good predictor of the outcome for patients under 18 but not as efficient for hemato-oncologic patients (11).

The performed study by EA Ozer et al. in 2004 on 105 babies less than 1-year-old in PICUs of hospitals in Turkey showed that PIM system is a better prognostic score than PRISM to predict mortality in countries with higher mortality rate and high prevalence of pre-existing medical conditions (6).

A more recent study on 190 under 18 patients showed that PIM is a more accurate predictive score than PRISM score (12), in contrast to the study by Ragia et al. in Egypt which stated both of the systems were effective but PRISM was a better predictor (7).

Another study performed on 145 PICU patients in India observed that PRISM3 score was associated with higher mortality rate and longer ICU stays, but it did not reliably estimate the survival rate, since sometimes patients with a very high PRISM3 score survived (11).

Balkin morel et al. studied 14268 patients diagnosed with pulmonary hypertension in 153 PICUs in America and concluded that PIM2 and PRISM3 are both efficient scores for early identification and early treatment of high-risk patients and they can be used to update children’s parents on their treatment progress and outcome (13). Same results were obtained in Gandhi et al ‘s study in 2013. They proved that PIM2 can help predict the mortality rate in children and counsel their parents in the earliest stages of disease (14).

Kadivar et al. declared that PRISM score measures the disease severity and identifies the triage level of the children who derive greater benefits from PICU services (15). These results support the results of Costa et al ‘s study in Brazil (16).

According to the present study it appears that PIM3 system by a cut-off point of 4 is an accurate tool to help predict the likelihood of mortality and triage level of patients who are at lower risk.

Our study suggests that sensitivity and specificity of this version of PIM enabled this scoring system as an effective clinical decision-making tool, because it can determine which patients actually need ICU admission and benefit the most from treatment in an intensive care unit. Meanwhile PRISM score had a greater specificity but very low sensitivity and this means PRISM score has a low positive predictive value for screening and predicting the outcome in the present study. On the other hand, the variables of PIM system require less laboratory testing, meaning It may be a faster and more efficient system to use.

In our study PRISM system lacked the efficiency to predict the prognosis of PICU patients. This matter could be due to prevalence of pre-existing conditions and different types of medical conditions leading to hospital admission. There is therefore a definite need for multi-center studies with bigger sample sizes to settle this claim.

Overall, clinical decision-making system is improving day by day to increase the quality of care and It seems that the future of clinical decision-making could be highly dependent on these scoring systems to enhance the patient’s outcomes.

5. Limitations

These findings are limited by the use of retrospective design because as we mentioned earlier profiles with missing or incomplete data regarding the required variables were excluded from the study. Further studies with bigger sample sizes may help validate these results.

6. Conclusion:

Based on the findings of present study PIM3 is significantly more accurate in predicting the in-hospital death of PICU patients than PRISM4 .It seems that considering the 100% sensitivity of PIM3 system in prediction of outcome, this model is a better screening tool for patients at risk of in-hospital mortality that actually need ICU admission and specialized care or need to be transported to a more equipped hospital and also allocate more resources to them to prevent their death and improve their outcome.

7. Declarations:

7.1. Acknowledgement

We would like to express our appreciation to the ICU staff in Qods Hospital in Qazvin for their valuable effort during the conduction of the study.

7.2. Author contributions

ViC: devised the study, the main conceptual ideas and proof outline.

HH and AM: Technical details, overall direction and planning

SJA and VeC: Data gathering and calculating the scores

MSM: Analysis and interpretation of the data

All authors read, discussed the results, commented on, and approved the final version of manuscript.

7.3. Funding

This study is not funded.

7.4. Conflicts of interest:

The authors declare no conflicts of interest in this study.