The prediction of mortality influential variables in an intensive care unit: a case study.

The intensive care units (ICUs) are among the most expensive and essential parts of all hospitals for extremely ill patients. This study aims to predict mortality and explore the crucial factors affecting it. Generally, in the health care systems, having a fast and precise ICU mortality prediction for patients plays a key role in care quality, resulting in reduced costs and improved survival chances of the patients. In this study, we used a medical dataset, including patients' demographic details, underlying diseases, laboratory disorder, and LOS. Since accurate estimates are required to have optimal results, various data pre-processings as the initial steps are used here. Besides, machine learning models are employed to predict the risk of mortality ICU discharge. For AdaBoost model, these measures are considered AUC= 0.966, sensitivity (recall) = 87.88%, Kappa=0.859, F-measure = 89.23% making it, AdaBoost, accounts for the highest rate. Our model outperforms other comparison models by using various scenarios of data processing. The obtained results demonstrate that the high mortality can be caused by underlying diseases such as diabetes mellitus and high blood pressure, moderate Pulmonary Embolism Wells Score risk, platelet blood count less than 100000 (mcl), hypertension (HTN), high level of Bilirubin, smoking, and GCS level between 6 and 9.

Introduction

The intensive care units (ICUs) are considered for the cure of the patients who require extensive monitoring status and parameters of their health and physiological [1], so it demands expert personnel, state-of-the-art facilities, emergency equipment for treatment, monitoring patients admitted to this unit, and it is much more specialized than other sections [2]. As a result, this hospital unit tends to play an important role and is considered one of the most high-priced parts of all hospitals [3-5]. ICU mortality prediction plays a significant part in the patient’s health care and allocation of hospital resources, leading to the improvement of patient survival [1]. Furthermore, it supplies a method to assess the performance of various medical facilities, services, and differences among them. It not only contributes in declining discrepancies of health care but also reduces the hospital costs due to discharging patients from the ICU as soon as possible [6, 7]. A growing number of researchers allocated a great deal of time to study to improve the accuracy in identifying ICU patient mortality since a huge amount of data, and a variety of complex attributes are produced every day in ICUs. This can be challenging to develop new algorithms in ICU mortality prediction. New technologies based on machine learning are considered one of the well-known modeling methods, including artificial neural network (ANN), support vector machine (SVM), and random forest (RF). These algorithms are capable of predicting the results more precisely and correctly for patients than other methods. Many researchers use machine learning to develop models for predicting complex clinical scenarios like ICU settings because clinical attributes and outcomes have nonlinear relationships, so the machine learning algorithms can demonstrate these nonlinear relationships [8-10]. In this work, machine learning methods such as decision tree (DT), neural network (NN), K-nearest network (K-NN), random forest (RF), linear regression(LR), AdaBoost, Bayesian Boosting, Vote (DT+K-NN), and Vote (DT+K-NN+LR) are employed to evaluate the rate of patients mortality who are admitted in ICUs. The prior studies used a rich set of features that is available at discharge. In contrast, we used commonly available data at admission time. The data includes the following (see Table 1): demographic information (age, gender, and skin type). The body mass index (BMI) is a value in which weight in kilograms (kg) relates to height in meters squared. It is divided into four groups by the National Institutes of Health (NIH): the normal weight which is 18.5 to 25 kg/m2, obesity, overweight, and underweight which have the BMI of more than 30, 25, to 30 and less than 18.5 kg/m2 respectively [11]. The Pulmonary Embolism Wells Score is a score of risk classification and clinical decision rule to predict the critical pulmonary embolism (PE) probable in patients who had the possibility of acute PE in their history and examination. A score is classified into three categories: a score greater than 6, a score between 2 and 6, and a score less than 2, considered “high risk,” “intermediate risk,” and “low risk” of PE, respectively [12]. The Wells score is a number that shows the risk of developing deep vein thrombosis (DVT) in the body. DVT occurs when blood clot forms in a vein that is deep inside the body. The Wells score is calculated based on several factors. Using this score, the doctor can determine the existence of DVT. This helps doctors in deciding whether to proceed with further diagnostic tests, such as CT scans. Deep Vein Thrombosis (DVT) is a normal complication in trauma patients, although there is a controversy concerning the appropriate use of venous duplex surveillance. It is mainly used for DVT identification. The Wells criterion is a clinically scoring system in an outpatient setting that offers the possible range between 2 and 8. It accounts for three risk classes: high with the range of 3 points or more, intermediate ranging from 1 to 2 point, and low, including less than 1 point. This is also sometimes mentioned a Hamilton score [13-15]. Paraplegia is an impairment of the legs and lower body resulting from injury to nerves in the lumbar or thoracic vertebrae areas of the body. Hemiplegia is an impairment of one vertical half of the body. Stroke is the most common cause of it. Anemia is a condition that the body lacks enough healthy red blood cells to transfer sufficient oxygen into the body’s tissues. Sensory processing disorder is also a condition when the brain has trouble in the process of sensory information (includes touch, taste, hear, smell, or see things), both in receiving the information and responding to. It means people who are too sensitive in their typical environment. Such things can be painful or overwhelming. The normal hemoglobin range for men, in general, is defined between 13.2–16.6 grams (g) of hemoglobin per deciliter (dL) of blood and between 11.6 and 15. G/dL for women. Anemia is a situation in which the hemoglobin level is lower than normal [16]. Bilirubin is an orange-yellow substance in the body that is generally formed during the normal breakdown of red blood cells. At first, it goes through the liver and then excretes out of the body. Normal Bilirubin level is less than 1.2 (mg/dL), and the higher level of Bilirubin is an indicator of different types of liver problems. A bilirubin test evaluates the amount of Bilirubin in the blood. It contributes to discovering the root cause of health conditions such as jaundice, anemia, and liver disease [16]. Serum Creatinine is a waste substance in the blood that arises from the activity of the muscle. The kidneys normally remove it from the blood; the creatinine level rises when the kidney function slows down. Its normal level in the blood in adults is from 0.5 to 1.2 milligrams (mg) per deciliter (dL). Severe kidney impairment is a sign when the Creatinine level reaches 2.0 or more in adults. Platelet blood test measures the number of platelets in the blood. The average platelet count blood test is commonly considered a part of a complete blood count (CBC) test. Platelets assist in healing blood wounds and preventing excessive bleeding. High or low platelet levels are indicators of severe conditions. Its normal range is between 150,000 and 450,000 platelets per microliter (mcL) of blood. Having more than 450,000 platelets is a state which is known as thrombocytosis; having less than 150,000 is called thrombocytopenia.

Table 1

Medical variable along with its definition which is used in this work

Variable	Definition
Hemiplegia/paraplegia	Paraplegia is an impairment of the legs and lower body resulting from injury to nerves in the lumbar or thoracic vertebrae areas of the body. Hemiplegia is the impairment of one vertical half of the body.
Anemia	It is a condition when the body lacks enough healthy red blood cells to transfer sufficient oxygen to the body’s tissues. The normal hemoglobin range for men, in general, is defined between 13.2–16.6 grams (g) of hemoglobin per deciliter (dL) of blood and between 11.6–15. G/dL for women.
Motor sensory disorder	It is a condition when the brain has trouble in the process of sensory information
Bilirubin	It is an orange-yellow substance in the body that is generally formed during the normal breakdown of red blood cells. Normal Bilirubin levels are less than 1.2 (mg/dL), and higher Bilirubin levels are an indicator of different types of liver problems.
Serum Creatinine	It is a waste substance in the blood that arises from the activity of muscle. Its normal level in the blood in adults is from 0.5 to 1.2 milligrams (mg) per deciliter (dL). Severe kidney impairment is a sign when Creatinine levels reach 2.0 or more in adults
Platelet blood count	Platelet blood test measures the number of platelets in the blood. High or low platelet levels are indicators of severe conditions. Its normal range is between 150,000 and 450,000 platelets per microliter (mcL) of blood.
Body mass index (BMI)	It is a value in which weight in kilograms (kg) relates to height in meters squared. The normal BMI range is from 18.5 to 25 kg/m2. Obesity, overweight, and underweight have a BMI of more than 30, 25, to 30, and less than 18.5 kg/m2.
Pulmonary Embolism Wells Score	It is a score of risk classification and clinical decision rule to predict the acute pulmonary embolism (PE) probable in patients who had the possibility of acute PE in their history and examination. A score is classified into three categories: a score greater than 6, a score of 2 to 6, and a score less than 2 considering as “high risk,” “intermediate risk,” and “low risk” of PE respectively.
Deep vein thrombosis (DVT)	A normal complication in trauma patients. It accounts for three risk classes: high with the range of 3 points or more, intermediate ranging from 1 to 2 points, and low including less than 1 point.
Hypertension (HTN)	A medical condition in which the blood pressure is continuously raised in the arteries. The range of 100–130 millimeters mercury (mmHg) systolic, and 60–80 mmHg diastolic is normal blood pressure at rest for most adults.
Glasgow Coma Scale (GCS)	It is a neurological scaling system describing the brain’s consciousness level and its changes to traumatic brain injury patients and further assessment. The criteria of the scale for assessing a person’s consciousness are ranging from 3 (a sign of deep unconsciousness) to 15 (normal scale of consciousness)
Intubation	It is a procedure that is used when one cannot breathe on their own
Diabetes mellitus	It is considered a metabolic disease that can result in high blood sugar. Normal blood sugar is 70–99 (mg)/(dl) and high blood sugar is between 80 and 130 (mg) and (dl) when fasting.
Multiple sclerosis (MS)	It is a disease that can potentially disable the brain and central nervous system (called the spinal cord).
Nutrition	It is the overall food that a person or other organism uses to maintain, grow, reproduce, health, and disease of an organism. It is divided into low in calories, NPO, and lose weight
Mobility	Is an average movement of patients during hospitalization. It includes agitation, slightly limited, completely immobile.
Bone fracture	It is a medical condition where the continuity of the bone is broken.
Cerebrovascular accident (CVA)	It is a medical condition in which blood cells’ flow in the brain suddenly deteriorates and stops suddenly.

Hypertension

This is a medical condition in which the blood pressure is continuously raised in the arteries [17]. It is also named high blood pressure (HBP) and normally does not cause symptoms [18]. However, in the long-term, it can be a major risk factor for disease including atrial fibrillation, vision loss, stroke, coronary artery, peripheral arterial, chronic kidney, heart failure, and dementia [19-22]. The systolic pressure and diastolic pressure are two measurements expressing the maximum and minimum of Blood pressure [18]. The range of 100–130 millimeters mercury (mmHg) systolic and 60–80 mmHg diastolic are normal blood pressure for most adults [23, 24]. For most adults, the blood pressure at or above 130/80 or 140/90 mmHg is a sign of high blood pressure [23, 25]. The Glasgow Coma Scale (GCS) is a neurological scaling system describing the brain's consciousness level and its changes to traumatic brain injury patients. It is a tool that needs nurses who equipped themselves with the latest information on its function. The criteria of the scale for assessing a person’s consciousness are ranging from 3 (a sign of deep unconsciousness) to 15 (normal scale of consciousness) [26]. Diabetes mellitus is considered a metabolic disease that can result in high blood sugar. Sugar is carried by hormone insulin from the blood into the body cells either to be stored or produced. Without diabetes, not only the body can make enough insulin, but also it can use insulin effectively. It is commonly known as diabetes; its normal range is 70–99 (mg)/(dl), and high blood sugar is between 80 and 130 (mg)/(dl) when fasting [27].

.The doctor puts a tube down the throat and into the windpipe to make it easier to get air into the lungs. The ventilator pumps the air with extra oxygen, and then, it helps breathe out air full of carbon dioxide (CO2). Multiple sclerosis (MS) is a disease that can potentially disable the brain and central nervous system (called spinal cord). In MS, the protective sheath (myelin) covers nerve fibers. The protective sheath is in danger of communication problems. Eventually, the disease can cause permanent damage or nerve deterioration [28]. Nutrition or diet is the overall food that a person or other organisms use to maintain, grow, reproduce, and healthy. Diet usually refers to the consumption of exact nutrition for some reasons, including health- or weight-management. It is divided into “low in calories,” “nothing by mouth” is a medical instruction that withholds food and fluids (known as nil per os (npo or NPO)) and loses weight. Mobility is an average movement of patients during hospitalization, which can be included: agitation, slightly limited, completely immobile. A bone fracture is a medical condition where the continuity of the bone is broken. A cerebrovascular accident (CVA) is a medical condition in which the flow of blood cells in the brain impairs and stops suddenly. It is also mentioned as a stroke. Stroke can weak one part of the body with partial or complete loss of voluntary movement or sensation in a leg or an arm such as numbness, speech, vision and balance problems, weak face muscles, and even unconsciousness(see Table 1). The remainder of this work is structured as follows: related work is represented in Section 2, followed by Section 3, which describes the proposed method, along with the data selection and the strategy of model evaluation. We also discussed the results and the underlying analyses in Section 4, and in Section 5, the conclusions and future works are drawn.

Literature review

The ICU commonly admits the most severely ill patients who need a particular cure or continuous monitoring. The most advanced monitoring equipment and device of the hospital are gathered inside the ICU. It plays a vital role in improving treatment and declining mortality. Mortality prediction in ICU can reflect the severity of disease or the prognosis of patients, get the reasonable allocation of clinical resources, and help clinicians make the right decision. Hence, the prediction of mortality for ICU patients is always one of the most remarkable topics in medical and healthcare research, which has attracted many scholars these days [29]. A growing number of them assess patient mortality with their organizations’ data source [30-34].

In contrast, public datasets are used by the rest [35-37]. MIMIC-II (The Multi-parameter Intelligent Monitoring in Intensive Care) database can be cited as an excellent example for such studies [38, 39]. It is one of the most available databases, including physiological signals and comprehensive clinical data for a cohort of ICU patients. Researchers carried out a considerable amount of research to predict the patient’s mortality in this section. In this section, we will review recent studies and methods in this area proposed by the researchers. Fika et al. (2018) [40] tried to develop and validate a new multivariate logistic regression model in order to predict mortality in an intensive care unit (ICU). Their model improved both the performance and the predictive capability regarding ICU mortality. Shukeri et al. (2017) [30] suggested a logistic regression model to demonstrate the biomarker’s potential. They aimed to use logistic regression to obtain a prediction model for sepsis mortality score (SMS). They made use of a multimarker approach including leukocytes count, PCT, IL-6, and PON and ARE activities of PON-1 for 30-day mortality in sepsis and compared its performance to the Sequential Organ Failure Assessment (SOFA) score. The mortality rate that they reached was 28.9%. Liu et al. (2018) [41], proposed ICU mortality prediction model. Their model includes pre-processing, feature extraction, predictive model, and parameter optimization. The stress was on data of high dimensionality and imbalance distribution that in feature extraction stage, a cost-sensitive principal component (CSPCA) strategy was suggested to handle these problems. The best AUC performance (0.77) belongs to support the vector machine (SVM). Davoodi et al. (2018) [35] used a modified deep rule-based fuzzy model (DRBFS) in forecasting the patient’s in-hospital mortality rate by MIMIC-III dataset. A supervised fuzzy clustering technique for training was employed. Their results demonstrated the superiority of DRBFS among various classifiers such as na¨ıve Bayes (NB), decision trees, gradient boosting (GB), deep belief networks (DBN), and D-TSK-FC. Ding et al. (2018) [31] showed a procedure of two-step JITL-ELM to mortality prediction for patients in ICU in which applied just-in-time learning (JITL) and extreme learning machine (ELM). The first step is clustering and the second one is for mortality prediction. Their model achieved 0.8568 AUC, which outperformed the existing traditional model such as ELM, LR, and BP neural network. Darabi et al. (2018) [32] extracted all the necessary information by using a subset of MIMIC III and combined gradient boosted trees and deep neural networks to estimate the mortality risk of patients who are admitted into an ICU. They used the AUC index to validate the proposed algorithm, and it was around 87.30 AUC on the 10% test set. Meadows et al. (2018) [42] have researched patients admitted to the ICU after cardiac surgery over a period of 10-year to predict LOS (length of stay) at the hospital. Logistic regression analysis was carried out to formulate. By using this approach, patients separate into two groups: less than 48 h and more than 48 h named short stay and long stay; the overall accuracy was 79.77%. Kutyrev et al. (2019) [43] used machine learning and neural network methods to predict patients mortality. Models such as CHF, RF, NB, DT classifier, and multilayer perceptron were used. The performance was improved by using machine learning methods (97% ROC curve). Five algorithms such as SVM, elastic-net penalized logistic regression, RF, NN, and stochastic gradient boosting were assessed by Karhade et al. (2019) [44] for prediction of 90-day mortality in a spinal epidural abscess (SEA). The first digital clinical prediction tool was established for mortality spinal epidural abscess. A logistic regression model was presented to predict the risk of ICU discharge based on snapshot measurements by MIMIC-II dataset by Xuea et al. (2019) [19]. They compared various widely used imputation methods and their impact on predictive models. The best AUC of their methodology was 0.661. Lin et al.(2019) [37] compared RF with other machine learning algorithms, including ANN and SVM models, and customized SAPS II model for acute kidney injury (AKI) patients in the ICU to obtain the best model for mortality prediction. They showed the RF model has the lowest Brier score (0.085, 95%, CI: 0.084–0.086) and the largest AUROC (0.866, 95%CI: 0.862–0.870). They achieved an acceptable accuracy and F1 score for RF among the models. Todd et al. (2019) [33] assessed high-frequency data and the performance of models based on model accuracy, calibration, and discrimination to improve ICU scoring systems. Three logistic regressions (baseline baseline + pulse baseline + MAP) were used. As a result, they have proved that the scoring system (APACHE III) can be raised by better frequency. Deliberato et al. (2019) [34] proposed a mortality prediction model for ICU by taking advantage of logistic regression. The dataset included two group patients: the former, namely the HIAE group, in which patients were elderly and had high BMI and lower rates of smoking and alcoholism compared to the latter group (HMVSC group). The threat of hospital mortality was higher in the latter group (18.7%) compared to the former group (8.4%). The use of the least absolute shrinkage and selection operator (LASSO), random forest (RF), gradient boosting machine (GBM), and the traditional logistic regression (LR) algorithms has been proposed by Kong et al. (2020) [45], to predict the risk of in-hospital death for sepsis patients. Data were collected from MIMIC III as an ICU database including the diagnostic codes, vital signs, laboratory tests, demographics, and some other clinical characteristics of each patient between 18 and 90 years old. Gradient boosting machine (GBM) model showed the best performance, and the AUROCs of GBM model was 0.845. They reached the decedents of patients who were older and they had higher SOFA and SAPS II scores compared to the survivors of patients. The results lead to more prolonged ICU stays and hospital stay reduction. Kang et al. (2020) [46], by using -nearest neighbor (KNN), support vector machine (SVM), multivariate adaptive regression splines (MARS), random forest (RF), extreme gradient boost (XGB), and artificial neural network (ANN) algorithms (10-fold cross-validation)for patients undergoing CRRT for AKI, have shown the power of machine learning algorithms. The used parameters include age, sex, application of mechanical ventilation, and co-morbidities including diabetes mellitus, hypertension, myocardial infarction, chronic heart failure, stroke, peripheral vascular disease, dementia, chronic obstructive pulmonary disease, connective tissue disease, peptic ulcer disease, cancer, ischemic heart disease, chronic kidney disease, atrial fibrillation, and Vital signs, such as mean arterial pressure, heart rate, respiratory rate, and body temperature were collected. As a result, random forest (RF) by Gini impurity to explore the necessary variables demonstrated better results than other machine learning models and also was the best among APACHE II, SOFA, and MOSAIC. Assaf et al. (2020) [47] aimed to explore the risk scale of factors affecting critical COVID (envisage patients with bacteria P. aeruginosa in the ICU) by making use of neural network, random forest, and classification and regression decision tree (CRT) models. They assessed clinical, hematological, and biochemical factors at admission. Different features including medical history, current complaints, measurement of vital signs, baseline testing for blood count, kidney and liver function tests, and inflammatory markers were selected in this work and SVM achieved the best performance and the best accuracy rates of 0.92%. Random forest regression, one of the best machine learning algorithms, is applied by Ripoli et al. (2020) [48], to envisage the candidemia possibility among patients admitted in IMWs. They collected demographic and clinical data of patients and reached C-statistics = 0.874 ± 0.003, sensitivity=84.24% ± 0.67%, and specificity =91% ± 2.63%.In their work, in-hospital MHIA therapy has been proven a frequent risk factor for candidemia as well as other factors such as TPN, previous hospitalization, previous antibiotic therapy, and CVC or PICC.5. Rodr´ıguez et al. (2020) [49] suggested supervised classification methods for mortality prediction among patients with sepsis. In their work, the best discrimination is provided by the supervised classification method via physiological variables. Some techniques were used in their work: decision tree, random forest, artificial neural networks (ANN), and support vector machine (SVM) models. The data was included in patients >18 years. They used two databases: the former includes clinical care variables in which random forest demonstrated the best performance; the latter is direct physiological variables. The best accuracy was achieved by the neural network (ANN) model with an AUC-ROC of 0.69 (95%CI: 0.62; 0.76) and AUC-ROC of 0.69 (95% CI: 0.61; 0.76), respectively. Liao et al. (2020) [50] tried to establish a prediction method for patients with pneumonia. They applied machine learning (K-nearest neighbors, Naive Bayes, decision tree, neural network, support vector matching including the linear kernel, polynomial kernel, and radial basis kernel, and random forest).The data were collected from the breath of the endotracheal tube via ventilator and then analyzed. Two groups were analyzed: first, patients with pneumonia, myocardial infarction, diabetes, aspiration pneumonia, hepatitis, endocarditis, heart failure, lung cancer, chronic obstructive pulmonary disease, hepatocellular carcinoma, idiopathic pulmonary fibrosis, colon cancer, necrotizing fasciitis, kidney injuries, hyponatremia, and cardiac arrest. Second, the intracranial hemorrhage, gastric cancer, traffic accident, fracture, gastric ulcer, coronary artery disease, acute kidney injury, traumatic brain injury, aortic dissection, lung cancer, Fournier’s gangrene, liver abscess, and some attributes demonstrated of low importance such as age, gender, smoking status, liver and renal function tests, and the number of comorbidities. In this case, random forest achieved the best AUC (%90). Betech et al. (2020) [51] established various supervised classification methodologies including SVM, LR, RF, and gradient boosted decision trees (XGBoost) by using Mexican Electronic Health Records information of patients who test for SARSCOV-2: demographics preconditions and outcomes of admission such as age, gender, diabetes, COPD, asthma, immunosuppression, hypertension, obesity, pregnancy, chronic renal failure, tobacco use, other diseases, and the SARS-CoV-2 test result to an ICU. They envisaged mortality and the importance of an ICU or ventilator. As a result, SVM and LR were better than RF and XGBoost. In their model, age, gender, immunosuppression, chronic renal insufficiency, obesity, and diabetes were significant attributes.

Our work is different from other methods due to some reasons: first, we advance an illustratable method that can envisage the outcomes. Second, it can evaluate the meaningful existence of different variables in making this prediction. Third, our method also clarifies the importance of various data processing scenarios in the pre-processing phase and strong feature selection ability, thereby differing our results from other previous studies mentioned above. Fourth, in our work, underlying diseases, age, gender, laboratory results, and nutrition are considered significant factors, so our results and analysis differ from previous reports. Fifth, although machine learning algorithms appear to be useful and have been used in different medical fields to predict critically ill mortality and outcomes, they do not adequately predict mortality because of low prediction accuracy and limited variables. Besides, this study explored whether machine learning algorithms are also applicable to predicting the mortality of ICU patients. We also compare the performance of several machine learning models and may explain the power of the AdaBoost algorithm, which is not considered in the previously mentioned works. Furthermore, the proposed method acquires high AUC, sensitivity (recall), Kappa, and F-measure for ICU patients. Our experimental results demonstrate that the proposed method is superior to state-of-the-art methodology.

We use the dataset of an ICU ward in a General Hospital of Isfahan in our study. The goal of our study is to predict the mortality rate of patients who are admitted to an intensive care unit. In this work, we applied the essential forecasting methods such as decision tree (DT), neural network (NN), K-nearest network (K-NN), random forest (RF), linear regression (LR), AdaBoost, Bayesian Boosting, Vote (DT+K-NN), and Vote (DT+K-NN+LR).

Methods

To evaluate the ability of the proposed model, we use a dataset of 180 patients collected from a general hospital (a ward in an ICUs) between 2017 and 2018. The dataset consists of detailed information about ICU patients’ stays, including demographic information and medication variables. The attributes were extracted, and a new dataset was created. We select 28 medical attributes of patients (see Table 2). We only include patients who have been admitted in a ward of ICUs. Each patient must have at least 12 h of data since we used data from the last 12 h before discharge to train our models. However, some attributes were eliminated from data analysis since their patient data records set were not available. As a result, a total of 180 patients opted in the final data set for data analysis to satisfy our criteria.

Table 2

Selected attributes for modeling

Variable	Type
Operation	Binominal
Smoking
Hemiplegia/paraplegia
Anemia
Diabetes mellitus
Motor sensory disorder
Hypertension (HTN)
Intubation
Multiple sclerosis (MS)
Bone fracture
Cerebrovascular accident (CVA)
Gender
Bilirubin (mg/dL)	Numerical
Serum Creatinine (mg/dL)
Platelet blood count (mcL)
Body mass index (BMI) (kg/m2)
Pulmonary Embolism Wells Score admit
Pulmonary Embolism Wells Score discharge
Deep vein thrombosis (DVT) score admit/discharge
Deep vein thrombosis (DVT) score discharge
Glasgow Coma Scale (GCS) admit
Glasgow Coma Scale (GCS) discharge
LOS
Age
Nutrition	Polynominal
Mobility
Skin Type

Methodology

Around 28 attributes used as input including body mass index (BMI), Pulmonary Embolism Wells score, deep vein thrombosis (DVT) Wells score, paraplegia/hemiplegia, anemia, diabetes mellitus, Bilirubin, Serum Creatinine, platelet blood count, hypertension (HTN or HT), GCS admit, GCS discharge (indicating discharged patients state and is considered true in the case of patients betterment), age, gender, skin type, having operation during hospitalization, length of stay (LOS) at ICU which is determined as the difference between the date of admission and discharge, smoking, intubation, diabetes mellitus, MS, nutrition, mobility, fracture, CVA. The selected attributes and their types are shown in Table 2. Also, Rapid Miner is a statistical tool and software which is used for data analysis, manipulation, calculation, and graphical representation. Figure 1 shows the overall figure which we consider as an ICU mortality prediction model. Health data frequently includes high dimensionality, imbalance, and time asynchronization data. Such complex properties and characteristics of data involve accurate methods for processing. The proposed ICU mortality prediction model is made up of four basic parts. The first step is data preprocessing, which is the most important step. Our study includes replacing missing values by k-NN-imputation and random forest-imputation, feature selection, data sampling, data reduction, and feature transformation stages. The next step is modeling includes decision tree (DT), neural network (NN), K-nearest network (KNN), random forest (RF), linear regression (LR), AdaBoost, Bayesian Boosting, Vote (DT+K-NN), and Vote (DT+K-NN+LR), then model testing that contains confusion matrix, finally, an evaluation of data analysis. Four core components are effectively combined to develop a prediction model of a patient’s mortality at the hospital.

Fig. 1

The proposed model, moving from selecting data to predicting discharge

Data preprocessing

Four steps for preprocessing are considered in the phase of data preparation to prepare clean data for modeling.

Replacing the missing value

In the beginning, we imputed the missing value with k-NN-imputation and random forest-imputation. Missing value usually happens for medical data set due to some reasons, and patients with incomplete data may bias our study. The k-nearest neighbor algorithm can be used for imputing missing data by discovering the k-closest neighbors. It not only predicts the most frequent value among the k-nearest neighbor as discrete attributes, but also it can predict the mean among the k-nearest neighbor as continuous attributes [52]. RF also puts a stop to data over-fitting. Moreover, it efficiently runs on many variables (except for dimensionality problems), and it assesses variable importance. It includes methods which are effective for missing data estimation [53-56] such properties make the k-NN and RF algorithm logical candidates for missing data in our study, and as a result, the k-NN-imputation method proved to be more effective than others.

Sampling of the existing data set

After data imputation, in the next step of pre-processing, sampling was conducted to achieve a more accurate model. Data sampling is a statistical analysis technique. It is mainly used to select, manipulate, and analyze a representative subset of data points to identify patterns and trends in the examined data set.

Feature generation

As the next step of preprocessing, the standardized features after feature generation were obtained. Table 3 provides the following information.

Table 3

Selected attributes and related ranges in our work

Variable	Range
Bilirubin (mg/dL)	A: Bilirubin ≤ 1.2 B : 1.2 < Bilirubin ≤ 1.9 C : 1.9< Bilirubin ≤ 5.9 D : 5.9 < Bilirubin ≤ 11.9 E : 11.9 < Bilirubin
Serum Creatinine (mg/dL)	A: Cr≤ 1.2 B : 1.2 < Cr ≤ 1.9 C : 1.9 < Cr ≤ 3.4 D : 3.4 < Cr ≤ 4.9 E : 4.9 < Cr
Platelet blood count (mcL)	A: Plt ≤ 20000 B : 20000 < Plt ≤ 50000 C : 50000 < Plt ≤ 10000 D : 10000< Plt ≤ 150000 E : 150000< Plt
Body mass index (BMI) (kg/m2)	Normal : 18.5 ≤ BM < 25 Obesity : 25 ≤ BM < 30 Overweight : BM < 30 Underweight : BM < 18.5
Pulmonary Embolism Wells Score admit/discharge	High probability: score >6 Moderate probability: score >2 and ≤ 6 Low probability : score < 2
Deep vein thrombosis (DVT) score admit/discharge low: 0–2 normal: 2–3 high: 3–8 Glasgow Coma Scale (GCS) admit	Deep coma/Death: GCS < 3 Severe: 3≤ GCS≤ 5 Low : 5 < GCS≤ 9 Moderate : 9 < GCS≤ 12 Mild: 12 < GCS≤ 14 Fully alert : 14 < GCS
Glasgow Coma Scale (GCS) discharge	False: 15≤ GCS ≤ 9 True : 9<GCS<2
Mobility	Agitation, slightly limited, completely immobile
Skin type	Dry, normal, moist

Feature selection

Feature selection (FS) procedure can be employed to maintain the feature subset that contributes to the classification accuracy and facilitates the classification task. Moreover, the removal of irrelevant and redundant features could significantly reduce the risk of complexity. It aims to be discarded in the case of imbalanced classification problems to achieve a subset of relevant features in compact models that are easier to interpret, so improve knowledge extraction [28, 57, 58]. In our work, by using FS, some features including skin type, BMI, MS, and CVA have been removed since they were of low importance.

Model evaluation

Eight different machine learning techniques based on data type are selected in this study. Decision tree (DT), K-NN, logistic regression (LR), random forest (RF), and ensemble methods such as Bayesian Boosting, Vote (DT, K-NN), Vote (DT, K-NN, and RF), and AdaBoost are mainly used among various machine learning techniques.

Decision tree (DT) classifies instances or examples into branchlike segments by taking paths from the root node through internal nodes to leaf nodes. In this method, the root node implies the predictor that gives the best split of the target class values. In this model, a set of training examples is broken down into smaller and smaller subsets. In contrast, an associated decision tree gets incrementally developed (in our case, the maximum depth of a tree is considered 20). At the end of the learning process, a decision tree that covers the training set is returned [59].

K-NN is a type of instance-based learning in situations that the function is only approximated locally and all computation is deferred until the classification phase. In this method, data is divided into training and test; then, K is selected (we have selected training set and test data 80% and 20% of total dataset respectively, and K=5). Determine which distance function is to be used. After that, a sample from the test data is selected to classify and it is computed the distance to its n training samples. Then, the obtained distances and the K-nearest data samples are sorted. In the end, based on the majority vote of its k neighbors, the test class is assigned to the class [60].

Logistic regression (LR) is used exclusively for two-class problems, so we convert our data into binary variables. Input values are combined by using weights or coefficient values to predict an output value. The first predicted output is mainly considered the bias or intercepts term; the next one is the coefficient for the single input value. Each column in input data has a constant real value that must be learned from training data. In all sections, we considered 80% and 20% of the total dataset for training and testing, respectively. This model uses maximum likelihood estimation to evaluate the probability of class membership [61].

Artificial neural network (ANN) is mainly based on the biological and processing nervous system, which includes a multilayer perceptron and a single hidden layer. It would be an appropriate choice for survival prediction. At first, some weights are assigned to input, and then their predicted output values are calculated (we selected the learning rate=0.3). After finding the value of the predicted output, it is compared with the target output. The aim is to reach a low error level and high accuracy for the model [61].

Random forest (RF) is considered an ensemble classifier that contains a multitude of decision trees. This method is taking advantage of the principle of “voting” to reach an accurate output. Multiple trees (in our study trees = 10) grows to the largest extent possible, and there is no pruning; then, in order to classify a new object and predict new data according to features, a classification is taken from each tree to vote for that class. As a result, the class with the most votes is selected [61, 62]. Voting is the simplest form of combining individual classification algorithms in which different multiple models and rules are selected. Then, they combined to build a strong classifier. We have selected the combination of DT and K-NN as a model, and DT, K-NN, and RF as another one.

AdaBoost is one of the most well-known algorithms of boosting, which combines the result of a series of the classifier algorithms. These algorithms are often decision trees in a single robust classifier; in each model, the data set is trained and assigned an amount of weight to achieve an acceptable accuracy score for the overall classifier. The AdaBoost algorithm is one of the best boosting techniques. It is a linear model that is mainly used as an ensemble method in ML. It is used to decrease both bias and variance in supervised learning. Firstly, it creates n number of decision trees during the training period of data, when the first tree is constructed. In this model, the errors of the first model are recorded by the algorithm. Then, the nodes which are incorrectly classified are considered the input for the next model. Each model aims to solve the errors of prior model. This process is repeated (models 1, 2, N) to meet the specified condition (see Fig. 2). The algorithm creates proper trees that include a start node with several leaves nodes, in our case, 4 number of trees [62, 63].

Fig. 2

Our proposed model evaluation in this study

Boosting is an ensemble method based on Bayes’ theorem which improves the predictive performance of base learning algorithms. Various classifiers are applied and trained. After many iterations, they are combined by boosting algorithm to obtain a strict rule. It implements a meta-algorithm that can be used in conjunction with many other learning algorithms to improve their performance. In our study, we have considered 1 as a subset for training [54, 64, 65]. The advantages and disadvantages of these methods are compared in Table 4.

Table 4

The advantages and disadvantages of our model with other models used in this study

Model	Advantages	Disadvantages
Bayesian Boosting	Is moderately robust to outliers Can be used on a small dataset Can learn nonlinear relationships	Vulnerable to overfitting Vulnerable to uniform noise
AdaBoost	Act remarkably in reality Is a type of reinforcement learning algorithm that can realize high precision classification by training several weak classifiers and assembling them into one strong classifier Can learn nonlinear relationships It is scalable Is moderately robust to outliers Its algorithm is fast, simple to implement, and easy to program Implicit feature selection Can remove overfitting (individual trees are inclined to overfitting since they can support branching till they memorize the training data)	Must adjust for cost-sensitive or imbalanced class problems Sensitive to noisy data & outliers Needs a termination condition Vulnerable to uniform noise
Vote (DT+K-NN)	• Can learn nonlinear relationships • It is scalable • Can be used on a small dataset	• Vulnerable to overfitting
K-nearest neighbors	• Not assume underlying data distribution • Is easy to implement	• Unable to construct a model • Unable to explore the relationship between a feature and the class • Computationally intensive recall • Require huge storage • With the existence of one class sample in excessive compared to other class, the control class will dominance the classification leading to an incorrect result
Decision tree	• Can be used on a small dataset • Can learn nonlinear relationships • Simple to analyze	• Over fitting or under fitting of the model is effortless • Negligible changes in the training data can generate huge changes in the result
Neural network	• Able to model complicated samples • Able to detect all possible interactions between predictor variables • Has the availability of multiple training algorithms	• Has a “black box” nature, • Has a great computational burden • Proneness to the overfitting • Inclined to an overfitting training dataset
Random forest	• Able to estimate noisy or missing data • Able to keep accuracy when a huge portion of the data is missing • Able to handle thousands of input variables • Can be Suitable for class imbalance issues	• Is not simply understandable model • It is like a black box approach for statistical modelers, the user has a low control level on what the model does • A large number of trees can make the algorithm too slow and ineffective for real-time predictions
Logistic regression(LR)	• Is easy to recognize and describe • Can be improved with no difficulty with new data • Does not require high computation power	• Acts weak when there are nonlinear relationships • Vulnerable to overfitting • Inflexible to absorb intricate patterns • Adding the right interaction terms or polynomials is risky and time-consuming.
Vote (DT+K-NN+LR)	• Is moderately robust to outliers • It is scalable • Can be used on a small dataset • Can learn nonlinear relationships	• Vulnerable to overfitting • Negligible changes in the training data can generate huge changes in the result • Require huge storage

Performance: evaluation of metrics

To define how a classifier is practical, various metrics can be evaluated. Although accuracy has been proven to be effective overall, it does not work for some classes. For instance, when there is imbalanced data with small amount of death patients and the majority of survivals like our dataset, so it can have a detrimental effect on the accuracy of the model. As a result, we used the area under receiver operating characteristics (AUC) curve as the most widely-used metric for imbalanced data [66]. Moreover, it refers to the area under the ROC curve. Furthermore, there are lots of other indices: Sensitivity in statistical analysis measures the proportion of true positives that the classifier detected correctly. It is also called recall, or true positive rate, (see Eq. (1.1)); it means correctly detected patients who have at least a condition.

The Kappa statistic is an agreement indication between the predicted outcomes based on Eq. (1.2). Po and Pe are

the observed agreement rate between the predictions and the actual values. These show percentage of chance agreement between the predictions and actual values, respectively.

F-measure, or F score according to Eq. (1.3), is a metric which is used in the accuracy of a test. The precision and the recall of the test are mainly taken into consideration to calculate F score. F-measure is the harmonic average of precision and recall. In F-measure, 1 is the best score (perfect precision and recall) and 0 is the worst [66-69].

In this case, such measures have provided a proper model evaluation. They also are utilized for model comparison. Comparison of the performance of various machine learning methods in our model is demonstrated in Table 5. In the next section, we analyze the findings. These measures for the AdaBoost model are AUC= 0.966, sensitivity (recall) = 87.88%, Kappa=0.859, F-measure = 89.23%, so AdaBoost accounts for the highest rate. We used 5-fold cross-validation (K=5) to determine the model performance, which means that the data is split into 5 folds. In one round of iteration, 1 fold is used to test, and the rest of the folds are served to its train. After five rounds in which a fold has been used as the testing set, the validation results are combined. In the end, the accuracy of the model is acquired. The confusion matrix is mainly used to show the performance of a model on two various categories (see Table 6). Classification problems are considered by using two classes. Generally, each instance is mapped into an element of the set p, n, which are labeled as positive and negative class. A classifier is a mapping from instances to predicted classes. The labels Y, N are used for distinguishing the actual class from the predicted class. There are four possible outcomes as follow: the instance is positive and it is classified as positive, the instance is positive and it is classified as negative, the instance is negative and it is classified as negative, and the instance is negative and it is classified as positive which are counted as true positive (TP), false negative (FN), true negative (TN), false positive (FP), respectively. The results of the confusion matrix eight models are presented in Table 7. Figure 3 is a bar chart of different factors affecting the rate of mortality in the present work. Overall, it is evident from the information provided that the highest amount of mortality allocated to the age group of more than 59 years old; in contrast, mortality risk for people between 2 and 59 years old is low (see figure (a)). Figure (b) shows that the mortality among men is less than women. By looking at figure (c), we can find the mortality for completely immobile is more than other mobility types (agitation and slightly limited). In figure (d), we can see the mortality for people with moderate PE wells score discharge is far more than mortality for people with low PE wells score. Based on (e), individuals with diabetes mellitus have a high level of mortality compared to people without diabetes mellitus. As we can see in figure (f), the length of stay more than 7 days can increase the mortality rate. Figure (g) shows that people who do not smoke have a higher mortality rate than others. The mortality of people with moderate PE wells score admit is far more likely than mortality for people with low PE wells score (h). Hemiplegia/paraplegia has an insignificant effect on mortality rate (figure (i)). Patients with GCS less than 9 are more in danger of death (j). Figure (K) illustrates Bilirubin less than 1.2 (mg/dl) leads to a higher mortality rate. Figure (l) depicts that almost a great number of patients have a normal rate of platelet blood count. As a result, it cannot play a key role in the mortality rate. Figure (m) displays a higher mortality rate that can be as a result of Creatinine more than 1.9. Patients with hypertension can be a key factor of high mortality rate (figure (n)). Figure (o) manifests the high risk of mortality that arise from moderate and high Pulmonary Embolism Wells Score admit. Figure (p) represents that high and moderate DVT wells score discharge puts individuals at risk of death.

Table 5

Comparison the performance of various machine learning methods

Model	AUC	Sensivity (recall) (%)	Kappa	F-measure
Bayesian Boosting	0.895	86.11%	0.847	88.57%
AdaBoost	0.966	87.88%	0.859	89.23%
Vote(DT+K-NN)	0.898	55.26%	0.606	68.85%
K-nearest neighbors	0.881	40.62%	0.440	53.06%
Decision tree	0.836	72.41%	0.764	82.35%
Neural network	0.888	47.62%	0.534	58.82%
Random forest	0.916	73.33%	0.614	68.75%
Logistic regression (LR)	0.923	81.25%	0.802	83.87%
Vote (DT+K-NN+LR)	0.875	70.59%	0.815	81.36%

Table 6

The performance of confusion matrix and related metrics.

Hypothesized class/true class	P	N
Y	True positive (TP)	False positive (FP)
N	False negative (FN)	True negative (TN)

Table 7

The results of the confusion matrix eight models

Model/confusion matrix	P	N	Model/confusion matrix	P	N
Bayesian Boosting			Random forest
Y	98	5	Y	112	19
N	3	31	N	2	4
AdaBoost			Neural network
Y	101	5	Y	63	4
N	3	31	N	6	11
Vote (DT, K-NN)			Regression
Y	97	17	Y	65	3
N	2	21	N	2	13
K-NN			Vote (DT, K-NN, RF)
Y	101	4	Y	102	10
N	3	13	N	1	24
Decision tree
Y	107	8
N	1	21

Discussion

The data analysis of the present study is discussed as follow:

Mortality and underlying diseases

High mortality can be as a result of underlying diseases (such as diabetes mellitus which is a metabolic disease that can result in high blood sugar and high blood pressure). To cite an example, patients with the Glasgow Coma Scale (GCS) less than 9 indicate the low level of brain consciousness and laboratory disorders, including the low level of platelets can give rise to discharge. In contrast, with such conditions, the existence of underlying diseases can lead to death. There is no denying fact that the more “completely immobile” patients are, the more rate of mortality they have, even the patients are “fully alert” in admission. Underlying diseases such as Hypertension (HTN) as a medical condition in which the blood pressure is continuously raised in the arteries can be a cause of high mortality. It really depends on three factors: the age of patients, laboratory disorder, and smoking. For example, it can lead to a high mortality rate among patients who are more than 76 years old and have a moderate Pulmonary Embolism Wells Score and low level of brain consciousness (GCS=6–9). Hypertension (HTN) among patients with a low level of brain consciousness (GCS= 9–6) and completely immobile and nutrition = NPO can cause death in comparison with patients without hypertension. As a result, hypertension can increase the mortality rate. Diabetes mellitus can be an influencing attribute in boosting mortality, especially among female patients.

Mortality and the Glasgow Coma Scale

The low level of brain consciousness (GCS= 9–6) plays a remarkable role in boosting discharge rate, regardless of underlying diseases. Underlying diseases include high blood pressure and impairment a vertical half of the body/impairment of the legs and lower body. These are as a result of the injury to nerves in the lumbar or thoracic vertebrae areas of the body. Mortality rate also can arise from the result of the moderate Pulmonary Embolism Wells Score risk (a score of risk classification and clinical decision rule to predict the critical pulmonary embolism (PE) probable in patients who had the possibility of acute PE in their history and examination, even though patients with low level of Glasgow Coma Scale (GCS), laboratory disorder, and low Pulmonary Embolism Wells Score risk can increase pre-discharge and reduce the mortality rate (discharge is by no means fully alert)). The higher level of “GCS admission” can lead to increasing the probability of discharge. It needs to be highlighted that the “GCS admission” level is a contributing factor in discharge.

Mortality and gender

Mortality rate increased among females who were 26–38 years old and had a low level of brain consciousness (GCS= 9–6) and moderate Pulmonary Embolism Wells Score risk with length of stay (LOS) = more than 16 days. Patients who were less than 26 years old and had a low level of brain consciousness (GCS=6–9), platelet blood count less than 100000 (mcl), and moderate Pulmonary Embolism Wells Score are more in danger of death than others. Gender is of low importance among patients with a low level of brain consciousness (GCS less than 5) and normal complication in trauma patients called moderate deep vein thrombosis (DVT) wells score; nonetheless, there is a risk of DVT wells score. Female patients who are 76 years old and have a normal rate of Serum Creatinine and are completely immobile have a higher mortality rate, whereas males with these conditions have a higher discharge rate.

Mortality and laboratory findings

Platelet blood count of less than 100000 (mcl) can cause death. Moreover, death can arise from laboratory findings, along with platelet blood count less than 70000 (mcl). The underlying disease solely cannot cause death unless it is accompanied by the blood count of less than 100000. Low level of platelet blood count among patients less than 29 years old leads to death and this factor has no effect on the death of patients more than 29 years old.

Mortality and serum creatinine

Mortality is likely to be affected by Serum Creatinine (a waste substance in the blood that arises from muscle activity) more than 3.4(mg/dl), indicating if the kidney’s function slows down, mortality rate increases.

Mortality and age

Patients who were 76 years of age and above and have creatinine 1.2–1.9 were fatal just if they had platelets less than 100000 (mcl). Otherwise, they have been discharged even with aging. In patients older than 76 years old, mortality is increased just if there is a laboratory disorder Serum Creatinine or being a smoker. In contrast, discharge is increased with the same conditions, just if the patient is a non-smoker.

Mortality and length of stay

Length of stay cannot be the sole reason for the high mortality rate. It should be along with laboratory disorder or smoking to increase mortality risk with the exception of the patients less than 76-year-old patients. Mortality also can arise from the high level of Bilirubin, low level of brain consciousness (GCS= 6–9), and length of stay for more than 12 days. In contrast, the normal level of Bilirubin along with the length of stay = 6–12 days accounts for discharge.

Mortality and smoking

Patients who are less than 72 years old with the low level of Glasgow Coma Scale (GCS= 6–9), Bilirubin levels less than 1.2 (mg/dL), low level of Serum Creatinine, and deep vein thrombosis (DVT) risk have length of stay 4–6 days. In contrast, these conditions for smoker patients who are less than 26 years old have increased length of stay up to 6–13 days. It means the smoking factor has increased the length of stay from 4–6 to 6–13 days for patients who are less than 26 years old.

Mortality and nutrition

Conditions, such as “GCS admission” between 6 and 9, slightly limited mobility and nutrition = NPO, leading to death after 6–11 days. In contrast, the level of Glasgow Coma Scale of patients who have nutrition = low in calorie, increased to 13–14, which means a normal scale of consciousness, so nutrition can be considered a significant factor in discharging alert patients.

Conclusion

Reaching a precise prediction of mortality risk for patients is of great importance in health care and hospital resource allocation, leading to an improvement in patients' durability. The model in our study included various scenarios of data processing. This model demonstrates the pre-processing as a significant step in data mining, and it improves the scale efficiency of the model. In this article, several machine learning methods were used; the AdaBoost method led to better results in discovering the influential factors in mortality with AUC= 0.966, sensitivity (recall) = 87.88%, Kappa=0.859, F-measure = 89.23%. These results show that the high mortality can be a result of underlying diseases such as diabetes mellitus and high blood pressure, moderate Pulmonary Embolism Wells Score risk, platelet blood count less than 100000 (mcl), hypertension (HTN), high level of Bilirubin, smoking, and GCS level between 6 and 9. GCS level also plays a remarkable role in the rise of patient discharge rate, regardless of underlying diseases like high blood pressure and hemiplegia/paraplegia. Diabetes mellitus can be an influencing attribute in boosting the mortality rate, especially among female patients. A low level of platelet blood count among patients less than 29 years old leads to death, and this factor does not affect the death of patients who are more than 29 years old. Mortality is likely to be affected by Serum Creatinine. In patients older than 76 years old, mortality is increased just if there is a laboratory disorder Serum Creatinine or being a smoker, while discharge is increased with the same condition, just if the patient is a non-smoker. The length of stay cannot be the sole reason for the high mortality rate; it should be along with laboratory disorder or smoking to increase mortality risk except for the patients less than 76-year-old. As a factor, smoking has increased the length of stay from 4–6 to 6–13 days for patients who were less than 26 years old. Nutrition also can be considered a significant factor in discharging with fully alert. Our suggestions for future researches can be listed as follow: considering the patients of alternative hospitals during a longer period of time, utilizing visual information, and determining whether alternative metrics are able to achieve better performance. Since AdaBoost is a reliable predictor of ICU mortality, further research seems possible in developing hybrid models (machine learning and statistical and deep learning models) with the aim of its evaluation, validation, and applicability to other ICUs to improve the performance of the forecast.