Explainable Artificial Intelligence for Predicting Hospital-Acquired Pressure Injuries in COVID-19-Positive Critical Care Patients.

Hospital-acquired pressure injury risk assessment is vital for prevention, but current risk assessment instruments such as the Braden Scale lack specificity in critical-care patients.

The current study shows good discrimination for predicting hospital-acquired pressure injuries in critical-care patients using machine learning algorithms combined into an ensemble SuperLearner.

Explainable artificial intelligence was used to create transparent machine learning models at the global and single-patient levels.

The most important variables in the top-performing model were hemoglobin, fragile skin, and serum albumin.

Hospital-acquired pressure injuries (HAPrIs) are areas of injury to the skin due to prolonged pressure or pressure in combination with shear. These injuries occur in 6% to 8% of critical-care patients and result in human suffering.[1-3] Most HAPrIs are preventable. Still, prevention may be better served with a more precise risk stratification approach and associated preventive interventions, given that every patient does not require the same level of care, nursing resources are limited and constrained by competing priorities (consider the COVID-19 pandemic), and cost-saving measures are further impacting care delivery. Risk stratification is essential in the ICU, but current risk assessment instruments, such as the widely used Braden Scale,[4] lack specificity and end up classifying most ICU patients as “high risk” and therefore hinder nurses from differentiating HAPrI risk among patients.[5-9] Moreover, special subgroups and conditions within the ICU population may have unique HAPrI risk profiles. For example, ICU patients with COVID-19 experience high severity of illness in the context of a unique constellation of HAPrI risk factors, including hypoxemia, altered perfusion, and care-related factors such as prone positioning.[10,11] Yet, little is known regarding HAPrI risk in COVID-19–positive ICU patients.

The National Pressure Injury Advisory Panel's (NPIAP's) 2019 Clinical Practice Guidelines call for research using artificial intelligence (AI) and machine learning (ML) to improve the accuracy of HAPrI risk assessment accuracy in the ICU.[3] Readily available electronic health record (EHR) data are used in modeling developed through ML methods, thus reducing documentation time and increasing the amount of nursing time available for patient care. Machine learning approaches are particularly relevant in the ICU setting because of the dynamic physiologic nature of critical-care patient conditions.[12] Unlike traditional prognostic instruments such as the Braden Scale,[4] an ML approach can incorporate nonlinear, complex interactions among variables and capture multiscale time dependencies, allowing for findings of new trends over time, thus producing a synergistic influence on HAPrI risk assessment and therefore prevention.[12,13]

The downside of ML algorithms is their “black box” nature—clinicians are unable to determine how the algorithm made the decision and are thus understandably unwilling to trust the algorithm for patient care decisions. Thus, the NPIAP's call for ML algorithms to predict HAPrI includes the specification that models must be transparent and interpretable.[3] Interpretability is defined as the ability of a human to understand the relationship between the features in an ML model and the model's prediction.[14] Explainable AI methods including the SHAP (SHapley Additive exPlanations) value are a way to increase transparency and interpretability.[15] The SHAP value assigns each feature (variable) in the model an importance value for a particular prediction by averaging the marginal contribution of a feature across all possible permutations (sets of features).[15] SHAP plots can be generated for global ML model interpretability (the collective SHAP values across a data set) and local interpretability (the SHAP values for one observation).

The purpose of this study was to evaluate HAPrI injury risk in COVID-19–positive ICU patients. The specific aims include the following: (1) develop an ML model to predict HAPrI risk and (2) apply the SHAP explainable AI method for global and local model interpretability.

METHODS

Design

This retrospective cohort study was conducted using EHR data extracted from one hospital system's enterprise data warehouse. Extracted data were limited to the duration of the patients' ICU stay and verified for accuracy by an informaticist and ICU nurse with Epic EHR system expertise (Epic Systems Corp, Madison, WI, USA). The study was approved by the facility's institutional review board.

Sample

Adult patients who tested positive for COVID-19 and admitted to one of two medical ICUs at a single level-1 trauma center and academic medical center between April 2020 and April 2021 were eligible for inclusion in the study. Patients with a pre-existing (community-acquired) pressure injury were included because of the increased likelihood of developing an additional pressure injury after hospitalization.[16]

Measures

The HAPrI outcome variable was defined according to the NPIAP staging definitions (stages 2–4, unstageable, or deep tissue injury).[3] Stage 1 HAPrIs were not included because stage 1 injuries are reversible and considered less severe.[17,18] Hospital-acquired pressure injuries were deemed to be hospital-acquired if occurring at least 48 hours after the ICU admission. All HAPrIs were verified by a certified wound nurse and evaluated to determine whether the injury was medical device–related. Medical device–related pressure injuries were excluded from this analysis because those injuries have different risk factors.[19,20]

Potential predictor variables were selected based on a review of the relevant literature and Coleman and colleagues'[21] conceptual framework for pressure injury etiology.[1,21,22] The conceptual framework classified variables based on a proposed causal pathway with immobility, skin status, and poor perfusion as direct causal factors.[21] Predictor variables were only recorded before an HAPrI occurred so that data in the ML models were limited to events preceding the HAPrI. Predictor variables and their operationalizations are described in Table 1.

Table 1

Characteristics of the Sample

	All Patients With COVID (N = 407)	Patients With HAPrI (n = 74 [18%])	No HAPrI (n = 333 [82%])	P	Missing Data
Demographic and discharge information
Age, mean (SD), y	59 (15)	63 (16)	58 (14)	<.001	0%
Sex, male, n (%)	256 (63)	47 (64)	209 (63)	1.0	0%
Race, n (%)	Native American or Alaska native: 22 (5%) Asian: 11 (3%) Black: 10 (2%) Native Hawaiian or Other Pacific Islander: 19 (5%) White: 229 (56%) Other, Unknown, or choose not to disclose: 116 (29%)	Native American or Alaska native: 5 (7%) Asian: 1 (1%) Black: 0 (0%) Native Hawaiian or Other Pacific Islander: 4 (5%) White: 58 (78%) Other, Unknown, or choose not to disclose: 6 (8%)	Native American or Alaska native: 17 (5%) Asian: 10 (3%) Black: 10 (3%) Native Hawaiian or Other Pacific Islander: 15 (5%) White: 171 (51%) Other, Unknown, or choose not to disclose: 110 (33%)	<.001	0%
Ethnicity, Hispanic, n (%)	98 (24)	4 (5)	94 (28)	<.001	0%
Hospital length of stay, mean (SD)	16 (16)	16 (14)	16 (16)	.89	0%
Died during hospitalization, n (%)	101 (25)	21 (28)	80 (24)	.74	0%
Time in the emergency department, mean (SD), hours	2.7 (2.6)	2.2 (2.9)	2.9 (2.5)	.10	0%
Braden Scale scores
Minimum Braden Scale total score, mean (SD)	11.3 (3.8)	11.0 (4.2)	11.4 (3.7)	.25	0%
Treatments
Ventilator days, mean (SD)	5 (10)	5 (12)	5 (10	.26	0%
Reintubation, n (%)	46 (11)	9 (12)	37 (11)	.96	0%
Dialysis, n (%)	89 (22)	21 (28)	68 (20)	.18	0%
Vasopressor infusion, n (%)	49 (12)	17 (23)	32 (10)	.003	0%
Laboratory values
Maximum lactate, mean (SD), mg/dL	3.81 (3.87)	4.37 (3.93)	3.69 (3.85)	<.001	9%
Maximum serum creatinine, mean (SD), mg/dL	2.16 (2.22)	2.66 (2.97)	2.05 (2.01)	<.001	0.01%
Maximum serum glucose, mean (SD), mg/dL	266 (128)	258 (125)	269 (129)	.53	0.01%
Minimum hemoglobin, mean (SD), g/dL	10.46 (3.00)	8.85 (2.85)	10.80 (2.93)	<.001	0.01%
Minimum albumin, mean (SD), mg/dL	2.68 (0.52)	2.24 (0.52)	2.74 (0.51)	<.001	9%
Mean Pao2, mean (SD), mm Hg	104 (62)	101 (62)	117 (63)	.06	6%
Maximum Paco2, mean (SD), mm Hg	53 (21)	59 (21)	52 (21)	.19	6%
Minimum pH, mean (SD)	7.44 (0.07)	7.44 (0.10)	7.44 (0.07)	.44	6%
Nursing skin assessments
Fragile skin: thin epidermis with subcutaneous tissue loss, n (%)	198 (49)	56 (76)	142 (43)	<.001	0%
Excessively moist skin, n (%)	81 (20)	16 (21)	65 (19)	.84	0%
Pitting edema, n (%)	84 (21)	18 (24)	66 (20)	.48	0%
Nutrition
Unplanned weight loss >10 lb before admission, n (%)	32 (8)	11 (15)	21 (6)	.26	0%
No intake >3 d, n (%)	28 (7)	6 (8)	22 (7)	.83	0%
Comorbid conditions and severity of illness
Charleston Comorbidity Index, mean (SD)	3.38 (3.32)	5.05 (4.06)	3.01 (3.02)	<.001	0%
Maximum MEWS score, mean (SD)	5.39 (2.01)	5.80 (2.14)	5.31 (2.14)	<.001	0%
Diabetes, n (%)	222 (55)	42 (57)	180 (54)	.77	0%
Spinal cord injury, n (%)	27 (7)	13 (18)	14 (4)	<.001	0%
Heart failure, n (%)	95 (23)	31 (42)	64 (19)	<.001	0%
Chronic obstructive pulmonary disease, n (%)	128 (31)	32 (43)	96 (29)	.21	0%

Analysis

All data analyses were performed using open-source R software version 4.1.2 (R Core Team, Vienna, Austria).[23] Missing data were quantified and assessed for patterns of missingness using graphical clustering displays. For prediction engineering, data were split into 80:20 training and testing data sets. Random forest (single value) imputation was applied independently to training and testing sets to avoid information leakage. Imputation was performed on variables not informatively missing; variables with potential informative missingness were given an indicator for whether the value was observed. Several competing predictive models (deep neural nets, extreme gradient boosting [xgboost], deep random forests, and logistic regression) were developed on the training data set using the H2O package in R and assembled into an ensemble (composite) SuperLearner.[24,25] Model performance was evaluated based on continuous performance on the receiver operating characteristic curve in the testing data set. Finally, the most important variables (features) were extracted from the best-performing model based on the mean decrease in accuracy.

Global and local (individual patient) SHAP plots were developed for the best-performing model in the ensemble algorithm (Deep Neural Network). The local SHAP plot was developed for a synthetic patient because of privacy concerns.

RESULTS

The final sample consisted of 407 patients. Seven patients were excluded from the analysis because of excessive missing data. The sample was predominantly male (n = 256 [63%]), and the mean age was 59 (SD, 15) years. Characteristics of the sample are presented in Table 1.

Hospital-Acquired Pressure Injury Outcome

Hospital-acquired pressure injuries (defined as stage 2 or worse) occurred in 18% of the sample (n = 74).

Predictor Variables

Relationships between the potential predictor variables and HAPrI formation are outlined in Table 1.

Predictive Models

The predictive models' discrimination based on area under the receiver operating characteristic curve is shown in Figure 1. The best-performing model was the ensemble SuperLearner with an area under the receiver operating characteristic curve of 0.807.

FIGURE 1

Predictive models discrimination.

Explainable AI

The global SHAP plot for ensemble SuperLearner is presented in Figure 2. The most important variables in the ensemble SuperLearner were, in descending order, hemoglobin, the presence of fragile skin (defined as thin epidermis with loss of subcutaneous tissue), and albumin. Note that red dots indicate negative correlations, and blue dots indicate positive correlations. For example, in Figure 2, low levels of hemoglobin were associated with risk for HAPrI, whereas higher levels were protective, and a positive value (1 = yes) for fragile skin conferred risk, whereas a negative value was associated with reduced risk. The local SHAP plot for a synthetic patient is presented in Figure 3. The model predicted that the synthetic patient would develop an HAPrI. The most important risk factor in the synthetic patient SHAP plot was the length of stay, followed by the presence of renal disease.

FIGURE 2

Global SHapley Additive exPlanations (SHAP) plot for the ensemble SuperLearner.

FIGURE 3

Local SHapley Additive exPlanations (SHAP) plot for a synthetic patient.

DISCUSSION

The purpose of this study was to evaluate HAPrI risk in ICU patients with COVID-19, to develop ML model to predict HAPrI risk, and apply explainable AI for model transparency and human interpretability. The best-performing ML model, an ensemble SuperLearner, showed good discrimination (area under the receiver operating characteristic curve = 0.807), and the global and local SHAP plots allow nurses to understand how the model is using the variables. This study adds to the body of literature showing ML approaches are useful for assessing HAPrI risk in critical-care patients,[26-29] and it is the first study to apply explainable AI for HAPrI risk prediction. The next step is model validation and development of associated clinical decision support.

Machine learning transparency and interpretability are essential for model implementation because clinicians will not—and should not—be willing to trust a model if they do not understand how the model reached its decision.[30] The global SHAP value is a human-interpretable way to visualize the relationships between the features in the ML model and its predictions. Yet, it is also necessary to consider that every patient is an individual with a unique constellation of risk factors, only some of which are represented in EHR data. For example, the clinician may be aware of individual contextual factors that may affect overall health and HAPrI risk (eg, unstable housing) that are invisible to the ML model. Moreover, ML models are generated on a data set that may or may not be representative of a given patient (consider racial minorities or unique disease states)[31,32]; therefore, it is necessary for the clinician to understand how the model decided for the individual patient in order to decide whether the model is trustworthy for that patient. The individual SHAP plot is one way to allow clinicians to see how a model decided and then choose whether to act on the risk prediction generated by the model.

Study findings show that COVID-19–positive critical-care patients have high risk for HAPrI compared with similar, non–COVID-19–positive ICU populations. The HAPrI incidence in the study sample (18%) was significantly higher than the incidence typically reported in non–COVID-19 ICU patients in the United States (6%-8%).[29,33] The high HAPrI incidence is particularly striking, given that the current study was limited to stage 2+ non–medical device–related injuries.

The most important variables in the top-performing model were hemoglobin, fragile skin, and serum albumin. Two of those—hemoglobin and serum albumin—are further evidence for the role of altered perfusion in HAPrI etiology.[1,22,34,35] Low levels of hemoglobin[36] and serum albumin[37,38] are previously identified HAPrI risk factors thought to affect tissue perfusion and therefore HAPrI risk through oxygen-carrying capacity (hemoglobin) and colloid osmotic pressure (serum albumin).[21] Furthermore, hemoglobin may be considered a modifiable factor, given that low levels can be corrected with blood transfusion; future research is needed to evaluate the effects of so-called permissive anemia[39] and blood transfusion on risk for HAPrI formation.

LIMITATIONS

This study is limited by its relatively small sample size (N = 407) and its single-site, retrospective design. The study was limited to HAPrI that occurred in the ICU, and therefore any HAPrIs that developed immediately after the ICU stay (and thus were formed in the ICU) were not captured.

CONCLUSIONS

Machine learning is a feasible approach for evaluating HAPrI risk in critical-care patients with COVID-19. Explainable AI methods such as SHAP plots are a way to ensure human interpretability and foster trust.