Predicting Time to Death After Withdrawal of Life-Sustaining Treatment in Children.

Accurately predicting time to death after withdrawal of life-sustaining treatment is valuable for family counseling and for identifying candidates for organ donation after cardiac death. This topic has been well studied in adults, but literature is scant in pediatrics. The purpose of this report is to assess the performance and clinical utility of the available tools for predicting time to death after treatment withdrawal in children. DATA SOURCES: Terms related to predicting time to death after treatment withdrawal were searched in PubMed and Embase from 1993 to November 2021. STUDY SELECTION: Studies endeavoring to predict time to death or describe factors related to time to death were included. Articles focusing on perceptions or practices of treatment withdrawal were excluded. DATA EXTRACTION: Titles, abstracts, and full text of articles were screened to determine eligibility. Data extraction was performed manually. Two-by-two tables were reconstructed with available data from each article to compare performance metrics head to head. DATA SYNTHESIS: Three hundred eighteen citations were identified from the initial search, resulting in 22 studies that were retained for full-text review. Among the pediatric studies, predictive models were developed using multiple logistic regression, Cox proportional hazards, and an advanced machine learning algorithm. In each of the original model derivation studies, the models demonstrated a classification accuracy ranging from 75% to 91% and positive predictive value ranging from 0.76 to 0.93.
CONCLUSIONS: There are few tools to predict time to death after withdrawal of life-sustaining treatment in children. They are limited by small numbers and incomplete validation. Future work includes utilization of advanced machine learning models.

Severely ill children are admitted to the ICU to receive disease-directed therapies. Unfortunately, for some patients, these therapies do not achieve the goal of survival with a good quality of life. If it is in line with the goals of care of the patient and family, medical providers may recommend a transition to exclusively comfort-focused care and withdrawal of life-sustaining treatment (1, 2). In these difficult situations, physicians are expected to counsel families about the dying process and the expected physiologic changes after treatment withdrawal. One of the more common questions from families in this position is an agonizing one: “How long will this take?” Despite the psychologic and logistical imports of estimating time to death after treatment withdrawal, physicians have very little to rely upon in providing an answer, aside from clinical experience.

Predictive modeling can help address this challenging question and provides two main benefits: 1) augmented family counseling at the end of life and 2) identification of good candidates for organ donation after cardiac death (DCD), who must die within a set time frame to be eligible for organ donation.

In 2015, Munshi et al (3) performed a comprehensive systematic review of adult studies that evaluate clinical factors influencing time to death after treatment withdrawal. Some studies had the goal of improving communication with families about the dying process; some studies developed models specifically to identify potential DCD candidates (4–9). In pediatrics, the literature is far more limited and has not yet been separately reviewed.

The purpose of this article is to compile and assess the available tools for predicting time to death after withdrawal of life-sustaining treatment in children as a focused supplement to the review by Munshi et al (3). We also aim to explore the untapped potential of advanced machine learning and autonomic nervous system features in creating such tools.

MATERIALS AND METHODS

We conducted a scoping literature review using the methodologies described by Arksey and O’Malley (10). We selected this framework because a systematic review is not feasible given the paucity of literature on the topic. Our goal of summarizing and disseminating research findings and identifying research gaps made a scoping review the more fitting approach. The primary medical literature databases of PubMed and Embase were reviewed from 1993 to November 2021. No DCD data were available by the Organ Procurement and Transplantation Network prior to 1993 (11).

The two databases were searched using the terms “predict” and “death” and “withdrawal of,” as well as “time to death” and “withdrawal.” As a scoping review rather than a systematic review, study inclusion was not limited by study quality or methodology. All studies that endeavored to predict time to death or describe factors related to time to death after withdrawal of life-sustaining treatment in adults and children were included. Abstracts with only preliminary data were excluded. Articles focusing on perceptions or practices of withdrawal of life-sustaining treatment were not considered relevant. Reference sections of each article were screened for additional citations.

Articles were screened by title and abstract. Retained articles were accessed for full-text screening. The Zotero software (Corporation for Digital Scholarship, Vienna, VA) was used to manage the publications.

RESULTS

Three hundred eighteen citations were identified from the initial literature search, resulting in 22 articles retained for full-text review (Fig. ). Eighteen of these studies included adults, and four of these studies focused on children. This search captured all the articles analyzed in the systematic review by Munshi et al (3) in 2015. This later review added five adult studies (4, 12–15) and two pediatric studies (16, 17), all of which were performed after the Munshi et al (3) publication date.

Flow diagram of study selection.

Updated Adult Literature

Munshi et al (3) systematically reviewed studies that endeavored to predict time to death after withdrawal of life-sustaining treatment. We identified five additional studies that met the inclusion criteria that were published after their review. Similar to the review by Munshi et al (3), we found that the primary outcome was time to death, usually within 1 hour of treatment withdrawal. Most studies identified risk factors associated with rapid time to death. Munshi et al (3) reported a total of seven new models that were derived for predicting time to death after treatment withdrawal (6–8,18–21). In our review, there were an additional two models: a nomogram developed by He et al (14) and a Classification and Regression Tree model developed by Okahara et al (15). Validation studies were performed by five groups (5, 20–22). Most multivariate analyses were performed via logistic regression and Cox proportional hazards models (4, 7, 8, 12–14, 22). Three studies used a Classification and Regression Tree approach (15, 18, 21). Munshi et al (3) found that predictors of time to death that were identified in five or more studies included mechanical ventilator settings, vasopressor use, and neurologic status. This was confirmed by our review. The new studies published since the review by Munshi et al (3) identified medications and imaging studies as additional candidate features with predictive capabilities (4, 14, 15).

Pediatric Literature Overview

Given the existing comprehensive review by Munshi et al (3), the focus of this article is on the literature dedicated to predicting time to death after withdrawal of life-sustaining treatment in the pediatric population. Table summarizes the study characteristics of the four articles that were reviewed (16, 17, 23, 24).

All studies used retrospective data of children who died after withdrawal of life-sustaining treatment. Similar to the adult studies, the main outcome was time to death after treatment withdrawal. Time to death was either a continuous variable or a binary outcome of whether a subject died within a set time period of 30 or 60 minutes. Patient age was reported as a median in three studies ranging from 10 to 15.6 months (16, 23, 24) and as an older mean in one study (5.3 yr) (17). Mortality rate of the ICU from which the candidate patients were extracted ranged from 1.5% to 5.3%. Median time to death ranged from 15 to 25 minutes (16, 17, 23, 24).

Model Development and Validation

In 2004, Zawistowski and DeVita (23) were the first to examine the association of demographics and treatment modalities discontinued with time to death after withdrawal of life-sustaining therapies in a single-center study with a sample size of 50 children. Multivariable analysis with the Kruskal-Wallis test demonstrated that simultaneous withdrawal of life-sustaining treatment, female gender, and absence of renal replacement therapy was associated with more rapid time to death. Their analysis was not accompanied by the development of a predictor tool.

In 2012, Shore et al (24) subsequently developed the first bedside tool to predict time to death after withdrawal of life-sustaining therapies in infants and children. They performed a retrospective chart review of 518 patients between 1996 and 2007 at Children’s Medical Center Dallas. Among their cohort, 87% of patients died within 1 hour. Using bivariate analysis, they identified significant predictors of death within 30 and 60 minutes and entered these parameters into a stepwise forward multiple logistic model. The significant predictors included age, vasopressor dose, extracorporeal membrane oxygenation (ECMO), positive end-expiratory pressure, and presence or absence of spontaneous respirations. Point-based values for each of these parameters were generated from the multiple logistic regression models; higher scores were associated with a higher chance of death within the set time period. Test positivity cutoffs were not provided; instead, score ranges were matched to a probability of death. Das et al (17) retrospectively validated the Dallas tool on an external cohort of 62 children at Rainbow Babies and Children’s Hospital in Cleveland, OH.

Winter et al (16) later used a complex machine learning model called a long short-term memory (LSTM) model to predict whether a child would die within 1 hour of terminal extubation. This was a retrospective single-center study at Children’s Hospital Los Angeles (CHLA) of 237 subjects who died after terminal extubation. More than 400 demographic, physiologic, respiratory, and laboratory parameters from admission to the ICU up to the time of extubation were input as possible predictors to train the LSTM model. The variables with the highest feature importance included heart rate, Glasgow Coma Score, pulse oximetry, Fio2, and acid-base status. The authors also derived a Cox Proportional Hazards model for comparison with both the LSTM model and the Dallas model. In the Cox Proportional Hazards model, statistically significant features that predicted death within 1 hour were low Glasgow Coma Score, high Pao2-to-Fio2 ratio, low pulse oximetry, and low serum bicarbonate. Test positivity cutoff was not provided; instead, positive predictive value (PPV) and number needed to alert (NNA) were generated at varying levels of sensitivity.

Model Assessment

The models were assessed with various metrics among the studies. Shore et al (24) presented sensitivity, specificity, and classification accuracy of two Dallas models, a 30-minute tool and a 60-minute tool. Das et al (17) validated the Dallas models and reported the same metrics, as well as area under the receiver operator curve (AUROC). Winter et al (16) presented AUROC, sensitivity, PPV, and NNA for the CHLA models. NNA is the inverse of the PPV (25). In this clinical situation, NNA corresponds to the number of operating rooms and surgical teams assembled per one viable organ. An NNA of 1.0 indicates that every prepared operating room yields a viable organ with no wasted institutional resources. Two-by-two tables were reconstructed with the pretest probability and reported performance data of each model in the aforementioned studies so that all calculable metrics could be compared head-to-head (Supplement 1, http://links.lww.com/CCX/B60). The performance metrics are presented in Table , including sensitivity, specificity, classification accuracy, PPV, NNA, and AUROC.

The reported metrics by Shore et al (24) that appear in Table 2 were based on the logistic regression models for the 30- and 60-minute models. The transformation of the logistic regression model into point-based scales yielded the two tools, which included the following parameters: age 1 month or younger, vasopressor dose greater than 0.2 ug/kg/min, use of ECMO, Fio2 greater than 0.5, positive end-expiratory pressure greater than 10, and absence of spontaneous ventilation. Total possible scores ranged from –17 to +27 for the 30-minute tool and from –21 to +38 for the 60-minute tool. Each tool showed that rapid death was more likely with higher scores. For example, among the quintile with the lowest scores (–21 to –10) on the 60-minute tool, 59% of the subjects were predicted to die; actual frequency of death was 63%. Among the quintile with the highest scores (+16 to +38), 98% of the subjects were predicted to die; actual frequency of death was 100%.

TABLE 2.

Performance Metrics of the Dallas 30-Minute Model, Dallas 60-Minute Model, Children’s Hospital Los Angeles Cox Proportional Hazards Model, and Children’s Hospital Los Angeles Long Short-Term Memory Model

Model	Pretest Probability of Death	Sensitivity	Specificity	Classification Accuracy	Positive Predictive Value	Number Needed to Alert	Area Under the Receiver Operator Curve
Dallas 30-min model (reported by Shore et al [24])	0.72	0.94	0.23	0.75	0.76	1.32	Not reported
Dallas 60-min model (reported by Shore et al [24])	0.87	0.99	0.08	0.87	0.88	1.14	Not reported
Dallas 30-min model (reported by Das et al [17])	0.60	0.76	0.52	0.66	0.70	1.42	0.69
Dallas 60-min model (reported by Das et al [17])	0.84	0.75	0.8	0.76	0.95	1.05	0.85
Dallas 60-min model for total cohort (reported by Winter et al [16])	0.68	0.94	0.07	0.66	0.68	1.47	0.72
Dallas 60-min model for DCD candidates (reported by Winter et al [16])	0.67	0.93	0.0	0.62	0.65	1.54	0.64
CHLA CPH model for total cohort (reported by Winter et al [16])	0.68	0.94	0.07	0.66	0.68	1.47	0.79
CHLA CPH model for DCD candidates (reported by Winter et al [16])	0.67	0.93	0.86	0.91	0.93	1.08	0.94
CHLA LSTM model for total cohort (reported by Winter et al [16])	0.68	0.94	0.53	0.81	0.81	1.23	0.85
CHLA LSTM model for DCD candidates (reported by Winter et al [16])	0.67	0.93	0.86	0.91	0.93	1.08	0.92

CHLA CPH = Children’s Hospital Los Angeles Cox Proportional Hazards, DCD = donation after cardiac death, LSTM = long short-term memory.

In the Das et al (17) validation study of the Dallas models, sensitivity and specificity were lower than that reported by Shore et al (24). The authors identified that the Dallas 30-minute tool had the best performance at a score of greater than or equal to 3 (sensitivity, 0.76 and specificity, 0.52); the Dallas 60-minute tool had the best performance at a score of greater than or equal to 9 (sensitivity, 0.75 and specificity. 0.8).

At CHLA, Winter et al (16) reported performance metrics of PPV and NNA at varying levels of sensitivity for their newly developed LSTM model, a Cox Proportional Hazards model, and the Dallas 60-minute model. At a set sensitivity of 0.93, both the Cox Proportional Hazards model and the LSTM model had a PPV of 0.93 and an NNA of 1.08 among DCD candidates. Performance was significantly better than the Dallas tool, which was tested on the same data and yielded a PPV of 0.65 and an NNA of 1.54. Among all patients including those who were not prospective donors, the LSTM model trended toward improved performance compared with the Cox Proportional Hazards model and the Dallas tool, but these results were not statistically significant.

DISCUSSION

This report summarizes and compares the few published studies that assess factors associated with time to death after withdrawal of life-sustaining treatment in children. Identification of these factors is important in order to facilitate end-of-life discussions and decision-making about DCD candidacy.

Parameters that Predict Rapid Time to Death

The predictive parameters reported in the reviewed pediatric studies are different from each other but generally clinically intuitive. The Zawistowski and DeVita (23) study identified simultaneous withdrawal of life-sustaining treatment as a predictor of rapid time to death. The Dallas model identified parameters of age, vasopressor dose, use of ECMO, positive-end-expiratory pressure, and spontaneous ventilation (24). The CHLA models found the predictive features to be heart rate, Glasgow Coma Score, measures of oxygenation, and degree of acidosis (16). All of these parameters have been identified in various adult studies investigating features associated with time to death after treatment withdrawal, with the exception of ECMO utilization (7–9, 18, 21, 26, 27). Several adult studies include absence of brain stem reflexes among the most important predictors of rapid death (7, 14, 22, 28). Brain stem reflexes were included in both CHLA models although were not statistically significant predictors; they were not included at all in the Dallas model. All of the pediatric studies were performed at single centers, and the differing predictive parameters may be due to unique patient populations or variation in end-of-life practices at each institution.

Implications for Practice

The ability to predict whether a patient will die within 1 hour after treatment withdrawal is particularly relevant for families considering organ DCD. DCD practice necessitates death within a short time interval after terminal extubation, typically 1 hour, to avoid ischemic damage to the organs (29). DCD is resource-intense; it requires preparation of an operating room and the assembly of a surgical team. The process also places emotional stress on a family, particularly if the child does not die within the organ recovery period and is unable to donate.

For the above reasons, an accurate estimate of time to death is valuable, and the performance metrics have direct clinical implications. A sensitive model maximizes the chance of true positives, thereby maximizing the chances of successful organ procurement. The cost is higher false positives, meaning that more patients will be brought to the operating room who do not die within the organ recovery period. A specific model minimizes these false positives, so operating rooms are not unnecessarily prepared; however, there will be missed opportunities to donate organs.

The existing tools for predicting time to death after treatment withdrawal in the pediatric population are the two models created at Dallas (30- and 60-min models) and the two models created at CHLA (Cox Proportional Hazards and LSTM models). In each of the original model derivation studies, the models had a classification accuracy ranging from 75% to 91%, PPV ranging from 0.76 to 0.93, and NNA ranging from 1.08 to 1.32 (Table 2) (16, 24). Sensitivity was greater than 0.90 for the models, aside from the Das et al (17) validation of the Dallas tools.

Limitations

Importantly, predictive models are only clinically useful if the PPV can exceed the pretest probability, and Table 2 shows the pretest probability to range from 0.60 to 0.87. Only the Dallas 60-min model as validated by Das et al (17) and the two CHLA models for DCD candidates have PPVs that are meaningfully higher than the pretest probability of death within the set time frames.

Validation of the models is limited, as well. In two external cohorts, the Dallas models did not achieve the predictive performance reported in the original study (16, 17). Further, the optimal cutoff scores in the Das et al (17) validation differed considerably from the original study, which makes the tool impossible to use universally without further calibration and modification. The Winter et al (16) study assessed the Dallas model and also reported lower performance metrics than the original study. These authors found counterexamples to each model parameter among the false positives. This study also excluded patients withdrawn from ECMO—this likely affected performance of the Dallas model, which included ECMO as one of the five predictive parameters. Neither of the CHLA models has been externally validated. Given the limited frequency of terminal extubation events in children, multi-institutional studies are needed for robust datasets for model training and validation.

Future Directions

Most models to predict time to death after treatment withdrawal have used multiple logistic regression to generate a tool (4, 6, 7, 26). Some adult studies have used basic machine learning such as Classification and Regression Tree modeling to create a tool (15, 18, 21), but the CHLA model was the first to use advanced machine learning for this type of prediction (16). There may be performance benefits of advanced machine learning models, and it is anticipated that use of machine learning to develop clinical decision support tools will continue to grow (30). The key will be implementation and integration with the electronic medical systems.

Along with the facilitation of advanced machine learning, our belief is that the advancement of electronic medical systems will also enable assessment of autonomic nervous system features. Assessment of its function and dysfunction has the potential to play an important role in predicting time to death. For example, multiple studies have demonstrated that absence of heart rate variability is correlated with organ failure and mortality in children and adults (31–34).

Widespread adoption of these types of metrics had previously been limited by challenges with standardization of measurements, access to continuous monitoring data, and real-time analysis. However, improvements in data extraction and archiving, as well as the advent of artificial intelligence-based applications, have made these metrics more accessible. With improving technology, clinician scientists are better equipped to test and incorporate autonomic nervous system metrics into predictive models and clinical practice.

CONCLUSIONS

There are limited tools in pediatrics to predict time to death after withdrawal of life-sustaining treatment. They require validation and ongoing calibration. Future tools should incorporate machine learning and investigate the metrics of autonomic nervous system dysfunction.

ACKNOWLEDGMENTS

We thank Drs. Patrick Ross and Christopher Newth for their review of this article, as well as their ongoing mentorship.

We thank Dr. Randall Wetzel for his expertise, guidance, and continued support.

TABLE 1.

Articles Predicting Outcome After Withdrawal of Life-Sustaining Treatment in Children

Study (Publication Date)	Study Design (Time Period, Location)	Sample Size	Age	Model or Test Type	Features Predicting Time to Death
Zawistowski and DeVita (23)	Single-center retrospective (1997–2001, Pittsburgh, PA)	n = 50	Pediatric	Multivariable analysis	Simultaneous withdrawal of multiple types of life-sustaining treatments
					Female gender
					Absence of renal replacement therapy
Shore et al (24)	Single-center retrospective (1996–2007, Dallas, TX)	n = 518	Pediatric	Multiple logistic regression	Age
					Vasopressor requirements
					Use of extracorporeal membrane oxygenation
					Positive end-expiratory pressure
					Presence or absence of spontaneous ventilation
Das et al (17)	Single-center validation study of Dallas Predictor Tool (2009–2014, Cleveland, OH)	n = 62	Pediatric	Multiple logistic regression	N/A (validation study)
Winter et al (16)	Single-center retrospective (2011–2018, Los Angeles, CA)	n = 237 (test set n = 47)	Pediatric	Long short-term memory model and Cox proportional hazards model	Heart rate
					Glasgow Coma Score
					Measures of oxygenation
					Degree of acidosis

N/A = not available.