External Validation of the DCD-N Score and a Linear Prediction Model to Identify Potential Candidates for Organ Donation After Circulatory Death: A Nationwide Multicenter Cohort Study.

BACKGROUND: Donation after circulatory death (DCD) is a procedure in which after planned withdrawal of life-sustaining treatment (WLST), the dying process is monitored. A DCD procedure can only be continued if the potential organ donor dies shortly after WLST. This study performed an external validation of 2 existing prediction models to identify potentially DCD candidates, using one of the largest cohorts.
METHODS: This multicenter retrospective study analyzed all patients eligible for DCD donation from 2010 to 2015. The first model (DCD-N score) assigned points for absence of neurological reflexes and oxygenation index. The second model, a linear prediction model (LPDCD), yielded the probability of death within 60 min. This study determined discrimination (c-statistic) and calibration (Hosmer and Lemeshow test) for both models.
RESULTS: This study included 394 patients, 283 (72%) died within 60 min after WLST. The DCD-N score had a c-statistic of 0.77 (95% confidence intervals, 0.71-0.83) and the LPDCD model 0.75 (95% confidence intervals, 0.68-0.81). Calibration of the LPDCD 60-min model proved to be poor (Hosmer and Lemeshow test, P < 0.001).
CONCLUSIONS: The DCD-N score and the LPDCD model showed good discrimination but poor calibration for predicting the probability of death within 60 min. Construction of a new prediction model on a large data set is needed to obtain better calibration.

INTRODUCTION

Organ transplantation improves quality of life and increases life expectancy of patients with end-stage organ failure.[1] The demand for organ transplantations is likely to increase because of an aging population and an increased patient survival, leading to more retransplantations.[1] Donation after circulatory death (DCD) is a procedure in which after planned withdrawal of life-sustaining treatment (WLST), the course of the dying process is precisely monitored at an intensive care unit (ICU).[2] DCD exposes organs to accumulate warm ischemic injury during the period between withdrawal of treatment and actual cardiac arrest, the agonal phase.[3] The agonal phase refers to a time period that begins after decrease of oxygen saturation below SpO2 < 80% or systolic blood pressure below SBP < 80 mm Hg.[3] After the circulatory arrest, there is a no-touch period of 5 min. After these 5 min, the donor will be transported to the operation room.[3]

Identification of patients who will die shortly (within the prespecified time interval) after withdrawal of treatment is therefore of great importance, as longer time periods will result in organs too damaged for transplantation.[2] In many countries, donors who do not fulfill the legal criteria for donation after brain death are considered for DCD donation. DCD has proven to be a valuable source of donor organs in times of shortage. The Netherlands has a relatively high percentage (>50%) of DCD donors compared with surrounding Eurotransplant countries.[3,4] In The Netherlands, the maximum duration of the agonal phase is 60 min for lung, liver, and pancreas retrieval and 120 min for kidney procurement. In the United States, 30 min is a common threshold for liver donation. In approximately 20% of potential DCD donors, circulation persists for >120 min after WLST and the donation procedure is canceled because of the expected organ damage.[5] Avoiding the initiation of unsuccessful DCD procedures may reduce the discomfort for the patients’ relatives and for the patients who are already called into hospital in anticipation of a donor organ. Early identification of the probability that a patient will die within 120 min will also ensure logistic benefits for transplant care teams. Additionally, reliably predicting the duration of this period may avoid unnecessary costs as well.

The challenges of DCD include as follows: screening and selecting patients, supporting and maintaining the trust of bereaved families, and managing and minimizing the consequences of warm ischemia in such a way that is both acceptable professionally and ethically, and according to national laws. In practice, ICU clinicians estimate the time to death by making use of physiological and neurological tests. Currently, they do not use a predictive scoring system to assess the likelihood of the patient dying within an acceptable time period. The Acute Physiology, Age, Chronic Health Evaluation (APACHE) tool is frequently used to predict the mortality risk for DCD donors, although this does not predict the length of the agonal phase.[4]

In 2012, a scoring system for circulatory death in patients in neurocritical state (DCD-N score) has been developed by Rabinstein et al.[2] This model was aimed at predicting the probability of death within 60 min after WLST, which is in general the maximum allowed length of the agonal phase in the United States. Some centers have been using the maximum duration of 120 min for kidney procurement. The DCD-N score takes 4 neurological variables into account, known to be associated with the time of death after WLST, absence of brain stem reflexes (cough reflex, cornea reflex, and extensor or motor reflex to pain), which, combined with the oxygenation index (OI) > 3.0, are used as predictors for the calculation of the probability of the patient’s death within 60 min. The same group also constructed a second model, the “linear prediction for donation after circulatory death model” (LPDCD model), in which they adjusted the weight of the variables on the basis of strength of associations identified in earlier studies.[2] Although this study has many strengths, no proper validation was performed, an essential step to assess reliability and applicability. The previous study also included nonpotential donors, and this study wanted to validate these models using only potential DCD donors. The aim of this study was to perform a proper external validation of the 2 models using one of the largest DCD cohorts to date.

MATERIALS AND METHODS

Study Design

We set up a multicenter retrospective observational cohort study with ICU patients aged 18 y and older who were medically and legally eligible for DCD. All procedures took place between 2010 and 2015 at 6 participating hospitals in The Netherlands. All potential donors who were at the ICU, waiting for WLST, were included in the database. The retrospective inclusion of patients in our study ended in 2015 because of the start of a new large multicenter prospective study, which aims at developing a novel prediction model for the length of the agonal phase after WLST.[6] Exclusion criteria were infections, lack of mechanical ventilation, and euthanasia. All variables were collected from medical files or electronic patient data management systems (EPIC, Epic Systems Corporation, Madison, WI; ChipSoft, HiX software, Amsterdam, the Netherlands; and Metavision provided by iMDsoft, Needham, MA). Demographic characteristics included: age, gender, body mass index, admission date and time of the ICU, and WLST date and time. Neurological diagnoses were classified according to the International Statistical Classification of Diseases and Related Health Problems by the World Health Organization. Hemodynamic parameters, necessary for the calculating of the OI, were assessed just before the patient’s WLST. Brain stem, motor, and extensor reflexes were tested by a neurologist according to the Dutch protocol for determining brain death.[7]

Our study was registered in the University Medical Center Groningen research register. Because of the descriptive character of this study, our institution’s Medical Ethics Committee granted dispensation for the Dutch law regarding patient-based medical research (WMO) obligation. Patient data were processed and electronically stored according to the Declaration of Helsinki for medical research involving human subjects. The clinical and research activities were consistent with the Principles of the Declaration of Istanbul as outlined in the “Declaration of Istanbul on Organ Trafficking and Transplant Tourism.”

Measurements

The OI was computed as 100× (FiO2 × mean airway pressure in cm H2O)/PaO2 in torr).[2] The mean airway pressure is half the combination of peak airway pressure (pPeak) in cm H2O and the positive end-expiratory pressure in Torr. An OI > 3.0 was defined as elevated.[2] These variables were assessed at the last examination before WLST. For the external validation of the LPDCD model, this study used the following formula:

Exp(logit)/1+Exp(logit). Logit = −2.49 + (0.90 * Absent cornea reflex) + (1.65 * Absent cough reflex) + (0.98 * Absent extensor or motor reflex) + (0.12 * OI).[2] Receiver-operator curve (ROC) analysis was used for the external validation of the DCD-N score.[8] For validation of the LPDCD model, all reflexes were used dichotomized and the OI as a categorical variable. The validation of the DCD-N score required absence of the neurological reflexes and the OI as dichotomized variables.

Statistical Analysis

Univariate and multivariate logistic regression analyses were used to calculate the odds ratio for death within 60 min as a binary outcome variable. ROC DeLong analysis was used to assess how well the DCD-N score predicts the chance for the potential donor to die within 60 or 120 min after WLST. The corresponding area under the curve (AUC) is then calculated as a measure for each model’s discrimination. An AUC of 0.7–0.8 was regarded as acceptable. An area of 0.8–0.9 was considered excellent, and >0.9 as outstanding.[2,8] Discrimination is the degree to which risk estimates from a model characterize different patient prognoses.[8] For calibration, this study used Hosmer and Lemeshow (HL) goodness of fit test to predict the chance of dying within 30, 60, or 120 min after WLST. The database had 2.6% missing data. All analyses were repeated after applying multiple imputations to deal with missing data.

RESULTS

Baseline Characteristics

This study enrolled a total of 406 patients at the University Medical Center Groningen, Elisabeth Twee Steden Hospital (Tilburg), Isala Hospital (Zwolle), Catharina Hospital (Eindhoven), Radboud University Medical Center (Nijmegen), and at Medisch Spectrum Twente (Enschede), all located in The Netherlands. Twelve patients who did not fulfill the inclusion criteria were excluded. Hence, analyses in this study are based on a total of 394 patients. All donations took place between 2010 and 2015. Two hundred eighty-three (72%) patients died within 60 min after WLST. Time to death after WLST ranged from 2 to 1253 min with a median of 32 min. There was no significant difference in age, sex, and physiological diagnosis between patients that died within or after 60 min. Within this cohort, 2.6% of data could not be reliably retrieved from the records. This concerned the variables of neurological reflexes and variables needed for the calculation of the OI. The scorings systems attributed points to absence of cough reflex (2 points), absence of cornea reflex (1 point), absent extensor or motor reflex to pain (1 point), and an OI >3.0 (1 point).

Table 1 summarizes the demographic variables of the study population divided into 2 cohorts: death within 60 min and death after 60 min. Table 2 summarizes the demographics divided into death within 120 min and death after 120 min. The LPDCD model with a cutoff score of a probability of 0.80 showed a sensitivity of 83.1%, a specificity of 50.8%, a positive predictive value of 37.5%, and a negative predictive value of 89.4% to predict death within 60 min after WLST.

TABLE 1.

Baseline demographics stratified into death within 60 min and death after 60 min

	Patients (n = 394)
Age, y	54.3 (18–75)
Sex, female	140 (36%)
	Death within 60 min (n = 283)	Death after 60 min (n = 111)	t test (P)	Chi-square (P)
Age, y	53.99 (18–75)	54.86 (22–75)	0.748	0.197
Sex, female	97 (34%)	43 (39%)	0.087	0.348
Sex, male	186 (66%)	68 (61%)
Primary diagnosis			0.031	0.138
TBI	74 (26%)	27 (24%)
ICH	23 (8%)	14 (13%)
SAH	46 (16%)	23 (21%)
Ischemic CVA	35 (12%)	9 (8%)
Anoxic damage after CPR/cardiac arrest	70 (25%)	34 (31%)
Spinal cord injury	24 (9%)	1 (1%)
Respiratory failure	3 (1%)	0 (0%)
Intoxication, suicide	5 (2%)	2 (2%)
Unknown	3 (1%)	1 (1%)

CVA, cerebrovascular accident; ICH, intracranial hemorrhage; SAH, subarachnoid hemorrhage; TBI, traumatic brain injury; CPR, Cardiopulmonary resuscitation.

TABLE 2.

Baseline demographics stratified into death within 120 min and death after 120 min

	Patients (n = 394)
	Death within 120 min(n = 319)	Death after 120 min(n = 75)	t test (P)	Chi-square (P)
Age, y	53.92 (18–75)	55.39 (22–75)	0.503	0.350
Sex, femaleSex, male	110 (34%)210 (66%)	30 (40%)44 (60%)	0.206	0.483
Primary diagnosis			0.151	0.296
TBI	86 (27%)	15 (20%)
ICH	29 (9%)	8 (11%)
SAH	56 (18%)	15 (20%)
Ischemic CVA	37 (12%)	7 (9%)
Anoxic damage after CPR/cardiac arrest	76 (24%)	26 (35%)
Spinal cord injury	24 (8%)	1 (1%)
Respiratory failure	3 (1%)	0 (0%)
Intoxication, suicide	5 (2%)	2 (2%)
Unknown	3 (1%)	1 (1%)

CVA, cerebrovascular accident; ICH, intracranial hemorrhage; SAH, subarachnoid hemorrhage; TBI, traumatic brain injury. CPR, Cardiopulmonary resuscitation.

The DCD-N score model showed with the same cutoff score a sensitivity of 87.6%, a specificity of 45.6%, a positive predictive value of 36.4%, and a negative predictive value of 91.7% to predict death within 60 min after WLST.

The binary regression analysis is shown in Table 3 with all the variables combined with death within 60 min. It showed that absence of cough reflex, cornea reflex, extensor—or motor reflex, and an OI >3.0 are associated with a higher probability of death within 60 min. ROC analysis for the DCD-N scorings system showed an AUC of 0.71 (95% confidene intervals [CIs], 0.66-0.77) for prediction death within 30 min (Figure 1A). ROC analysis for the DCD-N scorings system showed an AUC of 0.77 (95% CI, 0.71-0.83) for prediction death within 60 min (Figure 1B). The ROC analysis for the DCD-N scorings system of death within 120 min showed an AUC of 0.80 (95% CI, 0.74-0.86) (Figure 1C). ROC analysis for the LPDCD 30-min model showed an AUC score of 0.71 (95% CI, 0.65-0.76) (Figure 2A). Calibration of the LPDCD 30-min model showed that the model underpredicted and overpredicted the probability of death (HL test; P < 0.001) (Figure 2B). ROC analysis for the LPDCD 60-min model showed an AUC score of 0.75 (95% CI, 0.68-0.81) (Figure 2C). Calibration of the LPDCD 60-min model showed that the model underpredicted and overpredicted the probability of death (HL test; P < 0.001) (Figure 2D). ROC analysis for the LPDCD 120-min model showed an AUC score of 0.83 (95% CI, 0.77-0.88) (Figure 2E). Calibration of the LPDCD model showed that the model underpredicted the probability of death (HL test; P < 0.001) (Figure 2F). As the DCD-N score does not result in an actual probability, but merely stratifies risk in terms of an integer between 0 and 5, no calibration measures could be calculated for this model.

TABLE 3.

Binary logistic regression analysis with distribution of variables of interest according to time to death after WLST

	Death within 60 min (n = 283) (%)	Death after 60 min (n = 111) (%)	Odds ratio (95% CI)	P
Absent corneal reflex	212 (77)	35 (35)	3.237 (1.820-5.756)	<0.0005
Absent cough reflex	218 (80)	38 (37)	4.306 (2.419-7.664)	<0.0005
Extensor or absent motor response	267 (94)	92 (84)	2.468 (0.995-6.123)	0.051
Oxygenation index >3.0	187 (70)	76 (74)	0.688 (0.371-1.276)	0.236

CI, confidence interval; WLST, withdrawal of life-sustaining treatment.

FIGURE 1.

A, Receiver operating characteristic curve based on the multivariable DCD-N model for prediction of death within 30 min. B, Receiver operating characteristic curve based on the multivariable DCD-N model for prediction of death within 60 min. C, Receiver operating characteristic curve based on the multivariable DCD-N model for prediction of death within 120 min. DCD-N, circulatory death in patients in neurocritical; ROC, receiver-operator curve.

FIGURE 2.

A, Receiver operating characteristic curve based on the LPDCD multivariable model for prediction of death within 30 min. B, Calibration plot with the combination of the observed proportion and the predicted mortality for the LPDCD 30-min model. C, Receiver operating characteristic curve based on the LPDCD multivariable model for prediction of death within 60 min. D, Calibration plot with the combination of the observed proportion and the predicted mortality for the LPDCD 60-min model. E, Receiver operating characteristic curve based on the LPDCD multivariable model for prediction of death within 120 min. F, Calibration plot with the combination of the observed proportion and the predicted mortality for the LPDCD 120-min model. LPDCD, linear prediction for donation after circulatory death model.

DISCUSSION

The DCD-N and LPDCD models are originally made for prediction time to death within 60 min after WLST. However, this study shows that the DCD-N and LPDCD models can make an acceptable prediction for death within 30, 60 min but also 120 min after WLST. These models show good and excellent discrimination, which means that the models are able to predict which patients will die within 60 min. Inadequate discrimination is a more important failing than poor calibration because calibration can be improved by updating the model.[9] Calibration of the LPDCD model shows that the validation model underpredicted and overpredicted the probability of death. This can be due to the fact that more variables have to be combined as strong predictors for time to death within 60 min. Better calibration is necessary for making the model more suitable for daily practice. Construction of a new prediction model on this large data set is needed to obtain better calibration.

In order to obtain a better reflection of the clinical conditions in practice, this study included one of the largest cohorts of 394 patients. Previous external validation studies included smaller cohorts with a maximum of 211 patients.[2,5,10-12] These previous studies included also nonpotential donors or included a very selected population, whereas this study focused on validating these models using nothing but potential DCD donors.

The DCD program inevitably includes a number of potential donors who do not die within the established period of 60 min.[10] In these situations, identification of appropriate DCD candidates is essential. This large retrospective multicenter study confirms that loss of brain stem reflexes and an OI >3.0 are associated with death within 60 min after WLST. These results can support future donor management and provide information to relatives. In addition to the DCD-N and the LPDCD model, various other variables may have to be taken into account when prediction of death within 60 min is attempted. Several studies have demonstrated both a positive and negative effect on the use of sedatives and analgesics.[10,13,14] One study based on a Dutch population concluded that higher dosages of sedatives and opioids were associated with death in >60 min and concluded that it is useful as a predictor for death after 60 min.[10]

The APACHE prognostic system is the current tool to predict mortality risk for critically ill adults. This system is primarily used to determine the required level of care in ICU patients but is neither developed nor validated to predict the agonal phase in DCD patients. Nonetheless, the APACHE score is often applied for this purpose. Other studies set up to identify appropriate DCD candidates include the United Network for Organ Sharing (UNOS) and the University of Wisconsin criteria.[4] Interestingly, neither the University of Wisconsin criteria nor the UNOS criteria model incorporate the degree of neurological injury. These scoring systems cover neurological information at the level of consciousness, which is not a predictor of early death after WLST in neurocritical patients.[15] Moreover, these scoring systems require disconnection from mechanical ventilation longer than 10 min.[4,16] One of these studies used the UNOS criteria for predicting time to death within 60 min after WLST.[16] This study showed that the absence of any criterion was associated with a low probability of death within 60 min.[16] However, such prolonged ventilator disconnection may cause distress for patients and relatives. In contrast, both the DCD-N score and the LPDCD model facilitate assessment of patients while they remain supported by mechanical ventilation. Because of this difference, these models appear to provide a more efficient, easily applicable, and possibly more acceptable way of predicting time to death.

The absence of neurological variables such as corneal and cough reflexes and motor- or extensor response to pain as predictors for time to death have been reported in previous studies.[12,17] A study among 149 patients showed that absence of pupil reflexes was a significant predictor for the course after WLST.[12] Unfortunately, external validation with a smaller sample size (n = 82) showed no statistical differences, which was most likely an effect of low power or differences in sample size.[17]

Prediction of time to death solely based on clinical judgment is a proven inaccurate method with a fairly high sensitivity but low specificity.[2,10] Based on these outcomes, it was even concluded that for each medically and legally eligible potential DCD donor, a DCD procedure should be started to avoid loss of potential organs. Although this approach is very understandable given the increasing shortage of suitable organs, each noneffectuated donation procedure causes discomfort for the relatives of the donor and the patients who are waiting for the anticipated transplantation. Also, the effort in donor recruitment and management, as well as the resulting costs, are considerably higher, with a lower and more uncertain organ yield when no selection is applied to avoid noneffectuated DCD donors. Not-effectuated donations have a deep impact on already grieving families and put psychological and physical strain on procurement teams and ICU staff.[18,19] This study will support the management of expectations of both the donor family and the treating physician and may support clinical decisions on the feasibility of planning a certain DCD procedure. Given the current shortage of deceased donor organs, the latter should be done with great caution, as to avoid an increase in donor nonutilization due to predicted, but not fully reliable high odds of a prolonged agonal phase.

In conclusion, validation of both existing models showed acceptable discrimination, but poor calibration with underestimation and overestimation of the probability of death within 30, 60, and 120 min. Our external validation of the DCD-N and LPDCD model is the first step in the process of developing a new predictive model using a large prospective cohort that can more accurately identify potential DCD candidates without losing available, viable donors.

ACKNOWLEDGMENTS

None.