Delirium and mortality risk prediction: a story in evolution.

We read with interest the article by van den Boogaard and colleagues, which proposed that delirium measured within 24 hours of admission did not improve the Acute Physiology and Chronic Health Evaluation (APACHE) II in-hospital mortality prediction [1]. Their data should be interpreted after considering the study design and statistical limitations.

First, the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) measurements include assessing the level of consciousness (using any valid sedation scale), which is highly correlated with the Glasgow Coma Score. Therefore it is not surprising that addition of delirium to the APACHE score (which includes the Glasgow Coma Score) on the first intensive care unit day does not alter predictions; however, earlier detection of delirium at the initial evaluation of Emergency Department patients is an important predictor of death [2]. We have found that the level of consciousness (via the Richmond Agitation-Sedation Scale) has been predictive of in-hospital mortality, but this relationship is not as strong as the independent value of delirium duration (via the CAM-ICU) for predicting long-term survival, even after adjusting for APACHE II score and sedatives [3,4].

Second, the authors base their conclusions upon comparisons of areas under the curve using the c statistic. Recent insights suggest that this analytic method is insensitive and open to type II error [5]. A more sensitive method to assess additive predictive ability applies likelihood ratio testing between models with and without additional risk factors. In addition, substantial improvements in risk reclassification may be apparent despite limited increases in the c statistic.

In sum, it may be true (but confirmation is required) that adding delirium to a measurement such as the APACHE score is not of value. Clinicians and hospital quality officers should continue to consider early detection of delirium and ongoing delirium detection as an important prognostic tool.

Authors' response

Mark van den Boogaard, Pieter Leffers and Lisette Schoonhoven

We thank Dr Vasilevskis and coworkers for their interest in our publication [1]. We are fully aware of the limitations of the c statistic as a measure for clinical usefulness of a predictive model - that is why we did not base our conclusions only on the lack of improvement of the c statistic, but also on the deteriorating ability to predict mortality [1].

As Cook pointed out in her publication, the evaluation of the clinical usefulness of risk-stratification models is not at all straightforward [5]; others make it clear that the last word about proper analysis and its interpretations has not yet been written [6,7]. This complicated issue needs further methodological development and thorough discussion. In addition to this, we would like to stress that showing the independent contribution of delirium after controlling for covariables in a Cox regression model is not a valid method to show the clinical usefulness of delirium as a predictor of mortality, not even when the corrected hazard ratio is high [5,8]. Also, showing the improved model fit from adding a variable to a model with the log-likelihood test does not serve that purpose [8].

As Vasilevskis and colleagues correctly point out, the probable reason why delirium does not add to the predictive properties of the APACHE score is that the latter already contains variables that essentially measure the same information about the clinical state of the patient. The predictive validity of a model is usually and mainly determined by its power to discriminate and/or by its ability to predict outcome (calibration). The reclassification index is a potentially interesting tool for evaluation of predictive models. Unfortunately this index is highly dependent on the width of the chosen categories of predicted risk. We do not know of category boundaries that would have a direct meaning for clinical decision-making [5,8]. Because proper interpretation of the index will not be possible, we have chosen not to include such an analysis.

In summary, despite shortcomings of various methods to determine the predictive value, our conclusion remains that delirium does not improve the predictive value of the APACHE score.

Abbreviations

APCHE: Acute Physiology and Chronic Health Evaluation; CAM-ICU: Confusion Assessment Method for the Intensive Care Unit.

Competing interests

The authors declare that they have no competing interests.