Are Unselected Risk Scores in the Cardiac Intensive Care Unit Needed?

Modern cardiology, perhaps more than any field in medicine, has become permeated with clinical risk scores. Yet, there are few risk scores available to guide clinicians caring for the heterogeneous patient population within the cardiac intensive care unit (CICU). , Unlike disease‐specific scores that are common in cardiology, general intensive care unit (ICU) risk scores have been developed to predict mortality across a broad mix of diagnoses. However, the uniqueness of the contemporary CICU population raises important questions about the validity, usefulness, and applicability of scores developed and validated in non‐CICU populations. In addition, the foremost challenge with any risk prediction tool is establishing how to use the prognostic information it provides. Should these scores help providers establish eligibility for certain procedures or therapies? Should they identify futility? Are scores simply giving providers comfort? In this Viewpoint, we discuss the potential applications, limitations, and use of risk scores in the CICU.

What Scores Are Currently Available for the CICU Clinician?

Standard ICU risk scores, including the Acute Physiology and Chronic Health Evaluation‐III/IV, Sequential Organ Failure Assessment, and Oxford Acute Severity of Illness Score, vary substantially in complexity. Each score has shown reasonable discrimination but poor calibration for short‐term mortality in the CICU population (Figure). However, these scores are all less predictive in CICU patients with critical cardiac diagnoses such as cardiac arrest, shock, and respiratory failure. A recent analysis from our group compared a time‐updated electronic health record (EHR) score, the Rothman Index, to the Sequential Organ Failure Assessment score in a single‐center CICU. We found that the Sequential Organ Failure Assessment score had modestly better discrimination, but the Rothman Index was better calibrated in this population. To our knowledge, there is only one risk score developed and validated specifically for the CICU. Although promising and easily calculated using covariates available on admission, the Mayo CICU Admission Risk Score has only been described in one academic tertiary care hospital, and must be replicated in other centers before widespread use.

Figure 1

Intensive care risk scores validated in unselected CICU populations.

ACS indicates acute coronary syndrome; APACHE, Acute Physiology and Chronic Health Evaluation; CICU, cardiac intensive care unit; EHR, electronic health record; HR, heart rate; ICU, intensive care unit; MAP, mean arterial pressure; M‐CARS, Mayo CICU Admission Risk Score; MCS, mechanical circulatory support; OASIS, Oxford Acute Severity of Illness Score; and SOFA, Sequential Organ Failure Assessment.

Although the focus of our Viewpoint is on unselected ICU risk scores, it is important to mention disease‐specific risk scores for common admissions to the modern CICU such as acute myocardial infarction, cardiogenic shock, cardiac arrest, and decompensated heart failure. The Global Registry of Acute Coronary Events score may be the best validated acute myocardial infarction risk score but was not designed to predict ICU‐related outcomes. The Observatorie Régional Breton sur I'Infarctus risk score was developed in France to predict patients at increased risk of cardiogenic shock after presenting with ST‐segment–elevation myocardial infarction. Using 11 covariates, including clinical, laboratory, and angiographic variables, the Observatorie Régional Breton sur I'Infarctus risk score demonstrated high discrimination (C statistic=0.84) for predicting cardiogenic shock. There are also numerous risk scores for the prediction of short‐term mortality in patients with cardiogenic shock, such as the Intra‐Aortic Balloon Pump in Cardiogenic Shock II risk score, which was derived in patients presenting with acute myocardial infarction. Although not a risk score, the Society for Cardiovascular Angiographic and Intervention cardiogenic shock classification has shown robust risk stratification in an unselected CICU population. Various scores are also available to help predict outcomes for patients requiring short‐term mechanical support such as extracorporeal membrane oxygenation. For example, the Survival After Veno‐Arterial Extracorporeal Membrane Oxygenation risk score was developed in an international cohort to predict mortality for patients requiring extracorporeal membrane oxygenation for cardiogenic shock. The MIRACLE2 score includes covariates available upon admission and is highly predictive for poor neurologic outcomes among patients presenting with cardiac arrest. Finally, there are several decompensated heart failure risk scores, some of which have also been used to estimate risk in unselected CICU patients.

Overall, the variables included in these risk scores overlap substantially and encompass disease‐specific risk factors (such as coronary flow grade in the Intra‐aortic Balloon Pump in Cardiogenic Shock II score), nonspecific markers of illness severity, and nonmodifiable risk markers (such as age and comorbidities). There is additional overlap with generic ICU risk scores such as the Acute Physiology and Chronic Health Evaluation, which demonstrates similar performance to some disease‐specific risk scores in less‐sick CICU patient subgroups but tends to underperform in critically ill cohorts with cardiac arrest and cardiogenic shock. An important limitation of these disease‐specific risk scores is that CICU patients may present with multiple acute cardiovascular disease processes simultaneously, raising uncertainty about which risk score is most appropriate. For example, a patient presenting with acute myocardial infarction complicated by cardiogenic shock and cardiac arrest could be classified using one of several scores, but rarely have these been compared directly. The development of a single integrated risk score of unselected CICU patients that can predict outcomes across these diagnosis groups would simplify risk prediction in clinical practice by avoiding the need to use numerous scores.

Are There Benefits of an Unselected CICU Risk Score Beyond Mortality Prediction?

Given the increasing patient complexity in the CICU, an unselected CICU risk score offers several potential advantages beyond mortality prediction. A standardized risk score can facilitate benchmarking of CICU performance for research and quality improvement both between centers and within individual centers over time. With the lack of CICU‐specific scores or databases, CICU providers have historically lacked foundational epidemiologic data to compare outcomes or acuity across centers. Recent analyses from the Critical Care Cardiology Trials Network have demonstrated wide variability in both patient case mix and the use of common therapies such as advanced mechanical and respiratory support across CICUs. , The use of a score to standardize mortality risk between patient populations could improve our understanding of the effects of different care models and provide insights into temporal trends. Second, although risk scores do not always provide decision support, more advanced and properly calibrated models that can distinguish important components of risk may help inform both the choice and urgency of diagnostics and therapeutics. Notably, many risk factors portend a multitude of complications such as bleeding and recurrent infarction in patients with acute myocardial infarction; a model that can provide risk estimates for multiple clinically relevant end points could help clinicians decide on the risk/benefit profile of aggressive interventions. As a simple example, imagine a score integrated into the EHR that could calculate and display the predicted mortality based on the choice of a certain therapeutic (eg, intra‐aortic balloon pump versus extracorporeal membrane oxygenation for cardiogenic shock). Third, an unselected risk score may help providers more accurately triage admission and discharge from the CICU in an evidence‐based manner, particularly when considered in the context of interfacility transfers and high ICU‐bed occupancy. Finally, scores specifically validated for the CICU may help providers more accurately counsel families about expected outcomes and facilitate goals of care discussions.

Are New Methods Needed to Build Risk Scores?

Although baseline or admission risk scores can offer substantial value for clinicians, a dynamic or repeated score may more realistically capture the evolving risk seen in clinical practice. In comparison to most scores that include static variables, dynamic scores offer the potential to not only recalculate risk but also assess trends over time that are likely to carry even greater prognostic value. This is a critical limitation of general ICU severity‐of‐illness scores such as Acute Physiology and Chronic Health Evaluation and Sequential Organ Failure Assessment, which use the worst values on a given day without regard to whether the patient is improving or deteriorating. A patient who has high severity of illness on admission and responds to therapy will not be distinguished from a patient who is stable on admission and deteriorates to a similar level of illness severity. Trends such as these may play a considerable role when deciding on potential therapeutic interventions or when counseling family members of a deteriorating patient. Finally, dynamic risk scores offer the potential for automatic recalculation as new values become available; the proprietary Rothman Index is one such example of a dynamic risk score that appears to provide useful risk stratification in the CICU population. In the ideal circumstance, a worsening dynamic real‐time risk score suggesting patient deterioration could be linked to alerts within the EHR to automatically recommend potential interventions and therefore improve early recognition while streamlining workflow.

Traditionally, most risk scores have been created using standard statistical methods such as multivariable logistic regression (typically with stepwise variable selection). Before the medical record was digitized, it made sense to have simple, parsimonious risk scores that included readily available data points and could be easily calculated. However, this approach lost the richness and complexity of clinical data leading to modest performance. These techniques likely do not capture the enormous amount of data, sometimes unstructured (eg, notes), in the EHR or account for inconsistent completeness, complex nonlinear relationships, or repeat measures and recalibration. However, more sophisticated analytical techniques incorporating machine learning could potentially overcome these complex challenges by using continuous streaming data from the EHR. In addition, methods such as natural‐language processing can extract often neglected types of data such as the physical examination or radiographic findings and have shown encouraging results in intensive care prediction modeling.

If You Build It, Will They Use It?

Development of new CICU risk scores must balance the tension between including a limited number of easily obtained variables and inclusion of a more comprehensive list of variables to optimize prediction. Even for risk scores that are highly predictive and readily available, numerous barriers exist for busy clinicians to use them. Most ICU scores require 24 hours of data to optimize risk prediction. In addition, they only include the worst values within this time period, precluding these scores from distinguishing a patient who was severely ill on presentation and is clinically improving from a patient who deteriorated after initially appearing stable. Few clinicians stop to calculate these risk scores while treating critically ill patients, and their intuition and experience still drive most critical decisions.

Underlying all risk scores is the fallacy of applying population‐level mortality statistics to a single individual. It remains unclear what level of discrimination is necessary to be clinically useful, and small differences in predicted mortality by risk scores are rarely meaningful. Although scores may predict mortality, they typically do not help clinicians make medical decisions, because risk scores do not effectively link treatment efficacy to mortality risk. Unlike acute coronary syndromes, where risk scores can identify high‐risk patients who benefit from early coronary angiography, no such risk‐treatment paradigm exists for most common CICU subgroups such as cardiogenic shock and cardiac arrest. The highest‐risk patients with these diagnoses may have irreversible neurologic injury or organ failure, making them unsuitable for aggressive interventions. The crucial patient‐level factors that determine candidacy for advanced therapies are rarely included in risk scores, yet are primary drivers of individual outcomes. If predictors of mortality are not modifiable, using an elevated predicted mortality as justification for more aggressive interventions may lead to extensive and ultimately futile care in many cases without improving outcomes. Risk scores often quantify what clinicians already know, that providers can typically differentiate high‐risk and low‐risk patients without the need to calculate a precise mortality risk estimate, and in our experience, risk scores seldom identify unexpectedly high risk. Determination of futility based on a high predicted mortality is likewise not palatable to clinicians given the imperfect discrimination and calibration of currently available scores, although considering palliative medicine consultation for high‐risk patients may be reasonable.

Strategies developed to increase the adoption of risk scores include user‐friendly apps for smart devices and integration into the EHR, which has near universal adoption in the United States. Although app‐based formats add a level of convenience, they remain plagued by the same reasons that clinicians do not routinely use risk scores, including the need for manual data entry, especially when there is a prohibitive number of variables. The EHR offers the potential for automatic score calculation, recalculation as new data become available, and alerts to warn clinicians before clinical deterioration. EHR‐based scores facilitate assessment of prognostically important trends, enable the use of algorithms using more sophisticated statistical techniques, and may provide individualized risk prediction. However, EHR‐derived risk scores may have limited generalizability to different EHR systems. Given the multitude of alerts and common EHR alert fatigue, it is also unclear how clinicians would use an EHR‐derived score.

Do We Need CICU‐Specific Scores?

Given the shortcomings of general ICU risk scores for common CICU diagnoses, we believe that a CICU‐specific score could be helpful if we can learn how to harness the information to optimize and improve patient care. Ideally, a single risk stratification platform that provides individualized predictions specific to each patient's admission diagnoses could be developed by leveraging data within the EHR. Such an approach would require a large multicenter CICU cohort with detailed data using sophisticated analytical techniques, particularly if the goal was to provide real‐time risk prediction. Any future CICU‐specific risk score should be validated across hospital settings including geographically diverse patient populations and hospitals of different sizes and capabilities. The clinical usefulness of risk scores will be improved if future clinical trials prospectively use scores to predict the benefit/harm of potential treatments, finally providing the much‐needed link between risk stratification and individualized treatment.