Clinical Examination for the Prediction of Mortality in the Critically Ill: The Simple Intensive Care Studies-I.

OBJECTIVES: Caregivers use clinical examination to timely recognize deterioration of a patient, yet data on the prognostic value of clinical examination are inconsistent. In the Simple Intensive Care Studies-I, we evaluated the association of clinical examination findings with 90-day mortality in critically ill patients.
DESIGN: Prospective single-center cohort study.
SETTING: ICU of a single tertiary care level hospital between March 27, 2015, and July 22, 2017. PATIENTS: All consecutive adults acutely admitted to the ICU and expected to stay for at least 24 hours.
INTERVENTIONS: A protocolized clinical examination of 19 clinical signs conducted within 24 hours of admission. MEASUREMENTS MAIN
RESULTS: Independent predictors of 90-day mortality were identified using multivariable logistic regression analyses. Model performance was compared with established prognostic risk scores using area under the receiver operating characteristic curves. Robustness of our findings was tested by internal bootstrap validation and adjustment of the threshold for statistical significance. A total of 1,075 patients were included, of whom 298 patients (28%) had died at 90-day follow-up. Multivariable analyses adjusted for age and norepinephrine infusion rate demonstrated that the combination of higher respiratory rate, higher systolic blood pressure, lower central temperature, altered consciousness, and decreased urine output was independently associated with 90-day mortality (area under the receiver operating characteristic curves = 0.74; 95% CI, 0.71-0.78). Clinical examination had a similar discriminative value as compared with the Simplified Acute Physiology Score-II (area under the receiver operating characteristic curves = 0.76; 95% CI, 0.73-0.79; p = 0.29) and Acute Physiology and Chronic Health Evaluation-IV (using area under the receiver operating characteristic curves = 0.77; 95% CI, 0.74-0.80; p = 0.16) and was significantly better than the Sequential Organ Failure Assessment (using area under the receiver operating characteristic curves = 0.67; 95% CI, 0.64-0.71; p < 0.001).
CONCLUSIONS: Clinical examination has reasonable discriminative value for assessing 90-day mortality in acutely admitted ICU patients. In our study population, a single, protocolized clinical examination had similar prognostic abilities compared with the Simplified Acute Physiology Score-II and Acute Physiology and Chronic Health Evaluation-IV and outperformed the Sequential Organ Failure Assessment score.

Patients acutely admitted to the ICU have a high mortality, and survivors may suffer from long-term morbidity and reduced quality of life (1, 2). These critically ill patients frequently present with clinical signs of circulatory shock such as low blood pressure, oliguria, and skin mottling. These signs often guide treatment, assuming that they indicate vital organ hypoperfusion and are associated with increased mortality (3–7). Indeed, guidelines on the management of shock recommend treating patients based on clinical examination, supplemented with critical care ultrasonography (CCUS) (8).

Data on the prognostic value of clinical examination findings are inconsistent. Previous studies have identified different predictors of mortality such as low blood pressure (6, 9, 10), oliguria (3, 5), prolonged capillary refill time (CRT) (11, 12), and skin mottling (4, 13). They often evaluated one or two clinical signs in isolation, instead of assessing a combination of signs and symptoms, which would more accurately reflect daily clinical practice. Furthermore, most studies had relatively small sample sizes or included a selected subgroup such as patients with sepsis, cardiogenic shock, and severe trauma (eTable 1, Supplemental Digital Content 2, http://links.lww.com/CCM/E746).

The prognostic value of clinical examination remains to be established in a large, consecutive cohort of critically ill patients. Compared with well-established prognostic scores, which are complex to calculate and unsuited for individual patient prognostication (14, 15), a simple bedside clinical examination might better inform caregivers in their decision making. Accordingly, our aim was to evaluate which clinical examination findings were independently associated with 90-day mortality in acutely admitted ICU patients. In addition, we hypothesized that combined clinical examination findings would have similar prognostic value compared with existing prognostic scores.

MATERIALS AND METHODS

Design, Setting, and Patients

The prospective, observational, single-center Simple Intensive Care Studies-I (SICS-I) was conducted following a prewritten protocol and statistical analysis plan (SAP; see Supplement 1, Supplemental Digital Content 1, http://links.lww.com/CCM/E745, or clinicaltrials.gov: NCT02912624). All consecutive patients admitted to the ICU of the University Medical Center Groningen were eligible for inclusion. Adult patients who had an unplanned ICU admission and were expected to stay for at least 24 hours were included. Patients were excluded if their ICU admission was planned preoperatively, if acquiring research data interfered with clinical care due to continuous resuscitation efforts (e.g., mechanical circulatory support), or if informed consent was not provided. In unresponsive patients, informed consent was first obtained from the legal representatives and at a later time if the patient recovered consciousness. If the patient died before consent was obtained, the study data were used and legal representatives were informed on the study. The local institutional review board approved the study (M15.168207).

All included patients underwent clinical examination followed by CCUS within the first 24 hours of their ICU admission (eTable 2, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). Researchers conducted the clinical and CCUS examinations, and their findings were not revealed to caregivers.

Clinical Examination

All clinical examinations were standardized, and cutoff values for abnormal clinical signs were predefined in the protocol (clinicaltrials.gov: NCT02912624). A total of 19 clinical signs per patient were recorded (eTable 1, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). Respiratory rate, heart rate and rhythm, arterial blood pressures, and central venous pressures were recorded from the bedside monitor. Patients were auscultated for the presence of cardiac murmurs and crepitations. Clinical signs reflecting organ perfusion were obtained from the three organs readily accessible to clinical examination: cerebral (mental status), renal (urine output), and skin perfusion (CRT, central-to-peripheral temperature difference [ΔTc-p] and skin mottling). Mental status was assessed according to the categories “Alert,” “responsive to Voice,” “responsive to Pain,” and “Unresponsive” and was scored irrespective of sedation use. Urine output was scored 1 and 6 hours prior to the clinical examination, adjusted for body weight, and considered decreased if less than 0.5 mL/kg/hr. CRT was the time for skin color to fully return after applying firm pressure at the sternum, index finger, and knee for 15 seconds and considered prolonged if greater than 4.5 seconds (16). ΔTc-p was the difference between central temperature measured by a bladder thermistor catheter and peripheral temperature measured by a skin probe on the big toe and dorsum of the foot and considered abnormal if greater than 7°C (17, 18). The degree of skin mottling was rated at the knee according to a score from 0 to 5, where 0–1 was regarded as mild, 2–3 was regarded as moderate, and 4–5 was regarded as severe mottling (19).

Outcome Definition

The primary outcome (dependent) variable was 90-day all-cause mortality obtained through the municipal record database. Sensitivity analyses were conducted using all-cause mortality at 7- and 30-day follow-ups.

Sample Size and Missing Data

The sample size was based on the estimation that half of the number of acute ICU admissions per year (N = 1,500) would fulfill the inclusion criteria. The potentially detectable difference was calculated using skin mottling as an example for the case inclusion exceeded 1,000 patients: a significant mortality difference of 9% for skin mottling with 84% power and a maximal type 1 error risk of 0.015 could be detected (20). Missing values were considered missing at random because these depended on other observed patient characteristics (such as age and mechanical ventilation) and a significant Little’s test (21). Multiple imputations (20 times) for missing data were conducted, and parameter estimates and standard errors were combined using Rubin’s formula (22, 23).

Analytical Approach

The aims of our primary analyses were twofold: first, a multivariable logistic regression analysis was conducted to identify the clinical examination findings that independently predict mortality at 90-day follow-up and, second, the discriminative performance of this model was compared with that of the Simplified Acute Physiology Score-II (SAPS-II), Acute Physiology and Chronic Health Evaluation-IV (APACHE-IV), and Sequential Organ Failure Assessment (SOFA). Analyses were conducted with Stata Version 15.1 (StataCorp, College Station, TX, USA) on the imputed dataset following our published SAP (Supplement 1, Supplemental Digital Content 1, http://links.lww.com/CCM/E745).

Model Development and Validation.

Unadjusted and age- and sex-adjusted regression analyses were conducted on 19 clinical signs. A p-value of less than 0.25 threshold was used for inclusion in the multivariable models, which was constructed using forward stepwise regression by adding blocks of variables. The multivariable model was adjusted for age (covariate) and norepinephrine infusion rate (mediator) under the pathophysiological mechanism that norepinephrine alters most clinical signs. The final model was internally validated with bootstrap sampling. For bootstrap sampling, 1,075 patients were repeatedly drawn with replacement from the imputed dataset (for a more in-depth explanation, see eFig. 1, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). In total, 100 bootstrap samples were drawn, and the final model was reconstructed in each sample. Each variable from the final model was considered internally validated if it was significant in at least 80 of the 100 bootstrapped models (20, 24). Calibration of the multivariable models was checked with calibration plots and Hosmer–Lemeshow tests. Discrimination of the final model was evaluated with receiver operating characteristic (ROC) curves (25). Dominance analysis was used to determine the relative importance of independent variables in each multivariable model (26). Our multivariable model was compared with the SAPS-II (reference model), APACHE-IV, and SOFA scores by 1) analyzing differences between the area under the ROC curves (AUC) using the method proposed by DeLong et al (27) and 2) constructing reclassification tables and calculating the net reclassification improvement (28).

Sensitivity and Subgroup Analyses.

In sensitivity analyses, we assessed whether the statistically significant predictors of 90-day mortality were also predictive of 7- and 30-day mortalities. Time dependency was also investigated by conducting a multivariable Cox regression analysis on 90-day mortality.

Two planned subgroup analyses were conducted, in which only the clinical examination findings that were statistically significant in the primary analysis were evaluated. First, patients were stratified by vasopressor use. Second, patients were stratified by underlying pathology that could influence the clinical measurements, that is, acute liver failure or post-orthotopic liver transplantation (OLT), heart failure, septic shock, cardiac arrest, and CNS pathology.

Statistical Significance.

The SICS-I was designed to address multiple hypotheses on six different outcomes, and therefore, the mortality outcome was adjusted for multiple hypothesis testing (29). Supplement 1 (Supplemental Digital Content 1, http://links.lww.com/CCM/E745) contains the details or our SAP, but in short, a p-value of 0.015 indicated statistical significance and p-values between 0.015 and 0.05 indicated suggestive significance with an increased family-wise error rate (20, 30). For our secondary (subgroup) analyses, a p-value of less than 0.05 indicated statistical significance due to the hypothesis-generating purpose. Accordingly, primary analyses are presented with 98.5% CIs and secondary (subgroup) analyses with 95% CIs.

Amendments to the SAP.

For our primary analysis, multivariable logistic regression analyses were used instead of Cox regression because the outcome (90-day mortality) was fixed, time to event was considered less relevant, and our statistical methods would be more in line with that in the literature. Findings of the Cox regression analyses are reported in Supplement 1 (Supplemental Digital Content 1, http://links.lww.com/CCM/E745) and Supplement 2 (Supplemental Digital Content 2, http://links.lww.com/CCM/E746).

We intended to conduct a multivariable regression analysis of clinical examination findings adjusted for the SAPS-II. Since the SAPS-II also contains various clinical examination findings, we realized that such a model would have little clinical relevance and instead used this score as the reference model. We compared the performance of our clinical examination model to the SAPS-II, APACHE-IV, and SOFA.

RESULTS

Patient Characteristic and Outcome

A total of 1,442 patients were assessed for eligibility between March 27, 2015, and July 22, 2017. The inclusion criteria were fulfilled in 1,212 patients, of whom 137 patients were not included for various reasons (Fig. ). In the final analysis, 1,075 patients (89%) were included. The median time from ICU admission to inclusion was 15 hours (IQR = 8–20 hr). The proportion of missing values per variable is presented in eTable 2 (Supplemental Digital Content 2, http://links.lww.com/CCM/E746) and per case (eFig. 2, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). One third of the patients was admitted after acute or complicated surgery, and the most common admission diagnoses were of cardiovascular or respiratory origin (Table ).

Flow diagram of the SICS-I. CCUS = critical care ultrasonography, SICS = Simple Intensive Care Studies.

After 90 days, 298 patients (28%) had died and eight patients (1%) were lost from follow-up due to emigration to or residence in another country. Patients who died within 90 days were significantly older, were more often mechanically ventilated, had higher positive end-expiratory pressures and lower diastolic blood pressures and mean arterial pressures (p < 0.015; Table ). Most clinical signs reflecting organ perfusion differed: patients who died had significantly lower urine outputs, colder extremities, longer CRTs, and more severe skin mottling during the clinical examination.

Clinical Examination and 90-Day Mortality

Both unadjusted and age- and sex-adjusted analyses showed that most clinical examination findings were associated with 90-day mortality (eTable 3, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). Multivariable logistic regression adjusted for age and norepinephrine infusion rate showed that five clinical examination findings, that is, higher respiratory rate, higher systolic blood pressure, lower central temperature, altered consciousness, and decreased urine output, were independently associated with 90-day mortality (Fig. ). The variables atrial fibrillation, diastolic blood pressure, and severe skin mottling were of suggestive statistical significance due to a p-value of greater than 0.015 and a statistical significance in less than 80 of the 100 bootstrap replications (Fig. 2).

Figure 2.

Clinical examination findings associated with 90-day mortality. Five of the eight clinical examination findings in our model were independently associated with mortality (i.e., p < 0.015): respiratory rate, systolic blood pressure, central temperature, consciousness, and urine output. The model included all 1,075 patients of whom 298 died. Pseudo R2 = 0.14. Hosmer–Lemeshow goodness of fit test χ2 = 7.43; p = 0.68 (see plot in eFig. 2, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). area under the receiver operating characteristic curves = 0.74 (95% CI, 0.71–0.78). Mottling was scored according to Ait-Oufella et al (19). DBP = diastolic blood pressure, SBP = systolic blood pressure.

The multivariable logistic regression analysis was repeated with systolic blood pressure in quartiles because this variable had a U-shaped relationship with mortality (eFig. 3, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). In this model, only the highest quartile (i.e., a systolic blood pressure > 133 mm Hg) had a suggestive statistically significant association with mortality (OR = 1.65; 98.5% CI, 0.90–3.05; p = 0.046; eTable 4, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). In the complete case analysis, all variables except diastolic blood pressure remained statistically significant (eTable 5, Supplemental Digital Content 2, http://links.lww.com/CCM/E746).

Performance of Clinical Examination When Compared with Prognostic Scores

When comparing AUC’s to three established ICU prognostic risk scores, the clinical examination model was comparable to the SAPS-II and APACHE-IV and significantly better than the SOFA score (Fig. ). The clinical examination model distinguished 817 patients (76%) correctly into survivor or nonsurvivor. The number of patients correctly classified was 810 (75%) for the SAPS-II, 818 (76%) for the APACHE-IV, and 800 (74%) for the SOFA. The net reclassification improvement of the clinical examination model was 3.8% compared with the SAPS-II (p = 0.09), 5.0% compared with the APACHE-IV (p = 0.025), and 12% compared with the SOFA (p < 0.001).

The discriminative value of the multivariable models to distinguish 90-day survivors from nonsurvivors using area under the receiver operating characteristic curves (AUC) analyses. Figure legend presents the AUC with 98.5% CI. The DeLong method was used to compare our clinical examination model to three prognostic scores commonly used in the intensive care. APACHE = Acute Physiology and Chronic Health Evaluation, SAPS = Simplified Acute Physiology Score, SOFA = Sequential Organ Failure Assessment.

Sensitivity Analysis: Clinical Examination and Short-Term Mortality

The relation of clinical examination findings over time was studied using logistic regression analyses on 7- and 30-day mortalities and a Cox regression. Severe skin mottling had stronger associations with 7-day mortality (OR = 3.06; 95% CI, 1.34–6.98; p = 0.008) compared with 90-day mortality (OR = 2.45; 95% CI, 1.12–5.34; p = 0.025). Systolic and diastolic blood pressures were not statistically significantly associated with 7-day mortality (eTable 7, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). Atrial fibrillation and diastolic blood pressures were not statistically significantly associated with 30-day mortality (eTable 7, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). Results of the multivariable Cox regression were comparable to the logistic regression analysis used in the main analysis (eTable 8, Supplemental Digital Content 2, http://links.lww.com/CCM/E746).

Subgroup Analyses

In two predefined subgroup analyses, the patient population was stratified by vasopressor use and by underlying pathology. In these analyses, only the eight clinical examination findings that were statistically significant in the primary analysis were tested (eFig. 5, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). In patients without vasopressors, only a higher respiratory rate and an altered consciousness had statistically significant associations with mortality (p < 0.001; eTable 9, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). In patients receiving vasopressors, a higher respiratory rate, atrial fibrillation, lower central temperature, nonresponsiveness, decreased urine output, and severe skin mottling were independently associated with 90-day mortality (eTable 9, Supplemental Digital Content 2, http://links.lww.com/CCM/E746). In patients admitted with septic shock, only age and skin mottling over the knee were significantly associated with mortality (OR = 3.22; 95% CI, 1.31–7.94; p = 0.011). In the subgroups of patients admitted with acute liver failure or post-OLT, heart failure, with a CNS pathology, or after cardiac arrest, there were too few events (i.e., < 40) to assess any meaningful independent associations.

DISCUSSION

Clinical examination in 1,075 adult patients acutely admitted to the ICU had reasonable prognostic accuracy. Five of the 19 tested clinical examination findings, that is, increased respiratory rate, increased systolic blood pressure, lower core temperature, altered consciousness, and decreased urine output, were independently associated with 90-day mortality. The predictive and discriminative value of a simple clinical examination approached that of the SAPS-II and APACHE-IV and outperformed the SOFA score.

In line with previous studies, we found that clinical signs reflecting cerebral, renal, and skin hypoperfusion were independently associated with mortality in the critically ill (5, 6, 11). In our data, severe skin mottling had a suggestively significant association with an OR of 2.48 (95% CI, 1.13–5.44), whereas others who assessed the persistence of skin mottling over time found a stronger association with an OR of 16 (95% CI, 11–1,568) (19) and an OR of 3.29 (95% CI, 2.08–5.19) (4). The independent association of decreased urine output with 90-day mortality confirms findings from the FINNAKI studies (5, 31). Similar to the modified early warning score, an altered consciousness regardless of sedation significantly predicted mortality in the critically ill (32). All abovementioned variables may reflect the severity of critical illness on the first day of ICU admission and as such identify patients at higher risk for circulatory failure and mortality. An alternative explanation might be that patients with these clinical signs are treated differently.

The reasonable performance of our prediction models on 90-day mortality is in line with previous models derived from similar cohorts (33). All prognostic scores performed worse than expected from previous literature (14, 15). The inclusion criteria of the SICS-I may explain this discrepancy: we studied 90-day mortality in patients acutely admitted to the ICU, whereas most prognostic scores perform best in evaluating in-hospital mortality or specific populations such as patients with trauma or suspected infection (34, 35). The use of an unselected population may produce unbiased risk estimates and increases external validity (24). The main disadvantage of this approach is that average associations may be neutral or balanced out by different characteristics in different subgroups. The secondary analyses in clinically different subgroups were conducted to explore such associations: for example, a high systolic blood pressure was no longer independently associated with mortality in patients requiring vasopressors or in patients with septic shock (eTable 9, Supplemental Digital Content 2, http://links.lww.com/CCM/E746).

Implications and Generalizability

The SICS-I provides evidence that a thorough clinical examination conducted on the first day of ICU admission may be used to obtain a rough estimation of 90-day mortality. By establishing the prognostic value of 19 clinical examination findings, we set the first step for a parsimonious clinical examination, that is, the fewest number of clinical signs that yield the most prognostic value (36). These simple and easily obtainable clinical variables may better inform physicians in their clinical decision making. The examinations were conducted within 24 hours of ICU admission, usually in the morning, and after primary resuscitation efforts. There was no prespecified moment in time for the examination, which may decrease generalizability of the results. Nonetheless, this research practice does reflect daily clinical care where most patients are routinely assessed in the morning, regardless of the time that has passed since ICU admission.

The dynamic care process of the critically ill patient may limit an accurate prediction of 90-day mortality with a single clinical examination or a prognostic score, which reflects a baseline mortality risk based on medical history and findings from the first 24 hours of ICU admission. The clinical status and treatment of a critically ill patient change frequently, and repeated clinical examinations might predict the individual patient prognosis more accurately. Previous studies have already shown that prolonged mechanical ventilation with high pressures or persistently low blood pressures, skin mottling, a decrease in urine output and increasing central-to-peripheral temperature gradients have strong associations with mortality (4, 5, 9, 37–39). Our clinical examination was limited to a single time point, which could explain why not all these common prognostic variables were also statistically significant in the SICS-I. Future research should study the variation of clinical examination and associated interventions over time to assess its prognostic value (40).

Strengths and Limitations

The SICS-I was an unselected, single-center cohort of consecutive ICU patients, and its findings require external validation in an independent cohort. To address this limitation, we assessed the robustness of our findings by adjusting for multiple outcomes, conducting multiple imputations and sensitivity analyses, and internally validating each predictive variable by bootstrap sampling. We evaluated a heterogeneous ICU population, and certain prognostic associations may be more pronounced in patient subgroups, which is why we studied clinically relevant subgroups in our secondary analyses. Our findings do not apply to pediatric or electively admitted patients.

The clinical examination findings collected in our study were not shared with caregivers. However, some of these findings (i.e., blood pressure and heart rate) were also assessed by caregivers and may have informed subsequent treatment decisions or were influenced by their interventions. The predictive value of a clinical variable measured at baseline therefore included the value of this variable combined with the subsequent intervention(s) to correct such a value. Since treatment strategies between physicians and countries differ, this fact may explain why different studies identify different predictors of mortality, in addition to population differences and other confounders. The prognostic value of clinical variables will be more transparent in a randomized setting where interventions are given based on different clinical treatment targets (41, 42).

CONCLUSIONS

A simple clinical examination, which can be performed in any critically ill patient in any setting, has reasonable discriminative value for assessing 90-day mortality in a single-center cohort of acutely admitted ICU patients. In this study, a single, protocolized clinical examination had similar prognostic abilities compared with the SAPS-II and APACHE-IV and outperformed the SOFA score.

ACKNOWLEDGMENT

We would like to thank Chris H.L. Thio from the Department of Epidemiology of the University Medical Center Groningen for helping with the multiple imputations of our missing values. The SICS Study Group members include the following: project leaders: Geert Koster, MD; Frederik Keus, MD, PhD; and Iwan C.C. van der Horst, MD, PhD; research coordinator: Willem Dieperink, PhD; researchers who conducted patient inclusions: Roos Bleijendaal, MD; Yasmin F. Cawale, MD; Ramon P. Clement, MD; Devon Dijkhuizen, BSc; Ruben J. Eck, MD; Bart Hiemstra, MD; Anja Haker, BSc; Casper D.H. Hilbink, MD; Thomas Kaufmann, MD; Martiene Klasen; MD, Manon Klaver, MD; Laura J. Schokking, BSc; Victor W. Sikkens, MD; Madelon Vos, MD; Justin Woerlee, MD; and Renske Wiersema, BSc.

TABLE 1.

Clinical Characteristics

TABLE 2.

Clinical Examination Findings of Survivors and Nonsurvivors