AIM: The Acute Physiological and Chronic Health Evaluation (APACHE) II score is a popular illness severity scoring system for intensive care units. Scoring systems such as the APACHE II allow researchers and clinicians to quantify patient illness severity with a greater degree of accuracy and precision, which is critical when evaluating practice patterns and outcomes, both within or between intensive care units. The study aims to: assess changes in APACHE II scores and hospital-standardised mortality ratio at our ICU over a nine year period from 1 January 1997 to 31 December 2005; assess for changes in the performance of the APACHE II scoring system in predicting patient hospital mortality over the same period; and assess for any clinical subgroups in which APACHE II scoring was particularly inaccurate or imprecise. METHODS: Retrospective audit of a single centre relational database, with evaluation of the APACHE II scoring system by year through discrimination (ability to discriminate between the patients who will die or survive at hospital discharge) using receiver operating characteristic (ROC) curves, and calibration (ability to predict mortality rate over classes of risk) using goodness-of-fit as assessed by the Hosmer-Lemeshow statistic. RESULTS: Data from 7703 patients were available for analysis. There was a decrease in overall hospital mortality, from approximately 19% at the beginning of the period of observation to approximately 12% at the end. There was also a decrease in the hospital standardised mortality ratio from 0.94 (95%CI 0.82-1.06) to 0.66 (95%CI 0.55-0.76). In general, both the APACHE II score and risk of death model performed adequately in each year with ROC curve AUCs of >0.8, albeit with progressively poorer performance over time and 'model fade' that approached statistical significance. There was progressively poorer calibration with the APACHE II risk of death model as indicated by the Hosmer-Lemeshow statistic, with a statistically significant difference between the predicted and observed mortality from 2003 onwards. Overall, there was moderately poor model performance in the diagnostic groups with the largest number of patients (sepsis and post-surgical complications). CONCLUSION: This study shows the progressively worse performance of the APACHE II illness severity scoring system over time due to 'model fade'. This is especially so in common diagnostic categories, making this a clinically relevant finding. Future approaches to illness severity scoring should be tested and compared, such as re-estimating coefficients of the APACHE II diagnostic categories or using locally developed ones, moving to later evolutions of the system such as the APACHE III or APACHE IV, or developing novel artificial intelligence approaches.
AIM: The Acute Physiological and Chronic Health Evaluation (APACHE) II score is a popular illness severity scoring system for intensive care units. Scoring systems such as the APACHE II allow researchers and clinicians to quantify patient illness severity with a greater degree of accuracy and precision, which is critical when evaluating practice patterns and outcomes, both within or between intensive care units. The study aims to: assess changes in APACHE II scores and hospital-standardised mortality ratio at our ICU over a nine year period from 1 January 1997 to 31 December 2005; assess for changes in the performance of the APACHE II scoring system in predicting patient hospital mortality over the same period; and assess for any clinical subgroups in which APACHE II scoring was particularly inaccurate or imprecise. METHODS: Retrospective audit of a single centre relational database, with evaluation of the APACHE II scoring system by year through discrimination (ability to discriminate between the patients who will die or survive at hospital discharge) using receiver operating characteristic (ROC) curves, and calibration (ability to predict mortality rate over classes of risk) using goodness-of-fit as assessed by the Hosmer-Lemeshow statistic. RESULTS: Data from 7703 patients were available for analysis. There was a decrease in overall hospital mortality, from approximately 19% at the beginning of the period of observation to approximately 12% at the end. There was also a decrease in the hospital standardised mortality ratio from 0.94 (95%CI 0.82-1.06) to 0.66 (95%CI 0.55-0.76). In general, both the APACHE II score and risk of death model performed adequately in each year with ROC curve AUCs of >0.8, albeit with progressively poorer performance over time and 'model fade' that approached statistical significance. There was progressively poorer calibration with the APACHE II risk of death model as indicated by the Hosmer-Lemeshow statistic, with a statistically significant difference between the predicted and observed mortality from 2003 onwards. Overall, there was moderately poor model performance in the diagnostic groups with the largest number of patients (sepsis and post-surgical complications). CONCLUSION: This study shows the progressively worse performance of the APACHE II illness severity scoring system over time due to 'model fade'. This is especially so in common diagnostic categories, making this a clinically relevant finding. Future approaches to illness severity scoring should be tested and compared, such as re-estimating coefficients of the APACHE II diagnostic categories or using locally developed ones, moving to later evolutions of the system such as the APACHE III or APACHE IV, or developing novel artificial intelligence approaches.
Authors: Brent McSharry; Lahn Straney; Janet Alexander; Tom Gentles; David Winlaw; John Beca; Johnny Millar; Frank Shann; Barry Wilkins; Andrew Numa; Christian Stocker; Simon Erickson; Anthony Slater Journal: J Am Heart Assoc Date: 2019-05-07 Impact factor: 5.501
Authors: Lowell Ling; Chun Ming Ho; Pauline Yeung Ng; King Chung Kenny Chan; Hoi Ping Shum; Cheuk Yan Chan; Alwin Wai Tak Yeung; Wai Tat Wong; Shek Yin Au; Kit Hung Anne Leung; Jacky Ka Hing Chan; Chi Keung Ching; Oi Yan Tam; Hin Hung Tsang; Ting Liong; Kin Ip Law; Manimala Dharmangadan; Dominic So; Fu Loi Chow; Wai Ming Chan; Koon Ngai Lam; Kai Man Chan; Oi Fung Mok; Man Yee To; Sze Yuen Yau; Carmen Chan; Ella Lei; Gavin Matthew Joynt Journal: J Intensive Care Date: 2021-01-06
Authors: Johannes Mierke; Thomas Nowack; Tobias Loehn; Franziska Kluge; Frederike Poege; Uwe Speiser; Felix Woitek; Norman Mangner; Karim Ibrahim; Axel Linke; Christian Pfluecke Journal: Int J Cardiol Heart Vasc Date: 2022-03-30