BACKGROUND: TRISS has reigned as the preeminent trauma outcome prediction model for 20 years. Despite this endorsement, the calibration of TRISS has been poor in most data sets where it has been examined. We hypothesized that the lack of calibration of TRISS was because of the inappropriate mathematical specification of the model that TRISS is based on, rather than the predictors in the model. In particular, we hypothesized that the nonlinearity of the Injury Severity Score (ISS) in the log odds of death was responsible for the poor calibration of TRISS, and further, that this nonlinearity could be corrected by the simple addition of an ISS squared term to the TRISS model. METHODS: We examined ISS in the log odds of mortality for linearity in one large trauma data set, the National Pediatric Trauma Registry (NPTR) (n = 53,113 from 1985-1996; mortality, 1.3%); and two small data sets, the University of New Mexico (UNM) (n = 3,142 from 1991-1995; mortality, 8.6%) and Portland, Oregon (PORT) (n = 2,916 from 1990-1994; mortality, 1.75%). In addition, in the NPTR we compared the calibration of TRISS models with and without linearity in the log odds of death. RESULTS: In the NPTR, ISS was profoundly nonlinear in the log odds of death for both blunt and penetrating trauma (p < 0.001). Moreover, the overall calibration of the TRISS model for the NPTR data was significantly improved when the nonlinearity of ISS was corrected by the addition of a quadratic ISS term as demonstrated by a 70% reduction (improvement) in the Hosmer-Lemeshow statistic. Interestingly, the addition of the ISS squared term did not affect the discrimination of the model. The log odds of survival in the UNM and PORT data sets were also better modeled when an ISS squared term was added (UNM, p = 0 0.052; PORT, p = 0.014), but improvements in the Hosmer-Lemeshow statistic were smaller, possibly because of the small size of these data sets. CONCLUSION: The TRISS model for outcome prediction currently uses ISS in a mathematically inappropriate way that impairs the calibration, but not the discrimination, of its predictions. If TRISS is to continue as the prediction standard for trauma, a quadratic ISS term must be added to the model. In the future, outcome prediction models should undergo thorough statistical modeling and evaluation before being released. Injury severity descriptors other than ISS (such as ASCOT, ICISS, or NISS) may require other modeling techniques to optimize the calibration of survival models that use these injury scores.
BACKGROUND: TRISS has reigned as the preeminent trauma outcome prediction model for 20 years. Despite this endorsement, the calibration of TRISS has been poor in most data sets where it has been examined. We hypothesized that the lack of calibration of TRISS was because of the inappropriate mathematical specification of the model that TRISS is based on, rather than the predictors in the model. In particular, we hypothesized that the nonlinearity of the Injury Severity Score (ISS) in the log odds of death was responsible for the poor calibration of TRISS, and further, that this nonlinearity could be corrected by the simple addition of an ISS squared term to the TRISS model. METHODS: We examined ISS in the log odds of mortality for linearity in one large trauma data set, the National Pediatric Trauma Registry (NPTR) (n = 53,113 from 1985-1996; mortality, 1.3%); and two small data sets, the University of New Mexico (UNM) (n = 3,142 from 1991-1995; mortality, 8.6%) and Portland, Oregon (PORT) (n = 2,916 from 1990-1994; mortality, 1.75%). In addition, in the NPTR we compared the calibration of TRISS models with and without linearity in the log odds of death. RESULTS: In the NPTR, ISS was profoundly nonlinear in the log odds of death for both blunt and penetrating trauma (p < 0.001). Moreover, the overall calibration of the TRISS model for the NPTR data was significantly improved when the nonlinearity of ISS was corrected by the addition of a quadratic ISS term as demonstrated by a 70% reduction (improvement) in the Hosmer-Lemeshow statistic. Interestingly, the addition of the ISS squared term did not affect the discrimination of the model. The log odds of survival in the UNM and PORT data sets were also better modeled when an ISS squared term was added (UNM, p = 0 0.052; PORT, p = 0.014), but improvements in the Hosmer-Lemeshow statistic were smaller, possibly because of the small size of these data sets. CONCLUSION: The TRISS model for outcome prediction currently uses ISS in a mathematically inappropriate way that impairs the calibration, but not the discrimination, of its predictions. If TRISS is to continue as the prediction standard for trauma, a quadratic ISS term must be added to the model. In the future, outcome prediction models should undergo thorough statistical modeling and evaluation before being released. Injury severity descriptors other than ISS (such as ASCOT, ICISS, or NISS) may require other modeling techniques to optimize the calibration of survival models that use these injury scores.
Authors: Mathieu Raux; Michel Thicoïpé; Eric Wiel; Elisabeth Rancurel; Dominique Savary; Jean-Stéphane David; Frédéric Berthier; Agnès Ricard-Hibon; Frédéric Birgel; Bruno Riou Journal: Intensive Care Med Date: 2006-02-17 Impact factor: 17.440
Authors: Soon Yong Kim; Ki Jeong Hong; Sang Do Shin; Young Sun Ro; Ki Ok Ahn; Yu Jin Kim; Eui Jung Lee Journal: J Korean Med Sci Date: 2016-12 Impact factor: 2.153