Statistical process control for validating a classification tree model for predicting mortality--a novel approach towards temporal validation.

Prediction models are postulated as useful tools to support tasks such as clinical decision making and benchmarking. In particular, classification tree models have enjoyed much interest in the Biomedical Informatics literature. However, their prospective predictive performance over the course of time has not been investigated. In this paper we suggest and apply statistical process control methods to monitor over more than 5 years the prospective predictive performance of TM80+, one of the few classification-tree models published in the clinical literature. TM80+ is a model for predicting mortality among very elderly patients in the intensive care based on a multi-center dataset. We also inspect the predictive performance at the tree's leaves. This study provides important insights into patterns of (in)stability of the tree's performance and its "shelf life". The study underlies the importance of continuous validation of prognostic models over time using statistical tools and the timely recalibration of tree models.