Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice?

OBJECTIVE: Several methods exist for estimating the risk of perioperative mortality based on preoperative risk factors; graphical methods such as the variable life adjusted display (VLAD) can be used to examine how an individual surgeon's performance for a series of operations fares against what would be expected, given the case mix. This study aimed to devise a method for assessing the natural variation in outcome in order to assist with making judgements about individual performance, in particular whether seemingly poor performance could have occurred by chance.
METHOD: The risk scoring system has been derived and validated locally for cardiac surgery. A method is described for calculating the probability that an observed number of deaths occurs within a sequence of operations if perioperative mortality is regarded as a chance event with an expected value derived from the risk score. To illustrate this method, nested prediction intervals are superimposed onto VLAD plots for a series of 393 isolated coronary artery bypass and isolated valve operations performed by a single surgeon.
RESULTS: Using the locally derived risk score, the VLAD plot for the individual surgeon shows a net life gain of about 6 over the predicted number of survivors, which is observed to be within the 90% prediction interval. If the Parsonnet scoring system is used instead of the locally derived risk score, the net life gain is considerably overestimated.
CONCLUSIONS: The nested prediction intervals are straightforward to generate and can be integrated into a visually informative display. As an indication of the inherent variability in outcome, they have a valuable role in the monitoring of surgical performance.