Robert S Butler1, Douglas Johnston2, Michael W Kattan1. 1. Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH, USA. 2. Heart and Vascular Institute, Cleveland Clinic, Cleveland, OH, USA.
Abstract
BACKGROUND: Mandated assessment of medical personnel by comparing individual performance averages to external targets is standard practice in many health care systems. This method of assessment uses only raw or adjusted averages without considering the associated variation. Failure to correctly incorporate variation in the assessment of medical personnel results in evaluations which are neither accurate nor fair with respect to assessing personnel performance. METHODS: Accepted statistical methods for process evaluation and quality control, including regression, control charts, and adjusted means comparisons will be used to analyze hospital length of stay (LOS) patient data for the period between January and October 2010 for 12 physicians in the Cardiothoracic Surgery service line at the Cleveland Clinic. RESULTS: The analysis and interpretation of physician performance data using both targets and tolerances results in physician performance ratings which differ significantly from performance ratings based only on targets. CONCLUSIONS: Failure to include variation when assessing medical personnel performance results in a system of ranking, rewarding, and punishing based primarily on blind chance instead of one based on actual personnel performance.
BACKGROUND: Mandated assessment of medical personnel by comparing individual performance averages to external targets is standard practice in many health care systems. This method of assessment uses only raw or adjusted averages without considering the associated variation. Failure to correctly incorporate variation in the assessment of medical personnel results in evaluations which are neither accurate nor fair with respect to assessing personnel performance. METHODS: Accepted statistical methods for process evaluation and quality control, including regression, control charts, and adjusted means comparisons will be used to analyze hospital length of stay (LOS) patient data for the period between January and October 2010 for 12 physicians in the Cardiothoracic Surgery service line at the Cleveland Clinic. RESULTS: The analysis and interpretation of physician performance data using both targets and tolerances results in physician performance ratings which differ significantly from performance ratings based only on targets. CONCLUSIONS: Failure to include variation when assessing medical personnel performance results in a system of ranking, rewarding, and punishing based primarily on blind chance instead of one based on actual personnel performance.