Moon-Tong Cheung1, Kelvin-K W Yau. 1. Department of Surgery, Queen Elizabeth Hospital, Kowloon, Hong Kong. qehsurg@ha.org.hk
Abstract
BACKGROUND: Early assessment of surgical trainees in their training programme is essential. A strong subjective observer's bias occurs when using the mentor assessment form. In-service examinations and qualifying examinations are only valid measurements of the trainee's knowledge of factual material. An objective method to assess a surgical trainee's clinical performance is introduced in the present study. METHODS: We examined the Generalized Linear Mixed Model statistical methodology to obtain standardized performance scores that would identify good and poor performers while taking into account the effects of patients' risk factors. Data regarding the postappendectomy complication occurrence and operation duration were retrospectively collected for each trainee. These variables were analysed by applying logistic regression and proportional hazards regression models respectively. Standardized trainee scores were obtained as by-products that measured the residual effect of surgeons' performances after taking into account the effects of possible risk factors. RESULTS: A total of 37 trainees were involved in 638 appendectomy operations from 1994 to 1997. Age, concurrent medical disease and presence of pus were significant risk factors associated with complication occurrence. Concurrent medical disease, presence of pus, operative findings of gangrenous or ruptured appendix as well as the experience of the surgeon had a significant effect on the operation duration. The present study identifies four trainees that displayed good clinical performance, and one trainee that displayed poor clinical performance of appendectomy. CONCLUSION: Our proposed method makes use of recently developed statistical methodology for predicting the effect of surgeons' performance,taking into account variations in their case-mix and their patients' fitness. This approach may serve as a very powerful tool for early assessment of a trainee's clinical proficiency, and may also have implications for the future assessment of a surgeon's clinical standard.
BACKGROUND: Early assessment of surgical trainees in their training programme is essential. A strong subjective observer's bias occurs when using the mentor assessment form. In-service examinations and qualifying examinations are only valid measurements of the trainee's knowledge of factual material. An objective method to assess a surgical trainee's clinical performance is introduced in the present study. METHODS: We examined the Generalized Linear Mixed Model statistical methodology to obtain standardized performance scores that would identify good and poor performers while taking into account the effects of patients' risk factors. Data regarding the postappendectomy complication occurrence and operation duration were retrospectively collected for each trainee. These variables were analysed by applying logistic regression and proportional hazards regression models respectively. Standardized trainee scores were obtained as by-products that measured the residual effect of surgeons' performances after taking into account the effects of possible risk factors. RESULTS: A total of 37 trainees were involved in 638 appendectomy operations from 1994 to 1997. Age, concurrent medical disease and presence of pus were significant risk factors associated with complication occurrence. Concurrent medical disease, presence of pus, operative findings of gangrenous or ruptured appendix as well as the experience of the surgeon had a significant effect on the operation duration. The present study identifies four trainees that displayed good clinical performance, and one trainee that displayed poor clinical performance of appendectomy. CONCLUSION: Our proposed method makes use of recently developed statistical methodology for predicting the effect of surgeons' performance,taking into account variations in their case-mix and their patients' fitness. This approach may serve as a very powerful tool for early assessment of a trainee's clinical proficiency, and may also have implications for the future assessment of a surgeon's clinical standard.