J R Lough1, T S Murray. 1. Department of Postgraduate Medical Education, University of Glasgow, Glasgow, UK.
Abstract
OBJECTIVE: To develop and test a system for delivering a completed audit cycle for summative assessment of general practitioner registrars in the United Kingdom. DESIGN: A trainer-based questionnaire on criteria for a completed audit cycle, followed by two marking exercises of audit projects submitted by general practice registrars. SETTING: Training practices in the West of Scotland between 1997 and 1998. SUBJECTS: Trainers and registrars in the above practices. RESULTS: 116 (89%) agreed that two collections of data were an essential or desirable part of an audit project. All 57 registrars who started in August 1997 successfully completed an audit cycle, seven (12%) after resubmission. Using two rather than three independent assessors to screen the projects, the marking instrument was shown to have a sensitivity of 95% (95% confidence interval (CI) +/-3.9%) and a specificity of 77% (95% CI +/-7.5%). All assessors found the new system easier to mark and 47 registrars (87%) found completing an audit cycle as or easier than expected. CONCLUSION: Evidence from the pilot project has shown that a general practice registrar's ability to review and critically analyse a piece of his/her work, with appropriate management of any necessary change, can be tested feasibly by means of a completed audit cycle within the registrar year. The process retains adequate levels of sensitivity and specificity and requires fewer assessors for marking the projects.
OBJECTIVE: To develop and test a system for delivering a completed audit cycle for summative assessment of general practitioner registrars in the United Kingdom. DESIGN: A trainer-based questionnaire on criteria for a completed audit cycle, followed by two marking exercises of audit projects submitted by general practice registrars. SETTING: Training practices in the West of Scotland between 1997 and 1998. SUBJECTS: Trainers and registrars in the above practices. RESULTS: 116 (89%) agreed that two collections of data were an essential or desirable part of an audit project. All 57 registrars who started in August 1997 successfully completed an audit cycle, seven (12%) after resubmission. Using two rather than three independent assessors to screen the projects, the marking instrument was shown to have a sensitivity of 95% (95% confidence interval (CI) +/-3.9%) and a specificity of 77% (95% CI +/-7.5%). All assessors found the new system easier to mark and 47 registrars (87%) found completing an audit cycle as or easier than expected. CONCLUSION: Evidence from the pilot project has shown that a general practice registrar's ability to review and critically analyse a piece of his/her work, with appropriate management of any necessary change, can be tested feasibly by means of a completed audit cycle within the registrar year. The process retains adequate levels of sensitivity and specificity and requires fewer assessors for marking the projects.
Authors: Elizabeth K McClain; Stewart F Babbott; Terance T Tsue; Douglas A Girod; Debora Clements; Lisa Gilmer; Diane Persons; Greg Unruh Journal: J Grad Med Educ Date: 2012-06