Denise Castro1, Joseph Yang2, Mary-Louise Greer3, Benjamin Kwan1, Eric Sauerbrei1, Wilma Hopman4, Don Soboleski1. 1. Department of Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada. 2. Queen's School of Medicine, Queen's University, Kingston, ON, Canada. 3. Department of Diagnostic Imaging, The Hospital for Sick Children, Toronto, ON, Canada. 4. WJ Henderson Centre for Patient Oriented Research, Kingston, ON, Canada.
Abstract
RATIONALE AND OBJECTIVES: To introduce a process that allows for development of standardized competency based testing modules (CBTM) for evaluating resident progress and competence during their radiology training. This work focuses on the development of pediatric imaging CBTMs to be utilized during general radiology residency. MATERIALS AND METHODS: Multiple in-patient and ER imaging request audits along with surveys of training programs and text recommendations were obtained. A list of 200 total diagnoses accumulated by review was distributed into one of four CBTM folders. Imaging cases which made ≥90% of the indications of the audits were added to Folder 1. Distribution of remaining imaging diagnoses was based on consensus by three subspecialists. A pilot study was performed with residents dictating selected imaging cases in their usual manner mimicking a typical rotation. RESULTS: The pilot study demonstrated resident grading mean scores significantly associated with both the American College of Radiology (ACR) rank (rho = 0.636, p = 0.035) and the objective structured clinical examinations (OSCE) scores (rho = 0.694, p = 0.018). The mean scores positively associated with the ACR score (rho = 0.466), but fell short of statistical significance (p = 0.149). As expected, the ACR score, ACR rank and OSCE scores all significantly correlated with each other ( < 0.01). PGY also significantly correlated with the ACR score (rho = 0.683, p = 0.021) and the OSCE (rho = 0.767, p = 0.006) but not with the ACR rank (rho = 0.408, p = 0.213). CONCLUSION: The process utilized to develop a standardized CBTM can be used as a simulation tool to assess radiology resident competence during their training. The format allows for assessment of resident reasoning skills and knowledge base, which provides documentation of progression and throughout residency.
RATIONALE AND OBJECTIVES: To introduce a process that allows for development of standardized competency based testing modules (CBTM) for evaluating resident progress and competence during their radiology training. This work focuses on the development of pediatric imaging CBTMs to be utilized during general radiology residency. MATERIALS AND METHODS: Multiple in-patient and ER imaging request audits along with surveys of training programs and text recommendations were obtained. A list of 200 total diagnoses accumulated by review was distributed into one of four CBTM folders. Imaging cases which made ≥90% of the indications of the audits were added to Folder 1. Distribution of remaining imaging diagnoses was based on consensus by three subspecialists. A pilot study was performed with residents dictating selected imaging cases in their usual manner mimicking a typical rotation. RESULTS: The pilot study demonstrated resident grading mean scores significantly associated with both the American College of Radiology (ACR) rank (rho = 0.636, p = 0.035) and the objective structured clinical examinations (OSCE) scores (rho = 0.694, p = 0.018). The mean scores positively associated with the ACR score (rho = 0.466), but fell short of statistical significance (p = 0.149). As expected, the ACR score, ACR rank and OSCE scores all significantly correlated with each other ( < 0.01). PGY also significantly correlated with the ACR score (rho = 0.683, p = 0.021) and the OSCE (rho = 0.767, p = 0.006) but not with the ACR rank (rho = 0.408, p = 0.213). CONCLUSION: The process utilized to develop a standardized CBTM can be used as a simulation tool to assess radiology resident competence during their training. The format allows for assessment of resident reasoning skills and knowledge base, which provides documentation of progression and throughout residency.