BACKGROUND: This study explores whether long and short cases performed in the workplace during training could be integrated into an overall summative assessment. Less examiner training and a less formalized structure might compromise reliability, but increased testing time might improve it. METHODS: Results of practice long and short cases, undertaken in preparation for the Royal Australasian College of Physicians clinical examination, were compared with actual examination results. The effects on reliability of the examination were compared by modelling varying combinations of practice and examination long and short cases. RESULTS: Fifty-nine candidates in two centres undertook 256 practice long cases and 154 practice short cases. Two practice long cases correlated with two examination long cases (r= 0.46). The reliability of a single long case was 0.22 under practice conditions and 0.36 under examination conditions. The reliability of a single short case was similar under either condition (0.18 vs 0.21). Reliability of over 0.80 could be achieved by assimilating two examination long cases and four examination short cases with varying combinations of seven practice cases. CONCLUSIONS: Long cases undertaken in the workplace are not as reliable those undertaken under examination conditions, but short cases have similar reliability under either condition.
BACKGROUND: This study explores whether long and short cases performed in the workplace during training could be integrated into an overall summative assessment. Less examiner training and a less formalized structure might compromise reliability, but increased testing time might improve it. METHODS: Results of practice long and short cases, undertaken in preparation for the Royal Australasian College of Physicians clinical examination, were compared with actual examination results. The effects on reliability of the examination were compared by modelling varying combinations of practice and examination long and short cases. RESULTS: Fifty-nine candidates in two centres undertook 256 practice long cases and 154 practice short cases. Two practice long cases correlated with two examination long cases (r= 0.46). The reliability of a single long case was 0.22 under practice conditions and 0.36 under examination conditions. The reliability of a single short case was similar under either condition (0.18 vs 0.21). Reliability of over 0.80 could be achieved by assimilating two examination long cases and four examination short cases with varying combinations of seven practice cases. CONCLUSIONS: Long cases undertaken in the workplace are not as reliable those undertaken under examination conditions, but short cases have similar reliability under either condition.