BACKGROUND: The reproducibility of authentic assessment methods has been investigated for objective structured clinical examinations (OSCEs) and video assessment in general practice, but not for assessment with incognito standardized patients. PURPOSE: To investigate the reproducibility of assessment with incognito standardized patients. METHODS: A total of 27 Dutch rheumatologists in 16 hospitals were each visited by 8 incognito standardized patients presenting with different rheumatological disorders. After each visit, the standardized patient completed a case-specific checklist containing items on medical history, physical examination and management. Over a 20-month period, 254 incognito visits took place, of which 201 were first visits. The standardized patient was detected by the rheumatologist in 2 cases only. These encounters were not included in the analysis. Generalizability theory was used to investigate the reproducibility of the assessment. RESULTS: One fifth of the variance can be attributed to variation between rheumatologists. The largest variance is due to the variation in difficulty among cases. A reproducible assessment requires 3 hours of testing time (6 cases) if it is obtained through a norm-referenced interpretation of scores and 7 hours of testing time (14 cases) if it is obtained through an absolute interpretation of scores. CONCLUSION: The reproducibility of performance assessment in clinical practice by incognito standardized patients is similar to that of other authentic measurements for the assessment of clinical competence and performance.
BACKGROUND: The reproducibility of authentic assessment methods has been investigated for objective structured clinical examinations (OSCEs) and video assessment in general practice, but not for assessment with incognito standardized patients. PURPOSE: To investigate the reproducibility of assessment with incognito standardized patients. METHODS: A total of 27 Dutch rheumatologists in 16 hospitals were each visited by 8 incognito standardized patients presenting with different rheumatological disorders. After each visit, the standardized patient completed a case-specific checklist containing items on medical history, physical examination and management. Over a 20-month period, 254 incognito visits took place, of which 201 were first visits. The standardized patient was detected by the rheumatologist in 2 cases only. These encounters were not included in the analysis. Generalizability theory was used to investigate the reproducibility of the assessment. RESULTS: One fifth of the variance can be attributed to variation between rheumatologists. The largest variance is due to the variation in difficulty among cases. A reproducible assessment requires 3 hours of testing time (6 cases) if it is obtained through a norm-referenced interpretation of scores and 7 hours of testing time (14 cases) if it is obtained through an absolute interpretation of scores. CONCLUSION: The reproducibility of performance assessment in clinical practice by incognito standardized patients is similar to that of other authentic measurements for the assessment of clinical competence and performance.
Authors: Carol E Franz; Ron Epstein; Katherine N Miller; Arthur Brown; Jun Song; Mitchell Feldman; Peter Franks; Steven Kelly-Reif; Richard L Kravitz Journal: Health Serv Res Date: 2006-12 Impact factor: 3.402
Authors: Laura A Siminoff; Heather L Rogers; Allison C Waller; Sonja Harris-Haywood; Ronald M Esptein; Francesc Borrell Carrio; Gayle Gliva-McConvey; Daniel R Longo Journal: Patient Educ Couns Date: 2011-03
Authors: N Kevin Krane; Delia Anderson; Cathy J Lazarus; Michael Termini; Bruce Bowdish; Sheila Chauvin; Vivian Fonseca Journal: J Gen Intern Med Date: 2008-10-31 Impact factor: 5.128