H Sandvik1. 1. Department of Public Health and Primary Health Care, University of Bergen, Norway.
Abstract
BACKGROUND AND OBJECTIVES: Patient vignettes are often used for investigating the performance of physicians. The purpose of this study was to assess the criterion validity of vignettes. METHODS: A random sample of incontinent women were interviewed, and vignettes depicting their cases were later presented to their own physicians. Questionnaires sent to the physicians included open questions about what drug treatment the doctor would give and a checklist (cueing items) for indicating other possible actions. Analysis compared actions taken with vignette patients and actual actions taken with the real patients. RESULTS: Thirty-two general practitioners and 32 patients were included in the study. When cueing items were provided, the physicians claimed more actions with vignettes than were actually performed. Mean difference was 1.9 actions per case (95% confidence interval 1.3 to 2.4). No such difference was found for the open question. When cues were provided, the likelihood ratio for reporting a vignette action in comparison with actually performing the action with a similar real patient was 2.8 (2.2 to 3.5), and Kappa .39. When no cues were provided, the likelihood ratio was 5.1 (2.6 to 10), and Kappa .45. Total number of actions with vignettes and real patients correlated significantly (r = .65, P < .001). CONCLUSIONS: Responses to vignettes with cueing items overestimate real performance. The agreement may be better when no cues are offered. The method may have acceptable validity for group comparisons.
BACKGROUND AND OBJECTIVES:Patient vignettes are often used for investigating the performance of physicians. The purpose of this study was to assess the criterion validity of vignettes. METHODS: A random sample of incontinent women were interviewed, and vignettes depicting their cases were later presented to their own physicians. Questionnaires sent to the physicians included open questions about what drug treatment the doctor would give and a checklist (cueing items) for indicating other possible actions. Analysis compared actions taken with vignette patients and actual actions taken with the real patients. RESULTS: Thirty-two general practitioners and 32 patients were included in the study. When cueing items were provided, the physicians claimed more actions with vignettes than were actually performed. Mean difference was 1.9 actions per case (95% confidence interval 1.3 to 2.4). No such difference was found for the open question. When cues were provided, the likelihood ratio for reporting a vignette action in comparison with actually performing the action with a similar real patient was 2.8 (2.2 to 3.5), and Kappa .39. When no cues were provided, the likelihood ratio was 5.1 (2.6 to 10), and Kappa .45. Total number of actions with vignettes and real patients correlated significantly (r = .65, P < .001). CONCLUSIONS: Responses to vignettes with cueing items overestimate real performance. The agreement may be better when no cues are offered. The method may have acceptable validity for group comparisons.
Authors: Lindsay M Griffin; Michelle R Denburg; Justine Shults; Susan L Furth; Isidro B Salusky; Wenke Hwang; Mary B Leonard Journal: Pediatr Nephrol Date: 2012-10-20 Impact factor: 3.714
Authors: Carola M E Döpp; Maud J L Graff; Steven Teerenstra; Eddy Adang; Ria W G Nijhuis-van der Sanden; Marcel G M Olderikkert; Myrra J F J Vernooij-Dassen Journal: BMC Geriatr Date: 2011-03-30 Impact factor: 3.921
Authors: Geert M Rutten; Janneke Harting; L Kay Bartholomew; Angelique Schlief; Rob A B Oostendorp; Nanne K de Vries Journal: BMC Health Serv Res Date: 2013-05-25 Impact factor: 2.655