Jeff Luck1, John W Peabody. 1. Veterans Administration, Greater Los Angeles Healthcare System, 11 301 Wilshire Blvd, Los Angeles, CA 90073, USA.
Abstract
OBJECTIVE: To assess the validity of standardised patients to measure the quality of physicians' practice. DESIGN: Validation study of standardised patients' assessments. Physicians saw unannounced standardised patients presenting with common outpatient conditions. The standardised patients covertly tape recorded their visit and completed a checklist of quality criteria immediately afterwards. Their assessments were compared against independent assessments of the recordings by a trained medical records abstractor. SETTING: Four general internal medicine primary care clinics in California. PARTICIPANTS: 144 randomly selected consenting physicians. MAIN OUTCOME MEASURES: Rates of agreement between the patients' assessments and independent assessment. RESULTS: 40 visits, one per standardised patient, were recorded. The overall rate of agreement between the standardised patients' checklists and the independent assessment of the audio transcripts was 91% (kappa=0.81). Disaggregating the data by medical condition, site, level of physicians' training, and domain (stage of the consultation) gave similar rates of agreement. Sensitivity of the standardised patients' assessments was 95%, and specificity was 85%. The area under the receiver operator characteristic curve was 90%. CONCLUSIONS: Standardised patients' assessments seem to be a valid measure of the quality of physicians' care for a variety of common medical conditions in actual outpatient settings. Properly trained standardised patients compare well with independent assessment of recordings of the consultations and may justify their use as a "gold standard" in comparing the quality of care across sites or evaluating data obtained from other sources, such as medical records and clinical vignettes.
OBJECTIVE: To assess the validity of standardised patients to measure the quality of physicians' practice. DESIGN: Validation study of standardised patients' assessments. Physicians saw unannounced standardised patients presenting with common outpatient conditions. The standardised patients covertly tape recorded their visit and completed a checklist of quality criteria immediately afterwards. Their assessments were compared against independent assessments of the recordings by a trained medical records abstractor. SETTING: Four general internal medicine primary care clinics in California. PARTICIPANTS: 144 randomly selected consenting physicians. MAIN OUTCOME MEASURES: Rates of agreement between the patients' assessments and independent assessment. RESULTS: 40 visits, one per standardised patient, were recorded. The overall rate of agreement between the standardised patients' checklists and the independent assessment of the audio transcripts was 91% (kappa=0.81). Disaggregating the data by medical condition, site, level of physicians' training, and domain (stage of the consultation) gave similar rates of agreement. Sensitivity of the standardised patients' assessments was 95%, and specificity was 85%. The area under the receiver operator characteristic curve was 90%. CONCLUSIONS: Standardised patients' assessments seem to be a valid measure of the quality of physicians' care for a variety of common medical conditions in actual outpatient settings. Properly trained standardised patients compare well with independent assessment of recordings of the consultations and may justify their use as a "gold standard" in comparing the quality of care across sites or evaluating data obtained from other sources, such as medical records and clinical vignettes.
Authors: John S Baer; Kelly M Carpenter; Blair Beadnell; Susan A Stoner; Michelle Hansten Ingalsbe; Bryan Hartzler; David B Rosengren; Zach Drager Journal: J Stud Alcohol Drugs Date: 2012-01 Impact factor: 2.582
Authors: Carol E Franz; Ron Epstein; Katherine N Miller; Arthur Brown; Jun Song; Mitchell Feldman; Peter Franks; Steven Kelly-Reif; Richard L Kravitz Journal: Health Serv Res Date: 2006-12 Impact factor: 3.402
Authors: Lisa M Hooper; Steven A Epstein; Kevin P Weinfurt; Jamie DeCoster; Lixin Qu; Natalie J Hannah Journal: J Behav Health Serv Res Date: 2012-04 Impact factor: 1.505
Authors: M R Weaver; M Myaya; K Disasi; M Regoeng; H N Matumo; M Madisa; N Puttkammer; F Speilberg; P H Kilmarx; J M Marrazzo Journal: Sex Transm Infect Date: 2008-02-06 Impact factor: 3.519