Joshua J Fenton1,2,3, Anthony Jerant4,5, Richard L Kravitz5,6, Klea D Bertakis4,5, Daniel J Tancredi5,7, Elizabeth M Magnan4,5, Peter Franks4,5. 1. Department of Family and Community Medicine, University of California, Davis Medical Center, Sacramento, CA, USA. jjfenton@ucdavis.edu. 2. The Center for Healthcare Policy and Research, University of California, Davis Medical Center, Sacramento, CA, USA. jjfenton@ucdavis.edu. 3. Department of Family and Community Medicine, University of California, Davis Health System, Sacramento, CA, USA. jjfenton@ucdavis.edu. 4. Department of Family and Community Medicine, University of California, Davis Medical Center, Sacramento, CA, USA. 5. The Center for Healthcare Policy and Research, University of California, Davis Medical Center, Sacramento, CA, USA. 6. The Division of General Medicine, Department of Internal Medicine, University of California, Davis Medical Center, Sacramento, CA, USA. 7. Department of Pediatrics, University of California, Davis Medical Center, Sacramento, CA, USA.
Abstract
BACKGROUND: Patient experience measures are widely used to compare performance at the individual physician level. OBJECTIVE: To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. DESIGN: Repeated cross-sectional design. SETTING: Academic family medicine practice in California. PARTICIPANTS: One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). MEASUREMENTS: Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). RESULTS: The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0-25.2%)]. CONCLUSIONS: Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.
BACKGROUND:Patient experience measures are widely used to compare performance at the individual physician level. OBJECTIVE: To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. DESIGN: Repeated cross-sectional design. SETTING: Academic family medicine practice in California. PARTICIPANTS: One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). MEASUREMENTS: Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). RESULTS: The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0-25.2%)]. CONCLUSIONS: Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.
Authors: Dana Gelb Safran; Melinda Karp; Kathryn Coltin; Hong Chang; Angela Li; John Ogren; William H Rogers Journal: J Gen Intern Med Date: 2006-01 Impact factor: 5.128
Authors: Clemens S Hong; Steven J Atlas; Yuchiao Chang; S V Subramanian; Jeffrey M Ashburner; Michael J Barry; Richard W Grant Journal: JAMA Date: 2010-09-08 Impact factor: 56.272
Authors: Hector P Rodriguez; Ted von Glahn; Hong Chang; William H Rogers; Dana Gelb Safran Journal: Am J Med Qual Date: 2009 Jan-Feb Impact factor: 1.852
Authors: Anthony Jerant; Joshua J Fenton; Richard L Kravitz; Daniel J Tancredi; Elizabeth Magnan; Klea D Bertakis; Peter Franks Journal: JAMA Intern Med Date: 2018-01-01 Impact factor: 21.873
Authors: Anthony Jerant; Kevin Fiscella; Joshua J Fenton; Elizabeth M Magnan; Alicia Agnoli; Peter Franks Journal: J Gen Intern Med Date: 2019-05-29 Impact factor: 5.128