Alex H S Harris1, Cheng Chen2, Anna D Rubinsky2, Katherine J Hoggatt3, Matthew Neuman2, Megan E Vanneman2,4. 1. Center for Innovation to Implementation, VA Palo Alto Health Care System, 795 Willow Road, Menlo Park, CA, 94025, USA. Alexander.Harris2@va.gov. 2. Center for Innovation to Implementation, VA Palo Alto Health Care System, 795 Willow Road, Menlo Park, CA, 94025, USA. 3. VA Greater Los Angeles Health Care System and University of California Los Angeles, Los Angeles, CA, USA. 4. Center for Health Policy/Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA.
Abstract
BACKGROUND: Process measures of healthcare quality are usually formulated as the number of patients who receive evidence-based treatment (numerator) divided by the number of patients in the target population (denominator). When the systems being evaluated can influence which patients are included in the denominator, it is reasonable to wonder if improvements in measured quality are driven by expanding numerators or contracting denominators. OBJECTIVE: In 2003, the US Department of Veteran Affairs (VA) based executive compensation in part on performance on a substance use disorder (SUD) continuity-of-care quality measure. The first goal of this study was to evaluate if implementing the measure in this way resulted in expected improvements in measured performance. The second goal was to examine if the proportion of patients with SUD who qualified for the denominator contracted after the quality measure was implemented, and to describe the facility-level variation in and correlates of denominator contraction or expansion. DESIGN: Using 40 quarters of data straddling the implementation of the performance measure, an interrupted time series design was used to evaluate changes in two outcomes. PARTICIPANTS: All veterans with an SUD diagnosis in all VA facilities from fiscal year 2000 to 2009. MAIN MEASURES: The two outcomes were 1) measured performance-patients retained/patients qualified and 2) denominator prevalence-patients qualified/patients with SUD program contact. KEY RESULTS: Measured performance improved over time (P < 0.001). Notably, the proportion of patients with SUD program contact who qualified for the denominator decreased more rapidly after the measure was implemented (p = 0.02). Facilities with higher pre-implementation denominator prevalence had steeper declines in denominator prevalence after implementation (p < 0.001). CONCLUSIONS: These results should motivate the development of measures that are less vulnerable to denominator management, and also the exploration of "shadow measures" to monitor and reduce undesirable denominator management.
BACKGROUND: Process measures of healthcare quality are usually formulated as the number of patients who receive evidence-based treatment (numerator) divided by the number of patients in the target population (denominator). When the systems being evaluated can influence which patients are included in the denominator, it is reasonable to wonder if improvements in measured quality are driven by expanding numerators or contracting denominators. OBJECTIVE: In 2003, the US Department of Veteran Affairs (VA) based executive compensation in part on performance on a substance use disorder (SUD) continuity-of-care quality measure. The first goal of this study was to evaluate if implementing the measure in this way resulted in expected improvements in measured performance. The second goal was to examine if the proportion of patients with SUD who qualified for the denominator contracted after the quality measure was implemented, and to describe the facility-level variation in and correlates of denominator contraction or expansion. DESIGN: Using 40 quarters of data straddling the implementation of the performance measure, an interrupted time series design was used to evaluate changes in two outcomes. PARTICIPANTS: All veterans with an SUD diagnosis in all VA facilities from fiscal year 2000 to 2009. MAIN MEASURES: The two outcomes were 1) measured performance-patients retained/patients qualified and 2) denominator prevalence-patients qualified/patients with SUD program contact. KEY RESULTS: Measured performance improved over time (P < 0.001). Notably, the proportion of patients with SUD program contact who qualified for the denominator decreased more rapidly after the measure was implemented (p = 0.02). Facilities with higher pre-implementation denominator prevalence had steeper declines in denominator prevalence after implementation (p < 0.001). CONCLUSIONS: These results should motivate the development of measures that are less vulnerable to denominator management, and also the exploration of "shadow measures" to monitor and reduce undesirable denominator management.
Authors: Rachel Sayko Adams; Deborah W Garnick; Alex H S Harris; Elizabeth L Merrick; Keith Hofmann; Wendy Funk; Thomas V Williams; Mary Jo Larson Journal: J Subst Abuse Treat Date: 2020-05-07
Authors: Andrea K Finlay; Ingrid A Binswanger; Christine Timko; David Smelson; Matthew A Stimmel; Mengfei Yu; Tom Bowe; Alex H S Harris Journal: J Subst Abuse Treat Date: 2018-10-02
Authors: James H Ford; Amanda J Abraham; Nicoleta Lupulescu-Mann; Raina Croff; Kim A Hoffman; Kelly Alanis-Hirsch; Mady Chalk; Laura Schmidt; Dennis McCarty Journal: J Stud Alcohol Drugs Date: 2017-09 Impact factor: 2.582