Do Proposed Quality Measures for Carpal Tunnel Release Reveal Important Quality Gaps and Are They Reliable?

BACKGROUND: The American Academy of Orthopaedic Surgeons recently proposed quality measures for the initial surgical treatment of carpal tunnel syndrome (CTS). One measure addressed avoidance of adjunctive surgical procedures during carpal tunnel release; and a second measure addressed avoidance of routine use of clinic-based occupational and/or physical therapy (OT/PT) after carpal tunnel release. However, for quality measures to serve their intended purposes, they must be tested in real-world data to establish that gaps in quality exist and that the measures yield reliable performance information. QUESTIONS/PURPOSES: (1) Is there an important quality gap in clinical practice for avoidance of adjunctive surgical procedures during carpal tunnel release? (2) Is there an important quality gap in avoiding routine use of clinic-based occupational and/or physical therapy after carpal tunnel release? (3) Do these two quality measures have adequate beta-binomial signal-to-noise ratio (SNR) and split-sample reliability (SSR)?
METHODS: This retrospective comparative study used a large national private insurance claims database, the 2018 Optum Clinformatics® Data Mart. Ideally, healthcare quality measures are tested within data reflective of the providers and payors to which the measures will be applied. We previously tested these measures in a large public healthcare system and a single academic medical center. In this study, we sought to test the measures in the broader context of patients and providers using private insurance. For both measures, we included the first carpal tunnel release from 28,083 patients performed by one of 7236 surgeons, irrespective of surgical specialty (including, orthopaedic, plastic, neuro-, and general surgery). To calculate surgeon-level descriptive and reliability statistics, analyses were focused on the 66% (18,622 of 28,083) of patients who received their procedure from one of the 24% (1740 of 7236) of surgeons with at least five carpal tunnel releases in the database. No other inclusion/exclusion criteria were applied. To determine whether the measures reveal important gaps in treatment quality (avoidance of adjunctive procedures and routine therapy), we calculated descriptive statistics (median and interquartile range) of the performance distribution stratified by surgeon-level annual volume of carpal tunnel releases in the database (5+, 10+, 15+, 20+, 25+, and 30+). Like the Centers for Medicare & Medicaid Services (CMS), we considered a measure "topped out" if median performance was greater than 95%, meaning the opportunity for further quality improvement is low. We calculated the surgeon-level beta-binomial SNR and SSR for each measure, each stratified by the number of carpal tunnel releases performed by each surgeon in the database. These are standard measures of reliability in health care quality measurement science. The SNR quantifies the proportion of variance that is between rather than within surgeons, and the SSR is the correlation of performance scores when each surgeons' patients are split into two random samples and then corrected for sample size.
RESULTS: We found that 2% (308 of 18,622) of carpal tunnel releases involved an adjunctive procedure. The results showed that avoidance of adjunctive surgical procedures during carpal tunnel release had a median (IQR) performance of 100% (100% to 100%) at all case volumes. Only 8% (144 of 1740) of surgeons with at least five cases in the database had less than 100% performance, and only 5% (84 of 1740) had less than 90% performance. This means adjunctive procedures were rarely performed and an important quality gap does not exist based on the CMS criterion. Regarding the avoidance of routine therapy, there was a larger quality gap: For surgeons with at least five cases in the database, median performance was 89% (75% to 100%), and 25% (435 of 1740) of these surgeons had less than 75% performance. This signifies that the measure is not topped out and may reveal an important quality gap. Most patients receiving clinic-based OT/PT had only one visit in the 6 weeks after surgery. Median (IQR) SNRs of the first measure, which addressed avoidance of adjunctive surgical procedures, and the second measure, which addresses avoidance of routine use clinic-based OT/PT, were 1.00 (1.00 to 1.00) and 0.86 (0.67 to 1.00), respectively. The SSR for these measures were 0.87 (95% CI 0.85 to 0.88) and 0.75 (95% CI 0.73 to 0.77), respectively. All of these reliability statistics exceed National Quality Forum's emerging minimum standard of 0.60.
CONCLUSION: The first measure, the avoidance of adjunctive surgical procedures during carpal tunnel release, lacked an important quality gap suggesting it is unlikely to be useful in driving improvements. The second measure, avoidance of routine use of clinic-based OT/PT, revealed a larger quality gap and had very good reliability, suggesting it may be useful for quality monitoring and improvement purposes. CLINICAL RELEVANCE: As healthcare systems and payors use the second measure, avoidance of routine use of clinic-based OT/PT, to encourage adherence to clinical practice guidelines (such as provider profiling, public reporting, and payment policies), it will be critically important to consider what proportion of patients receiving OT/PT should be considered routine practice and therefore inconsistent with guidelines. The value or potential harm of this measure depends on this judgement.