Impact of the Publication of Appropriate Use Criteria on Utilization Rates of Myocardial Perfusion Imaging Studies in Ontario, Canada: A Population-Based Study.

BACKGROUND: Concern regarding overutilization of cardiac imaging has led to the development of appropriate use criteria (AUC). Myocardial perfusion imaging (MPI) is one of the most commonly used cardiac imaging modalities worldwide. Despite multiple iterations of AUC, there is currently no evidence regarding their real-world impact on population-based utilization rates of MPI. Our goal was to assess the impact of the AUC on rates of MPI in Ontario, Canada. We hypothesized that publication of the AUC would be associated with a significant reduction in MPI rates. METHODS AND
RESULTS: We conducted a retrospective cohort study of the adult population of Ontario from January 1, 2000, to December 31, 2015. Age- and sex-standardized rates were compared from 4 different periods intersected by 3 published iterations of the AUC. Overall, 3 072 611 MPI scans were performed in Ontario during our study period. The mean monthly rate increased from 14.1/10 000 in the period from January 2000 to October 2005 to 18.2/10 000 between November 2005 and June 2009. After this point in time, there was a reduction in rates, falling to a mean monthly rate of 17.1/10 000 between March 2014 and December 2015. Time series analysis revealed that publication of the 2009 AUC was associated with a significant reduction in MPI rates (P<0.001). This translated into ≈88 849 fewer MPI scans at a cost savings of ≈72 million Canadian dollars.
CONCLUSIONS: Our results reflect a potential real-world impact of the 2009 MPI AUC by demonstrating evidence of a significant effect on population-based rates of MPI.

Clinical Perspective

What Is New?

Concern regarding overutilization of cardiac imaging has led to the development of AUC.

Myocardial perfusion imaging (MPI) is a commonly utilized cardiac imaging modality.

Our time series analysis of the adult population of Ontario, Canada (≈10.7 million population) revealed that publication of the 2009 AUC was associated with a significant reduction in MPI rates.

This translated into ≈88 849 fewer MPI scans performed at a cost savings of ≈72 million Canadian dollars.

What Are the Clinical Implications?

Our results reflect a potential real‐world impact of the 2009 MPI AUC by demonstrating evidence of a significant effect on population‐based rates of MPI.

Our study is the first to report such an impact of the AUC on population‐based utilization rates of cardiac imaging tests.

We hope that studies such as this will provide the impetus for further consideration of AUC in future funding and health policy decisions in Ontario and beyond.

Introduction

Rapid growth in healthcare costs has provided the impetus for understanding utilization patterns of diagnostic tests. Specifically, the costs of medical imaging have risen sharply in recent years.1, 2, 3, 4, 5 Myocardial perfusion imaging (MPI) with single photon emission computed tomography is a commonly utilized cardiac imaging modality. In fact, in 2002, ≈9.3 million such tests were performed in the Unites States.6 Concern about the potential overutilization of noninvasive cardiac imaging has led to the development of health policy initiatives such as appropriate use criteria (AUC) publications geared to curb utilization.7, 8, 9, 10, 11 AUC have been published on MPI exclusively on 2 occasions (2005 and 2009) and as part of a multimodality AUC publication in 2014.12, 13, 14 Despite the multiple iterations of the AUC, the impact of these publications on physician behavior as reflected by population‐based rates of MPI utilization is unknown. The goal of our study was to describe the utilization of MPI in Ontario over the past 15 years and to assess the impact of publication of the AUC on population‐based rates of MPI utilization. We hypothesized that AUC publication would be associated with a reduction in MPI utilization.

Methods

Design

We conducted a retrospective cohort study of the population of Ontario aged ≥20 years from January 1, 2000, until December 31, 2015.

Data Sources

Information regarding receipt of MPIs and graded exercise stress tests (GXTs) was obtained through medical claims data from the Ontario Health Insurance Plan (OHIP) Physicians Claim Database. Given that MPIs may be conducted over one or more consecutive days, we applied a 2‐day window on either side of the date of an MPI claim to avoid duplicate counting. The diagnostic codes used to identify the MPI studies and our methods to avoid duplicate counting have been validated previously and used in prior studies.15, 16, 17 The Registered Persons Database was used to obtain patient demographic information, including age and sex. Patients’ postal codes of residence were also obtained from the Registered Persons Database. Postal codes were linked to the Census Area Profile using the Postal Code Conversion File to obtain median neighborhood income in a manner described previously.15, 16, 17 Hospitalizations and receipt of invasive coronary angiography were determined using the Canadian Institute for Health Information Discharge Abstract Database (CIHI‐DAD). The CIHI‐DAD is an administrative database containing summary data of discharge abstracts of every inpatient hospital admission in Ontario.18, 19 Ontario has universal insurance for all residents of the province and a single‐payer system. Thus, these databases record all of the health service use for the current study.

Outcome

The primary outcome was the age‐ and sex‐standardized monthly rate of MPI scans per 10 000 adults in Ontario, Canada. To provide context, we also calculated the age‐ and sex‐standardized rates of GXT and invasive angiography as secondary outcome measures.

Interventions

There were 3 interventions representing the publication of the 3 iterations of the AUC: (1) October 2005, (2) June 2009, and (3) February 2014.

Costs

Case costs were estimated using the OHIP schedule of benefits and included both professional and technical fees that were billed to OHIP for each MPI scan.20, 21 The total fee billed to OHIP per MPI scan was ≈811 Canadian dollars.

Statistical Analysis

Population‐based rates of MPI, GXT, and invasive angiography were calculated for each year by dividing the number of diagnostic tests by the corresponding adult population of Ontario and were then estimated per 10 000 individuals. Age‐ and sex‐standardized rates were obtained through direct standardization using the 2006 Ontario population as the standard population, as described previously.15, 16 Characteristics of patients receiving MPI scans were compared over time using the χ2 test for categorical variables and ANOVA for continuous variables.

An autoregressive integrated moving average (ARIMA) model was used to compare mean monthly utilization rates before and after publication of the guidelines (ie, the interventions). The impact of these interventions were assessed after accounting for seasonality (if present), background trends, and autocorrelation. Linear spline functions with knots at the dates of publications of the AUC were incorporated into this model.22, 23 The ARIMA model accounted for existing temporal trends in the use of MPI. Existing temporal trends in use of MPI could be due to temporal changes in patient characteristics, changes in the utilization of other noninvasive diagnostic tests, and changes in pharmacological therapy for coronary artery disease. Accounting for existing temporal trends permits one to isolate the effect of publication of the AUC from existing temporal trends.

To graphically represent the overall trend and any change in the rates before and after the intervention, we estimated a smoothing line using robust locally weighted regression (LOESS).24, 25 Reductions in numbers of MPI scans as a potential result of publication of the AUC were estimated by multiplying the change in monthly rate between the periods of any statistically significant intervention by the yearly population of Ontario aged ≥20 years. That number was then multiplied by 12 to obtain an annual estimate. In years when the statistically significant intervention occurred, we accounted for only those months that occurred after publication. Specifically, for the June 2009 intervention, the annual number was divided by 2 to account for the ≈6 months that followed AUC publication in that year. Potential cost savings were then determined by multiplying the fees billed to OHIP per scan by the number of scans potentially reduced by the intervention. Analyses were performed with SAS version 9.3. This study was approved by the research ethics board at Sunnybrook Health Sciences Center.

Results

Derivation of the Study Population

Figure 1 displays the derivation of the study population. There were 3 257 454 OHIP claims for MPI studies between the calendar years 2000 and 2015. Of these, 154 072 were excluded due to multiple billings within 2 days of the initial claim. A further 30 771 claims for patients aged <20 years were also excluded. The final sample consisted of 3 072 611 MPI scans performed in Ontario during our study period.

Figure 1

Derivation of the study cohort. MPI indicates myocardial perfusion imaging; OHIP, Ontario Health Insurance Plan.

Characteristics of Patients Undergoing MPI

Table 1 summarizes the characteristics of patients undergoing MPI and compares these characteristics in the 4 periods straddling the 3 interventions. Over time, MPI scans were performed on older patients, on a greater proportion of female patients, and on increasingly sicker patents, as measured by their hospital admission rate within 30 days of the test (overall P value <0.001 for all comparisons).

Table 1

Characteristics of Patients Undergoing MPI in Ontario, Canada, 2000–2015

	January 1, 2000–October 18, 2005	October 19, 2005–June 9, 2009	June 10, 2009–February 4, 2014	February 4, 2014–December 31, 2015	Total	P Value
n	827 594	749 564	1 051 647	443 806	3 072 611
Age (mean±SD)	60.1±13.2	61.4±13.4	62.9±13.5	64.2±13.4	62.0±13.5	<0.001
Female sex	395 092 (47.7%)	363 449 (48.5%)	526 269 (50.0%)	221 461 (49.9%)	1 506 271 (49.0%)	<0.001
Income quintile						<0.001
1 (lowest)	161 219 (19.5%)	148 974 (19.9%)	205 170 (19.5%)	85 575 (19.3%)	600 938 (19.6%)
2	168 052 (20.3%)	153 797 (20.5%)	214 605 (20.4%)	89 562 (20.2%)	626 016 (20.4%)
3	163 050 (19.7%)	147 027 (19.6%)	210 107 (20.0%)	89 318 (20.1%)	609 502 (19.8%)
4	161 046 (19.5%)	147 933 (19.7%)	214 391 (20.4%)	91 793 (20.7%)	615 163 (20.0%)
5 (highest)	171 503 (20.7%)	149 353 (19.9%)	203 194 (19.3%)	85 431 (19.2%)	609 481 (19.8%)
Self‐referral	65 514 (7.9%)	50 532 (6.7%)	73 224 (7.0%)	35 560 (8.0%)	224 830 (7.3%)	<0.001
Diabetes mellitus	141 485 (17.1%)	165 320 (22.1%)	265 894 (25.3%)	120 138 (27.1%)	692 837 (22.5%)	<0.001
Hypertension	310 907 (37.6%)	295 978 (39.5%)	403 867 (38.4%)	163 680 (36.9%)	1 174 432 (38.2%)	<0.001
Hospital admission within 30 d	34 644 (4.2%)	35 200 (4.7%)	53 376 (5.1%)	23 926 (5.4%)	147 146 (4.8%)	<0.001

Age‐ and Sex‐Standardized Rates of MPI Between 2000 and 2015

MPI rates increased steadily from ≈2000 to 2009 (see Figure 2). The mean age‐ and sex‐standardized monthly rate between January 2000 and October 2005 was 14.1/10 000, and this increased to 18.2/10 000 in the November 2005 to June 2009 period. After this point in time, there was a reduction in rates, falling to a mean monthly rate of 17.1/10 000 in the time period from March 2014 to December 2015.

Figure 2

Mean monthly age‐ and sex‐standardized rates of myocardial perfusion imaging (MPI) in Ontario; 2000–2015. The red vertical lines indicate the interventions (ie, publication of the appropriate use criteria). The blue dotted line indicates the LOESS smoothing curve.

Age ‐and Sex‐Standardized Rates of GXT Between 2000 and 2015

Mean monthly age‐ and sex‐standardized GXT rates increased from 26.3/10 000 in the January 2000 to October 2005 period to 29.2/10 000 in the November 2005 to June 2009 period. After this point in time, there was a reduction in rates to 28.3/10 000 in the time period from March 2014 to December 2015 (see Figure 3A).

Figure 3

A, Mean monthly age‐ and sex‐standardized rates of graded exercise stress tests (GXTs) in Ontario, 2000–2015. The red vertical lines indicate the interventions (ie, publication of the appropriate use criteria [AUC]). The blue dotted line indicates the LOESS smoothing curve. B, Mean monthly age‐ and sex‐standardized rates of invasive angiography in Ontario, 2000–2015. The red vertical lines indicate the interventions (ie, publication of the AUC). The blue dotted line indicates the LOESS smoothing curve.

Age‐ and Sex‐Standardized Rates of Invasive Angiography Between 2000 and 2015

Mean monthly age and sex‐standardized rates of invasive angiography increased from 4.8/10 000 in the January 2000 to October 2005 period to 5.4/10 000 in the June 2009 to February 2014 period. After this point in time, there was a slight decline to 5.1/10 000 in the time period from March 2014 to December 2015 (see Figure 3B).

The Impact of AUC Publication on Rates of MPI Utilization

In the time series analysis, we found that the second intervention (publication of the 2009 AUC) was associated with a significant reduction in MPI rates after accounting for the background trend, seasonality, and autocorrelation (P<0.001). In contrast, publication of the first (P=0.69) and third (P=0.76) AUC interventions were not associated with a significant reduction in the rate of MPI utilization. Furthermore, we found that the second intervention was also associated with a significant reduction in GXT rates (P<0.001); however, it was not associated with a significant reduction or increase in invasive angiography rates (P=0.22).

Estimation of the Reduction in MPI Scans and Potential Associated Cost Savings Resulting From Publication of the 2009 AUC

The reduced utilization rate translated into ≈88 849 fewer MPI scans performed after publication of the 2009 AUC. In addition, total cost reductions were estimated at 72 056 539 Canadian dollars during that time frame (see Table 2).

Table 2

Potential Numbers of MPI Scans Reduced and Associated Estimated Cost Savings as a Result of Publication of the 2009 AUC

	2009	2010	2011	2012	2013	2014	2015	Total
Population of Ontario aged ≥20 y	9 884 042	10 026 236	10 160 346	10 322 805	10 480 945	10 623 748	10 753 497
Number of MPIs potentially reduced per y as a result of publication of the 2009 AUC	6 523a	13 235	13 412	13 626	13 835	14 023	14 195	88 849
Potential cost savings per y (Canadian dollars)	5 290 153	10 733 585	10 877 132	11 050 686	11 220 185	11 372 653	11 512 145	72 056 539

AUC indicates appropriate use criteria; MPI, myocardial perfusion imaging.

Reflects estimates for 6 months after publication of the AUC (July–December 2009).

Discussion

Over time, MPI scans were performed on patients who were older, were more likely to be female, and were sicker overall, as measured by an increasing frequency of admission to hospital within 30 days of the test. Age‐ and sex‐standardized utilization rates of MPI increased in Ontario between 2000 and ≈2009. After 2009, rates began to decrease. Publication of the 2009 AUC was associated with a significant decrease in the population‐based rates of MPI in Ontario. This translated into ≈88 849 fewer MPI scans at a cost savings of ≈72 million Canadian dollars.

There is limited available evidence on the impact of AUC on utilization and physician behavior. A recent population‐based study reported that the publication of AUC for invasive coronary angiography was associated with a decreasing trend in the utilization of diagnostic angiography for the diagnosis of coronary artery disease.26 There have been mixed reports with regard to the efficacy of MPI‐based AUC. After publication of the 2005 MPI AUC guidelines, Mehta et al reported that although the majority of studies performed at a single large academic center were appropriate, a large proportion of ordered MPI studies were categorized as uncertain, inappropriate, or no category.27 Similarly, an American quality improvement project, focused on feedback, physician education, and remeasurement, did not reduce the rate of inappropriate stress MPI studies in a single academic medical center.28 A recent metaregression analysis involving 59 studies and 103 567 tests published from 2000 to 2012 reported no significant difference in the rate of appropriate MPI scans before and after publication of the AUC.29 In contrast, Gibbons et al reported a significant decrease in the proportion of inappropriate MPI scans performed in a single academic center after publications of the 2009 AUC.30 Furthermore, a recent budget impact analysis showed that application of the 2009 AUC could result in savings of 18.6% in a single tertiary care hospital in Brazil.31 To our knowledge, no studies have assessed the impact of the AUC on utilization rates of noninvasive cardiac diagnostic testing (including MPI) at the population level. This is a relevant metric to examine the potential impact of the AUC, given that publication of the AUC was driven, in part, by the need to control the rapid expansion of imaging utilization.

Our results provide evidence to suggest a significant effect of the 2009 AUC publication on population‐based MPI rates in Ontario, Canada. In contrast, publication of the 2005 AUC was not associated with a significant change in rates. This is consistent with previous work documenting that adherence to AUC among clinicians may gradually increase over time.32, 33 We speculate that although the 2005 AUC may have introduced the concept of appropriateness to most clinicians, the 2009 iteration may have cemented the importance of these criteria, ultimately culminating in the effect that we observed. In addition, there was no significant impact of the 2014 multimodality criteria on MPI rates. In the observed data, rates continued to decline after publication of these guidelines; however, after accounting for trend and seasonality, publication of the criteria themselves were not associated with a significant change in the rate of change of MPI rates. A potential explanation may lie in the fact that these were multimodality AUC not addressing MPI exclusively. Thus, the effect on MPI may not have been as profound as modality‐specific AUC. Furthermore, our results indicate that it is unlikely that MPI scans were simply replaced by another test. Up until 2011, billing information was only available for GXT and MPI in Ontario. Our analysis did not show a significant increase in rates of GXT coinciding with a reduction in MPI rates. Moreover, previous published work by our group reported that there was no increase in total noninvasive testing rates (including GXT, MPI, stress echocardiography, and computed tomography angiography) between 2011 and 2014.16

Our findings of a significant impact of the 2009 AUC are important. We are the first group to examine the relationship between AUC and rates of MPI utilization in the population. As such, our results may be meaningful from a health policy perspective by providing evidence of the real‐world impact of the AUC. Currently, AUC are not factored into funding decisions by the OHIP. Despite this, there is growing awareness of the need to limit unnecessary testing in Ontario and in Canada. We hope that studies such as this will provide the impetus for further consideration of AUC in future funding and health policy decisions in Ontario and beyond.

Limitations

Our study needs to be interpreted in the context of its limitations. First, we did not have clinical information regarding the appropriateness of tests. As such, we cannot comment on changes in inappropriate MPI utilization. Second, this is a population‐based study and our observations were at the population and not the individual level; therefore, our results are ecological in nature. Third, we did not specifically adjust for potential confounders such as changing patient demographics and risk factor profiles; however, our ARIMA model incorporated such factors into the background trends and tests for an impact on an intervention (in our case, publication of the AUC) above and beyond these background trends. Fourth, billing codes for newer noninvasive diagnostic tests, namely, stress echocardiography and computed tomography angiography, only became available in Ontario in 2011. Consequently, we were unable to track utilization of these tests during the time of the observed significant reduction in MPI rates to determine if MPIs were being replaced by either of these tests. With that said, a previously published study from our group reported that between 2011 and 2014 (ie, when data for all 4 tests were available), the vast majority of tests performed in Ontario were still GXTs and MPIs, accounting for ≈80% to 85% of all noninvasive diagnostic tests performed. Furthermore, these data indicated that there was a slight reduction in the overall testing rate during that time16; therefore, it is unlikely that the reduction in MPI was caused by replacement with other testing modalities. Fifth, our study did not assess the impact of test‐specific AUC on the rates of other important noninvasive diagnostic tests such as GXT, stress echocardiography, and computed tomography angiography. Future work targeting each of these modalities is needed to assess the impact of test‐specific AUC publication on the rates of utilization of each individual testing modality. Last, we report results from Ontario, Canada. These results may not be representative of other jurisdictions.

Conclusions

Publication of the 2009 AUC was associated with a significant reduction in the MPI utilization rates in Ontario after accounting for background trend and seasonality. This reduced rate translated into ≈88 849 fewer MPI scans performed at a cost savings of ≈72 million Canadian dollars. These population‐based results illustrate a potential real‐world impact of the MPI AUC.

Sources of Funding

This article was funded by operating funds from Schulich Heart Program and the Sunnybrook Research Institute. This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long‐Term Care (MOHLTC). The funding organizations did not have any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.

Disclosures

None.