Emergency department use following incentives to provide after-hours primary care: a retrospective cohort study.

BACKGROUND: Access to primary care outside of regular working hours is limited in many countries. This study investigates the relation between the after-hours premium, an incentive for primary care physicians to provide services after hours, and less-urgent visits to the emergency department in Ontario, Canada.
METHODS: We analyzed a retrospective cohort of a random sample of Ontario residents from April 2002 to March 2006, and a subcohort of patients followed from April 2005 to March 2016. We linked patient and primary care physician data with emergency department visit data. We used fixed-effects regression models to analyze the association between the introduction of the after-hours premium, as well as subsequent increases in the value of the premium, and the number of monthly emergency department visits.
RESULTS: The sample consisted of 586 534 patients between 2002 and 2006, and 201 594 patients from 2005 to 2016. After controlling for patient and physician characteristics, seasonality and time-invariant patient confounding factors, introduction of the after-hours premium was associated with a reduction of 1.26 less-urgent visits to the emergency department per 1000 patients per month (95% confidence interval -1.48 to -1.04). Most of this reduction was observed in after-hours visits. Sensitivity analysis showed that the monthly reduction in less-urgent visits to the emergency department was in the range of -1.24 to -1.16 per 1000 patients. Subsequent increases in the after-hours premium were associated with a small reduction in less-urgent visits to the emergency department.
INTERPRETATION: Ontario's experience suggests that incentivizing physicians to improve access to after-hours primary care reduces some less-urgent visits to the emergency department. Other jurisdictions may consider incentives to limit less-urgent visits to the emergency department.

One prominent health policy issue confronting many countries is overcrowding of the emergency department.1 Not only does overcrowding result in longer wait times in the emergency department, but it may be associated with patient dissatisfaction and higher risk of death,2–4 as well as contribute to higher health system costs.5–7 Use of the emergency department by patients with conditions treatable in primary care may be a factor that contributes to emergency department overcrowding,4,8,9 and improved after-hours access to primary care is a potential solution.10

Following Canada’s initiatives on primary care reform in the early 2000s, the Ontario government introduced several patient enrolment models (PEMs) for primary care delivery.11 These models were characterized by mandatory patient enrolment, group-based practice and blended remuneration, including retrospective and prospective payments and pay-for-performance incentives. In July 2003, the Ontario government introduced the after-hours premium, an incentive for physicians practising in PEMs to claim an additional 10% on specific services provided to enrolled patients after regular business hours (5 pm to 8 am on weekdays, and any time on weekends and holidays). The after-hours premium increased to 15% in April 2005, 20% in April 2006 and to 30% in September 2011. One study examined the effect of enrolment in a PEM on overall emergency department visits, thereby masking any differential effects on urgent and less-urgent visits.12 Using physician-level data, a recent study examined the impact of the increase in the after-hours premium from 10% to 20%.13 We build on this literature and examine whether the introduction of Ontario’s after-hours premium, and subsequent increases in the premium, were associated with changes in emergency department visits, stratified by visit urgency. In particular, we examine whether the premium was associated with reductions in less-urgent visits to the emergency department.

Methods

We conducted a retrospective cohort study using linked health administrative data to investigate the effect of the after-hours premium on emergency department visits over 2 periods. The results are reported according to the Strength in Reporting of Observational Studies in Epidemiology guidelines.14 We examined a 10% random sample of Ontario residents in 2002, following them in 2 discrete periods up to 2016. We excluded patients who did not have complete data over the entire study period, as well as patients who lived in rural areas because of rural–urban differences in primary care practice.15 In the first period, Apr. 1, 2002, to Mar. 31, 2006, we examine the effect of the premium’s introduction and of the change in its value from 10% to 15%. Follow-up was limited to 2006 as several other incentives and new PEMs were introduced after 2006.11 In the second period, we followed a subcohort of patients, all of whom were enrolled in a PEM from Apr. 1, 2005, to Mar. 31, 2016, investigating the effect of subsequent increases in the premium.

Data sources and variables

Seven health administrative databases were linked using encoded identifiers, as follows: the Corporate Provider Database, for physician demographic characteristics and physician model type; the ICES Physician Database, for additional physician practice characteristics; the Client Agency Program Enrolment database, for linking enrolled patients with their physicians; the Ontario Health Insurance Plan Claims Database, for physician billings; the Registered Persons Database, for patient demographic characteristics; Canadian Census data, for dissemination area-level income; and the National Ambulatory Care Reporting System database, for emergency department visits.

The Client Agency Program Enrolment database captures all patients enrolled to family physicians in PEMs, enabling us to link patients with physicians in the PEM group.16 All other patients were assigned as fee-for-service (FFS) patients to the physician who claimed the highest billings in the previous 2 years.17,18 Patients were categorized by their physician’s eligibility to bill for the after-hours premium each month. Physicians who practised in PEMs were eligible to bill for the after-hours premium for their enrolled patients; physicians who practised in the traditional FFS model were not. PEMs are comprised of blended FFS, blended capitation and other specialized models, the details of which are described elsewhere.11

The outcome was the number of emergency department visits per patient per month, stratified by urgency and timing. Urgency was defined using the Canadian Triage and Acuity Scale scores.19 Emergency department visits with a score of 1 or 2 were defined as very urgent, a score of 3 was defined as urgent and scores of 4 or 5 were defined as less urgent (hereafter referred to as very urgent, urgent and less-urgent visits, respectively). Timing was split into regular hours (weekdays 8 am to 5 pm) and after hours (5 pm to 8 am on weekdays, and weekends and statutory holidays).

We included patient age, sex, income and comorbidity in our models. We defined low income as being in the 2 lowest area-level income quintiles. We used the Johns Hopkins Adjusted Clinical Groups software (Version 11.0) to define comorbidity as the number of adjusted diagnosis groups (maximum 32 groups). Physician characteristics included age, sex, international medical graduate status, experience (defined as years since graduation) and physician practice group size.

Statistical analysis

We used fixed-effects linear regression models to assess the effect of the introduction of the after-hours premium and increases in the premium on monthly emergency department visits at the patient level. Under certain reasonable assumptions, the results of a fixed-effects model may be interpreted to show causality.20 Fixed-effects models control for the observable and unobservable patient confounding factors that do not vary over time. Clustering of patients within physicians was allowed, to obtain valid standard errors.21 The after-hours premium was constructed as a categorical variable and set to the value of the premium (i.e., 10%, 15%, 20% or 30%) for patients whose physician was eligible to bill for the after-hours premium. The value of the premium for all patients before its introduction and for patients whose physician practised in the FFS model was set to 0. We adjusted for patient and physician characteristics, as well as for seasonal effects using monthly dummies.

One issue with the fixed-effects approach is that patients whose physician switched to a PEM between 2002 and 2006 and those who remained in the traditional FFS may be different. To address this potential selection bias, we used inverse probability weighting based on estimated propensity scores22–25 and entropy balancing weights26,27 to ensure that all observable physician characteristics, patient characteristics and emergency department visits were similar across both groups of patients before the introduction of the premium. Thus, any change in visits to the emergency department could be attributed to the after-hours premium. This methodological approach has been used in several recent papers to study the effect of reform on other outcomes.28–31 We also assessed the effect of the after-hours premium for a subsample of patients whose physician billed the premium at least once in a month.

Analysis was conducted at the patient level using separate models for each level of urgency and timing. The estimated coefficients were multiplied by 1000 for ease of interpretation. The β coefficient is the impact of the after-hours premium on the change in the number of emergency department visits per 1000 patients per month. Analysis was performed using Stata 15.1 and the user-written Stata program, reghdfe.

Ethics approval

The use of data in this study was authorized under Section 45 of Ontario’s Personal Health Information Protection Act; thus, review by a research ethics board was not needed.

Results

The construction of our study cohorts is presented in Figure 1. The sample for the first study period (2002–2006) consisted of 586 534 patients. The rate of emergency department visits increased from 28.19 per 1000 patients per month in 2002/03 to 28.99 per 1000 patients per month in 2005/06 (Table 1). This trend was driven by an increase in very urgent and urgent visits, while less-urgent visits decreased from 15.27 to 12.09 per 1000 patients per month. A slightly larger reduction in less-urgent visits was found during after-hours periods compared with regular hours (Figure 2).

Figure 1:

Flowchart showing creation of the study cohort and subcohort. Note: FFS = fee-for-service, PEM = patient enrolment model.

Table 1:

Descriptive characteristics of patients and their physicians between 2002/03 and 2005/06

Characteristic	Year; no. (%) of patients*n = 586 534
	2002/03	2003/04	2004/05	2005/06
Patients enrolled in PEM	0 (0.0)	167 400 (28.5)	288 321 (49.2)	449 481 (76.6)
Age group, yr
0–20	123 634 (21.1)	116 772 (19.9)	109 830 (18.7)	102 803 (17.5)
21–65	378 955 (64.6)	380 342 (64.9)	381 595 (65.1)	383 006 (65.3)
≥ 66	83 945 (14.3)	89 420 (15.3)	95 109 (16.2)	100 725 (17.2)
Sex, female	328 290 (56.0)	328 290 (56.0)	328 290 (56.0)	328 290 (56.0)
Low income area	223 160 (38.1)	217 889 (37.1)	230 563 (39.3)	226 166 (38.6)
ADG score, mean ± SE	3.650 ± 0.003	3.634 ± 0.003	3.580 ± 0.003	3.606 ± 0.003
No. of physicians	8128	8224	8186	8164
Physician characteristic
Age, yr, mean ± SE	48.71 ± 0.12	49.08 ± 0.12	49.63 ± 0.12	50.10 ± 0.12
Sex, female	2728 (33.6)	2822 (34.3)	2840 (34.7)	2888 (35.4)
Years since graduation, mean ± SE	22.24 ± 0.13	22.58 ± 0.13	23.12 ± 0.13	23.60 ± 0.13
Group size, mean ± SE	1.00 ± 0.00	1.37 ± 0.03	6.43 ± 0.15	10.71 ± 0.22
IMG status	1032 (12.7)	1056 (12.8)	1099 (13.4)	1170 (14.3)
ED visits per 1000 patients per month, mean ± SE
Overall ED visits	28.19 ± 0.11	25.98 ± 0.11	27.88 ± 0.11	28.99 ± 0.11
Regular hours	9.87 ± 0.05	9.23 ± 0.05	10.26 ± 0.05	10.90 ± 0.05
After hours	18.33 ± 0.08	16.75 ± 0.08	17.62 ± 0.08	18.09 ± 0.08
Very urgent visits	2.35 ± 0.02	2.68 ± 0.02	3.62 ± 0.03	4.35 ± 0.03
Regular hours	0.82 ± 0.01	0.93 ± 0.01	1.31 ± 0.01	1.58 ± 0.02
After hours	1.53 ± 0.02	1.75 ± 0.02	2.31 ± 0.02	2.77 ± 0.03
Urgent visits	10.58 ± 0.05	10.29 ± 0.05	11.68 ± 0.06	12.55 ± 0.06
Regular hours	3.56 ± 0.03	3.57 ± 0.03	4.17 ± 0.03	4.58 ± 0.03
After hours	7.02 ± 0.04	6.72 ± 0.04	7.51 ± 0.04	7.97 ± 0.04
Less-urgent visits	15.27 ± 0.07	13.01 ± 0.07	12.58 ± 0.06	12.09 ± 0.07
Regular hours	5.48 ± 0.04	4.73 ± 0.03	4.78 ± 0.03	4.74 ± 0.03
After hours	9.78 ± 0.05	8.28 ± 0.05	7.81 ± 0.05	7.35 ± 0.04

Note: ADG = aggregated diagnosis group, ED = emergency department, IMG = international medical graduate, PEM = patient enrolment model, SE = standard error.

Except where indicated otherwise.

Figure 2:

Monthly emergency department (ED) visits between April 2001 and March 2016, stratified by timing and urgency. After-hours premium values over time are indicated at the top of each graph.

We found that the introduction of the after-hours premium was associated with a reduction in less-urgent visits by 1.26 per 1000 patients per month (95% confidence interval [CI] −1.48 to −1.04) (Table 2). However, the reduction was larger after hours than during regular hours, with a reduction of 0.87 (95% CI −1.03 to −0.72) and 0.39 (95% CI −0.51 to −0.26) less-urgent visits per 1000 patients per month, respectively (Table 3). Although a reduction was found in less-urgent visits, there were small increases in urgent and very urgent visits. The increase in the after-hours premium from 10% to 15% was not associated with a change in less-urgent visits; however, the premium was associated with a small decrease in less-urgent visits after hours by 0.17 per 1000 patients per month (95% CI −0.32 to −0.02). The increase in the value of the premium was associated with increases in urgent and very urgent visits.

Table 2:

Impact of the after-hours premium value on emergency department visits between 2002/03 and 2005/06 based on fixed-effects regression models with and without monthly effects

Variable	All ED visits		Very urgent ED visits		Urgent ED visits		Less-urgent ED visits
	No monthly effects β (95% CI)*	With monthly effects β (95% CI)*	No monthly effects β (95% CI)*	With monthly effects β (95% CI)*	No monthly effects β (95% CI)*	With monthly effects β (95% CI)*	No monthly effects β (95% CI)*	With monthly effects β (95% CI)*
After-hours premium†
10%	−0.09 (−0.41 to 0.23)	0.23 (−0.10 to 0.55)	0.73 (0.63 to 0.82)	0.65 (0.55 to 0.75)	0.87 (0.70 to 1.05)	0.84 (0.66 to 1.02)	−1.69 (−1.91 to −1.48)	−1.26 (1.48 to −1.04)
15%	1.31 (0.98 to 1.64)	1.50 (1.17 to 1.83)	1.15 (1.05 to 1.24)	1.09 (0.99 to 1.19)	1.74 (1.55 to 1.93)	1.71 (1.52 to 1.90)	−1.58 (−1.80 to −1.35)	−1.29 (−1.52 to −1.07)
Change from 10% to 15%	1.40 (1.07 to 1.73)	1.28 (0.95 to 1.60)	0.42 (0.32 to 0.52)	0.44 (0.34 to 0.54)	0.87 (0.68 to 1.06)	0.87 (0.68 to 1.06)	0.11 (−0.10 to 0.32)	−0.04 (−0.24 to 0.18)
Month‡
February		−1.26 (−1.58 to −0.93)		−0.18 (−0.28 to −0.07)		−0.38 (−0.58 to −0.18)		−0.69 (−0.91 to −0.48)
March		1.29 (0.95 to 1.62)		0.28 (0.17 to 0.39)		0.63 (0.42 to 0.84)		0.38 (0.16 to 0.60)
April		−0.93 (−1.27 to −0.60)		−0.47 (−0.57 to −0.36)		−0.54 (−0.74 to −0.34)		0.07 (0.15 to 0.30)
May		1.48 (1.14 to 1.82)		−0.15 (−0.26 to −0.04)		0.22 (0.02 to 0.43)		1.40 (1.17 to 1.64)
June		1.12 (0.78 to 1.46)		−0.24 (−0.35 to −0.14)		−0.17 (−0.38 to 0.03)		1.54 (1.31 to 1.77)
July		3.18 (2.82 to 3.54)		−0.16 (−0.27 to −0.05)		0.46 (0.25 to 0.67)		2.884 (2.639 to 3.129)
August		3.27 (2.91 to 3.63)		−0.14 (−0.25 to −0.03)		0.56 (0.36 to 0.77)		2.84 (2.60 to 3.09)
September		1.58 (1.24 to 1.92)		−0.02 (−0.13 to 0.09)		0.21 (< 0.01 to 0.41)		1.39 (1.16 to 1.63)
October		0.73 (0.39 to 1.08)		−0.08 (−0.19 to 0.02)		0.07 (−0.14 to 0.28)		0.75 (0.52 to 0.98)
November		−0.44 (−0.77 to −0.10)		−0.14 (−0.24 to −0.03)		−0.159 (−0.362 to 0.044)		−0.14 (−0.36 to 0.08)
December		1.01 (0.68 to 1.35)		−0.08 (−0.18 to 0.02)		0.37 (0.16 to 0.58)		0.72 (0.50 to 0.94)
Patient age, 5 yr	0.25 (−0.02 to 0.52)	0.18 (−0.09 to 0.45)	0.82 (0.75 to 0.90)	0.84 (0.76 to 0.92)	0.78 (0.63 to 0.94)	0.80 (0.64 to 0.95)	−1.36 (−1.54 to −1.17)	−1.45 (−1.64 to −1.27)
Low income	−0.43 (−0.80 to −0.06)	−0.43 (−0.80 to −0.06)	< 0.00 (−0.11 to 0.11)	< 0.00 (−0.11 to 0.11)	−0.23 (−0.45 to −0.02)	−0.23 (−0.45 to −0.02)	−0.19 (−0.43 to 0.04)	−0.20 (−0.43 to 0.04)
ADG score	−2.01 (−2.08 to −1.94)	−2.01 (−2.08 to −1.94)	−0.22 (−0.24 to −0.20)	−0.22 (−0.24 to −0.20)	−0.85 (−0.90 to −0.81)	−0.85 (−0.90 to −0.81)	−0.94 (−0.98 to −0.90)	−0.94 (0.98 to −0.90)
Physician age, yr	0.03 (−0.37 to 0.43)	0.03 (−0.37 to 0.42)	0.04 (−0.08 to 0.16)	0.04 (−0.07 to 0.16)	−0.01 (−0.24 to 0.21)	−0.01 (−0.24 to 0.21)	> 0.00 (−0.27 to 0.27)	< 0.00 (−0.27 to 0.26)
Physician age, yr squared	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)	< 0.00 (< 0.00 to > 0.00)
Years since graduation	−0.09 (−0.28 to 0.11)	−0.09 (−0.28 to 0.11)	−0.07 (−0.12 to −0.01)	−0.07 (−0.12 to −0.01)	−0.03 (−0.13 to 0.08)	−0.03 (−0.13 to 0.08)	0.01 (−0.13 to 0.14)	0.01 (−0.13 to 0.14)
Years since graduation, squared	> 0.00 (< 0.00 to 0.01)	> 0.00 (< 0.00 to 0.01)	> 0.00 (< 0.00 to > 0.00)	> 0.00 (< 0.00 to > 0.00)	> 0.00 (< 0.00 to > 0.00)	> 0.00 (< 0.00 to > 0.00)	0.00 (< 0.00 to > 0.00)	0.00 (< 0.00 to > 0.00)
Female physician	0.60 (0.07 to 1.13)	0.59 (0.06 to 1.12)	0.10 (−0.04 to 0.23)	0.10 (−0.04 to 0.23)	0.35 (0.05 to 0.64)	0.35 (0.05 to 0.64)	0.16 (−0.19 to 0.51)	0.15 (−0.20 to 0.50)
International medical graduate	0.04 (−0.66 to 0.74)	0.05 (−0.65 to 0.75)	0.16 (−0.02 to 0.34)	0.16 (−0.03 to 0.34)	0.09 (−0.30 to 0.48)	0.09 (−0.30 to 0.48)	−0.21 (−0.66 to 0.24)	−0.20 (−0.65 to 0.25)
Group size	0.01 (> 0.00 to 0.02)	> 0.00 (< 0.00 to 0.01)	0.01 (> 0.00 to 0.01)	0.01 (> 0.00 to 0.01)	< 0.00 (−0.01 to > 0.00)	< 0.00 (−0.01 to > 0.00)	> 0.00 (0.00 to 0.01)	> 0.00 (< 0.00 to > 0.00)

Note: ADG = aggregated diagnosis group, CI = confidence interval, ED = emergency department.

β coefficients interpreted as the change in the number of ED visits per 1000 patients per month. For confidence intervals, standard errors were clustered at the patient and physician levels.

Reference group: after-hours premium value of 0%.

Reference group: January.

Table 3:

Impact of changes in the after-hours premium value on emergency department visits by timing, between 2002/03 and 2005/06*

Change in after-hours premium	All ED visits β (95% CI)†	Very urgent ED visits β (95% CI)†	Urgent ED visits β (95% CI)†	Less-urgent ED visits β (95% CI)†
Regular hours‡
0% to 10%	0.18 (0.01 to 0.35)	0.21 (0.15 to 0.26)	0.36 (0.26 to 0.46)	−0.39 (−0.51 to −0.26)
10% to 15%	0.69 (0.51 to 0.86)	0.17 (0.12 to 0.23)	0.38 (0.28 to 0.49)	0.13 (0.01 to 0.25)
After hours‡
0% to 10%	0.05 (−0.20 to 0.29)	0.44 (0.36 to 0.52)	0.48 (0.34 to 0.62)	−0.87 (−1.03 to −0.72)
10% to 15%	0.59 (0.35 to 0.83)	0.27 (0.18 to 0.35)	0.49 (0.34 to 0.63)	−0.16 (−0.32 to −0.02)

Note: CI = confidence interval, ED = emergency department.

Controlling for patient characteristics (patient age, low-income status, and comorbidity using aggregated diagnosis group score), and physician characteristics (physician age, physician age-squared, years since graduation, years since graduation-squared, physician sex, international medical graduation status and group size) as well as the monthly dummy variable.

β coefficients interpreted as the change in the number of ED visits per 1000 patients per month. For confidence intervals, standard errors were clustered at the patient and physician levels.

Regular hours refers to 8 am to 5 pm weekdays; after hours refers to 5 pm to 8 am weekdays, and any time on weekends and statutory holidays.

Results using inverse propensity score weights and entropy balancing weights (Appendices 1–3, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200277/tab-related-content) produced slight attenuation of the effects on less-urgent visits, in the range of −1.24 (95% CI −1.46 to −1.02) and −1.16 (95% CI −1.39 to −0.94) per 1000 patients per month (Table 4). However, with entropy balancing weights, increases in the value of the premium were no longer associated with a reduction in less-urgent visits after hours. The effect of introduction of the premium was slightly attenuated when examining only patients whose physician billed the after-hours premium (Appendix 4, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200277/tab-related-content). In this subsample, the effect of increases in the premium value on less-urgent visits after hours was no longer significant.

Table 4:

Impact of changes in the after-hours premium on emergency department visits between 2002/03 and 2005/06, using propensity score and entropy balancing weights*

Change in after-hours premium	All ED visits			Very urgent ED visits
	IPW β (95% CI)†	CBPS β (95% CI)†	Entropy balancing β (95% CI)†	IPW β (95% CI)†	CBPS β (95% CI)†	Entropy balancing β (95% CI)†
Any timing
0% to 10%	0.24 (−0.08 to 0.57)	0.25 (−0.08 to 0.57)	0.23 (−0.10 to 0.57)	0.65 (0.55 to 0.75)	0.65 (0.55 to 0.75)	0.61 (0.51 to 0.71)
10% to 15%	1.29 (0.96 to 1.62)	1.29 (0.96 to 1.62)	1.26 (0.93 to 1.60)	0.44 (0.34 to 0.54)	0.44 (0.34 to 0.55)	0.41 (0.30 to 0.51)
Regular hours‡
0% to 10%	0.19 (0.02 to 0.36)	0.19 (0.02 to 0.36)	0.16 (−0.01 to 0.34)	0.21 (0.16 to 0.26)	0.21 (0.16 to 0.26)	0.20 (0.14 to 0.25)
10% to 15%	0.69 (0.52 to 0.87)	0.70 (0.52 to 0.87)	0.67 (0.49 to 0.85)	0.17 (0.12 to 0.23)	0.17 (0.12 to 0.23)	0.16 (0.10 to 0.22)
After hours‡
0% to 10%	0.06 (−0.19 to 0.30)	0.06 (−0.19 to 0.30)	0.07 (−0.18 to 0.32)	0.44 (0.37 to 0.52)	0.44 (0.37 to 0.52)	0.41 (0.33 to 0.49)
10% to 15%	0.59 (0.35 to 0.84)	0.59 (0.35 to 0.84)	0.59 (0.35 to 0.84)	0.27 (0.18 to 0.35)	0.27 (0.18 to 0.35)	0.24 (0.16 to 0.33)
	Urgent ED visits			Less-urgent ED visits
	IPW β (95% CI)†	CBPS β (95% CI)†	Entropy balancing β (95% CI)†	IPW β (95% CI)†	CBPS β (95% CI)†	Entropy balancing β (95% CI)†
Any timing
0% to 10%	0.84 (0.66 to 1.02)	0.84 (0.66 to 1.02)	0.79 (0.60 to 0.98)	−1.24 (−1.46 to −1.02)	−1.24 (−1.46 to −1.02)	−1.16 (−1.39 to −0.94)
10% to 15%	0.87 (0.68 to 1.06)	0.87 (0.68 to 1.06)	0.83 (0.63 to 1.02)	−0.02 (−0.24 to 0.18)	−0.02 (−0.24 to 0.18)	0.03 (−0.18 to 0.24)
Regular hours‡
0% to 10%	0.36 (0.26 to 0.46)	0.36 (0.27 to 0.46)	0.35 (0.24 to 0.45)	−0.38 (−0.51 to −0.26)	−0.38 (−0.51 to −0.26)	−0.38 (−0.50 to −0.25)
10% to 15%	0.39 (0.28 to 0.49)	0.39 (0.28 to 0.49)	0.37 (0.26 to 0.47)	0.13 (0.01 to 0.26)	0.14 (0.01 to 0.26)	0.14 (0.02 to 0.26)
After hours‡
0% to 10%	0.47 (0.33 to 0.61)	0.47 (0.33 to 0.61)	0.44 (0.30 to 0.58)	−0.86 (−1.02 to −0.70)	−0.86 (−1.02 to −0.70)	−0.78 (−0.95 to −0.62)
10% to 15%	0.49 (0.34 to 0.63)	0.49 (0.34 to 0.63)	0.46 (0.31 to 0.61)	−0.16 (−0.31 to −0.01)	−0.16 (−0.31 to −0.01)	−0.11 (−0.26 to 0.04)

Note: CBPS = covariate-balancing propensity score weighting, CI = confidence interval, ED = emergency department, IPW = inverse probability weighting.

Controlled for patient characteristics (patient age, low-income status, and comorbidity using aggregated diagnosis group score), and physician characteristics (physician age, physician age-squared, years since graduation, years since graduation-squared, physician sex, international medical graduation status, and group size) as well as the monthly dummy variable.

β coefficients interpreted as the change in the number of ED visits per 1000 patients per month. For confidence intervals, standard errors were clustered at the patient and physician levels.

Regular hours refers to 8 am to 5 pm weekdays; after hours refers to 5 pm to 8 am weekdays, and any time on weekends and statutory holidays.

Of the initial sample, 201 594 patients remained enrolled in a PEM for the entire period between Apr. 1, 2005, and Mar. 31, 2016 (Appendix 5, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200277/tab-related-content). The rate of emergency department visits for this subcohort increased over time, from 23.9 per 1000 patients per month in 2005/06 to 33.2 per 1000 patients per month in 2015/16. Similar to the first period, this increase was driven by very urgent and urgent visits, while less-urgent visits decreased from 11.2 to 9.7 per 1000 patients per month, with a larger reduction in visits after hours compared with regular hours.

Although the increase in the premium from 15% to 20% was not associated with reductions in less-urgent visits, the increase from 20% to 30% was associated with a small reduction in less-urgent visits, especially after hours (Table 5). We found a reduction of 0.17 (95% CI −0.31 to −0.03) and 0.13 (95% CI −0.25 to −0.02) emergency department visits per 1000 patients per month after hours when the premium increased from 15% to 20% and from 20% to 30%, respectively. Although less-urgent visits decreased, we found an increase in very urgent and urgent visits to the emergency department.

Table 5:

Impact of changes in the after-hours premium on emergency department visits between 2005/06 and 2015/16*

Change in after-hours premium	All ED visits β (95% CI)†	Very urgent ED visits β (95% CI)†	Urgent ED visits β (95% CI)†	Less-urgent ED visits β (95% CI)†
Any timing
15% to 20%	−0.08 (−0.39 to 0.24)	0.10 (> 0.00 to 0.20)	−0.24 (−0.42 to −0.05)	0.06 (−0.14 to 0.26)
20% to 30%	3.08 (2.78 to 3.37)	1.31 (1.20 to 1.42)	1.99 (1.81 to 2.17)	−0.22 (−0.39 to −0.06)
Regular hours‡
15% to 20%	0.27 (0.10 to 0.43)	0.04 (−0.02 to 0.09)	> 0.00 (−0.09 to 0.10)	0.22 (0.11 to 0.34)
20% to 30%	1.26 (1.10 to 1.43)	0.46 (0.41 to 0.52)	0.89 (0.79 to 0.99)	−0.09 (−0.19 to 0.01)
After hours‡
15% to 20%	−0.34 (−0.57 to −0.11)	0.07 (−0.02 to 0.15)	−0.24 (−0.38 to −0.10)	−0.17 (−0.31 to −0.02)
20% to 30%	1.81 (1.60 to 2.02)	0.84 (0.76 to 0.93)	1.10 (0.97 to 1.23)	−0.13 (−0.25 to −0.02)

Note: CI = confidence interval, ED = emergency department.

Model of subcohort of patients with physicians using patient enrolment models (n = 201 594). Controlled for patient characteristics (patient age, low-income status, and comorbidity using aggregated diagnosis group score), and physician characteristics (physician age, physician age-squared, years since graduation, years since graduation-squared, physician sex, international medical graduation status, and group size) as well as the monthly dummy variable.

β coefficients interpreted as the change in the number of ED visits per 1000 patients per month. For confidence intervals, standard errors were clustered at the patient and physician levels.

Regular hours refers to 8 am to 5 pm weekdays; after hours refers to 5 pm to 8 am weekdays, and any time on weekends and statutory holidays.

Interpretation

The introduction of an after-hours premium for primary care practitioners in Ontario reduced less-urgent visits to the emergency department, with much of the reduction found in visits after hours rather than during regular hours. The effect of the introduction of the after-hours premium was similar when accounting for potential selection bias. Using the lower bound from the entropy-balanced weighted estimate of 1.16 fewer less-urgent visits per 1000 patients per month (95% CI −1.39 to −0.94), and given that about 10 million residents were enrolled in a PEM by March 2011,32 the introduction of the after-hours premium was associated with a reduction of 139 600 less-urgent visits (95% CI 112 600 to 166 700) per annum. Subsequent increases in the premium were associated with a very small reduction in less-urgent visits after hours. We found that the introduction of the after-hours premium reduced some less-urgent visits, suggesting that some emergency department visits can be avoided through improved access to primary care.

Previous studies found that interventions to improve access to after-hours primary care can be associated with a reduction in emergency department use.33–38 Our results are consistent with studies that found that incentivizing physicians to be available for longer hours was associated with a small reduction in nonurgent emergency department visits.39 However, our study conflicts with another that found that enrolment in a PEM was associated with an increase in emergency department visits in Ontario.12 That study’s findings are likely owing to the use of overall emergency department visits without a comparison group, and the lack of control for patient-specific time-invariant confounding factors. In contrast, a more recent study at the physician level found that increases in the after-hours premium from 10% to 20% were associated with a small reduction in less-urgent visits to the emergency department.13 Many studies have explored the effects of interventions not directly related to the emergency department on its use; however, the impact of these programs varies vastly in the context of managed care setting with financial incentives.40 Our results regarding less-urgent visits to the emergency department fall closer to the lower end of the effectiveness of financial incentives found in the literature.

Further research is required to understand the cost-effectiveness of the after-hours premium and determine whether the after-hours premium is a cost-saving measure for the health care system. Further research is also required to examine whether it has led to improved access to primary care for patients with unmet needs and improved patient satisfaction.

Limitations

First, we were unable to account for the role of individual-level socioeconomic factors that may influence emergency department visits. Second, while the first period analyses reflect the effect of the after-hours premium, as it was not contaminated with other policies, the analyses in our second period capture the combined effects of the after-hours premium and other pay-for-performance incentives introduced in the post-2006 era. We were not able to disentangle the effects of the increased after-hours premium (20% to 30%) from other incentives as some services are covered by multiple incentives in the post-2006 period. Third, whether the premium is actually associated with urgent visits or whether other factors contribute to this association requires further investigation. About 25% of the sample were excluded with incomplete follow-up data, including patients who had died, who had moved out of province, or who could not be traced in ICES databases. Although exclusion of patients who died is unlikely to affect the number of less-urgent visits to the emergency department, it may bias our results of very urgent and urgent visits. Fourth, there may be some residual confounding if there is unobservable physician selection into PEMs. Fifth, we are unable to account for the role of walk-in clinics as we did not have accurate data on these clinics. Finally, our data set was unable to capture whether patients attempted to seek care from a primary care physician before seeking care in the emergency department or whether they were referred to the emergency department by their physicians.

Conclusion

The after-hours premium reduced some less-urgent visits to the emergency department in Ontario. Interventions to reduce potentially avoidable emergency department visits are of considerable interest to policy-makers; the use of incentives to promote access to after-hours primary care and divert patients away from the emergency department in a cost-effective manner is one policy option to consider.