Primary care performance of alternatively licenced physicians in Ontario, Canada: a cross-sectional study using administrative data.

OBJECTIVES: Medical Regulatory Authorities (MRAs) provide licences to physicians and monitor those physicians once in practice to support their continued competence. In response to physician shortages, many Canadian MRAs developed alternative licensure routes to allow physicians who do not meet traditional licensure criteria to obtain licences to practice. Many physicians have gained licensure through alternative routes, but the performance of these physicians in practice has not been previously examined. This study compared the performance of traditionally and alternatively licenced physicians in Ontario using quality indicators of primary care. The purpose of this study was to examine the practice performance of alternatively licenced physicians and provide evaluative evidence for alternative licensure policies.
DESIGN: A cross-sectional retrospective examination of Ontario health administrative data was conducted using Poisson regression analyses to compare the performance of traditionally and alternatively licenced physicians.
SETTING: Primary care in Ontario, Canada. PARTICIPANTS: All family physicians who were licenced in Ontario between 2000 and 2012 and who had complete medical billing data in 2014 were included (n=11 419). OUTCOME MEASURES: Primary care quality indicators were calculated for chronic disease management, preventive paediatric care, cancer screening and hospital readmission rates using Ontario health administrative data.
RESULTS: Alternatively licenced physicians performed similarly to traditionally licenced physicians in many primary care performance measures. Minimal differences were seen across groups in indicators of diabetic care, congestive heart failure care, asthma care and cancer screening rates. Larger differences were found in preventive care for children less than 2 years of age, particularly for alternatively licenced physicians who entered Ontario from another Canadian province.
CONCLUSIONS: Our findings demonstrate that alternatively licenced physicians perform similarly to traditionally licenced physicians across many indicators of primary care. Our study also demonstrates the utility of administrative data for examining physician performance and evaluating medical regulatory policies and programmes. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study to examine the primary care performance of alternatively licenced family physicians in Ontario.

The use of population-level data across multiple indicators of primary care allowed for a comprehensive comparison of alternatively licensed physicians (ALPs) and traditionally licensed physicians (TLPs); the use of multivariable analysis enabled statistical adjustment of factors associated with primary care performance.

A limitation of this study is that ALPs and TLPs were compared to each other, not to a gold standard; thus, findings do not indicate whether physicians are meeting performance benchmarks, only whether ALP performance is comparable to TLP performance.

Secondly, results are based on one year of health administrative data which depicts a point in time and only represents elements of care that are funded by the Ontario Ministry of Health and Long-Term Care; other important aspects of primary care are not accounted for.

Lastly, quality indicators are proxies for delivery of care; therefore, some of the variance in the indicators may be, in part, attributable to differences in billing practices or factors unrelated to the physician, such as patient preference.

Introduction

A safe and effective healthcare system relies on high quality physician performance. Medical Regulatory Authorities (MRAs) support such performance by issuing licences to qualified physician applicants and monitoring those physicians once in practice to ensure their continued competence. MRAs also support high quality performance by examining factors that influence physician performance in order to identify physician subgroups that may benefit from educational support.1–8 There has been discussion about the efficacy of regulatory processes for serving professional and public interests9–11 and calls for evidence-informed regulation through the evaluation of regulatory processes and programmes.11–14 This study heeds such a call by examining the primary care performance of family physicians in Ontario as a way of evaluating regulatory licensure policies and exploring the influence of licensure route on physician performance.

In Canada, physicians traditionally complete a Canadian residency programme and the Canadian qualifying and certification examinations to be granted a licence to practice. However, in response to projected physician shortages in the early 2000s, many Canadian MRAs developed alternative licensing criteria to facilitate the licensure of physicians who do not meet the traditional criteria.15 16 Alternative licensure routes were developed, primarily for international medical graduates (IMGs), based on previous experience or licensure, postgraduate training, and/or eligibility to write the Canadian certification exams.15 Often, these physicians were recruited to work in specific underserviced areas and given provisional licences to practice despite not meeting the traditional qualifications.17 Smaller Canadian provinces, such as Newfoundland and Saskatchewan, have been prominent issuers of provisional licences due to their longstanding health human resource needs18 19; however, provisionally licenced physicians often move to other parts of the country after completing their service terms, as most are able to practice anywhere in Canada once licenced.16–20 As such, it is thought that smaller provinces may serve as entry points to larger provinces such as Ontario.17 18

In addition to the migration of provisionally licenced physicians across Canada, alternative licensure routes also allow entry of physicians from the US into Canada and the licensure of physicians who completed Canadian residency but did not immediately write or pass the national certification exams. In these cases, provisional licences are given with the stipulation of successful exam completion within 3 years. Although these routes were initially developed to increase access for IMGs, they are now also used by domestic medical graduates (DMGs) who have not successfully completed exams at the time of licensure.

Collectively, alternatively licenced physicians (ALPs) represent physicians who did not meet the licensure criteria at the time of entering independent practice in a given province but who were considered to have comparable qualifications to traditionally licenced physicians (TLPs), based on their postgraduate training and/or professional experience. The performance of ALPs in practice, however, has not been previously examined. Given that many ALPs are IMGs, a review of IMG literature may offer insight into ALP practice performance; however, research comparing IMGs and DMGs has been equivocal. Some studies show IMGs perform less well than DMGs on certification and licencing examinations21–24 and that such performance is associated with practice performance.5 25 Yet, IMGs and DMGs have been shown to be comparable on practice outcomes such as patient mortality,26 27 readmission rates,27 surgical outcomes28 and cardiac care.29

While these conflicting findings may reflect the different outcomes being measured, they may also stem from the limited definition of IMG being employed. IMGs are typically defined by and compared on the location of their undergraduate medical training, but this only represents one step in an often long and diverse path of training and experience to independent practice.15 Examining physicians as defined by later steps in this process, such as point of licensure, may shed light on the impact of postgraduate medical training and early career practice experiences on subsequent performance and how physicians entering practice through alternative licensure routes may be better supported at different stages of their career.

In this study, we sought to understand the impact of alternative licensure routes on the delivery of primary care in Ontario. We used primary care quality indicators derived from health administrative data that were developed and validated by health services researchers to examine physician performance in areas such as chronic disease management, screening rates and hospital readmissions using accepted practice guidelines.30 31 We focused on the performance of a cohort of family physicians licenced through three main alternative routes: those licenced in another Canadian province, those licenced in the USA and those who trained in Canada but did not complete certifying examinations at the time of licensure. The research question guiding this study was: does licensure route influence the primary care performance of physicians in Ontario?

Methods

Approach

The objective of this study was to examine the independent contribution of being licenced through various alternative routes on primary care performance. Given that licensure route has not been previously studied as a potential factor influencing performance, we were interested in isolating its effect by comparing each ALP group to TLPs on a variety of quality care indicators while adjusting for a number of covariates. We do not address the independent impact of the other variables that were adjusted for, as, practically, we could either focus on a small number of outcomes and explore the full multivariable models, or examine a broad spectrum of indicators representative of general family practice and narrow our focus to licensure route. We chose the latter, as we were interested in primary care performance as a whole rather than performance on any individual quality indicators. Additionally, the indicators do not have validated thresholds or gold standard rates at the individual level or population level, thus performance is better assessed globally as opposed to focusing on individual tests, screens or prescriptions. By focusing on the independent contribution of licensure route while adjusting for covariates, we aimed to understand if a physician’s licensure route is associated with subsequent performance, irrespective of other demographic or practice characteristics. The goal of this approach was to explore whether regulatory licensure policies independently influence the primary care provided to patients in Ontario, thus offering evaluative evidence for the impact and outcomes of these policies.

Study cohorts

The study population included all practising family physicians in Ontario who were registered with the College of Physicians and Surgeons of Ontario between 2000 and 2012 and billed the Ontario Health Insurance Plan (OHIP) in 2014. All publicly funded health services provided by physicians are submitted to OHIP. This population included TLPs and ALPs. TLPs are physicians who obtained a licence to practice by meeting the traditional criteria, namely the completion of postgraduate training in Canada and successful completion of the national qualifying and certification examinations (the Medical Council of Canada Qualifying Examinations parts 1 and 2, and either the College of Family Physicians of Canada or the Royal College of Physicians and Surgeons of Canada examinations). ALPs are those physicians who were missing one or more of the traditional requirements but met an alternative set of criteria at the time of licensure. There are many alternative licensure routes; in this study, we have focused on the three most commonly used by family physicians, described in table 1. A more comprehensive description of these routes has been described previously.15

Table 1

Description of alternatively licenced physician (ALP) subgroups

Out-of-province ALPs	Physicians who obtained a licence in another Canadian province and thus were given an equivalent licence in Ontario despite missing one or more traditional licencing requirements* or who gained eligibility to write the CFPC examinations by gaining 2 years of practice experience in another Canadian province and were thus eligible for a provisional licence in Ontario.†
US-trained ALPs	Physicians who completed postgraduate training in the USA but had not successfully completed the Canadian certification examinations at the time of licensure.†‡
Canadian-trained ALPs	Physicians who completed postgraduate training in Canada but had not successfully completed the Canadian certification examinations at the time of licensure.†

*The Agreement on Internal Trade is an interprovincial agreement that was incorporated into Ontario legislation enabling physicians migrating from other Canadian provinces be granted equivalent licences to practice without assessment or examination.

†Physicians are granted restricted (provisional) licences and have up to 3 years to write the Canadian certification examinations.

‡Physicians may be granted restricted (provisional) licences due to successful completion of a practice assessment.

CFPC, College of Family Physicians of Canada.

Data sources

Ontario health administrative datasets held at the Institute for Clinical Evaluative Sciences (ICES) were used in this study. These datasets were linked using unique encoded identifiers and analysed at ICES under data security and privacy policies and procedures that are approved by the Office of the Information and Privacy Commissioner of Ontario.32 The following administrative databases were used: Canadian Institute for Health Information hospital Discharge Abstract Database (providing diagnostic information regarding hospital admissions), OHIP physician claims database (containing physician billings and diagnoses from 1991), the National Ambulatory Care Reporting System database (providing information on hospital-based and community-based ambulatory care, including emergency department (ED) visits, from 2000 and same-day surgery from 1991) and the Ontario Drug Benefit program database (containing information on all drug therapies dispensed to eligible individuals 65 years of age and older).

Variables

Physician demographic characteristics included age, sex, medical school region and the Human Development Index (HDI) associated with the physician’s country of medical school, which is a composite score based on life expectancy, education and per capita income that rank orders all countries.33 Physician practice characteristics included practice type (comprehensive or not), group type (family health team [FHT], non-FHT no group), scope of practice (percent providing any of the following: postnatal visits, obstetrical deliveries, postnatal visits, ED visits and long-term care visits) and practice location (urban and suburban/rural). Comprehensive family physicians are those who met specific criteria regarding the type and scope of services they provide.34 FHTs are group practices that include comprehensive family physicians working alongside primary providers such as nurses, social workers, pharmacists and nutritionists. A detailed description of the physician demographic and practice characteristics is included in online supplementary appendix A.

Primary care quality indicators based on health administrative data were calculated for chronic disease management, preventive paediatric care, cancer screening and hospital readmission rates. Chronic disease management indicators included measures for diabetes care (haemoglobin A1c (glycated haemoglobin) [HbA1c] testing, cholesterol testing, ophthalmology examinations, the receipt of prescriptions for ACE inhibitors and angiotensin II receptor blockers), congestive heart failure (CHF; echocardiogram testing within 12 months of diagnosis and ED visits), asthma (spirometry testing within 12 months of diagnoses and ED visits) and chronic obstructive pulmonary disease (COPD; spirometry testing within 12 months of diagnoses and ED visits). Paediatric care indicators include well-baby visits, the 18-month enhanced developmental assessment and the absence of paediatric vaccinations (defined as no billing for any immunisation in OHIP). Cancer screening indicators included cervical, breast and colorectal cancer screening. Hospital readmission rates were calculated at 30 days and 1 year for patients with a hospital admission. These primary care quality indicators are described in online supplementary appendix B.

For each family physician, patients who were either rostered (enrolled) or virtually rostered to them (attributed to the physician based on the majority of their billings) were included. All outcomes denote whether a patient received a given type of care, rather than whether the physician they were rostered to provided it. Therefore, patients who received care from a physician other than their family physician (eg, a walk-in clinic physician or another family physician in their practice) would appear in the data as having received that care, and this would be attributed to the family physician they are rostered to.

Statistical analysis

Demographic and practice characteristics are presented as proportions, means, percentage with any and mean percentages (table 2). Absolute rates for the primary care quality indicators are presented as means and mean percentages unless otherwise noted (table 3). Unadjusted rates are included for comprehensiveness, but only the adjusted rates are discussed to answer our research question. Confidence limits are presented where applicable. To help with interpretation of results, statistically significant differences less than 5% were considered small and statistically significant differences greater than 5% were considered larger.

Table 2

Demographic and practice characteristics of TLPs and ALPs

Characteristic	All TLPs	Out-of-province ALPs	US-trained ALPs	Canadian-trained ALPs
Total (n)	11 127	87	91	114
Sex, n (%)
Male	6303 (56.6)	60 (69)	38 (41.8)	62 (54.4)
Female	4824 (43.4)	27 (31)	53 (58.2)	52 (45.6)
Age (years, mean)	50.6	49.5	42.1	45.3
Medical school region, n (%)
Canada/USA	8656 (77.8)	9 (10.3)	13 (14.3)	54 (47.4)
All others	2471 (22.2)	78 (89.7)	78 (85.7)	60 (52.7)
HDI group, n (%)
Very high/High	10 065 (90.5)	32 (36.8)	62 (68.1)	83 (72.9)
Medium/low	1062 (9.5)	55 (63.2)	29 (31.9)	31 (27.2)
Practice type, n (%)
Comprehensive	7355 (66.1)	60 (69)	64 (70.3)	80 (70.2)
Not comprehensive	3772 (33.9)	27 (31)	27 (29.7)	34 (29.8)
Group type, n (%)
FHT	2273 (20.4)	12 (13.8)	16 (17.6)	20 (17.5)
Non-FHT	5635 (50.6)	47 (54)	60 (65.9)	70 (61.4)
No group	3219 (28.9)	28 (32.2)	15 (16.5)	24 (21.1)
Rurality, n (%)
Urban	8596 (77.3)	61 (70.1)	71 (78)	77 (67.5)
Suburban/rural	2531 (22.7)	26 (29.9)	20 (22)	37 (32.5)
Scope of practice (N, % with any)
Prenatal visits	6131 (55.1)	46 (52.9)	52 (57.1)	77 (67.5)
Obstetrical delivery	1224 (11)	6 (6.9)	7 (7.7)	21 (18.4)
Postnatal visits	3093 (27.8)	24 (27.6)	17 (18.7)	33 (28.9)
ED visits	2292 (20.6)	17 (19.5)	8 (8.8)	27 (23.7)
LTC visits	2181 (19.6)	7 (8)	13 (14.3)	27 (23.7)
Patient age distribution
<18 years	18.5	23.0	21.0	20.4
18–64 years	63.8	63.4	63.1	62.5
65+ years	17.7	13.6	15.9	17.1
Patient SES
% low income	38.0	42.6	42.2	38.9

ALPs, alternatively licenced physicians; ED, emergency department; FHT, family health team; HDI, Human Development Index (2013); LTC, long-term care; SES, socioeconomic status; TLPs, traditionally licenced physicians.

Table 3

Adjusted and unadjusted relative rates from Poisson modelling, primary care quality indicators, ALPs versus TLPs

Population/measure	Out-of-province ALPs				US-trained ALPs				Canadian-trained ALPs
	Unadjusted RR (CL)	Sig.	Adjusted RR(CL)	Sig.	Unadjusted RR (CL)	Sig.	Adjusted (RR CL)	Sig.	Unadjusted RR (CL)	Sig.	Adjusted RR (CL)	Sig.
Total (n)	87				91				114
Diabetes
HbA1c	0.95 (0.91 to 0.98)	**	0.96 (0.92 to 0.99)	*	1.09 (1.06 to 1.12)	****	1.08 (1.05 to 1.11)	****	1.03 (1.00 to 1.06)	*	1.03 (1.01 to 1.06)	*
Eye exam	0.97 (0.94 to 1.00)	*	0.98 (0.95 to 1.01)		0.98 (0.96 to 1.01)		0.99 (0.97 to 1.01)		1.00 (0.98 to 1.03)		1.00 (0.98 to 1.02)
Lipids	1.03 (1.00 to 1.06)	*	1.00 (0.97 to 1.03)		1.12 (1.09 to 1.14)	****	1.08 (1.05 to 1.10)	****	1.02 (1.00 to 1.05)	*	1.02 (0.99 to 1.04)
ACE/AARB	1.02 (0.95 to 1.11)		1.06 (0.97 to 1.14)		0.96 (0.90 to 1.02)		0.98 (0.92 to 1.04)		1.05 (1.00 to 1.11)		1.04 (0.98 to 1.10)
Statin	1.00 (0.95 to 1.04)		0.98 (0.94 to 1.03)		1.03 (1.00 to 1.07)		1.01 (0.97 to 1.04)		1.01 (0.98 to 1.04)		1.00 (0.97 to 1.03)
CHF
Echo within 12 months of dx	1.00 (0.93 to 1.08)		1.00 (0.93 to 1.08)		1.03 (0.97 to 1.10)		1.02 (0.96 to 1.08)		1.01 (0.95 to 1.06)		1.00 (0.95 to 1.06)
COPD
Spiro within 12 months of dx	0.80 (0.69 to 0.92)	**	0.82 (0.71 to 0.95)	**	1.10 (0.99 to 1.22)		1.10 (0.99 to 1.23)		0.88 (0.80 to 0.97)	*	0.89 (0.80 to 0.98)	*
Asthma
Spirometry (ever)	0.96 (0.93 to 0.98)	**	1.03 (1.01 to 1.06)	*	0.98 (0.96 to 1.00)		1.00 (0.98 to 1.02)		1.00 (0.98 to 1.02)		1.03 (1.01 to 1.06)	**
CHF to COPD or asthma
ED visits per person	1.08 (1.06 to 1.10)	****	1.07 (1.05 to 1.09)	****	1.02 (1.01 to 1.04)	**	0.97 (0.95 to 0.99)	***	1.18 (1.16 to 1.19)	****	1.09 (1.07 to 1.10)	****
Paediatric care
Well-baby visits	0.90 (0.87 to 0.92)	****	0.86 (0.83 to 0.88)	****	0.93 (0.90 to 0.95)	****	0.93 (0.90 to 0.95)	****	0.98 (0.96 to 1.01)		0.97 (0.95 to 0.99)	*
18-month enhanced assessment	0.70 (0.63 to 0.78)	****	0.76 (0.68 to 0.84)	****	0.99 (0.92 to 1.07)		0.99 (0.91 to 1.07)		1.03 (0.96 to 1.10)		1.06 (0.99 to 1.14)
No immunisation	1.20 (0.99 to 1.46)		1.38 (1.13 to 1.68)	**	0.65 (0.52 to 0.81)	***	0.73 (0.58 to 0.92)	**	0.66 (0.54 to 0.81)	***	0.66 (0.54 to 0.82)	***
Cancer screening
Mammography	0.92 (0.90 to 0.94)	****	0.96 (0.93 to 0.99)	**	0.98 (0.96 to 1.01)		1.01 (0.99 to 1.03)		0.96 (0.94 to 0.98)	***	0.98 (0.96 to 1.00)
Pap test (3 years)	0.94 (0.92 to 0.96)	****	1.00 (0.98 to 1.02)		1.00 (0.98 to 1.01)		1.02 (1.01 to 1.03)	**	0.98 (0.97 to 0.99)	**	1.01 (0.99 to 1.02)
Any colon CA screening	0.98 (0.96 to 1.00)	****	1.04 (1.02 to 1.06)	****	0.99 (0.98 to 1.01)	*	1.02 (1.00 to 1.03)	*	1.00 (0.99 to 1.02)		1.03 (1.02 to 1.05)	****
Hospital readmissions
30 days	0.91 (0.79 to 1.03)		0.96 (0.84 to 1.09)		0.91 (0.82 to 1.01)		0.94 (0.84 to 1.05)		1.00 (0.92 to 1.10)		1.03 (0.94 to 1.13)
1 year	0.84 (0.78 to 0.91)	****	0.91 (0.84 to 0.98)	*	0.91 (0.85 to 0.96)	**	0.98 (0.93 to 1.05)		0.93 (0.88 to 0.99)	*	0.98 (0.93 to 1.04)

Adjusted for age group, sex, IMG status, urban–rural status, number of years in practice, whether the physician was in a patient enrolment model, HDI group, median patient age and per cent of patients in low-income neighbourhoods.

Sig, significance level. *p<0.05, **p<0.01, ***p<0.001, ****p<0.0001.

ACE/AARB, ACE inhibitors/angiotensin II receptor blockers; ALPs, alternatively licenced physicians; CL, confidence limit; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; CA, cancer; dx, diagnosis; ED, emergency department; HbA1c, haemoglobin A1c (glycated haemoglobin); HDI, Human Development Index; IMGs, international medical graduates; Pap test, Papanicolau test; TLPs, traditionally licenced physicians.

Our multivariable analysis modelled the relationship between physician licensure cohort (ALP group or TLP) and clinical practice outcomes. Before carrying out the modelling, we tested the outcome measures for normality and found that many, such as the proportion of a physician’s patients with diabetes who received an eye exam within the previous year, were not normally distributed but became so after being log transformed. Based on this, we chose to use proc genmod in SAS to model the number with each characteristic (rather than the proportion) based on the Poisson distribution and including a log offset. Exponentiating the resulting parameter estimate gave us the relative rate for each outcome.

Each outcome was modelled individually. Covariates were entered into each model in a stepwise fashion, with only the significant variables retained in the final model. These included grouped age, sex, number of years in practice, urban–rural status, IMG status, whether the physician was in a patient enrolment model, HDI group, the proportion of their patients who were low income and the median age of their patients. The relative rates estimated by the models indicate the difference in outcome between each ALP group and the TLPs (reference group). All analyses were conducted using SAS V.9.3.

Patient and public involvement

There was no patient or public involvement in this study.

Results

Demographic and practice characteristics

A total of 292 ALPs and 11 127 TLPs were included in the study (table 2). The largest group of ALPs were the Canadian trained (n=114), followed by the US trained (n=91) and the out of province (n=78). The majority of TLPs were men (56.6%) and were older (50.6 years) than all three groups of ALPs. TLPs had fewer IMGs (22.2%) and overwhelmingly came from countries with very high HDI (90.5%). All ALPs were slightly more likely than TLPs to be in comprehensive practice and were less likely to be working in an FHT. Patient age and income distributions were similar across all groups.

The ALP groups’ average ages ranged from 42.1 years to 50.6 years. The out-of-province ALPs had the highest proportion of men (69%) compared with the other ALP groups. In the out-of-province ALP group, the majority were IMGs (89.7%) and completed medical school in countries considered medium/low on the HDI (63.2%). Seventy per cent (70.1%) practised in urban environments, and they had the largest proportion in solo practice (32.2%). Similar to out of province, the US-trained ALPs were mostly IMGs (85.7%); however, they graduated primarily from medical schools from countries with a very high/high HDI (68.1%). They had the largest proportion practising in non-FHT groups (65.9%) and were the most urban group (78%). Contrary to the other ALPs, almost half (47.4%) of the Canadian-trained ALPs were non-IMGs and 72.9% came from countries with very high/high HDI. Seventy per cent were in comprehensive practice, and they had the lowest percentage in solo practice. They also had the lowest proportion practising in urban areas compared with all other groups (67.5%).

Primary care quality indicators

Table 3 shows the results of the unadjusted and adjusted comparisons between each ALP group and the TLPs (unadjusted mean numbers are included in online supplementary appendix C). Each ALP group had a unique profile of primary care quality indicators. Patients of the out-of-province ALPs had the most substantial statistically significant differences in the quality care indicators compared with patients of TLPs after multivariable adjustments. These family physicians’ patients with diabetes were 4% less likely to have received HbA1c testing, and their patients with COPD were 18% less likely to have received spirometry testing. Their patients with CHF, COPD or asthma were 7% more likely to visit an ED for any reason (ie, all-cause) than those of TLPs. Additionally, their female patients aged 50–69 years were 4% less likely to have received a mammogram in previous 2 years and their paediatric patients had 14% fewer well-baby visits, were 24% less likely to have had an 18-month enhanced well-baby visit and were 38% more likely to have received no immunisations. However, their patients were 3% more likely to have received spirometry testing, 4% more likely to receive colon cancer screening and their hospitalised patients were 9% less likely to be readmitted in 1 year.

In contrast, US-trained ALPs were comparable with their TLP counterparts, with some statistically significant differences. Their diabetic care and cancer screening rates were similar, although US-trained ALP patients were 8% more likely to have received HbA1c and lipids testing than TLPs’ patients and 2% more likely to have received a pap test or any colon cancer screening. Their patients with CHF, COPD and asthma were also 3% less likely to visit the ED, and their paediatric patients were 27% less likely to have not received any immunisations; however, they were 7% less likely to receive well-baby visits. Canadian-trained ALPs were also similar to their TLP counterparts across most indicators; however, some statistically significant differences were seen: their paediatric patients were 3% less likely to have received a well-baby visit but were 34% less likely to have not received any childhood immunisations; their patients with COPD were 11% less likely to have had spirometry testing within 12 months of diagnosis; and their patients with CHF, COPD or asthma were 9% more likely to visit an ED (all-cause) than those of TLPs. Minor differences were seen also seen with HbA1c testing, spriometry testing for asthma patients, and colon cancer screening, with Canadian-trained ALP’s patients being 3% more likely to have received testing or screening.

Discussion

Our analysis of primary care quality indicators suggest that ALPs perform similarly to TLPs in many areas of primary care practice when controlling for a number of covariates. Small differences were seen across groups in indicators of diabetic care, CHF care, asthma care and cancer screening rates. Larger differences were found in preventive care for children less than 2 years of age and COPD management, particularly in patients of out-of-province ALPs. While individual family physician performance is contextual and influenced by many factors,6 35 health administrative data are useful for gaining a system-level impression of family physicians’ quality of care and broadly identifying areas that may need improvement.36 Overall, our findings suggest that alternative licensure route is not a strong independent predictor of family medicine performance on the majority quality indicators examined. For a small number of newly licenced family physicians, education pertaining to Ontario-specific guidelines and expectations may be of benefit.

Out-of-province ALPs

Compared with other subgroups, the primary care performance of out-of-province ALPs was the most different from TLPs after adjustments. Most notably, their patients less than 2 years of age were significantly less likely to receive well baby visits, enhanced 18-month assessments or immunisations, highlighting a trend in preventive paediatric care. These differences may reflect provincial differences in guidelines and schedules for paediatric care. For example, there is significant variation in how 18-month assessments are approached globally and across Canada.37 Ontario has supported a longer and more comprehensive enhanced 18-month assessment by providing financial incentives through a unique billing code.37–39 It is possible that out-of-province ALPs were unaware of Ontario’s enhanced 18-month assessment or of the paediatric care expectations of family physicians in the province. It is also possible that these physicians provided 18-month assessments but did not bill for it using the Ontario-specific code. Previous research has shown that male IMGs who have been in practice for over 10 years are less likely to provide 18-month assessments in Ontario.40 In this study, age, gender and HDI were controlled for, suggesting these factors are not accountable for the differences, thus entering Ontario from another province through an alternative route appears to be an independent risk factor.

Similar to the differences seen in 18-month assessments, the lower childhood immunisation rates in the out-of-province ALPs may be in part due to interprovincial variation in policies. For example, childhood vaccine schedules differ across provinces,41 which may have implications for how physicians bill. Furthermore, in Ontario, immunisations for children under 2 years of age are predominantly done in physician offices,42 while they may be administered by nurses or other allied health professionals in other provinces. Thus, the norms and conventions from their prior jurisdictions may be reflected in the billing practices of these ALPs once in Ontario.

In addition to differences in preventive paediatric care, out-of-province ALPs differed from TLPs in rates of spirometry testing for patients with COPD, which is recommended to confirm a diagnosis of COPD.43–46 Previous research has found that spirometry test ordering among family physicians in Ontario is generally low,47 and our findings suggest that it is even lower among out-of-province ALPs (and Canadian-trained ALPs) compared with TLPs, highlighting potential provincial differences in utilisation. Out-of-province ALPs’ patients with CHF, COPD or asthma also had higher rates of all-cause ED visits, but their hospitalised patients were 9% less likely to have been readmitted within 1 year. Rates of all-cause ED visits and readmissions are sometimes associated with access to primary care48–51 but can be influenced by many factors51–54 and should thus be interpreted cautiously.

Overall, our findings highlight that ALPs entering Ontario from another Canadian province perform differently than TLPs in certain indicators of primary care. However, the performance differences noted in this study may be due to provincial differences in care expectations and reflect the context of their recent work environments. Such provincial differences in how physicians provide primary care have implications for the migration of physicians across provinces since physicians can typically practice anywhere in Canada once licenced. Given that provisional licences have been seen as a way for physicians to gain entry to larger provinces through smaller ones,17 the Federation of Medical Regulatory Authorities of Canada has begun to standardise provincial licensure requirements, and the Medical Council of Canada is facilitating a common approach to practice ready assessments for IMGs across the country. While these efforts will help to mitigate potential performance differences in Canadian physicians, our findings suggest that ALPs entering Ontario from another province may still benefit from information about Ontario care expectations at the time of licensure. Focused knowledge translation for family physicians migrating across provinces may help to educate physicians about province-specific expectations and support their adoption of provincially supported programmes and guidelines, reducing the potential for future performance differences.

US-trained and Canadian-trained ALPs

US-trained and Canadian-trained ALPs performed similarly to TLPs on most primary care quality indicators after adjustments, though some notable differences were seen. For both groups, patients less than 2 years of age were less likely to receive well-baby visits but more likely to receive immunisations. In contrast to the out-of-province ALPs, whose patients were more likely to have not received any early childhood immunisations, US-trained and Canadian-trained ALP patients were much more likely to receive them compared with the rest of the province: this 27%–34% difference was the largest difference seen between these groups and the TLPs. US-trained ALPs’ patients were also more likely to receive pap tests and colon cancer screenings, their patients with diabetes were more likely to receive HbA1c and lipid testing, and their patients with CHF, COPD or asthma were less likely to visit an ED. The higher rates of testing and screening may be reflective of their American training, as previous research has found American physicians tend to have lower thresholds for diagnostic and therapeutic interventions.55 Similar to out-of-province ALPs, Canadian-trained ALPs’ patients with COPD were less likely to receive spirometry testing, and their patients with CHF, COPD or asthma were more likely to visit an ED. Overall, the performance of both of these ALP groups was comparable with TLPs. This is perhaps unsurprising given that these physicians have similar postgraduate training and that postgraduate training has been found to be predictive of patient outcomes.56 Our findings provide evidence that their practice performance is in fact similar to TLPs in Ontario.

Conclusions

Our findings illustrate that ALPs perform similarly to TLPs across many indicators of primary care, suggesting that route of licensure is not a strong predictor of family physician performance in Ontario. These findings provide support for alternative licensure policies and also demonstrate the utility of health administrative data for examining physician performance and evaluating regulatory processes. As transparency and accountability are increasingly emphasised in healthcare,57 and as physician migration and the use of alternative licensure routes continues to increase,15 it is imperative that processes for licencing and monitoring physicians are rigourously evaluated. The ongoing assessment of physician performance is critical for understanding the effects of medical regulatory policies and, ultimately, for ensuring high-quality patient care.

Strengths and limitations

This is the first study to examine the primary care performance of alternatively and traditionally licenced family physicians in Ontario. Our use of population-level data across multiple indicators of primary care allowed for a robust and comprehensive comparison of ALPs and TLPs, and our use of multivariable analysis enabled statistical adjustment of physician demographics, practice environments and patient factors, such as socioeconomic status, that are associated with primary care performance. While this approach contributes to our understanding of ALP performance, it is not without limitations. First, ALPs and TLPs were compared with each other, not to a gold standard. As such, our findings do not indicate whether physicians are meeting performance benchmarks, but rather whether ALP performance is comparable with TLP performance. Second, our results are based on 1 year of health administrative data that depict a point in time and also only represents elements of care that are funded by the Ontario Ministry of Health and Long-Term Care. Other important aspects of primary care such as the doctor–patient relationship or interprofessional collaboration with other primary healthcare providers are thus not accounted for. Last, billing data introduces unique interpretation challenges as these quality indicators are proxies for delivery of care; therefore, some of the variance in the indicators may be, in part, attributable to differences in billing practices or factors unrelated to the physician, such as patient preference.

Implications for future research

This study offers insight into the primary care performance of alternatively licenced physicians. Primary care is an important area of study given that approximately half of physicians in Ontario specialise in family medicine15; however, future research is needed to examine the practice performance of ALPs practising in other specialties. Performance is also multifaceted and must be studied using a variety of measures. Future studies could include other measures of performance, such as practice assessments or complaints profiles, to gain a comprehensive picture of ALPs’ practices.

This study also demonstrates that licensure route is a useful way of stratifying and comparing physicians. IMG studies typically define physicians based on their country of undergraduate medical school, whereas licensure route accounts for the influence of postgraduate training and previous practice experience on performance. Examining the impact of all of a physician’s training and experience on future practice performance allows for a more robust understanding of the predictors of performance and may enable more nuanced IMG research in the future.

Finally, this study represents a collaboration between a medical regulator and system partners. Such collaborations are important for linking performance data across the continuum of medical education and practice58 59 and for providing evaluative evidence for regulatory processes and policies, such as alternative licensure routes.12 14 Further collaborations of this nature will allow for robust examinations of the influence of each stage of a physician’s training on future practice performance.