Longitudinal costs and health service utilisation associated with primary care reforms in Ontario: a retrospective cohort study protocol.

INTRODUCTION: Over the last 20 years, the Canadian province of Ontario implemented several new models of primary care focusing on changes to physician remuneration, clinics led by nurse practitioners and the introduction of interprofessional primary care teams. Health outcome and cost evaluations of these models thus far have been mostly cross-sectional and in some cases results from these studies were conflicting. The aim of this population-based study is to investigate short, medium and long-term effectiveness of these reforms over the past 15-20 years. METHODS AND ANALYSIS: This is the protocol for a retrospective cohort study including fee-for-service (FFS) and community health centre cohorts (control cohorts) or patients who switched from either being unattached or from FFS to a new practice model (eg, capitation, enhanced FFS, team, nurse practitioner-led) from 1997 to 2020. The primary outcome is total healthcare costs and secondary outcomes are primary care costs, other (non-primary care) health costs, hospitalisations, length of stay, emergency department visits, accessibility and mortality. A combination of hard and propensity matching will be used where relevant. Outcomes will be adjusted for demographic and health factors and measured annually. Interrupted time series models will be used where data permits and difference-in-differences methods will be used otherwise. ETHICS AND DISSEMINATION: Ethics approval has been received from Queens University and Memorial University. The dissemination plan includes conference presentations, papers, brief evidence summaries targeted at select audiences and knowledge brokering sessions with key stakeholders. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is a longitudinal, population-based study of Ontario (the most populous Canadian province) residents using a comprehensive healthcare costing methodology that accounts for virtually all health costs expended at the individual patient level.

We will examine primary care reforms including the introduction of patient rostering, capitation-based physician payments, enhanced fee-for-service payments, interdisciplinary primary care teams and nurse practitioner-led clinics.

We will use comprehensive hard and propensity matching to mitigate selection bias, and will be the first Canadian study to compare nurse practitioner-led and physician-led clinics.

In addition to costs, we will also examine accessibility, emergency department utilisation, hospitalisation and mortality.

The study will primarily use administrative data which was not originally designed for research, and are therefore subject to measurement errors, and may limit our ability to fully adjust our analyses for all relevant covariates.

Introduction

Evidence suggests that individuals with better access to primary care have better health outcomes, lower healthcare costs and reductions in both emergency department (ED) visits and hospitalisations.1–7 Reorganisation of primary care to improve access and quality has therefore become a focus of international policy implementation in recent years, and all Canadian provinces are exploring reforms to varying degrees. In Canada, healthcare is primarily a responsibility of provincial governments and there is some variability in the way services are organised between provinces. Over the last 20 years, The province of Ontario implemented the greatest number of new models of care, primarily focused on changing the way physicians are remunerated and the introduction of nurse practitioner-led (NP-led) clinics and interprofessional teams. These interprofessional team models closely mirror the principles of the Patient Centred Medical Home (PCMH),8 and the extent of introduction has led some to refer to this as the largest example of PCMH implementation in North America.8 9

Primary care reform in Ontario included three major policy initiatives: new physician reimbursement or organisational models, patient enrolment with primary care providers, and support for interprofessional team-based care.10 During the last two decades, more than one-third of primary care physicians have voluntarily transitioned from traditional fee-for-service (FFS) practice to blended capitation payment and in some cases received additional funding to support interprofessional team members. Between 2006 and 2009, the cost of primary care in Ontario more than doubled, in part because of the introduction of these more costly models of care.11 Despite this, there is limited evidence showing whether these reforms have translated into improvements in health outcomes or cost savings in other sectors. Current analyses of primary care reforms in Ontario are mostly cross-sectional and have therefore been limited in their ability to distinguish whether differences in outcomes are associated with the reforms themselves or to confounding associated with dramatic differences between the patients cared for under those models of care. Between the models with the lowest and highest prevalence, there is a 4-fold difference in serious mental illness, 2.5-fold difference in chronic obstructive pulmonary disease, 1.8-fold difference in diabetes and 5.1-fold difference in receipt of social assistance.12 Longitudinal methods are better able to account for these group differences and attribute outcomes to the reforms themselves. While some longitudinal analyses of Ontario reforms have been published,13–18 no studies to date have examined costs or mortality as we propose to do.

No Canadian studies have comprehensively compared the costs and health service utilisation of patients cared for within NP and physician-led or community-governed interdisciplinary primary care teams. Given that a diverse array of reforms to primary care in other provinces are either underway or being considered, there is an urgent need to study the effectiveness of existing models to help inform this policy change. This project will leverage the massive natural primary care reform experiment in Ontario to examine the effectiveness of the reforms implemented there, information that will be valuable to inform policy decisions and programme planning in other provinces.

Objectives

The primary objective of this project is to assess the short, medium and long-term effectiveness of interprofessional teams and other primary care reforms implemented in Ontario over the past 15–20 years. We define effectiveness through four outcomes: (1) healthcare costs (and cost categories); (2) ED utilisation; (3) hospital admissions; (4) accessibility and (5) mortality.

Our specific objectives are as follows:

To estimate the effects of changes to a patient’s primary care including:

A change from traditional FFS to another practice model.

A change from being unattached to being enrolled with or attached to a consistent clinician.

A change from a non-team to an interdisciplinary team-based practice model.

These analyses will be stratified to determine the effectiveness of reforms on more complex patients, including socially marginalised patients and those with greater comorbidity.

Finally, we will assess the broader impact on the health system by examining the association between changes to primary care and outcomes over time in a population-based cohort.

Methods

Patient and public involvement statement

This study addresses a priority that was identified by the Primary and Integrated Healthcare Innovations Network19 which includes a large number of patients. We also consulted and received support from the Patient Council of the Newfoundland and Labrador Support for People and Patient-Oriented Research and Trials Unit, and the Primary Healthcare Research and Integration to Improve Health System Efficiency Network Steering Committee (including patients) at the funding application and project design stages. Feedback from these groups was used to add an additional objective to this study—to determine the change in accessibility to care associated with different reforms. Once we have completed our initial data analyses, we will present to each of these groups again to get their input on follow-up data analyses, interpretation and presentation.

Study population and practice models

This is a retrospective cohort study involving longitudinal record linkage of Ontario residents and their family physicians/NP. The study period of interest for this project is from 2001 (1–2 years before most new primary care models were introduced in Ontario) to 2020. Unfortunately, costing methods have only been developed for 2001 forward but we will access data for other outcomes for several years prior to 2001 in order to establish a preintervention trend for use in time-series models.

Using the Registered Persons Database, a separate dataset of patients meeting the following criteria will be created for each year of the study period:

Were alive and age 18 or older on 1 January of that year.

Were Ontario residents.

Were eligible for coverage by the Ontario Health Insurance Programme (OHIP).

Several models of primary care in Ontario have been introduced over the last number of years, some of which were only in place for a short time before being rolled into other models and others of which are only available to a limited segment of the population (eg, Aboriginal Access Centres, Rural Northern Physician Group Agreement). We will study the outcomes associated with models that were broadly implemented in diverse settings as outlined in table 1. These models are more completely described elsewhere.10

Table 1

Models of Ontario primary care included in this study

Model names	FFS	CHC	eFFS		Capitation		NP
			CCM	FHG	FHN, FHO	FHT
First introduced		1979	2005	2003	2002	2005	2007
Patients (1000s)*	224	60	50†	2300	1066	1162	40†
Most responsible	MD	MD	MD	MD	MD	MD	NP
Interprofessional		✓				✓	✓
Patient enrolment		✓	✓	Optional	✓	✓	✓
Remuneration	FFS	Salary	eFFS	eFFS	Capitation	Capitation	Salary
Bonuses			✓	✓	✓	✓
After hours		✓	Optional	✓	✓	✓	✓
Minimum # MDs	1	1	1	3	3	3	n/a

*Most recent data available are 2011/2012.16

†Estimate.

CCM, comprehensive care model; CHC, community health centre; eFFS, enhanced FFS; FFS, fee-for-service; FHG, family health group; FHN, family health network; FHO, family health organisation; FHT, family health team; MD, medical doctor; n/a, not available; NP, nurse practitioner.

Data linkage

Patient demographic data will be linked to healthcare visit records from various sources including OHIP billings and both Community Health Centre (CHC) and NP-Led Clinic electronic visit records. Records will also be linked to clinician descriptors included in several different Institute for Clinical Evaluative Sciences (ICES) holdings including practice model, sex, age and year of graduation from clinical studies. Finally, hospitalisation and emergency visits will be linked to our database to assess outcomes and to include diagnoses in an adjustment for comorbidities.

Most responsible provider and practice models

Patients will be assigned to a most responsible primary care provider for each year of the study period. Several of the primary care reform models in Ontario require that patients enrol with their family physician or NP. In this protocol, we will use the term ‘enrolled’ to refer to patients that have signed such an agreement with their provider, and ‘assigned’ to refer to all methods of attributing patients to a particular provider. Enrolled patients will usually be assigned to the physician with whom they are enrolled, except that patients who see a different primary care provider for greater than 60% of their core primary care visits will be assigned to that provider.20 Patients who were not enrolled in a given year will be assigned either to a NP or to a primary care physician who is the most frequent provider of core primary care visits during the index year and year prior. Only providers with a specialty code of general practice/family medicine, NP or community health will be eligible to have patients assigned to them, excluding providers with a focused practice as previously defined.20 Patients will be assigned the practice-type of their most responsible primary care provider. Physicians practicing within enrollment models in Ontario are also able to bill FFS for patients that are not enrolled with them. Anecdotally, these patients are considerably different than enrolled patients and we will therefore consider these groups separately.

Patients who are not assigned to any provider in a given year (ie, they have no healthcare visits) but who are assigned to the same provider at any point before and after will be assigned to that provider for the missing year(s) as well, as long as they maintained their eligibility for OHIP coverage during that time. For patients and/or physicians who changed practice models in a given year, the model that was in place for the greatest length of time that year will be assigned. When transition occurred half way through the year, the model from the beginning of the year will be assigned. The year of practice transition will be excluded from most statistical analyses.

Outcomes

The primary outcome for most substudies will be total individual-level, inflation-adjusted health costs including the following services: inpatient hospitalisations, physician services (primary care and specialist), ED, rehabilitation, home care, long-term care (nursing home), surgical daycare, assistive devices, medications, mental health services, laboratory, and diagnostic imaging as calculated previously.21 Secondary outcomes will include primary healthcare costs, cost categories listed above, hospitalisations, total length of acute-care hospital stay as well as hospitalisations and length of stay for ambulatory care sensitive conditions, ED visits, low acuity ED visits, accessibility and mortality. Given the limitations to the administrative data we are using, we will be constrained to estimating accessibility by measuring the number of patients cared for under different models and the change in those numbers over time.

Covariates

Johns Hopkins Adjusted Clinical Groups (ACG) software will be used to adjust for health status and comorbidity.22 Additionally, postal codes of each individual from our databases will be mapped to census dissemination areas using the Postal Code Conversion File which will allow us to adjust for small area census variables such as per cent aboriginal,23 percent visible minority and several other socioeconomic indicators, marginalisation24 and a comprehensive indicator of rurality, the Rurality Index of Ontario.25 The accessibility of primary care physicians is measured in several ways in a recent report,26 and we will adjust for this factor using the marker of access which has the strongest predictive ability in our models (lowest Akaike information criterion). Hospitalisation models will include an adjustment for acute care beds per capita in the patient’s census subdivision. Finally, analyses will be adjusted for the most commonly used indicator of continuity of care and characteristics of the primary care provider such as sex, age and years in practice.27 We will complete additional analyses to determine the degree to which the outcomes associated with a given practice type are attributable to the core features of the model (payment model, group practice, multidisciplinary teams) or to aspects of care that vary with the model (eg, continuity, volume of patients).

Creation of cohorts

Separate analyses (substudies) will be completed to address each of the study objectives identified above, and each of the substudies will involve the comparison of several different cohorts. Individuals will be eligible for membership in at most one cohort for the same sub-study, but they may belong to multiple cohorts across different substudies. Cohorts will be defined to address study objectives 1–3 as described above, but analyses will be carried out on the entire study population to address objective 4.

Objective 1 (a): a change from traditional FFS to another practice model

For this objective, we will select all individuals who were community-living, 18 years or older, and had the same family physician assigned for at least 6 years of the study period. For patients assigned to physicians who changed practice models, we will only include those whose physician remained the same between the year prior and 6 years after the change. For the patients assigned to the consistent FFS or CHC groups, we will require that they be assigned to the same physician from 1 year before to 5 years after the median model switch year for the changed groups. Individuals will be divided into specific groups for comparison (3–7 below) plus two control groups (1 and 2 below). Recent ICES reports suggest that FFS continues to be an important source of income for primary care physicians28; however, anecdotal accounts suggest that the bulk of these physicians have a focused practice (eg, psychotherapy, surgical assisting) and do not provide comprehensive primary care, were providing short term practice coverage (ie, locums), or were approaching retirement.20 Because our interest is comprehensive primary care, our analyses will exclude physicians who have a focused practice or who changed practice model after 2015,20 potentially making comparisons with the FFS control group underpowered.

Objective 1 (a) cohorts:

Consistent FFS (Main control group).

Consistent CHC.

Switch from FFS to enhanced FFS.

Switch from FFS to capitation.

Switch from FFS to capitation with interdisciplinary teams.

Switch from FFS to enhanced FFS to capitation.

Switch from FFS to capitation to capitation with teams.

Objective 1 (b): a change from being unattached to enrolled or associated with a clinician

The cohorts to address this objective will be limited to patients who were previously unattached to a provider. NP-led clinics are interprofessional team-based models introduced in 2007 and access was primarily limited to patients who did not previously have a primary care clinician. Objective 1(b) was included in this project both to measure the effect of introducing a consistent source of care, as well as to have a similar comparison group for the NP-led clinic cohort. Included individuals must have had a 3-year period without a consistent FFS provider followed by a minimum of 5 years assigned to a single primary care model. The control for this substudy will include patients not enrolled in a model and without a consistent FFS provider for the entire study period.

Objective 1 (b) cohorts:

Patients newly assigned to NP-led interprofessional teams.

Patients newly assigned to physician-led interprofessional teams.

Persistent unattached patients (patients with no clinician or a different clinician assigned each year).

Objective 1 (c): a change from a non-team to an interdisciplinary team-based practice model

Patients for this substudy will be limited to those with a consistent family physician from the year before until 5 years after the relevant practice change. Because all interprofessional team-based physicians in Ontario are paid by capitation, the relevant control group includes non-team-based physicians paid by capitation.

Objective 1 (c) cohorts:

Switch from FFS to capitation.

Switch from FFS to capitation plus teams.

Switch from FFS to capitation, then switch to capitation plus teams at a later date.

Objective 2: stratified analyses

All of the analyses described above will be stratified to determine the effectiveness of reforms on more complex patients, including socially marginalised patients using the Ontario Marginalisation Index (ON-Marg) and those with greater comorbidity using the ACG comorbidity score.22 24 ON-Marg is an area-based index that measures marginalisation across four dimensions: residential instability, material deprivation, dependency and ethnic concentration using census data. The index was derived from many different census variables using factor analysis, and a separate index was also developed for each dimension. Studies have shown that ON-Marg is associated with health outcomes such as depression, youth smoking, injuries and infant birth weight.24

Objective 3: association between changes to primary care and outcomes over time

The analyses described above will likely exclude a substantial proportion of the population in order to estimate the effect of individual reforms or the effects of reforms on specific populations as cleanly as possible. To estimate the effects of Ontario primary care reform on the health system, substudy 3 will include virtually all community-dwelling adult residents of Ontario with a valid health card at any point between 2003 and 2018. Residents will be assigned to a primary care model and individual health costs will be determined as described previously for each year of the study period. Models will be compared using the methods described below except that patient matching will not be used.

Cohort matching

In order to control for potential confounders, we will match patients in each cohort to patients in the appropriate control cohort initially with hard matching for age group (±2.5 years), sex and health cost category in the year prior to practice change. After hard matching, we will match using a propensity score that includes ACG comorbidity score,22 marginalisation index,24 the rurality index for Ontario25 and an indicator for new health card holder within 10 years (a surrogate for new immigrants). We will explore different types of propensity score matching but will likely use nearest neighbour matching without replacement because most groups are sufficiently large that replacement will not be required.29 Where the comparator (reform) groups are sufficiently large, we will match each control case to four randomly selected reform patients to maximise power, but will otherwise match one to one.

Statistical analyses

Cost and service utilisation outcomes at each time-point will be compared using generalised linear models with generalised estimating equations, and longitudinal data will be compared using generalised linear mixed models. All models will control for covariates mentioned above and adjust for clustering within primary care providers. Because health costs increase over time and year of model introduction will vary with each clinic, we will also capture year as a covariate in our analyses. Mortality will be analysed using Cox proportional hazards models allowing time-varying characteristics as covariates.

For the cohorts described above (objectives 1–3), difference-in-difference methods will be used to compare the change in outcomes between the year before introduction and: (1) the year after introduction (short term), (2) Year 2 after introduction, (3) Year 5 after introduction (medium term) and (4) Year 10 after introduction, where possible.30 All costs will be adjusted for inflation and reported in 2001 dollars. ICES costing methods have only been validated for 2001 onwards and many of the PHC models were introduced shortly after that. For the early adopting physicians, there will therefore not be sufficient time to establish a preintervention trend necessary for interrupted time series methods, but we will consider using these methods for later adopters.

We will assess the predictive validity of regression models using k-fold cross-validation with 10 samples. In this method, the full sample is randomly divided into 10 subsamples of equal size. One of the 10 subsamples is removed from the dataset, and the models are trained with the remaining nine subsamples, then validated on the excluded subsample. This process is repeated with each of the remaining subsamples, and the parameters from each of the 10 models are averaged to obtain more accurate estimates. Parameters from the model with the smallest validation average squared error will be presented. This method is more statistically efficient than split sample validation techniques.31

Data presentation

Most outcome data will be presented on a line graph with outcome on the vertical axis and year on the horizontal. In substudies with more than two cohorts, outcomes will be compared with a single reference group. Where other group comparisons are deemed to be of interest, we will run different models to present the significance of those comparisons.

Power

The Canadian Institute for Health Information estimates that individual health costs increased by about US$40 per year,32 ED visits from 360 to 380 visits per 1000 patients per year, and hospitalisation days increased from about 579 to 614 days per 1000 patients per year between 2000 and 2012.33 Even with conservative estimates of variance and intra-cluster correlation in the smallest group comparison (NP led and FFS models), we estimate power of greater than 80% to detect a difference in change in costs over 5 years of US$80 per patient, ED visits of 40/1000 patients, and hospitalisation days of 40/1000 patients. Our stratified (by comorbidity and marginalisation) analyses should be adequately powered to assess costs, but analyses of ED visits and hospitalisation days may not be possible because of the smaller sample sizes in the strata.

Ethics and dissemination

This research was approved by the Health Research Ethics Board in Newfoundland and Labrador and the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board in Ontario. The research team has also received approval from the ICES and the Ontario Alliance for Healthier Communities. We are currently negotiating with NP-led clinics to link and analyse their data.

Our dissemination plan will evolve over the course of the grant but at minimum will include: four papers for each of the sub-studies, presentations to groups such as the provincial Canadian Institutes of Health Research (CIHR) Primary and Integrated Healthcare Innovations network, provincial health funders, medical associations, advocacy groups and the College of Family Physicians of Canada. Study findings will also be disseminated through social media platforms of the Primary Healthcare Research Unit (PHRU) as well as in PHRU’s highly regarded series of ‘Research Snapshots’ to summarise the evidence in an accessible and visually engaging format that is tailored specifically for different audiences.