Regional variation in primary care improvement strategies and policy: case studies that consider qualitative contextual data for performance measurement in three Canadian provinces.

OBJECTIVE: To explore regional primary care improvement strategies that are potentially determinants of primary care performance.
DESIGN: Multiple comparative embedded case study.
SETTING: Three regions in Canada: Fraser East, British Columbia; Eastern Ontario Health Unit, Ontario; Central Zone, Nova Scotia. DATA SOURCES: (1) In-depth interviews with purposively selected key informants (eg, primary care decision-makers, physician leads, regulatory agencies) and focus groups with patients and clinicians (n=68 participants) and (2) published and grey literature (n=205 documents). OUTCOME MEASURES: Variations in spread and uptake of primary care improvement strategies across the three study regions. NVivo (V.11) was used to manage data and perform content analysis to identify categories within and across cases. The coding structure was developed by researchers through iterative collaboration, using inductive and deductive processes.
RESULTS: Six overarching primary care improvement strategies, differing in focus and spread, were implemented across the three study regions: interprofessional team-based approaches, provider skill mix expansion, physician groups and networks, information systems, remuneration and performance measurement and reporting infrastructure.
CONCLUSION: The addition of information on regional improvement strategies to primary care performance reports could add important contextual insights into primary care performance results. This could help identify possible drivers of reported performance outcomes and levers for change in practice, regional and system-level settings. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Cross provincial case study using multiple sources of data including documents, interviews and focus groups.

Interdisciplinary, international research team using intensive electronic and face-to-face discussions focussed on analysis and synthesis of findings.

Limited regional level documents meant a reliance on provincial level documents.

Small numbers of interview and focus group participants and differences in the positions of stakeholders interviewed across study regions.

Using multiple data sources enabled triangulation of findings.

Introduction

There is a widespread call for rigorous and systematic performance measurement to guide primary care (PC) improvement.1 2 When governance of health is decentralised, and reforms are a state, provincial or territorial responsibility, considerable within-country diversity exists. This is the case in Canada, where structural and policy differences across all 10 provinces and 3 territories, each with their own independent healthcare system, constrain the development of a comprehensive national PC performance measurement system which could be used to gain insight into determinants of variations in patient and provider experience and outcomes. For the past 20 years unprecedented federal and provincial/territorial investments in PC reforms have resulted in a mosaic of improvement strategies, sometimes referred to as reforms.3–5 In Canada, systematic purposeful evaluation of PC improvement strategies has not occurred.6 Assessing and reporting on regional variations in improvement strategies and their impact on performance is important so decision-makers can compare and learn from similar systems to identify which investments to maintain, expand or discard.

However, knowledge of regional variations is meaningless in the absence of an understanding of the context in which health system results are achieved. There is a dearth of information reporting on how different PC structures or policies might be associated with differential PC performance. Contextual information, such as population-level demographics7 or practice-level organisational information,8 can add depth to understanding quantitative performance data. For example, in the UK, national reports of patients’ experience of PC are adjusted for with important case-mix variables.9 Although health system context at the level of regional management areas within provinces and territories is identified as important in several performance measurement frameworks,10–13 few PC performance measurement studies collect contextual data at the regional level.14

This paper reports on the case study component of the TRANSFORMATION Study, a large, three province, 5 year programme of research (http://www.transformationphc.ca/about/). TRANSFORMATION seeks to advance the mechanisms for regional-level PC performance measurement and reporting. The TRANSFORMATION team includes interdisciplinary researchers, decision-makers and clinicians from each province participating in the study, as well as an International Scientific Advisory Committee and Regional Stakeholder Committees, which included patient representatives. In this paper, we describe the spread and uptake of PC improvement strategies in comparable regions in three provinces to address the research question, how can the context of regional PC structures and policies explain differences in PC system performance across regions. This contextual information is being utilised to inform TRANSFORMATION’s approach to comprehensive regional PC performance measurement and reporting.

Methods

Our study used a multiple comparative embedded case study design.15 The case study settings were three regions in Canada: Fraser East, British Columbia (BC); Eastern Ontario Health Unit, Ontario (ON) and Central Zone, Nova Scotia (NS). These provinces were chosen for their varied approaches to PC reform.3 16 Provincial and territorial governments are responsible for their own healthcare systems and must adhere to a set of principles outlined in federal legislation, known as the Canada Health Act.4 PC is publicly funded and privately delivered, mainly by family physicians, a legacy dating back to the 1960s when family physicians and governments negotiated a payment plan for their services. In 2004, federal policy and funding incented PC reforms, which, in turn, have been shaped through negotiations between physician groups and provincial governments. Whereas ON used a model-based organisational approach with changes in physician remuneration, BC used a system integration approach establishing networks and division of family physicians.4 17 BC, and to some extent NS, also used a principle-based approach emphasising quality improvement. ON’s reforms have been the most far-reaching followed by BC and lastly NS. Another reason for their selection was because in 2013, the first year of the TRANSFORMATION Study, these health regions had similar socio-demographic profiles, for example, with respect to ethnicity, age and socioeconomic status.17 The embedded units of analysis were the improvements in PC service delivery and organisation in each region.18 Over the 5 years of the study, three full-team meetings enabled important input into key decisions about the case study methods.

Anticipating a large range of PC improvement strategies considered in the three regions, our multi-stakeholder group agreed that to identify the priority strategies for the case study we would conduct an initial review of Canadian papers that address PC reforms to identify improvement strategies with significant provincial financial investments and legislative or regulatory policy change.3 19–21 We sought those strategies which were most likely to be priorities across all three regions or within at least one. We identified six broad improvement strategies and considered these our embedded units for further exploration in the case studies: interprofessional team-based approaches, provider skill mix expansion, physician groups and networks, information systems, remuneration and performance measurement and reporting infrastructure (table 1).

Table 1

Working definitions for six areas of primary care improvement strategies

Interprofessional team-based approaches	Defined models of primary care teams involving at least two healthcare professionals (eg, family health teams, community health centres, nurse practitioner-led clinics)
Provider skill mix	Introduction of new providers (eg, nurse practitioners, physician assistants) or expansion of existing provider roles (eg, pharmacists prescribing certain medications; registered nurses) in primary care
Physician groups and networks	Organisation of physicians into groups or networks with formal governance and/or accountability mechanisms
Physician remuneration	Payment models and incentives paid to primary care physicians
Information technology	Information systems and data management tools used in primary care settings (eg, electronic medical records, telehealth)
Performance measurement and reporting infrastructure	Organisations and initiatives committed to primary care performance measurement and reporting at regional and/or provincial levels.

Patient and public involvement

Patient advisors participated in two initial face-to-face meetings of the full research team to plan implementation of the 5 year study following securement of grant funding. During these meetings, the research questions and methods for the case study were refined through group discussions that included patients. In addition, regional Stakeholder Advisory Groups that included patients and clinicians were convened annually in the first half of the project. Their role was to provide general advice and consultation including document and interview participant selection. A patient advocacy group partnered with the research team while the grant was being written but this partnership did not continue after the grant was funded.

Our full programme of study included day-long deliberative dialogue sessions with members of the public in each region during which they were provided with background material and an explanatory introduction from content experts on performance measurement and reporting methods. They were then asked to discuss how the public uses primary care performance information and what information is most useful. In the next phase of our case study work, we will seek further feedback from patients and the public on the design features of an actual draft regional primary care performance portrait.

Document review

We conducted a document review to find sources that described or evaluated PC improvement strategies in each region. We included peer-reviewed research articles and ‘grey literature’ documents (dissertations, policy descriptions or analyses, reports, regulations, acts, mission statements, funding agreements, strategic planning documents and logic models) all published between 2003, the year of the Accord on Healthcare Renewal which catalysed renewed investments in PC across the country,22 and 2014, the year in which documents were extracted. Research team members conducted the keyword search (British Columbia, Ontario, Nova Scotia, Fraser Health, Eastern Ontario, Capital Health, primary health care, primary healthcare, primary care, innovation, policy, model of care, teams, group practice, networks, patient enrollment, funding, financial incentives, payment models, governance, nurse practitioner, physician assistant, midwife, EMR, quality improvement, telehealth, web-based services), selecting documents based on a scan of the title and abstract (peer-reviewed literature) or a preliminary scan of the document (grey literature), using the following criteria: (1) published no earlier than 2003, (2) discussed a PC improvement strategy (table 1) and (3) improvement strategy implemented in all or most of the case regions (figure 1). As table 2 shows, the number of documents ranged from 95 in BC to 46 in NS. The study team developed a data extraction form to identify key information from each selected document (online supplementary file 1 appendix A). Team training for data extraction was provided and progress was discussed at monthly team meetings. Findings from the document review were discussed at team meetings and informed the guide for individual interviews and focus groups. We probed for missing and/or unclear information obtained from documents.

Figure 1

Document review flow diagram.

Table 2

Number and type of data sources in each case

	British Columbia	Ontario	Nova Scotia	Total
Data extraction forms from document review*	95	64	46	205
Key informant interviews	6	6	8	20
Clinician focus group (1/region) participants†	6	6†	15	27
Patient focus group (1/region) participants	5	7	5	17

*In documents where multiple study regions were discussed, the document was recorded as a data source of the study region where PC improvement strategy(s) were most fully discussed.

†In Ontario, individual semi-structured interviews were conducted with PC clinicians in lieu of a focus group.

PC, primary care.

Focus groups and interviews

Team members (RMM, STW, SJ) and trained research assistants conducted focus groups with patients and clinicians and individual interviews with key stakeholders in 2015 and 2016. The aim of the focus groups was to solicit clinicians’ and patients’ awareness of and experiences with the improvement strategies in their region. Both groups have unique perspectives and provided insights into the extent to which improvement strategies were being experienced on the ground.

Patients were recruited by email or telephone from a convenience sample of participants who filled out a patient experience survey in the waiting room of their practice23 and consented to be contacted for future related studies. We sought to recruit patient participants with different demographic characteristics and prioritised those with multiple chronic conditions, given these individuals would likely have more experience with the PC system. Clinicians (eg, family physicians, family practice nurses, nurse practitioners, pharmacists) were interviewed in focus groups in BC and NS. In ON, we conducted individual, semi-structured telephone interviews with PC clinicians in lieu of a clinician focus group because a mutually convenient time among clinicians could not be found. Focus groups for patients and clinicians were held at central meeting spaces and were approximately 1-1/2 to 2 hours duration (online supplementary file 2 appendix B).

Individual interviews were conducted with purposively selected key informant (eg, PC decision-makers in provincial health departments or health authorities or, heads of regulatory colleges or associations). The purpose of these interviews was to obtain current information about the improvement strategies that may not have been included in documents and to probe for missing and/or unclear information obtained from documents.

Stakeholders were recruited by email or telephone from a list of potential participants created with our Regional Advisory Stakeholder Committees and other regional contacts. We conducted the in-depth, semi-structured, hour-long interviews either in-person or over the telephone (online supplementary file 2 appendix B).

Focus group and stakeholder interviews were audio-recorded and transcribed, and field notes recorded. Across regions, the number of key informant participants ranged from six to eight, clinician participants from six to fifteen and patient participants from five to seven. Interview and focus group participants gave informed consent.

Researcher characteristics and reflexivity

Several of the Canadian researchers on our team are experienced PC researchers who are well known and have longstanding relationships with decision-makers and clinicians in their regions which facilitated recruitment for interviews and focus groups. Discussions among our international, interdisciplinary and multijurisdictional team enabled us to capitalise on this knowledge while maintaining a critical lens.

Data analysis

Data extraction forms (document review) and transcripts (focus groups and interviews) were imported into NVivo V.1124 for data management and to perform content analysis.

Researchers from each province with experience in qualitative methods jointly developed the coding structure using inductive and deductive processes.25 Deductive coding began with the document review and was informed by established frameworks that describe the structures of PC practice.10 26 Inductive coding of the interviews and focus groups occurred through organised monthly team discussions or more often if needed. The final coding structure was discussed and tested with members of the full research team. Two team members in each study region coded the data. Data were analysed within each region and across regions.15 The research team discussed coding and emerging categories at monthly telephone and three face-to-face meetings. We describe below the variations across study regions in spread and uptake in the PC improvement strategy areas using quotes from the interviews and focus groups to illustrate findings. Explaining the variability in how and why the provinces decided to tackle the six different PC improvement strategies is beyond the scope of this article.

Results

Table 2 describes the number and type of data source in each region. The document review provided information on the implementation of PC policies and improvement strategies, involvement of stakeholders and the results of policy evaluations. Interviews and focus groups confirmed the selected improvement strategies and provided insights on their spread, uptake and acceptability within the regions and illuminated examples. Participants were closely split between male and female and most were aged 40 and over. Most patient participants had two or more chronic conditions.

Table 3 outlines the number of data sources coded to the six improvement strategies. The most commonly discussed improvement strategies were interprofessional teams and provider skill mix. Further information on the specific PC improvement strategies identified under each broad area of improvement (eg, expanded scopes of practice as part of provider pool) is available on request from the corresponding author.

Table 3

Number of unique references coded to each primary care improvement strategy

Improvement strategy	Documents	Interviews	Focus groups
	Total N (n from BC, ON, NS)
Interprofessional team-based approaches	71 (16, 26, 29)	22 (6, 8, 8)	2 (0, 1, 1)
Provider skill mix	77 (36, 19, 22)	7 (2, 2, 3)	5 (2, 1, 2)
Physician groups and networks	39 (19, 15, 5)	14 (6, 4, 4)	3 (2, 0, 1)
Remuneration	40 (18, 17, 5)	13 (4, 2, 7)	2 (1, 0, 1)
Information technology	38 (15, 17, 6)	18 (6, 5, 7)	3 (1, 1, 1)
Performance measurement and reporting infrastructure	34 (17, 12, 5)	21 (6, 7, 8)	2 (1, 0, 1)

BC, British Columbia; NS, Nova Scotia; ON, Ontario.

Interprofessional team-based approaches

All three provinces have introduced interprofessional team-based approaches (table 4). ON has more formal and better resourced approaches to the implementation of interprofessional teams than both BC and NS.16 Teams in ON are government funded with model-specific defined mandates and accountabilities, and some also have requirements related to team structure and/or governance.6 Patients from Ontario reported appreciation for interprofessional team-based care:

Table 4

Examples of interprofessional team-based approaches in primary care settings by study region

British Columbia: Fraser East	Ontario: Eastern Ontario Health Unit (EOHU)	Nova Scotia: Central Zone
1 Community health centre in Fraser East / 29 in the province68	CHCs: 5–7 CHCs in EOHU / 74–87 in the province69 70 Family health teams were implemented in 2004.71 2 in EOHU / 184 in province72 Nurse practitioner-led clinic were first implemented in 200773 1 in EOHU / 27 in province74	CHCs: 2 in Central Zone / 25 in the province75 Collaborative emergency centres were first implemented in 2011.76 2 in Central Zone / 8 in province77

Note: The number of CHCs in each province registered with the Canadian Association of Community Health Centres is less than the number in the above references https://www.cachc.ca/ourmembers/?sort=3&dir=asc.

CHC, community health centre.

So I have to go at one place, you know, my needs are taken care of right there. Everybody knows everything. (ON patient)

BC and NS have introduced some interprofessional team-based models/approaches, although on a smaller scale, though usually without the defining mandates, deliverables and governance requirements of models seen in Ontario, and, in both provinces, most activity is in rural areas.

When we started looking around the province, there's collaborative practice teams all over the province, except in the densely populated areas. So we have a misaligned sort of distribution right now. (NS key informant)

As the following quote explains, this misalignment was attributable to a shortage of rural healthcare providers, and the more recent shift to including team-based care in urban areas is driven by chronic disease management goals.

So the system factors were initially probably dominated by problems with access to service in rural areas. So, much of the initial work was focusing on using alternate payment plans and structuring team-based care to make those settings more attractive to providers, and to have some deliverables for teams that were in turn supported to work in small rural communities. (…) I think more recently, the focus on chronic disease management has highlighted the importance of restructuring primary care, to do that more effectively. And most of the evidence there fits with the policy directions of moving to teams but also trying to structure those teams based on a population model that recognises community differences as well as including everybody in a geographic (health region). (NS key informant).

Similarly, speaking about the desire to move forward with an interprofessional team-based approach, a BC key informant said ‘… the beauty of that, though, is that Northern Health (a more rural region than Fraser East) is a bit ahead of us in that model, so there’s a wealth of opportunities for us to learn from some of the lessons that they’ve either uncovered or tripped over or whatever that training is.’ Key informant interview participants in BC and NS indicated their jurisdictions were moving towards increasing the number of interprofessional team-based models of PC.

Provider skill mix

All study regions had implemented strategies to expand the PC provider skill mix by introducing new provider roles (eg, nurse practitioners, physician assistants) or expanding the practice of existing providers (eg, pharmacists prescribing certain medications; registered nurses in family practice) (table 5). Much of the focus was on integrating nurse practitioners into PC settings. However, the provinces differed in the extent to which this had occurred. ON was the first of the study regions to implement legislation authorising the nurse practitioner role in PC in 1995; NS and BC followed suit in 2002 and 2005, respectively.27–29 In 2005, nurse practitioner-led clinics were funded in ON.30–32 Between 2005 and 2010, the number of PC nurse practitioners in ON increased from 381 to 741, an increase of 95%.27 In NS, PC nurse practitioners mostly worked in rural areas where physician shortages have been acute and population needs high. ‘We have some in Central (Region, NS) but not in the primary healthcare…not embedded in primary healthcare. They tend to be more in acute.’ (NS key informant). In BC in 2012 the government budgeted $22.2 million to hire 190 NPs over 3 years33; in 2014, the total number of NPs was 574.34 According to a BC key informant, few nurse practitioners were being implemented in general PC. ‘Most ( nurse practitioners ) are assigned to special populations, so they’re not necessarily accessible to every British Columbian.’

Table 5

Examples of new and optimised provider skill mix by region

British Columbia: Fraser East	Ontario: Eastern Ontario Health Unit (EOHU)	Nova Scotia: Central Zone
Nurse practitioners (NPs) in primary care: ~14 in Fraser East / 169 in province.78	NPs in primary care: 1426 in province79	NPs in primary care: 6–1580 in Central Zone / 8780 81 in province
Physician assistants (PAs) ~24 in province82	PAs: ~170 PAs in province82	PAs: ~25 PAs in province82
Midwives:~26 in Fraser East/ 315 in province83	Midwives: ~15 in EOHU / 817 in province84	Midwives: ~6 in Central Zone / 10 in province85
Family medicine physicians:316 in Fraser East / 6189 in province86	Family medicine physicians:15 417 in province86	Family medicine physicians: 636 in Central Zone / 1215 in province86

Registered nurses were present in all three study regions, though the spread of this role in PC and the scope of its contribution varied widely between regions.27 35 In ON, the self-reported number of registered nurses working in PC rose from 2419 in 2005 to 2873 in 2010.27 Based on registered nurse registration data, in 2000 there were 3179 registered nurses working in primary healthcare in BC; however only 2% of these nurses worked in physician offices where most PC occurs, the remainder were in public or community health workplaces.36 More recent data were not found. NS is moving towards the incorporation of new roles for registered nurses (eg, family practice nurses) into PC interprofessional collaborative family practice teams37; however, in 2007 there were only 35 in Central Zone35 and more recent data were not found. There was also evidence of growing experimentation with expanded roles for other clinicians in PC (eg, pharmacists, paramedics) in all three study regions.38–42

Physician groups and networks

We identified physician groups and/or networks that work together to provide healthcare services and those organised for health system planning and governance infrastructure. British Columbia invested significant resources in the formation of the General Practice Services Committee and Divisions of Family Practice, structures that link physicians at regional and provincial levels to improve health service delivery and increase physician influence.43 44 The funding for these improvement strategies was largely contained within the Physician Master Agreement.45 The Divisions are incorporated as non-profit societies with their own board structure and have representation at the health authority and provincial government levels. As the following two quotes demonstrate, BC key informant interview participants held conflicting reports about benefits of these initiatives. One participant stated, ‘No question the (Division of Family Practice) has been helpful. They’ve changed the way we do (primary) care.’ Another participant said, ‘I think the amount of change for dollars and time and effort that’s been spent is a fraction of what could have been achieved.’ Ontario has a number of different PC physician groups, for example, Family Health Networks, Family Health Groups and Family Health Organisations, each of which have requirements such as provision of after hours care, rostering of patients and access to telephone health advisory service.16 46 Key informant interview participants did not speak about the various types of physician groups but rather about the need for physicians to work more closely with regional health authorities to integrate PC within the healthcare system. In Nova Scotia there was one District Department of Family Practice, the purpose of which was to support physicians to strengthen PC in the district. One NS key informant stakeholder interview participant described it as a ‘dedicated primary care leadership structure that had been created in a council of family physicians and what I believe to be strong primary healthcare leadership and innovation in this what was the former Capital District Health Authority.’

Remuneration

Physicians across the three provinces, were still commonly remunerated through fee-for-service (FFS) payments. In 2015 to 2016, 55.7% of clinical payments in family medicine were billed through alternative (non-fee-for-service) clinical payments in ON, compared with 46.1% in NS and 17.4% in BC.47 Physician payment structures in ON vary by the type of interprofessional model. For example, in ON Community Health Centres, which serve a higher proportion of disadvantaged marginalised populations, the payment structure is based on a salaried approach, whereas, in ON Family Health Teams (interprofessional team with family physicians, nurses, nurse practitioners, pharmacists and others),48 it is based on capitation.49 The General Practice Services Committee in BC has driven major investment into expanded billing code options within the FFS model (eg, financial incentives for out-of-office consultation, maternity services, etc).50–52 One BC clinician described FFS as a ‘juggernaut’ that is difficult to change. According to NS key informant interview participants, most physicians (an estimated 80%) were still paid through FFS, with some expanded billings (eg, chronic disease management and comprehensive care). This is a higher proportion than what NS documents suggest and may reflect physicians whose income includes but is not exclusively FFS.47

Key stakeholder interview participants in ON and NS identified disadvantages of the FFS model. An ON participant noted, ‘Yeah, I think so. I mean, one for sure that people don’t necessarily like to talk about is cost containment. Second is simply recognition that fee-for-service doesn’t work as well for people with chronic medical problems and who are more complex.’ And from a NS participant:

‘But taking innovation in the full sense of it, the fee-for-service model has a huge impact. And so if you’re struggling in the trenches that we are in a fee-for-service model, which most of us are in primary care … I have to make a living here … Do you really think that my brain is capable of giving much attention to innovation even though spiritually, intellectually, ideologically, I’m all for it?’

Information technology

While electronic medical record implementation is high across all three regions (table 6), significant limitations and barriers to implementation persist, particularly regarding time, cost, interoperability and privacy concerns. As noted by key stakeholder participant in BC, data privacy issues impact the potential uses for such systems ‘…to the point where you can’t share information to get best outcomes.’ There is variation across cases in how electronic medical records function within PC practices and across PC systems. A key stakeholder in NS noted that some practices in the region utilise a portal to assess and support patients who are frail. Further, some practices’ electronic medical records have capacity to make external connections (eg, pharmacies and labs in BC, hospitals and pharmacies in ON). Providing access for patients to view their health records is still in an early stage of implementation across the three provinces. Telehealth and other health information technologies are also being used within the study regions to a limited extent (eg, linkages between clinicians across geographies in BC; clinician after-hours access to care across clinics and e-consult mechanisms to access specialists in ON; telehealth triage and advice services in all study regions).

Table 6

Examples of information technology in use by region

British Columbia: Fraser East	Ontario: Eastern Ontario Health Unit	Nova Scotia: Central Zone
FPs who capture patient information using87: Exclusively electronic records: 45.8% A combination of paper and electronic charts: 39.5% Paper charts only: 14.8% FPs whose patients can view their health record online87: 15.0%% of FPs who use telemedicine/telehealth in their practice87: 17.7%	FPs who capture patient information using87: Exclusively electronic records: 51.5% A combination of paper and electronic charts: 34.1% Paper charts only: 14.4% FPs whose patients can view their health record online87: 4.1%% of FPs who use telemedicine/telehealth in their practice87: 23.1%	FPs who capture patient information using87: Exclusively electronic records: 46.9% A combination of paper and electronic charts: 38.7% Paper charts only: 14.4% FPs whose patients can view their health record online87: 6.5%% of FPs who use telemedicine/telehealth in their practice87: 31.8%

FPs, family physicians.

Performance measurement and reporting infrastructure

National initiatives, such as the Canadian Institute for Health Information and Statistics Canada53 provide data on the healthcare system in Canada but have limited information about PC. ON is the only province of the three in our study with a funded organisation, Health Quality Ontario,54 dedicated to performance measurement across health systems. While BC has implemented strategies with an aim to improve the quality of PC, there has been little investment in performance measurement. Quality improvement initiatives in BC focused on physicians' concerns with family practice (eg, Professional Quality Improvement Days), services in rural communities (eg, Joint Standing Committee on Rural Issues) and clinician learning modules (eg, Practice Support Program).45 55 56 ON has implemented provincial strategies (eg, Excellent Care for All Act; Primary Care Performance Measurement Framework for Ontario) that serve as an accountability mechanism.57 58 Health Quality Ontario, the Ministry for Health and Long-term Care, and other partners like Cancer Care Ontario, drive performance measurement in ON and these organisations collaborate with several other organisations to provide a significant number of performance measurement and quality improvement initiatives. Family physicians in ON can opt to receive reports on quality indicators from administrative data, and annual performance reports provide an overview of the province's health system.59 In NS, our interviews and focus groups suggested the Department of Health and Wellness and the Nova Scotia Health Authority are central policy drivers in system performance. While NS has put some effort into practice improvement,60 the province has put less focus on efforts to measure and supply data. Nascent performance measurement and reporting infrastructures have been developed in all three provinces, with ON being the most developed.

Discussion

We identified and compared six common PC improvement strategies, implemented to varying degrees between 2003 to 2014 across the study regions, for the purposes of obtaining contextual data that helps explain regional differences and which can be incorporated into regional PC performance reports. We identified variations in implementation of interprofessional team-based approaches, provider skill mix, physician groups and networks, remuneration and performance measurement and reporting infrastructure. ON focused substantially to implement all six of the key strategies. BC and NS focused attention in each of the six key improvement strategies, but with observable differences in focus and spread. BC had more of an emphasis on infrastructure for physician involvement in health system planning and governance at provincial and regional levels of governance than either ON or NS. Explaining the variability in how and why the provinces decided to tackle the six different improvement strategies is beyond the scope of this article. Possible reasons might include extent or availability of financial investments and government and/or professional leadership.3 20 Growth in the use of information systems was seen across all three regions, a finding that is in keeping with international studies.61–64

While innovation and further implementation of these policies has continued in the study regions since our data collection ended, the results generated from our three provincial case studies identify clear differences illustrating that performance measurement is taking place against very different policy contexts within the same country. These variations matter for performance measurement because these improvement strategies vary with respect to their impact on outcomes. For example, interprofessional team-based care reduces emergency department use but the impact on hospital admissions is less clear.5 Meaningful reports of performance that include regional and provincial contextual information can inform decisions about the suitability or relevance of geographical comparisons.65 Furthermore, understanding PC contexts enables important questions to be considered when PC performance measurement identifies that there are or are not differences. If there is a difference in performance, can any of the difference be attributed to context? If there are no differences in performance, does it mean that investments should be questioned? This type of comparison may also stimulate decision-makers to reflect on why each province strategises PC improvement so differently and the relative advantages of a more comprehensive approach to PC investment. In this way performance reporting can become a practical tool to support decision-making for policy implementation.

Our results highlight the need for contextualisation of both the interpretations of performance data and the strategies for performance improvement. Lessons learnt across these cases are of value, not just for other provinces in Canada, but also for international audiences trying to strike the balance between creating systematic approaches in the development of PC performance measurement systems and acknowledging the influence of local context. The value in incorporating context into performance reporting is to help jurisdictions make sense of their performance, see ‘like/fair comparisons’ areas where there might be learnings on what to implement, and help them implement improvement strategies faster if the contexts are similar.26

The incorporation of contextual data into PC performance reports is still nascent and needs to be more strongly incorporated into future systems. In the UK, demographic details have been incorporated into their system for performance measurement and reporting of Clinical Commissioning Groups and individual PC practices.66 A further step would be to incorporate the type of contextual data on PC improvement strategies described above. As Wong et al 67 suggest, performance reports should be made actionable to healthcare decision-makers. While PC contextual data in and of itself cannot infer causation of PC performance, when considered with complementary sources of performance data, it provides information for decision-makers, clinicians and the public to reflect on what may be driving high or low performance in their regions.

Attempting to understand PC improvement strategies at the regional level was limited by the dearth of regional-level documentation; most of the documents included in our analysis are provincial in scope. While we captured more regional-level detail in the focus groups and interviews, these were limited by small numbers of participants and differences in the positions of key stakeholders interviewed across study regions. Patients’ focussed on their experience with team-based care but had little to contribute in relation to the other improvement strategies. This may be attributable to the small number of patients interviewed or to the scarcity of strategies to inform the public about primary care reforms. However, using multiple data sources allowed us to triangulate our findings by looking for thematic parallels in the three data sources. Our approach to documenting context was time and resource intensive. Development of a regional primary healthcare context survey would be a more practical and sustainable strategy for future performance measurement and reporting activities.

Despite these limitations, our findings suggest the importance of layering in contextual data to help identify the possible drivers of reported performance outcomes and levers for change in practice, regional and system-level settings. The addition of information on regional improvement strategies to primary care performance reports could add important contextual insights into primary care performance results. This could help identify possible drivers of reported performance outcomes and levers for change in practice, regional and system-level settings.