Examining Primary Healthcare Performance through a Triple Aim Lens.

PURPOSE: This study sought to apply a Triple Aim framework to the measurement and evaluation of primary healthcare (PHC) team performance.
METHODS: Triple Aim components were populated with 10 dimensions derived from survey and health administrative data for 17 Family Health Teams (FHTs) in Ontario, Canada. Bivariate analyses and rankings of sites examined the relationships among dimensions and among Triple Aim components.
RESULTS: Readily available measures to fully populate the Triple Aim framework were lacking in FHTs. Within sites, there was little consistency in performance across the Triple Aim components (health, patient experience and cost).
CONCLUSIONS: More and better measures are needed that can be readily used to examine the Triple Aim performance in PHC teams. FHTs, in this study, are partially achieving Triple Aim goals; however, there was a lack of consistency in performance. It is essential to collect appropriate measures and attend to performance across all components of the Triple Aim.

The Institute for Healthcare Improvement (IHI) Triple Aim is a framework describing an approach to optimizing health system performance (Berwick et al. 2008). The Triple Aim focuses on (1) improving the health of populations; (2) improving the patient experience (including quality, patient-centredness, safety and timeliness of care); and (3) reducing the per capita cost of healthcare (IHI 2014). The goal of the Triple Aim is to achieve improvement simultaneously in all three components. Although initially conceived as a framework for health system improvement, the Triple Aim can be adapted to primary care at both the sector and practice/organization levels (Kates et al. 2012).

Ontario's Family Health Teams (FHTs), launched in 2005, serve more than three million Ontarians, approximately 22% of the provincial population (Hutchison and Glazier 2013; Ontario Ministry of Health and Long-Term Care 2009–2010). In total, 200 FHTs were established between 2006 and 2011. FHTs are inter-professional organizations that include family physicians and other primary healthcare (PHC) professionals, most commonly nurses, nurse practitioners, social workers, dietitians, pharmacists and registered practical nurses, as well as administrative support staff (Accreditation Canada 2013). The main objectives of the provincial government's FHT program are improved access to PHC, quality and comprehensiveness of care (with an emphasis on chronic disease management, health promotion and disease prevention), interdisciplinary teamwork, patient engagement, and integration and coordination of care (system navigation) (Accreditation Canada 2013). Physicians working in FHTs are remunerated through a blended capitation or blended salary payment model (Accreditation Canada 2013; Hutchison and Glazier 2013).

The extent to which it is possible to populate the Triple Aim framework with measures of PHC performance and thereby assess the extent to which PHC teams achieve the Triple Aim has not been previously reported. The principal objectives of this study were to: (1) determine the extent to which it was possible to populate the Triple Aim framework using readily obtainable data to measure the performance of PHC teams; and (2) examine the extent to which FHT performance is consistent across the components of the Triple Aim. A secondary objective was to explore anticipated relationships among the 10 dimensions of Triple Aim performance measured in this study.

Methods

Study design and sample

This paper reports results from a larger mixed-methods study that assessed the relationship between FHT characteristics and performance based on an examination of 17 FHT sites. Sites recruited by the Ontario College of Family Physicians were selected to reflect a range of locations (urban and rural), years in operations as an FHT, mix of health professionals and practice configurations (e.g., single site, multi-site). The Triple Aim components were populated with 10 dimensions of PHC performance derived from patient survey data and health administrative data sets that were linked using unique, encoded identifiers and analyzed at the Institute for Clinical Evaluative Sciences. In the remainder of this paper, we describe the methods, analyses and results for the patient survey data collection, the health administrative data and the Triple Aim analysis (Figure 1). Findings from the qualitative portion of this study are reported elsewhere (Brown et al. 2015). Ethics approval for this study was received from The University of Western Ontario's Review Board for Health Sciences Research Involving Human Subjects and through Sunnybrook Health Sciences Centre Research Ethics Board.

Figure 1.

Triple Aim framework components and associated dimensions

Populating the Triple Aim outcomes

The IHI provides a guide for the types of measures that can be used to populate the Triple Aim framework (Stiefel and Nolan 2012). This study attempted to populate the framework with measures appropriate to PHC. We selected measures based on the current availability of relevant data and the feasibility of primary data collection given the overall mixed methods research design and study budget.

Improving the health of populations: We used proxy measures of population health rather than direct measures. The latter are unlikely to be sensitive to PHC performance at the team level given the powerful impact of other influences on population health, including social determinants, other healthcare sectors and the illness burden of the population being served. Appropriate proxy measures – over which PHC teams have substantial control – are clinical processes of care and intermediate outcomes linked in logic and evidence to health outcomes. In this study, we populated the “health” component of Triple Aim with cancer screening (colorectal, cervical and breast) and diabetes care (HbA1c testing, eye examination, cholesterol testing and ACE or ARB prescription), measures that were readily available in health administrative data, although limited in scope.

Improving the patient experience: We populated “patient experience” with patient-centredness, access to after-hours care and timely access to care.

Reducing the per capita cost of healthcare: Finally, we populated “cost” with per capita physician services costs, emergency department (ED) visit costs, hospital in-patient costs, rates of low-urgency ED visits and potentially avoidable hospitalizations.

Appendix 1 provides the data sources and the definitions for the 10 dimensions of primary care performance we used to represent the three components of the Triple Aim.

Patient surveys

Patients were approached in the waiting room by the researchers after being invited to participate by the receptionist. Their acceptance of the invitation to participate was signalled by their acceptance of an information postcard. Patients completed the surveys immediately after their appointments. Inclusion criteria were: being 18 years of age or older and having a visit with a family physician, nurse practitioner or inter-professional healthcare provider (e.g., dietitian, pharmacist or social worker). Exclusion criteria were: (1) non-English speaking; (2) too ill or cognitively impaired; (3) attending an appointment for strictly administrative reasons; or (4) having an appointment with a healthcare trainee at the FHT. Data collection took place in 2012 and 2013.

The patient survey was used to capture the following dimensions of the Triple Aim “patient experience” component – patient-centredness, after-hours access and timely (same- or next-day) access. The patient survey included: (1) demographic questions; (2) a validated 14-item Patient Perception of Patient-Centredness (PPPC) Survey (Stewart et al. 2014b) that is based on the conceptual framework of the Patient-Centred Clinical Method (Stewart et al. 2014a); and (3) two questions concerning access to PHC that were taken from the 2013 Commonwealth Fund International Policy Survey of Adults in 11 countries (Schoen et al. 2013). See Appendix 1 for definitions. The analysis was conducted using SPSS21 (SPSS Statistics 2012). The mean PPPC score, standard deviation, median and range of scores were calculated. Bivariate analyses (t-tests and ANOVA as appropriate) examined whether there were differences between the PPPC scores for different groups of patients. The percentage of patients who achieved access was calculated, and only those patients who tried to access services were included in the denominator.

Health administrative data

Administrative data profiles were generated for the 17 FHT sites based on patients of site physicians as of the year ending 31 March 2011. These profiles included the following dimensions of the Triple Aim components: “health” – cancer screening and diabetes care and “cost” – physician visit and capitation costs, ED visit costs, in-patient hospital costs, low-urgency ED visit rates and potentially avoidable hospitalization rates. Means and frequencies were computed for each of these dimensions. Profiles also included practice demographics and summary statistics across the 17 sites. As healthcare utilization is expected to vary according to practice demographics, the Triple Aim performance dimension scores were adjusted for age, sex, rurality, income quintile and morbidity. See Appendix 1 for definitions. In these health administrative databases, patients were linked to a family physician if they were enrolled with the family physician. Those who were not enrolled were linked to the family physician from whom they received the majority of their primary care.

Triple Aim analysis

For each of the 10 dimensions used to populate the Triple Aim performance, a bivariate analysis calculating Pearson's correlation coefficients was conducted in SPSS21 (SPSS Statistics 2012) to compare each of the dimensions against the other dimensions.

For the Triple Aim component analysis, standardized z-scores for each of the “health,” “patient experience” and “cost” components were calculated by summing the standardized z-scores for the performance dimensions representing each of the Triple Aim components. This was necessary because the units of measurement varied among the 10 dimensions. A bivariate analysis calculating Pearson's correlation coefficients (r) was conducted in SPSS21 to compare each of the Triple Aim components against the other components. Additionally, sites were ranked by the Triple Aim Components, with 1 indicating the highest rank, that is, the most desirable outcome, to 17 indicating the lowest rank, that is, the least desirable outcome. The ranks were then divided into tertiles, with each site being in the top, middle or bottom third of the rankings on the Triple Aim components.

Results

The survey sample consisted of 398 patients; 70% were females, with 24% between the ages of 18 and 34 years; 34% between the ages of 35 and 64 years; and 25% aged 65 years and older. The majority of patients (77%) had seen family physicians, 19% had seen either a nurse or inter-professional healthcare provider and 4% reported seeing two providers where one was a family physician. The mean number of surveys per site was 25. The bivariate analysis found no statistically significant differences between patient-centredness (PPPC scores) and patient sex or patient age group, or for the type of provider seen. As well, there were no significant differences in PPPC scores across the practice sites. The descriptive statistics for the Triple Aim performance dimensions derived from the patient survey (patient-centredness, after-hours access and timely [same-day/next-day] access) are reported in Table 1.

Table 1.

Descriptive statistics for Triple Aim component dimensions (n = 17 sites)

Component Dimension	Source	Mean	SD	Minimum	Maximum	IQR
Health
1. Cancer screening (%)	Health administrative data	68.9	6.6	56.7	78.5	11.7
2. Diabetes care (%)	Health administrative data	68.8	3.9	62.1	75.2	6.2
Patient experience
3. Patient-centredness (score out of 4)	Patient survey	1.4	0.1	1.2	1.5	0.1
4. After-hours access (%)	Patient survey	55.2	25.1	10.0	100.0	34.6
5. Timely access (%)	Patient survey	71.5	19.5	28.6	100.0	28.5
Cost
6. Physician visit and capitation costs ($/patient)	Health administrative data	1,150.40	94.40	1,050.10	1,315.10	167.00
7. ED visit costs ($/patient)	Health administrative data	196.80	47.60	134.60	309.60	68.10
8. In-patient hospital costs ($/patient)	Health administrative data	970.90	122.50	771.80	1,214.40	164.50
9. Low-urgency ED visits (per 100)	Health administrative data	16.3	7.4	8.2	37.5	10.2
10. Potentially avoidable hospitalizations (per 10,000)	Health administrative data	33.9	13.2	14.2	69.1	13.8

Administrative data

In the administrative data for the 17 sites, over half of the patients were women. Patients aged 65 years and older represented 13% of patients, and 46% of patients were in the two highest-income quintiles. The majority of patients (67%) were in a major urban centre. There was wide variation across the 17 sites with regard to ED visits, potentially avoidable hospitalizations, cancer screening, diabetes care and primary care-related costs. The descriptive statistics for the Triple Aim performance dimensions derived from the health administrative data are reported in Table 1. Table 2 describes the patient characteristics of the 17 FHT sites.

Table 2.

Characteristics of patients – health administrative data (n = 17 sites)

	Mean	SD	Minimum	Maximum
Number of patients	14,515	8,594	3,165	38,262
Male (%)	46.5	4.7	38.0	53.3
Age in years (%)
≤18	22.6	4.4	14.4	32.7
19–49	43.8	6.8	36.6	63.6
50–64	19.8	3.0	12.6	23.6
≥65	13.0	4.5	4.8	20.1
Income quintile (%)
1 (low)	16.1	5.7	4.9	25.9
2	18.2	4.2	8.1	25.4
3	19.8	5.7	11.5	39.4
4	23.1	7.8	12.1	46.2
5 (high)	22.6	7.1	6.0	33.0
Rurality (%)
Major urban	67.5	37.6	4.3	98.1
Non-major urban	26.7	32.0	0.8	94.3
Rural	5.3	9.0	0.3	32.7
Resource utilization ban (RUB) (Mean)	2.6	0.1	2.4	2.9

Triple Aim Analysis

Within each site, there was little consistency in the rankings of the Triple Aim components. Only one FHT scored in the top tertile on all Triple Aim components and only two scored in the bottom tertile on all three components. The remaining 14 FHTs scored in at least two different tertiles for the three components. Figure 2 provides one example of this variation within the sites. The five sites that scored in the top tertile for “patient experience” are used as an illustration. The standardized z-scores for the 10 performance dimensions representing the “health,” “patient experience” and “cost” components of the Triple Aim are plotted for these sites. This illustrates that performing in the top tertile for “patient experience” did not necessarily mean that the sites performed well in all 10 dimensions.

Figure 2.

Dimensions of Triple Aim components – standardized z-scores for five sites*

The standardized z-scores for the 10 performance dimensions representing the “health” (Cancer screening and Diabetes care), “patient experience” (Patient-centredness, After-hours access and Next appointment), and “cost” (remaining five) components of the Triple Aim are plotted for the five sites that performed in the top tertile for “patient experience.”

Across the sites, when the 10 dimensions were aggregated to form the overall Triple Aim components of “health,” “patient experience” and “cost,” there were no statistically significant correlations among the three components. There were, however, some significant correlations among the 10 dimensions used to populate the Triple Aim components. Some of these correlations reflected anticipated relationships. Greater after-hours access was associated with: (1) more timely access (r = 0.502, p = 0.040), (2) lower rates of low-urgency ED visits (r = −0.496, p = 0.043) and (3) lower overall ED visit costs (r = −0.543, p = 0.024). Lower rates of low-urgency ED visits were associated with lower overall ED visit costs (r = 0.828, p = 0.001). Better cancer-screening performance was associated with better diabetes care (r = 0.634, p = 0.006). Higher potentially avoidable hospitalizations were associated with higher ED visit costs (r = 0.555, p = 0.021). However, other relationships were not anticipated. Better diabetes care was associated with higher rates of potentially avoidable hospitalizations (r = 0.505, p = 0.039) and with higher in-patient hospital costs (r = 0.547, p = 0.023). Higher scores on patient-centredness were associated with higher access (r = 0.500, p = 0.041). Higher physician visit and capitation costs were associated with lower in-patient hospital costs (r = −0.633, p = 0.006).

Discussion

This study examined the extent to which it was possible to populate the Triple Aim framework with dimensions of healthcare performance using readily available data for 17 Ontario FHTs. Ten dimensions of PHC performance were identifed to populate the Triple Aim components of “health,” “patient experience” and “cost”. Analyses examined both the Triple Aim components as a whole and the 10 individual dimensions. There were two clear messages from this study: (1) readily available measures to populate the Triple Aim framework were lacking in FHTs and (2) there was a lack of consistency in FHT performance across and within different FHT practice sites.

Data availability and quality

With respect to measurement, the two dimensions that were used to represent “health” were narrow in scope (cancer screening and recommended diabetes care) and were not in themselves direct measures of health. Rather, they measured care that is expected to be related to good health outcomes. More comprehensive measurement of outcome-related clinical processes would require clinical record review (Green et al. 2012). Identification of appropriate measures of population health is a major challenge when applying the Triple Aim framework at the PHC team level. The measures of population health recommended by IHI (e.g., mortality, health and functional satus, healthy life expectancy, disease burden, health behaviours and physiological measures) (Stiefel and Nolan 2012) are applicable at the health system level and for substantially large populations. However, they are almost certain to be insensitive to PHC team performance. Clinical processes of care and intermediate outcomes are feasible alternatives, although these would normally fall within the Experience of Care component in IHI's typology (Stiefel and Nolan 2012). Further theoretical and empirical work is needed to define a suitable set of measures of PHC team-level population health in the context of Triple Aim, including those that are patient-reported (McGrail et al. 2012).

Patient experience measures included patient-centredness and access. Patient-centredness must be measured from the patient's perspective (Stewart 2001). Patient access could be measured from both the patient's perspective and the practice's perspective through internal audits of wait times, use of after-hours care and reports of the number of patients who access ED for care that could have been provided by the practice.

Similar to limitations reported in other studies (Green et al. 2012), there were also limitations in measuring cost. Several costs are not yet captured by the health administrative data algorithm including, for example, primary care (as opposed to total) physician costs and inter-professional and administrative staff team member salaries.

In Ontario and elsewhere in Canada, primary care clinicians and managers have routine access to only the most limited practice-level data on their performance, other than the data they collect themselves. FHTs now have access to administrative data profiles similar to those used in this study. Health Quality Ontario, in partnership with a broad range of relevant stakeholders, has recommended a comprehensive set of practice- and system-level primary care performance measures covering the full scope of primary care, drawing on health administrative, electronic medical record, patient survey, and provider and organizational data (Health Quality Ontario 2015). Of the 112 recommended measures, only 15 (13%) will be available to all primary care practices in the province within the near future. Clearly, major investments in infrastructure for primary care data collection, analysis and reporting are required to equip primary care providers and organizations with feedback on their performance over time and in comparison with peers that will enable them to identify opportunities for improvement and to track the impact of their improvement efforts.

Relevance of the Triple Aim framework

A major strength of the Triple Aim framework is its emphasis on improvement and on balancing potential trade-offs between health, patient experience and cost. The Triple Aim framework addresses key aspects of the Ontario government's objectives for FHTs and its February 2015 Patients First: Action Plan for Health Care, particularly access, quality and comprehensiveness, integration/coordination and improved patient experience (Ontario Ministry of Health and Long-Term Care 2015). Future research should focus on identifying a parsimonious set of performance measures to adequately capture the Triple Aim at the practice level, to further explore the relationships among the Triple Aim components and to identify the key policy and organizational drivers of the Triple Aim.

Triple Aim performance in FHTs

With respect to performance, across FHT practice sites, we found a wide range in scores for the different dimensions of the Triple Aim components. As examples, for “health,” cancer screening varied by a magnitude of 1.4; for “patient experience,” there was a 10-fold variation in patients reporting after-hours access; and for “cost,” there was almost a twofold variation in in-patient adjusted hospital costs. No consistent pattern was found within practice sites. Only one FHT scored in the top tertile on all of the Triple Aim components and two scored in the bottom tertile on all three components.

Despite a lack of overall consistency in performance across and within FHT practice sites, there were specific correlations among some of the 10 dimensions of the Triple Aim performance we assessed. Not previously reported in the literature were two significant relationships between: (1) after-hours access and the rates of low-urgency ED visits and (2) after-hours access and overall ED visit costs. It is to be expected that provision of after-hours care would result in lower ED visits and lower ED visit costs. This study demonstrated these relationships at the practice level by bringing together health administrative data and patient-reported data, suggesting the possibility of a significant system impact through having access to more after-hours PHC.

The correlation between diabetes care and hospital care was not in the expected direction. Higher percentages of patients receiving diabetes care according to guidelines was associated with higher potentially avoidable hospitalizations and higher in-patient hospital costs. One possible explanation for this finding is that a higher percentage of patients receiving appropriate diabetes care is a proxy for having sicker patients with diabetes that could lead to more hospital care.

An interesting and unexpected finding was the correlation between higher physician costs and lower in-patient hospital costs. This suggests that more intensive physican care may reduce the need for hospital care. This finding cannot be attributed entirely to primary care physician care because this dimension included all physican costs and it was not possible to separate primary care physician costs in these data.

Limitations

A limitation of this study was the small sample size of 17 FHT practice sites. The modest number of patient surveys at each site resulted in some imprecision in measurement for each site. Further, generalizability to all FHTs is also limited due to the wide variations in FHT structure within the province of Ontario. Health administrative data are always older than data that can be collected at a practice site in real time. In this study, the administrative data were older by a year than the patient survey data. The other limitation is the cross-sectional nature of these data that prevents implying causality from any associations and prevents drawing conclusions about improvement in care which is the focus of the Triple Aim framework. Finally, research of this nature is time-consuming, costly and resource-intensive. However, research has shown that chart audit and access are excellent meaures to utilize (Harris et al. 2015), but they were not within the budget of this study. Future research in this area should consider the use of chart audits, organizational surveys and interviewing patients directly.

Conclusion

This study highlights the need for more and better measures that can be readily used to examine FHT (and more generally, PHC team) performance on the Triple Aims of patient health, patient experience and healthcare costs. The findings also reveal how the FHTs in this study are partially achieving the goals of the Triple Aim framework. It cannot be determined from this study why FHTs do not perform consistently across all the components of the Triple Aim. It may be that as FHTs evolve and grow, they choose to focus their efforts in particular ways at particular times in response to their patient population's needs. Perhaps, given limited resources, an increase in performance in one dimension may come at the expense of performance in another dimension. This study provides evidence that performance in one area of the Triple Aim is not necessarily associated with performance in another area. Therefore, it is essential at both the practice and health system levels to collect appropriate measures and be attentive to performance across all components of the Triple Aim framework.