Optimising the changing role of the community pharmacist: a randomised trial of the impact of audit and feedback.

OBJECTIVE: To evaluate the impact of comparative performance feedback to community pharmacists on provision of professional services and the quality of patients' medication use.
DESIGN: Randomised, controlled, single-blind trial.
SETTING: All 1833 community pharmacies in the Quebec province, Canada. PARTICIPANTS: 1814 pharmacies not opting out and with more than 5 dispensings of the target medications during the 6-month baseline were randomised by a 2×2 factorial design to feedback first for hypertension adherence (907 control, 907 intervention) followed by randomisation for asthma adherence (791 control, 807 intervention). 1422 of 1814 pharmacies had complete information available during the follow-up for hypertension intervention (706 intervention, 716 control), and 1301 of 1598 had the follow-up information for asthma (657 intervention, 644 control). INTERVENTION: Using provincial billing data to measure performance, mailed comparative feedback reported the pharmacy-level percentage of dispensings to patients non-adherent to antihypertensive medications or overusing asthma rescue inhalers. PRIMARY AND SECONDARY OUTCOME MEASURES: The number of hypertension/asthma services billed per pharmacy and percentage of dispensings to non-adherent patients over the 12 months post intervention.
RESULTS: Feedback on the asthma measure led to increased provision of asthma services (control 0.2, intervention 0.4, RR 1.58, 95% CI 1.02 to 2.46). However, this did not translate into reductions in patients' overuse of rescue inhalers (control 45.5%, intervention 44.6%, RR 0.99, 95% CI 0.98 to 1.01). For non-adherence to antihypertensive medications, feedback resulted in no difference in either provision of hypertension services (control 0.7, intervention 0.8, RR 1.25, 95% CI 0.86 to 1.82) or antihypertensive treatment adherence (control 27.9%, intervention 28.0%, RR 1.0, 95% CI 0.99 to 1.00). Baseline performance did not influence results, and there was no evidence of a cumulative effect with repeated feedback.
CONCLUSIONS: Comparative pharmacy performance feedback increased the provision of asthma pharmacists' services but did not improve the performance on medication-use measures. Billing data can be used to evaluate the impact of billable services rendered by pharmacists on the quality of patients' medication use. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

RCT Entities:   Population Interventions Outcomes

A large, representative number of community pharmacies participated in this study.

The trial used objective, standardised measures of pharmacists’ provision of professional services and of the quality of patients’ medication use.

The performance was measured and reported on only two quality of medication-use measures, and this may not be representative of a pharmacy's overall performance.

Administrative data allow only the measurement of billing for services and, therefore, the provision of services that were not billed could not be measured.

Administrative data are limited in the extent to which they measure actual medication use by patients.

Introduction

Background

Policymakers continue to seek solutions that optimise the role of healthcare professionals in managing the increasing numbers of patients with medication-use problems and the healthcare costs associated with the misuse of medications.1 Internationally, community pharmacists are being recognised as a relatively untapped resource for improving patients’ use of medications, leading to revisions in healthcare policies to expand the authority of pharmacists.2–4 As healthcare payers around the world implement schemes to reimburse community pharmacists for provision of these services, they are also seeking systems to evaluate the quality of care provided and the impact of pharmacists’ services on the inappropriate use of medications.5–8 To enable such evaluations, nationally endorsed quality of medication-use indicators have been developed for use as standardised measures of the outcomes of pharmacists’ care.9–11 Systems are also being developed for pharmacists to be held accountable for the achievement of defined performance on these indicators, and healthcare payers are incorporating these performance measures into revised reimbursement schemes.12 13 The Pharmacy Quality Alliance (PQA) in the USA has been instrumental in defining standardised quality of medication-use outcome measures that focus on patient adherence to medications, use of high-risk medications in the elderly, appropriate selection of medications for common diseases, management of drug interactions, appropriate dosing of high-risk medications and completion of comprehensive medication reviews for at-risk patients.14 Methodologies have now also been developed to use community pharmacy billing data to measure and report community pharmacy-level performance on such quality of medication-use measures.14–16 Similarly, where there is appropriate documentation for reimbursement of pharmacists’ service provision, billing data-derived methodologies have been developed to objectively measure pharmacists’ provision of expanded services in healthcare systems.16 17

Despite these advances, there is limited evaluation of the impact of pharmacists’ expanded services on the quality of medication use.6 The Pennsylvania Collaborative project, a pioneer in this area, showed that pharmacists’ screening of patients for risk of non-adherence and brief motivational counselling increased adherence to five classes of medications for the management of chronic diseases.18 Pharmacists were supported by a combination of intensive direct training, audit and feedback of their performance on the quality of medication-use measures, regular on-site visits and telephone calls to support and sustain the implementation of the service throughout the evaluation. Although the added costs of such resource-intensive support can be maintained during research evaluations, it is challenging to incorporate these costs into a business model that enables sustainable, scalable provision of the service. Indeed, even with resource-intensive support during research trials, service provision by community pharmacists is often low and limited numbers of new services delivered have been identified as a major challenge in community pharmacy research.19–22 It is, therefore, important to evaluate whether strategies that require less resources could increase pharmacists’ provision of expanded services and improve performance on the quality of medication-use measures.

Audit and feedback to healthcare professionals of a summary of their clinical performance measured over a specified period of time has been shown to lead to small but potentially important improvements in care.23–25 For pharmacists, although feedback on performance with standardised patients has been piloted to improve quality of care, no studies have evaluated the use of audit and feedback alone to either increase pharmacists’ provision of professional services or improve performance on standardised quality of medication-use measures.21 26 If community pharmacy billing data are used to electronically provide real-time feedback, the use of such audit and feedback to community pharmacists is an attractive option. This is because electronic systems can be fully automated, reducing the resource requirements for adoption and sustainability. Such automated technologies have been introduced and the adoption of these electronic performance dashboards has been rapid in the USA, with a wide range of pharmacies and healthcare plans using the EQuIPP platform to audit and provide pharmacy-level performance feedback on PQA quality of medication-use measures.27–31 As there has been no evaluation of the impact of these technologies, what remains unknown is whether performance feedback alone effectively improves community pharmacists’ provision of professional services and quality of medication use.

Objectives

The purpose of this study was to determine whether comparative feedback to community pharmacists of their pharmacy's performance on the quality of medication use by hypertensive and asthmatic patients increases the provision of targeted pharmacists’ services and performance on the quality of medication-use measures.

Methods

Setting

This study was conducted in Canada, where the organisational structure of the healthcare system offers several advantages. First, all provinces offer publically funded health insurance programmes that cover the drug costs and pharmacists’ fees for prescription medications dispensed to seniors and the economically disadvantaged. Second, consistent with healthcare policy trends in Australia, the UK and the Netherlands, over the past several years all Canadian provinces have expanded the authority and fee-for-service reimbursements for pharmacy services to community pharmacists.32 33 Third, all provinces maintain central, electronic databases of information about the medications dispensed and services provided if they are reimbursed by the publically funded insurance programme. Although varying in format and the level of detail across the provinces, the information retained in the community pharmacy billing databases is sufficiently detailed to allow the measurement of the provision of reimbursable pharmacists’ services targeted at managing specific medication-use problems and the patients’ quality of medication-use.15 These electronic databases provide ready access to the information required to audit performance and prepare feedback reports to community pharmacists.

The province of Quebec is the largest in Canada, with a population of 8 million patients; of whom ∼3.5 million receive government support for payment of their medications via the Régie de l'Assurance Maladie du Québec (RAMQ). Medication-related data maintained by the RAMQ include payments made for reimbursable medications and for provision of defined pharmacist services that aim to resolve specific medication-use problems. Similar to many developed countries, the date, name, strength, dosage form and quantity of the dispensed medication, prescriber, pharmacist and pharmacy identification, and associated costs are recorded. Reimbursable medications include more than 85% of medications available in Canada. For reimbursable pharmacists’ services, the date, type of pharmacists’ service provided (eg, refusal to dispense, review and recommendations for changes in therapy to the prescribing physician, initiation of therapy) and the medication-use problem that the service was meant to resolve are recorded in a coded format. The Quebec Order of Pharmacists, the provincial authority who regulates the performance of more than 5000 community pharmacists in over 1800 community pharmacies, is authorised to access the information contained in the electronic databases as part of its mandate to ensure the quality of care provided by community pharmacists.

Trial design

A single-blind, randomised, controlled trial was conducted to determine the effect of comparative feedback of pharmacy-specific performance on two quality of medication-use measures related to medication adherence. Outcomes were measured and compared between the intervention and control groups in the 12 months post intervention. A 2×2 factorial design also allowed the estimation of the cumulative effect of receiving sequential feedback on the two measures. There are ∼1800 community pharmacies in the province of Quebec. A sample of this size was expected to detect an absolute difference of 7% in the pharmacy-level percentage of dispensings to non-adherent patients, assuming a type I and II errors of 5% and 20%, respectively. The McGill University Faculty of Medicine Institutional Review Board provided ethics approval.

Data sources

To assess the impact of relative performance feedback, we received anonymised pharmacy billing data from RAMQ for all Quebec community pharmacies every 6 months starting from January 2010 to September 2012. These data provided all billings for dispensings of antihypertensive and asthma medications and the provision of community pharmacists’ services over the time period of 1 October 2008–30 April 2012. Each pharmacy was assigned a unique encrypted identifier by RAMQ that enabled all activity within a pharmacy to be measured by the investigators, without revealing the identity of the pharmacy.

Participants

All community pharmacies were advised about the study through communications from the pharmacy regulatory authority and were provided with the opportunity to opt out of participating. Community pharmacies were eligible if they had dispensed the targeted medications more than five times during the baseline period.

Intervention

Pharmacy-specific performance on two quality of medication-use measures was calculated using previously described methods.15 Quality measures included were pharmacy-specific rates of dispensing: (1) antihypertensive medications to non-adherent patients (defined as those who had used <80% of their required medication over the previous 90 days) and (2) short-acting β agonists (SABAs) to patients with demonstrated overuse of these medications (defined as more than 200 doses of SABAs over the previous 90 days). Feedback reports of comparative performance on each quality of medication-use measure were generated and mailed once to the intervention pharmacies. Reports provided a graphic and numeric display of performance at the pharmacy on the quality measure over a 6-month baseline in relationship to the performance at all other pharmacies in the province (figure 1A[i]). Specific colour coding was assigned for each quartile of performance and the report recommended how pharmacists could provide and bill for professional services to address the relevant medication-use problem (figure 1B). Feedback reports were developed in conjunction with the Quebec Order of Pharmacists and included a review by practising pharmacists. The design of the feedback reports was based, with permission, on the Pharmaceutical Society of Australia's Targeted Interventions publications.34 The graphical representation of performance was derived from the best practices used at the time by the Quebec College of Physicians within their practice enhancement programme.35

Figure 1

(A and B) Sample feedback report.

Continued

Outcomes

The impact of feedback was measured by two outcomes in the 12 months postintervention: 1. the pharmacy-specific number of billings for pharmacists’ services for the management of the specific medication-use problem and 2. the pharmacy-level percentage of dispensings to patients non-adherent with their hypertension and asthma medications. The total numbers of all pharmacists’ expanded services billed were also calculated to determine whether the feedback lead to a general change in the provision of pharmacists’ services or a change in services targeting the specific medication-use problems addressed in the feedback reports.

Randomisation

Pharmacies that did not opt out were randomised sequentially, using a random numbers table by the statistician, starting with the hypertension non-adherence measure. Six months later, pharmacies were randomised again to receive feedback or not on the asthma measure. Sequential randomisation ensured that the dual intervention group was not receiving feedback on two quality indicators at the same time, which could have diluted pharmacists’ efforts to improve their performance. Sequential randomisation also allowed the evaluation of the impact of the repeated provision of feedback as pharmacies fell into one of the four feedback groups: no performance feedback; hypertension alone; asthma alone; and both hypertension and asthma feedback.

Blinding

Reports were prepared for each intervention pharmacy, identified to the research team by an anonymised study pharmacy number only. The pharmacists’ regulatory authority served as a trusted third party to retain the look-up codes provided by RAMQ that linked the anonymised pharmacy study number with the real pharmacy name and address. Feedback reports were provided to the regulatory authority in sealed envelopes labelled only with the anonymised study pharmacy number to ensure the confidentiality of results was maintained. Consistent with the ethics approval requirements, control pharmacies received their relative performance feedback reports on completion of the study.

Statistical methods

For each quality of medication-use measure, Poisson regression with the log link function and adjustment for baseline performance was used to determine the impact of performance feedback on the number of pharmacists’ services billed for managing non-compliance with antihypertensive medications and SABAs, respectively, and the impact on the number of all pharmacists’ services billed in the follow-up period. Setting control pharmacies as the reference, exponentiation of the estimate allowed the calculation of the relative risk of intervention pharmacies’ billing for pharmacists’ services during the follow-up period as compared to that of control pharmacies. Negative binomial regression was also used as a sensitivity analysis to correct for overdispersion.

Generalised linear regression with adjustment for baseline performance was used to test for differences between pharmacies receiving performance feedback and controls in the per cent of dispensings that were provided to non-adherent patients during the follow-up period. Given that results were bounded between 0 and 100, the binomial distribution was used. The log link function was selected to enable the calculation of the relative risk of intervention pharmacies dispensing medications to non-adherent patients during the follow-up period as compared to that of control pharmacies.36 To test whether changes in performance in response to feedback were modified based on the baseline level of performance, pharmacies were divided into quartiles according to their performance during the baseline period. Setting the lowest performing quartile as the reference (ie, the quartile of pharmacies with the highest per cent of dispensings provided to non-adherent patients), differences between intervention and control pharmacies were compared among quartiles. To determine whether there is a cumulative effect of feedback, we used a generalised linear regression model with binomial distribution, log link function and dummy variables to evaluate changes in performance on the asthma quality measure if pharmacies received no feedback, feedback only for hypertension, feedback only for asthma or feedback for asthma and hypertension. The four dummy variables represented the four intervention groups, with the no feedback intervention being the reference. These analyses allowed the determination of whether: feedback was effective in producing a change in performance; the change in performance was greater depending on initial levels of performance; and there was an increasing change in performance in response to receipt of repetitive performance feedback on multiple quality of medication-use measures.

Results

Study participants

Of the 1833 eligible community pharmacies, 19 opted out of the audit and feedback study (see online supplementary appendix Consort Flow Diagram). All 1814 pharmacies had sufficient dispensings of antihypertensive medications over the baseline period to be included in the hypertension intervention. Of the 1814 pharmacies, 1598 had sufficient SABA dispensings over the baseline period to be included in the asthma randomisation.

For the 12-month follow-up, 1422 of the 1814 pharmacies had complete information provided from RAMQ for the hypertension intervention (N=706 intervention, N=716 control), and 1301 of the 1598 pharmacies for the asthma intervention (N=657 intervention, N=644 control). Lack of complete data occurred when the coded pharmacy identifier was not found in the billing data during the follow-up period. Changes in pharmacy identifiers occur when pharmacies close or undergo a change in ownership or management, thereby accounting for attrition over time.

Baseline characteristics

Intervention and control pharmacies had baseline patient populations that were comparable in their characteristics known to influence medication adherence such as patient age and sex, or in the use of multiple medications, multiple prescribing physicians or multiple dispensing pharmacies (table 1).37 Intervention and control pharmacies were also comparable in their baseline performance on the quality of medication-use measures and provision of pharmacists’ services, including those targeted at managing the specific medication-use problems. The 216 pharmacies excluded from the asthma randomisation due to dispensing of low numbers of SABAs during the baseline period also had low total numbers of dispensings and pharmacists’ services, and fewer pharmacists employed, over the 6-month baseline period. Pharmacies without complete information during follow-up (hypertension n=392, asthma n=297) were comparable to the pharmacies with complete information for the respective intervention.

Table 1

Baseline characteristics of participating community pharmacies

	Managing non-compliance in hypertensive patients		Managing medications in asthma patients
Number of pharmacies	ControlN=716	InterventionN=706	ControlN=644	InterventionN=657
Characteristics	n (%)	n (%)	n (%)	n (%)
Total number of dispensings of targeted drugs during baseline	387 287 (50.1)	385 829 (49.9)	87 908 (48.9)	91 797 (51.1)
Pharmacy clients*
Sex
Female	214 967 (55.5)	214 323 (55.5)	50 143 (57.0)	52 676 (57.4)
Male	172 320 (44.5)	171 506 (44.5)	37 765 (43.0)	39 121 (42.6)
Age(years)
<65	385 589 (99.6)	384 168 (99.6)	74 467 (84.7)	77 905 (86.8)
65–69	593 (0.2)	646 (0.2)	2388 (2.7)	2642 (3.2)
70–79	860 (0.2)	768 (0.2)	4709 (5.4)	2872 (3.2)
>79	245 (0.1)	247 (0.1)	6344 (7.2)	6378 (6.9)
Drug therapy for condition
New therapy (<6 months)	77 334 (20.0)	77 726 (20.1)	6472 (7.4)	6810 (7.4)
Chronic therapy (≥6 months)	309 953 (80.0)	308 103 (79.9)	81 436 (92.6)	84 987 (92.6)
Single drug	178 953 (46.2)	178 863 (46.4)	86 217 (98.1)	89 978 (98.0)
Multiple drugs	208 334 (53.8)	206 966 (53.6)	1691 (1.9)	1819 (2.0)
Prescription duration<2 months	374 740 (96.8)	373 609 (96.8)	87 059 (99.0)	90 852 (99.0)
Prescription duration ≥2 months	12 547 (3.2)	12 220 (3.2)	849 (1.0)	945 (1.0)
Single pharmacy dispensed	346 442 (90.4)	346 606 (89.8)	77 616 (88.3)	80 611 (87.8)
Multiple pharmacies dispensed	40 845 (9.6)	39 223 (10.1)	10 292 (11.7)	11 186 (12.2)
Single prescriber	326 482 (84.3)	325 716 (84.4)	76 998 (87.6)	80 185 (87.4)
Multiple prescribers	60 805 (15.7)	60 113 (15.6)	10 910 (12.4)	11 612 (12.6)
Pharmacy characteristics	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Total number of Prescriptions/day	264 (189.2)	262 (189.9)	270 (186.5)	275 (204.5)
Number of pharmacists employed over 6 months	8.1 (5.7)	8.3 (5.9)	9.1 (6.4)	9.3 (6.7)
Pharmacists’ performance and provision of professional services
Performance on quality measure (% dispensings to non-adherent patients)	27.2 (4.6)	27.4 (5.1)	48.1 (12.0)	48.3 (11.9)
Number of pharmacist services for the medication-use problem†	0.7 (3.4)	0.7 (2.6)	0.1 (0.5)	0.2 (0.9)
Number of all pharmacist services per 100 prescriptions	0.2 (0.2)	0.2 (0.1)	0.2 (0.1)	0.2 (0.1)

*Considering all dispensings included in the calculation of performance on relevant indicator.

†As a majority of pharmacies did not bill any pharmacist services targeted at the medication-use problem, the average number billed is <1.

Impact of audit and feedback

For the first outcome measure of the number of pharmacists’ services billed for managing the targeted medication-use problem, pharmacists receiving asthma feedback had 1.6 times the chance of billing for services recommending changes to patients’ asthma medications as compared to pharmacists not receiving asthma feedback (control 0.2, intervention 0.4, RR 1.58, 95% CI 1.02 to 2.46) (table 2). Of interest, during the same period, control pharmacies billed more pharmacists’ services for all medications compared to intervention pharmacies (control 39.8, intervention 38.9, RR 1.17 95% CI 1.00 to 1.37), suggesting that the intervention effect was specific to asthma management.

Table 2

Impact of single provision of relative performance feedback on the provision of pharmacists’ services over 12-month follow-up

	Managing non-compliance in hypertensive patients					Managing medications in asthma patients
	ControlN=716	InterventionN=706		Poisson regression		ControlN=644	InterventionN=657		Poisson regression
	Mean count (SD)	Mean count (SD)	Difference	Relative risk* (95% CI)	p Value	Mean count (SD)	Mean count (SD)	Difference	Relative risk* (95% CI)	p Value
Pharmacists’ services for targeted medication use problem†	0.7 (3.3)	0.8 (2.7)	0.1	1.25 (0.86 to 1.82)	0.25	0.2 (1.0)	0.4 (1.6)	0.2	1.58 (1.02 to 2.46)	0.04
Pharmacists’ services for all medication-use problems	30.5 (70.1)	27.4 (58.8)	−3.1	1.05 (0.89 to 1.23)	0.58	39.8 (80.7)	38.9 (70.6)	−0.9	1.17 (1.00 to 1.37)	0.05

*Relative risk of intervention pharmacies billing for the service during the follow-up period as compared to that of control pharmacies.

†As a majority of pharmacies did not bill any pharmacist services targeted at the medication-use problem, the average number billed is <1.

The same trends were seen for the hypertension feedback with intervention pharmacists billing more services to improve compliance with antihypertensive medications despite billing for fewer pharmacists’ services overall, but the differences were not statistically significant (HTN pharmacist services, control 0.7, intervention 0.8, RR 1.25, 95% CI 0.86 to 1.82; all pharmacist services, control 30.5, intervention 27.4, RR 1.05, 95% CI 0.89 to 1.23).

For the second outcome measure of the pharmacy-level per cent of dispensings to non-adherent patients, baseline performance on each measure was a significant predictor of performance during the follow-up period; so, all analyses were adjusted for prior performance (table 3). Feedback on hypertension performance had no impact on the follow-up performance over 12 months (control 27.9%, intervention 28.0%, RR 1.0, 95% CI 0.99 to 1.00) (table 3). Similarly, for asthma performance, the 12-month follow-up performance was virtually identical in control and intervention pharmacies (control 45.5%, intervention 44.6%, RR 0.99, 95% CI 0.98 to 1.01). The impact of hypertension performance feedback differed significantly among the quartiles of pharmacies, but there was no trend for feedback to have a greater impact on pharmacies with lower initial levels of performance (table 3). Although the quartile of pharmacies performing the worst on the asthma measure demonstrated the largest improvement in performance post feedback (control 55.5% of dispensings to SABA-overuse patients compared to intervention 52.9% of dispensings to SABA-overuse patients), there were no statistically significant differences in the response to asthma feedback among the quartiles.

Table 3

Impact of single provision of relative performance feedback on patient quality of medication use over 12-month follow-up

	Managing non-compliance in hypertensive patients*					Managing medications in asthma patients†
	ControlN=716	InterventionN=706		GLR‡		ControlN=644	InterventionN=657		GLR‡
	Mean % (SD)	Mean % (SD)	Differencein percent	Relative risk (95% CI)	p Value	Mean % (SD)	Mean % (SD)	Differencein percent	Relative risk(95% CI)	p Value
All pharmacies	27.9 (5.1)	28.0 (4.4)	0.1	1.0 (0.99 to 1.00)	0.59	45.5 (12.9)	44.6 (13.2)	−0.9	0.99 (0.98 to 1.01)	0.42
By performance during baseline
Worst quartile	31.8 (4.2)	32.2 (3.7)	0.6	Reference		55.5 (11.3)	52.9 (14.0)	−2.6	Reference
Second quartile	28.6 (2.1)	28.5 (2.9)	−0.1	0.98 (0.97 to 1.00)	0.01	47.5 (8.7)	45.8 (10.8)	−1.7	0.98 (0.96 to 1.01)	0.28
Third quartile	26.3 (2.4)	26.9 (2.9)	0.3	0.97 (0.96 to 0.99)	<0.001	43.0 (10.7)	44.0 (8.5)	1.0	0.98 (0.95 to 1.02)	0.41
Best quartile	24.9 (7.4)	24.1 (3.3)	−0.8	0.94 (0.92 to 0.96)	<0.001	36.5 (12.6)	35.5 (12.3)	−0.9	0.97 (0.91 to 1.03)	0.29

*Calculated as a per cent of all dispensings of antihypertensive medications that were provided to patients documented to have taken <80% of the prescribed dose over the previous 90 days.

†Calculated as a per cent of all dispensings of SABAs that were provided to patients who had used >250 doses in the previous 90 days.

‡Generalised linear regression using binomial distribution and log link function.

There was no evidence of a cumulative effect of providing comparative feedback on more than one quality measure (table 4). There were no significant differences in medication-use measures among pharmacies who had received no feedback, feedback only on hypertension non-adherence, feedback only on asthma management or feedback on both. In relationship to asthma pharmacy services, the group receiving no feedback billed the lowest number of pharmacists’ services for asthma management. Pharmacies who received feedback for both conditions did not bill for significantly more asthma services relative to the pharmacies who received no feedback. However, pharmacies who received only asthma feedback had more than a twofold chance of billing for asthma services compared to those with no feedback (no feedback 0.18, asthma-only feedback 0.46, RR 2.28, 95% CI 1.19 to 4.39). These higher billings for asthma services occurred despite significantly lower billings for all pharmacists’ services, again supporting that the intervention effect was specific to asthma management.

Table 4

Impact of repeated provision of relative performance feedback on patient quality of medication use over 12-month follow-up

	Pharmacies n	Managing medications in asthma patients			Pharmacist services billed for SABA overuse			Total pharmacist services billed for all medication-use problems
		Mean % (SD)	Generalised linear model		Mean number billed (SD)	Poisson regression		Mean number billed (SD)	Poisson regression
			Relative risk* (95% CI)	p Value		Relative risk† (95% CI)	p Value		Relative risk† (95% CI)	p Value
No performance feedback	323	46.4 (12.5)	Reference		0.18 (0.62)	Reference		41.51 (83.28)	Reference
Performance feedback on hypertension non-compliance, asthma control	321	44.5 (13.1)	0.99 (0.97 to 1.02)	0.48	0.30 (1.19)	1.60 (0.80 to 3.20)	0.19	38.03 (78.12)	1.18 (0.94 to 1.48)	0.14
Performance feedback on asthma management, hypertension non-compliance control	328	44.1 (12.5)	0.99 (0.97 to 1.02)	0.64	0.46 (1.85)	2.28 (1.19 to 4.39)	0.01	40.35 (79.93)	1.33 (1.06 to 1.66)	0.01
Performance feedback on both hypertension non-compliance and asthma management	329	45.1 (13.8)	0.98 (0.96 to 1.01)	0.18	0.34 (1.31)	1.84 (0.94 to 3.60)	0.08	37.37 (59.95)	1.22 (0.98 to 1.53)	0.08

*Relative risk of pharmacies in the respective intervention group dispensing a SABA during the follow-up to a patient who had used >250 doses in the previous 90 days relative to pharmacies receiving no feedback on either measure.

†Relative risk of pharmacies in the respective intervention group billing for the provision of pharmacist services during the follow-up relative to pharmacies receiving no feedback on either measure.

Discussion

Statement of principal findings

A population-wide, randomised, intervention trial of audit and feedback to more than 1400 community pharmacies documented that the provision of comparative performance feedback alone led to a significant increase in the number of pharmacists’ services billed for the management of asthma. However, audit and feedback had no impact on the provision of pharmacist services for managing non-compliance with antihypertensive medication, or on the overall performance on the quality of medication-use measures related to non-adherence in patients with asthma or hypertension.

Strengths and limitations

The main strengths of this study are the large sample of pharmacies and the use of objective outcome measures. In particular, the ability to objectively measure the pharmacists’ provision of targeted services allowed the evaluation of the value of pharmacists’ provision of these services.21 At present, there is little literature available that evaluates pharmacists’ services, and in turn there is limited evidence to guide policy decisions.6 38 This has led to substantial variability across and within countries as to the services community pharmacists are authorised and reimbursed to provide.2 6 33 Mossialos et al 6 recognised the inherent difficulties in completing robust evaluations of the impact of expanded pharmacists’ services, and similar concerns lead to Patwardhan et al's conclusion that evaluations must include objective measures of pharmacists’ service provision and standardised outcome measures.21 Since this study included these two objective measures, the impact of pharmacists’ services could be evaluated. As many provinces and countries continue to expand the services pharmacists are authorised to provide, while creating and maintaining electronic databases to document and reimburse these services, the methodologies from this study could be used to evaluate the value and impact of these expanding services. Limitations include that we evaluated performance on only two quality of medication-use measures that differed in the medication-taking behaviours targeted (increased use of antihypertensive medications vs decreased use of asthma rescue medications). These two measures may not be representative of the overall performance at a pharmacy and ongoing analysis is evaluating whether the predictors of performance are consistent for the two different medication-taking behaviours targeted in our study. In addition, administrative databases are limited in the extent to which they can measure whether pharmacists provided a service but did not bill for it, whether patients actually consumed dispensed medications or the impact of the feedback and pharmacist's services on patient health and well-being.39–41

Interpretation

Our results of the impact of audit and feedback are similar to those reported for other healthcare professions that indicate variable impact of the provision of feedback on performance.23–25 Criteria for the effective use of feedback to improve performance have been defined, and begin with the recipients having confidence that the performance being measured is important, within their professional scope of practice and amenable to change by the services or care they provide.42 Recipients must also be convinced that the feedback is based on valid, reliable measures of performance and be able to understand the feedback provided.23 The opportunity to discuss their relative performance with the feedback providers enables thoughtful reflection, which is proposed to be instrumental to the integration of external feedback with self-perceptions of performance, and the subsequent acceptance, use and integration of feedback to improve performance.42 43 These latter steps require that recipients believe that performance improvement is possible, understand how to improve their performance, set performance goals and plan/take action to improve performance.42

For the current study, management of patient's adherence to antihypertensive and asthma medications is readily identified as important and a core responsibility of pharmacists, thereby supporting pharmacists’ acceptance of the significance of the performance feedback.44 Confidence in the measures reported was increased through the use of objective measures from pharmacy billing databases, rather than subjective self-reports, of both service provision and patient outcome. Collaboration with the provincial regulatory authority also increased pharmacists’ acceptance of the fairness and credibility of the outcome measures and reports. Initial work of the PQA documented that pharmacists are able to understand performance feedback that is based on medication-adherence reports, although they are less certain how they can improve their performance.27 45 Feedback that is associated with increased acceptance and incorporation into practice improvement is timely, individualised, consequential but non-punitive and ‘actionable’, which supports recipients’ understanding of how to improve their practice.23 42 The asthma reports in our study provided specific advice to review the use of inhaled corticosteroids and prepare a recommendation for adding these medications if appropriate. It may be that this recommendation was more actionable, leading to the increase in billing of pharmacists’ services for asthma management.

The low numbers of pharmacists’ services billed during the follow-up period of our study is consistent with the existing literature.21 46 Both these low numbers and the overall limited impact of our performance feedback on billing of pharmacists’ services targeted at managing patient non-adherence could be explained if pharmacists did not believe that the current process of providing written recommendations to the prescriber is an effective means of improving patient adherence.47 Our results show that increasing numbers of asthma-related recommendations did not lead to improved adherence. The origin of the practice of pharmacists billing for medication-related recommendations in Quebec dates back to 1978, with a goal of promoting the optimal use of medications.48 Pharmacists experienced in providing pharmaceutical opinions over the subsequent 5 years concluded that these written opinions were a good means of communication about a range of medication-related problems encountered in community pharmacy.49 Policy reviews in 1983 and 1992 aimed to focus the process on the provision of patient-specific recommendations and to decrease the administrative burden for pharmacists.48 The current process requires that pharmacists send their written treatment recommendations to the original prescriber, who receives them at a variable time point after having seen the patient—and when the patient is not present for further consultation. As no direct discussion or follow-up between the pharmacist and prescriber is required, the prescriber's response to the recommendation, actions taken with the patient to address medication adherence and the patient's response to these actions are not communicated to the pharmacist. These numerous steps, delays and lack of direct discussion all have the potential to decrease the likelihood that the pharmacist's written recommendations will have an impact on the patient's medication use. It would be valuable to determine the impact of audit and feedback when pharmacists’ services involve direct communication with prescribers or where pharmacists are authorised and reimbursed to take direct actions with patients to resolve their medication-use problems.50 The recently expanding scope of practice in a number of jurisdictions, including Quebec, that authorise pharmacists to modify prescription drug therapy offers rich combinations of advanced pharmacists’ services and detailed administrative billing databases that could serve as an ideal site for such evaluations.51 52

An additional factor that could have led to the low impact of the performance feedback is that the pharmacists were not provided the opportunity to discuss or reflect on their performance. Changes in the role of pharmacy practice inspectors to function primarily as practice mentors are providing opportunities for future studies to evaluate the impact of peer discussion in combination with performance feedback.53 Finally, when placed in the larger context of the factors known to influence pharmacists’ provision of professional services, it may be that lack of awareness of their relative performance is not a primary factor influencing pharmacists’ service provision. Consistent with the theory that there are multiple barriers and facilitators to the provision of pharmacists’ professional services, feedback of relative performance may be insufficient to overcome the more significant barriers related to, for example, relationships with prescribers, remuneration, insufficient time and lack of management support.46 54–56

Implications and future research

Given the evidence that the provision of targeted pharmacists’ services did not lead to improvements in medication use, modifications in the services provided by pharmacists and the associated required processes should be considered. Such modifications should be grounded in a conceptual framework that incorporates the theories of factors influencing pharmacists’ professional practice and the evidence supporting these theories.57 To date, much of community pharmacy practice research has focused on single influencers such as pharmacist competence, pharmacist motivation and reimbursement.5 58 Integrated frameworks that consider the range and source of influencers, including patient and context factors, may be more useful for understanding pharmacists’ practice and developing services that are effective at improving patients’ medication use and sustainable within the community pharmacy environment.55 59 60 Future research should also focus on using objective measures and strong methodologies to evaluate the effectiveness of services that are reimbursed—both new and old—to optimise the reimbursement of services that have a meaningful impact on health outcomes. Such research is of importance as provinces, countries and private insurers consider the value offered by community pharmacists’ provision of targeted professional services.