Evaluating an implementation programme for medication review with follow-up in community pharmacy using a hybrid effectiveness study design: translating evidence into practice.

OBJECTIVES: To evaluate an implementation programme of a community pharmacy medication review with follow-up (MRF) service using a hybrid effectiveness-implementation study design, and to compare the clinical and humanistic outcomes with those in a previously conducted cluster randomised controlled trial (cRCT).
SETTING: Community pharmacies in Spain. PARTICIPANTS: 135 community pharmacies and 222 pharmacists providing MRF to polymedicated patients aged 65 or over. INTERVENTION: The intervention was an implementation programme for the MRF service. A national level group was established, mirrored with a provincial level group. A series of interventions were defined (1) to engage pharmacy owners with the implementation model and (2) to provide training to pharmacists consisting of clinical case studies, process of MRF, communication skills and data collection methods and (3) practice change facilitators. PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcomes for the implementation programme were progress, reach, fidelity and integration. The secondary outcomes were number of medications, non-controlled health problems, emergency visits, hospitalisations and health-related quality of life, which were compared with a previous 6-month cluster RCT.
RESULTS: 55% of pharmacies reached the implementation phase and 35.6% remained in the testing phase at 12 months. A reach of 89.3% (n=844) was achieved. Fidelity average score was 8.45 (min: 6.2, max: 9.3) out of 10. The integration mean score was 3.39 (SD: 0.72) out of 5. MRF service outcomes were similar to the cluster RCT study; however, the magnitude of the outcomes was delayed.
CONCLUSIONS: The implementation of pharmacy services is a complex multifactorial process, conditioned by numerous implementation factors. In the absence of remuneration, the implementation of the MRF service is a slow process, taking at least 12 months to complete. TRIAL REGISTRATION NUMBER: CGFTRA-2017-01. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Community pharmacist-led medication review with follow-up (MRF) has the objective of improving medication management, managing chronic health problems and improving patients’ quality of life.

MRF service has previously been shown to be clinically effective and cost-effective in cluster randomised controlled trials.

The impact of a multilevel implementation programme was evaluated in 135 sites, with a 55% implementation success with high reach, fidelity and integration scores.

Service benefits were similar to those previously obtained in a cluster randomised controlled trial; however, the magnitude of the outcomes was delayed from 6 to 12 months.

A limitation of the study was the self-reporting nature of some variables for patients and pharmacists.

Background

Over the last two decades many research programmes have evaluated professional pharmacy services delivered from community pharmacies. Systematic reviews and meta-analysis are being published proving or attempting to evaluate the clinical and humanistic impact on patients and the economic benefits to healthcare systems.1 2 As new professional pharmacy services are being designed, randomised controlled trials (RCTs) continue to be undertaken in controlled environments.3 However, much less attention has been paid to the development and evaluation of implementation programmes associated with the widespread adoption and scaling up of these services. Nevertheless, national and provincial governments have incorporated pharmacy services as part of their healthcare policies. In many countries such as England, Canada, Australia and the USA, community pharmacies are being remunerated for a range of medication management and disease-related programmes and services.4–7

Implementation of innovations, new practices and services, which is often unsuccessful, is a complex and long-term process that requires holistic approaches tailored at multiple domains. The lack of translation of evidence to practice phenomenon is common across many different disciplines ranging from agriculture, health, education and teaching, to pharmacy, among others.8 Successful implementation requires the use of models and frameworks that provide a structured and sound theoretical approach to implementation processes and outcomes.9 Current knowledge on implementation of innovations in healthcare shows that an implementation framework is crucial to successful implementation and long-term sustainability. A commonly used framework, the Consolidated Framework for Implementation Research by Damschroder et al,10 was adapted by Moullin et al so as to be pharmacy discipline-specific. An evaluation framework was also proposed by Moullin et al to measure the effect of implementation programmes on professional pharmacy services.11

Community pharmacist-led medication review with follow-up (MRF), prioritised as a national pharmacy service by Spanish pharmaceutical organisations, is a service with the objective of improving medication management, managing chronic health problems and improving patients’ quality of life.12 MRF was designed and developed through consensus by a body of organisations which included universities and professional organisations and met under the auspices of the General Council of Colleges of Pharmacy.13 MRF can be classified as a type 3 or advanced medication review service using the Pharmaceutical Care Network definition of medication review services.14 A study on the impact of MRF has shown positive clinical, economic and humanistic outcomes.15 In 2019 a systematic review conducted by the National Institute for Health and Care Excellence in the UK on the cost-effectiveness of advanced pharmacy services was published confirming these findings.16

Due to MRF’s proven efficacy, the next phase of the research was to design, develop and evaluate an implementation programme which would be used to scale up to a national level. Some dissemination of MRF services had transpired during the national process of prioritisation of services. The prospect of patient benefits and increased professional satisfaction has not been sufficient to drive implementation nor had passive diffusion strategies such as training.17–19 There was also a lack of direct financial incentives to adopt the practice, an important factor particularly in the community pharmacy business setting.20 Importantly there is a high probability that widespread implementation had not been achieved because a comprehensive multilevel programme has not been used.21–23 Alongside using an implementation model, the literature suggests that combinations and more active strategies need to be employed particularly if the clinical, economic and humanistic benefits shown during its impact research phase are to be translated to the target population.18 The objective of the study was to evaluate the impact of an implementation programme for MRF in community pharmacies and to compare the clinical and humanistic outcomes with those achieved in the impact research phase.

Methods

The ‘Standards for Reporting Implementation Studies’ checklist with the accompanying explanation and elaboration documentation and guidelines were used in preparing this paper.24 However, we did not have sufficient data to report on item 8, that is, the ‘full characteristics of the target sites’, as there were 135 different sites, and on items 13 and 20, ‘economics’ of the standards.

Study design and context

A hybrid effectiveness-implementation design was used.25 The complete methodology of both the impact study and the implementation programme has been previously described.26 27 The implementation programme study was undertaken during 12 months in 2015–2016.

MRF intervention

The MRF service was the intervention to be implemented. MRF is a professional pharmacy service whose ultimate objective is to detect drug-related problems in order to prevent and solve negative outcomes associated with medications. The service is delivered according to the following phases: (1) patient recruitment; (2) first patient interview (with the objective of retrieving relevant clinical and medicines information); (3) comprehensive medication review (where drug-related problems and negative outcomes associated with medications are identified); (4) delivery of a care plan agreed with the patient and other healthcare professionals (targeted at the problems identified in phase (3)); and (5) follow-up visits (to monitor the outcomes achieved and assess potential new drug-related problems and negative outcomes associated with medications).

During this study community pharmacists did not have access to the formal clinical patient record. However, during the interview the pharmacists solicited hard copies of reports, which patients have as part of normal medical practice in Spain. These reports included diagnostic and analytical laboratory results. There were a number of ways in which the pharmacist evaluated the control of the health condition: data from the documentation provided by the patient, symptoms reported by the patient, in relevant cases measuring blood pressure and other tests, and from responses to disease-specific questionnaires. Using this clinical information, the pharmacist applied his/her clinical judgement to the decision.

At baseline patients were asked to provide the number of hospital admissions and emergency visits undertaken in the previous 6 months. At subsequent visits they were asked to report on the number of visits undertaken during the intervening time. In the case of hospitalisations, the patient unique identifier was used and the information was validated through hospital records as well as accessing the Diagnosis Related Group. In a previous work15 similar data were correlated with the influence of drug-related problems on the hospitalisation rate. In an additional study these data were also analysed with the opinion of three internal specialists having obtained a good correlation.28

Setting and study population

The service was to be implemented in community pharmacies in 11 Spanish provinces (A Coruña, Albacete, Ciudad Real, Córdoba, Gipuzkoa, Granada, Guadalajara, Huelva, Las Palmas, Santa Cruz de Tenerife and Valencia). Community pharmacy owners were offered to participate in the study and a maximum of 14 pharmacies were randomly selected in each province, according to the following criteria: (1) pharmacy with patients aged 65 or over and (2) pharmacy owner available to attend an initial training and willing to implement the service. Community pharmacists working in the participating community pharmacies offered eligible patients to enter the study. Eligible patients were aged 65 or over using five or more chronic medications.

Implementation framework

The Framework for the Implementation of Services in Pharmacy model,11 which had been adapted from the consolidated framework by Damschroder et al,10 was used as part of the implementation programme. In this model the progress of the service implementation passes through different phases: exploration, preparation, testing and implementation. For this study each phase was operationally defined as follows: exploration: the number of pharmacies (a pharmacist can only own one pharmacy in Spain) who had decided to enrol in the programme; preparation: the number of pharmacies represented by pharmacists attending the face-to-face training: and the testing phase, which started when a pharmacy had at least one patient receiving the service with at least one pharmacist intervention (with the patient or with the physician). Finally, the implementation phase was reached when there were at least seven patients receiving the service. This number of patients was based on the results of the impact study which indicated that a pharmacy without extra resources, that is, without payment, could maintain this number of patients.

Implementation strategy

At the national level an implementation group with senior personnel provided high-level political and management leadership. At the meso level, this was mirrored with the establishment of a provincial-level college of pharmacy group with the same objectives as the national group. For the meso level a series of interventions were designed. Pharmacy owners were requested to attend a 4-hour information session which included the descriptions of international modes of professional services, the implementation model, and the professional and business advantages of implementing the MRF service. Over 3 days, face-to-face training for service providers consisted of clinical case studies, process of MRF, communication skills and data collection methods. Third, practice change facilitators (PCF) were employed by local colleges of pharmacy as part of the implementation programme. These PCFs were trained for 5 days and were educated on the implementation model, motivational and communication skills, and to identify barriers and facilitators for the practice change in situ in the pharmacy. PCFs were provided with a list of 43 implementation factors previously identified from the literature and from the impact study that were relevant to the implementation of MRF services (see online supplemental file 1). An observation guide was designed to allow PCFs to identify, systematically and individually in each pharmacy, the determinants, that is, barriers or facilitators and their causes. The PCF also quality-controlled data collection. Participant observation, through direct observation, collective discussion and document analysis were conducted during these visits in order to gain an understanding of relevant implementation factors within the pharmacy. Post visit, the PCF analysed the data collected. An electronic documentation programme was provided where all these activities could be formally documented and analysed by PCFs and the research team. PCFs visited the pharmacy monthly and also provided telephone and email support. The research team held regular Skype meetings with PCFs to assist in the discharge of their duties.

Evaluation of implementation process and outcomes

The Moullin et al 11 evaluative framework (figure 1) was used to assess the progress of the implementation. The implementation outcomes measured were progress of the implementation, that is, stages, rate and reach. The ‘climate indicators’ are reported in a separate paper. The validated questionnaires published in English by Moullin et al 29 were used to measure fidelity and integration. The fidelity and integration questionnaires were to be completed by each pharmacist provider of the MRF service.

Figure 1

Evaluative framework as adapted from Moullin et al. 11

Evaluation of service outcomes

Finally, clinical and humanistic outcomes similar to those of the impact research phase were measured. A medication history was undertaken by the pharmacist as part of the MRF service and the number of medications extracted from these interviews. Each patient’s clinical history was also taken and the patient’s results compared with approved clinical guidelines to determine whether the underlying chronic disease was controlled or not. The number of emergency visits and hospitalisations was provided through a patient interview during the 6 months prior to receiving the service and at two timepoints (6 and 12 months) during the service. For perceived health-related quality of life, European Quality of Life Scale-5D and Visual Analogue Scale were used.30

Data analysis

To describe quantitative variables, mean and SD were used. Student’s t-test was used to test independent variables. To describe qualitative data, frequency measures (percentages) were used. χ2 and Fisher’s tests were used to analyse the relationship between categorical variables.

Consent to participate

Written consent was obtained from all participants.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

A total of 246 pharmacies responded to the request for information on the project, with 155 pharmacies from 11 provinces accepting to participate in the implementation programme. Due to restriction of human resources 135 were randomly selected and participated in the implementation programme. Training sessions were attended by pharmacy owners and pharmacists from these pharmacies.

Progress of implementation

Within the first month 63% (n=85) of pharmacies had undertaken an intervention with either a medical practitioner and/or a patient and were thus classified to be in the testing phase, while 2.2% (n=3) had moved to implementation of the service, meeting the full quota of seven patients. Between the third and fourth months, the largest percentage (76.3%, 103) of pharmacies had moved to testing phase, with 13% (n=18) moving to implementation phase. Only seven pharmacies remained in the preparation phase by month 5 (figure 2). The percentage of pharmacies in the implementation phase steadily rose to a maximum of 55% of the original 135 reaching this phase, with 35.6% (n=48) remaining in the testing phase 12 months after the commencement of the programme. The first dropouts, three pharmacies, occurred in month 5, and after 1 year of the programme there were a total of 13 dropouts (3.4%).

Figure 2

Progress of pharmacies through the different phases of the Framework for the Implementation of Services in Pharmacy model.

Reach in implementation programme

As 135 participating pharmacies and at least seven patients per pharmacy were expected, 945 was the target patient reach. At the end of the study, a reach of 89.3% (n=844) was achieved. After the first month of the study, 210 patients (approximately 25%) were recruited, with 50% recruitment within 5 months of commencing the programme and 75% within 7 months (figure 3).

Figure 3

Reach achieved in the implementation programme.

Fidelity to the MRF service

There were 222 pharmacist providers in the study (1.64 per pharmacy). A total of 161 questionnaires on fidelity were completed by participating pharmacist providers, resulting in a response rate of 72.5%. A total of 145 questionnaires were included in the final analysis (90.0%), with 16 (9.9%) discarded due to missing data. The overall average score of the items on the questionnaire was 8.45 (min: 6.17, max: 9.33) (table 1). The stage with the highest score was ‘general service’ aspects and the lowest was ‘service offer’. The validated questionnaire in Spanish is provided in online supplemental file 2.

Table 1

Fidelity after 12 months of MRF service provision

Fidelity at each stage of the MRF service	Mean	SD
Service offer	7.95	1.13
First patient interview	8.62	0.42
Determination of patient’s current health status	8.74	0.19
Case study phase	8.60	0.17
Evaluation phase	8.87	0.36
Intervention phase	8.37	0.42
Successive interviews and evaluation of outcomes	8.19	0.10
General service aspects	8.88	0.39

MRF, medication review with follow-up.

MRF integration

From the 222 participating pharmacist providers, 108 questionnaires were completed, resulting in a response rate of 48.6%. The average score obtained for the total number of items was 3.39 (SD: 0.72) out of 5 (table 2). The best rated dimension was resources, with 4.39±0.63, and the worst rate was evaluation, with 2.99±0.11, with item 31 (‘The operation of the Service is evaluated periodically’, 2.87±1.22) performing the worst. The validated questionnaire in Spanish is provided in online supplemental file 3.

Table 2

Integration achieved at 12 months by the implementation programme

Integration dimensions	Mean	SD
Routinisation	3.07	0.99
Teamwork	3.07	0.99
Resources	4.39	0.63
Evaluation	2.99	0.11

Service outcomes

The differences in the sample characteristics and health outcomes for the impact study undertaken for 6 months and the implementation programme at 6 and 12 months are shown in tables 3–8. Six hundred and eight patients were recruited and followed for 6 months in the implementation programme. Of these patients, data for the 12-month point were available only for 176 patients as the fieldwork ended 12 months from initial commencement of the programme.

On comparing the baseline patient characteristics for patients followed for 6 months, there were significant differences in the mean number of medications (9.05 (3.0) vs 7.74 (2.5)) and the number of health problems (5.98 (2.0) vs 4.96 (1.8)), both of which were higher for recruited patients in the implementation programme. For patients followed for 12 months in the implementation programme, in addition to these significant differences, differences were also found in the mean age of patients approximately 2 years (73.4 (5.7) vs 75.3 (6.5)) (table 3).

Table 3

Baseline characteristics of the sample: impact study and implementation programme (6 and 12 months)

	Baseline for patients completing 6 months					Baseline for patients completing 12 months
	Impact (n=688)		Implementation (n=608)		P value	Implementation (n=176)		P value
	Mean	SD	Mean	SD		Mean	SD
Age (years)	75.3	6.5	75.4	6.4	0.978	73.4	5.7	≤0.05
Gender (female), n (%)	409	60.1	517	59.2	0.718	99	56.9	0.448
Number of medications	7.74	2.5	9.05	3.0	<0.001	9.38	3.1	<0.001
Number of health problems	4.96	1.8	5.98	2.0	<0.001	6.08	2.2	<0.001
Non-controlled health problems	1.46	1.3	1.39	1.4	0.351	1.57	1.5	0.159
Percentage of non-controlled health problems (%)	29.4		23.2		NA	25.8		NA
Quality of life related to health	64.97	18.6	64.53	19.2	0.676	63.6	19.6	0.389

NA, not applicable.

Even though the baseline number of medications was statistically significantly higher in the sample for the implementation programme (9.05 (3.0)) and (9.38 (3.1)) than the impact study population (7.74 (2.5)), the pattern of decrease in the number of medications in the implementation programme did not become significant until the 12-month point (table 4).

Table 4

Number of medications: impact study (6 months) and implementation programme (6 and 12 months)

	Baseline mean		6 months		Mean change		P value	12 months		Mean change		P value
	Mean	SD	Mean	SD	Mean	SD		Mean	SD	Mean	SD
Impact study (n=688)	7.74	2.5	7.45	2.4	−0.29	1.3	≤0.05	NA		NA		NA
Implementation programme (n=608)	9.05	3.0	8.99	3.1	−0.06	1.6	NS	NA		NA		NA
Implementation programme (n=176)	9.38	3.1	9.23	3.2	−0.15	1.2	NS	8.99	3.4	0.39	2.3	≤0.05

NA, not available; NS, not significant.

At baseline, there were significant differences in the number of non-controlled health problems between the three samples. The decrease at 6 months, although significantly different, was lower in the implementation programme than in the impact study (−57.5% vs −38.1%). At 12 months, the percentage decrease was of similar magnitude (−57.5% vs −52.2%) between the impact study and at 12 months for the implementation programme (table 5).

Table 5

Non-controlled health problems: impact study (6 months) and implementation programme (6 and 12 months)

	Baseline*		6 months		Mean change		Percentage change	P value	12 months		Mean change (SD)		Percentage change	P value
	Mean	SD	Mean	SD	Mean	SD			Mean	SD	Mean	SD
Impact study (n=688)	1.46	1.3	0.62	0.9	−0.84	1.1	−57.50	≤0.05	NA		NA		NA	NA
Implementation programme (n=605)	1.39	1.4	0.86	1.1	−0.53	1.2	−38.10	≤0.05	NA		NA		NA	NA
Implementation programme (n=176)	1.57	1.5	NA		NA		NA	NA	0.75	1	−0.82	1.4	−52.20	≤0.05

*Baseline data are the number of non-controlled health problems at the initiation of the study.

NA, not available.

The percentage decrease in emergency visits at 6 months, even accounting for baseline differences, was lower in the implementation programme than in the impact study (−49.1% vs −43.0%). At 12-month point, the percentage decrease was greater by about 10% than the impact study (−49.1% vs −56.8%) (table 6).

Table 6

Emergency visits: impact study (6 months) and implementation programme (6 and 12 months)

	Baseline		6 months		Percentage change	P value	Between 6 and 12 months		Percentage change	P value
	n	%	n	%			n	%
Impact study* (n=667)	193	28.9	90	14.7	−53.40	≤0.05	NA		NA	NA
Implementation programme† (n=575)	121	20.2	69	12	−43.00	≤0.05	NA		NA	NA
Implementation programme‡ (n=160)	44	25.4	NA		NA	NA	19	11.9	−56.80	≤0.05

*Impact study baseline is the number of emergency department visits in the 6-month period prior to the study, and from baseline to 6 months are the emergency department visits during the 6 months of the study.

†Implementation programme (6 months) baseline is the number of emergency department visits in the 6 months prior to the study. The 6-month period is from baseline to 6 months.

‡Implementation programme (12 months) baseline is the number of emergency department visits during the first 6 months prior to the study. The 12 months is the number of emergency visits during the 6–12 months of the programme.

NA, not available.

There were no significant differences at baseline for the reported number and/or percentage of hospitalisations in the three samples (table 7). Significant differences were seen at 6 and 12 months in the decrease in hospitalisations, reaching a decrease of −63.2% in hospitalisations in the implementation programme at 12 months.

Table 7

Hospitalisations: impact study (6 months) and implementation programme (6 and 12 months)

	Previous 6 months		6 months		Percentage change	P value	Between 6 and 12 months n (%)		Percentage change	P value
	n	%	n	%			n	%
Impact study* (n=663 baseline, n=613 at 6 months)	89	13.4	36	5.9	−59.6	≤0.05	NA		NA	NA
Implementation programme† (n=594 baseline, n=570 at 6 months)	64	10.8	41	7.2	−35.9	≤0.05	NA		NA	NA
Implementation programme‡ (n=170 at 6 months basal, n=154 at 12 months)	19	11.2	NA		NA	NA	7	4.5	−63.2	≤0.05

*Impact study baseline is the number of hospitalisations in the 6-month period prior to the study. The 6-month period is from baseline to 6 months.

†Implementation programme (6 months) baseline is the number of hospitalisation visits in the 6 months prior to the study. The 6-month period is from baseline to 6 months.

‡Implementation programme (12 months) baseline is the number of hospitalisations visits during the first 6 months prior to the study. The 12 months is the number of hospitalisation visits during the 6–12 months of the programme.

NA, not available.

There were statistical differences in changes from baseline quality of life in all three samples, with an increase of 6.74 (18.7) in the 12-month point of the implementation programme. At the 6-month time, the change in the impact study was higher than that of the implementation programme (5.51 (15.3) vs 2.89 (17.0)) (table 8).

Table 8

Health-related quality of life: impact study (6 months) and implementation programme (6 and 12 months)

	Baseline		6 months		Percentage change		P value	12 months		Percentage change		P value
	HRQL	SD	HRQL	SD	%	SD		HRQL	SD	%	SD
Impact study (n=688)	64.97	18.5	70.48	17.1	5.51	15.3	≤0.05	NA		NA		NA
Implementation programme (n=597)	64.53	19.2	67.42	18.6	2.89	17	≤0.05	NA		NA		NA
Implementation programme (n=170)	63.1	19.5	NA		NA		NA	69.84	17.2	6.74	18.7	≤0.05

HRQL, health-related quality of life; NA, not available.

Discussion

This study in community pharmacies provided the results of the implementation process and the evolution of pharmacies through the different stages of an implementation model for the MRF service. In community pharmacies, the application of a theoretical implementation model following a rigorous hybrid effectiveness design is novel. This approach allowed the analysis of the effectiveness of the implementation programme in a structured holistic way, with the aim of not only evaluating and measuring implementation effectiveness but importantly the clinical benefits previously identified in the impact study on the target population. At the same time, it allowed the identification of areas for improvement in the future, including the type of training provided at different levels and the length of time required for the scaling-up programme.

By the end of the 12-month programme, just over half of the original 135 pharmacies reached the predefined implementation stage. Over a third of the pharmacies remained in the testing phase, highlighting the need for both a longer scaling-up period and the necessity to review the support systems used. Reach is an important variable that is critical to measure in order to ensure testing of the universality of the programme in target populations. In many ways it determines the success of the implementation programme. Most participating pharmacies (80%) recruited at least a patient within the service after 2 months of the programme, indicating that initial uptake was not an issue. However, the overall reach achieved was 90%, indicating the variability of recruitment by pharmacies. An average of 7.5 patients were reached per pharmacy, indicating the success of the preparation intervention. Service delivery, that is, recruitment, between months 0 and 3 accounted for a third of the patients participating, and by the end of month 6 two-thirds of the patients were receiving the medication review programme with follow-up. This speed of recruitment may be due to the motivation with which the majority of the pharmacist providers adopted the service. Their wish to differentiate and professionalise the pharmacy has been identified as a relevant facilitator for the implementation of new pharmacy service.31 While it is valid to suggest that this motivation is essential to implement any complex service as the MRF service, motivation as an isolated factor does not ensure an effective implementation. The presence of remuneration for provision of services has been described as an essential factor in the implementation process.31 Therefore, in its absence, the implementation of the MRF service may be considered challenging. Lack of remuneration of the service, due to the economic nature of the community pharmacy, could have had a decisive influence on the progress of implementation; however, the magnitude of this effect is unknown.

Community pharmacist providers self-reported a high fidelity to the service, that is, they reported delivering the service according to how it was protocolised. The outcome indicators in the comparison of the impact study versus the implementation programme provide additional face validity to these self-reports. The major problem reported in many impact studies is the heterogeneity of the intervention; however, in this programme the process and dose of the intervention appear to be uniform. It is important for usual practice to achieve high quality and have a uniform and standardised service to the target population. Fidelity scores can be used at the level of the individual pharmacy service provider to reinforce those weakest or most difficult areas for the pharmacist during the delivery of the service. It can be used as a surrogate indicator of the quality and appropriateness of the educators, pedagogy and content of the training programme in the preparation stage. The work of PCFs can also be assessed using this fidelity measure.

The extent of integration of the service throughout the programme seemed appropriate with mean scores of about 4 out of 5 in a Likert-type scale. It should be noted that the dimension that obtained the lowest score was related to the continuous evaluation of the service within each participating pharmacy, with an average score of 2.99 (SD: 0.11), reflecting that in current practice practitioners are not accustomed to evaluating their practice. The routinisation of the service, with an average score of 3.07 (SD: 0.99), also indicated some difficulties in adopting new practices. These results may be related again to the absence in current practice of a portfolio of services that would constitute an essential part of the daily activity of community pharmacy. However, improvements in the training programmes should be considered in any future implementation programme. Nevertheless, these results provide some evidence that it is possible to integrate the MRF service on a large scale to usual practice in Spain.

The evaluation of the clinical, economic and humanistic effectiveness of this MRF service innovation was the main objective of the first phase of the research. However, a critical point of any implementation process and/or programme32 is to ensure that any potential adaptation for the generalisation of the service to the pharmacy population does not significantly diminish the previously identified patient and system benefits.15 26 28 One of the major innovations and added value of our implementation programme was evaluating these clinical and humanistic benefits. Interestingly, patients recruited during the implementation programme were more complex, that is, with higher statistically significant baseline number of medications and percentage of uncontrolled health problems. The patients followed up for 12 months were also more complex. The probable reason is that pharmacists are identifying higher risk patients for the longer follow-up period and/or these patients are remaining longer in the programme. Despite this issue the results obtained with the implementation programme follow the same trend as those in the impact study, a positive impact on the clinical and humanistic outcomes both at 6 and 12 months of providing the service. However, the magnitude of the outcomes is delayed. Most outcomes at 6 months of the impact study show a slightly higher effect than those at 6 months of the implementation programme. For example, the decline in emergency visits in the impact study and implementation programme was reduced by 53.4% and 43.0%, respectively. At 12 months of the implementation programme the percentage of visits had been reduced by 56.8%. A similar trend can be seen in the reduction in the number of medications, in the impact study at 6 months by 0.29, and in the implementation programme at 6 months by 0.06 and at 12 months by 0.39. These results obtained at 12 months of MRF are very similar to the findings of Jódar-Sánchez et al.33 In their study of 12 months, the MRF service was delivered to patients 65 years of age or older institutionalised in geriatric residences in the autonomous community of Andalucía. It achieved a statistically significant decrease of 0.47 (p≤0.001) in the average number of medications. Other studies confirm these findings.34–37 Despite this decline in medications, the impact study reported a decline in the number of uncontrolled health problems of 57% at 6 months and the implementation programme of 52% at 12 months.

One of the more interesting effects is the number of uncontrolled health problems of patients at baseline. It would be logically assumed that since these are polymedicated patients their chronic diseases should be adequately managed. However, it is evident that this is not the case since both the impact study and the implementation programme at baseline report polypharmacy with limited therapeutic results, reinforcing the need to implement services such as MRF.

Limitations

There were various limitations to this study, in particular the self-reporting nature of some variables for patients and pharmacists. The number of reported hospital admissions was verified with hospital records for the impact study but not for the implementation study. To ensure comparativeness, the raw data for both studies have been reported in this paper. For the impact study, the data published by Jódar-Sánchez et al 15 for the reported number of hospitalisations excluded any prescheduled hospitalisations. However, in the Malet-Larrea et al the association of the number of hospitalisations with drug-related problems as ascertained by a panel of medical internists was reported. Due to the unavailability of the causes of emergency visits, it was not possible to link them to Drug Related Problems (DRP) in both studies. The determination of controlled or not controlled health problems, in a small number of cases, where objective clinical data were unavailable, was dependent on the clinical judgement of the pharmacist. The fidelity of the intervention was, for practical reasons, only episodically observed by the PCFs.

Conclusion

The approach taken in this study allowed for the analysis of the effectiveness of the implementation programme in a holistic way, with the aim of identifying areas for improvement, including the training provided at different levels and the length of the final scaling-up programme to the target population. The resource allocation and the closer monitorisation during a cluster randomised trial than during an implementation programme may account for the delay in achieving clinical and humanistic benefits. The implementation of professional pharmaceutical services is a complex multifactorial process, conditioned by numerous implementation factors at different levels. What appears evident is that in the absence of remuneration, the implementation of the MRF service is a slow process, taking at least a minimum of 12 months. Without remuneration its long-term sustainability can also be questioned. The use of a structured, multilevel, theory-based approach permitted a practical and rigorous assessment of the programme in a holistic manner, providing areas for improvements in the future.

The evidence provided in this comparative study should encourage policy makers to allocate resources to the healthcare system to adopt MRF with an implementation programme due to the high morbidity of an elderly population with multiple chronic diseases, which is increasingly challenging the sustainability of most countries’ healthcare systems.