Post hoc evaluation of a common-sense intervention for asthma management in community pharmacy.

OBJECTIVES: The aim was to evaluate a common-sense, behavioural change intervention to implement clinical guidelines for asthma management in the community pharmacy setting.
DESIGN: The components of the common-sense intervention were described in terms of categories and dimensions using the Intervention Taxonomy (ITAX) and Behaviour Change Techniques (BCTs) using the Behaviour Change Wheel (BCW), Capability, Opportunity and Motivation-Behaviour (COM-B) System and Behaviour Change Techniques Taxonomy (BCTTv1). The retrospective application of these existing tools facilitated evaluation of the mechanism, fidelity, logistics and rationale of the common-sense intervention. INTERVENTION: The initial intervention study was conducted in 336 community pharmacies in the metropolitan area of Perth, Western Australia. Small-group workshops were conducted in 25 pharmacies; 162 received academic detailing and 149 acted as controls. The intervention was designed to improve pharmacy compliance with guidelines for a non-prescription supply of asthma reliever medications.
RESULTS: Retrospective application of ITAX identified mechanisms for the short-acting β agonists intervention including improving knowledge, behavioural skills, problem-solving skills, motivation and self-efficacy. All the logistical elements were considered in the intervention design but the duration and intensity of the intervention was minimal. The intervention was delivered as intended (as a workshop) to 13.4% of participants indicating compromised fidelity and significant adaptation. Retrospective application of the BCW, COM-B system and BCTTv1 identified 9 different behaviour change techniques as the rationale for promoting guideline-based practice change.
CONCLUSIONS: There was a sound rationale and clear mechanism for all the components of the intervention but issues related to logistics, adaptability and fidelity might have affected outcomes. Small group workshops could be a useful implementation strategy in community pharmacy, if logistical issues can be overcome and less adaptation occurs. Duration, intensity and reinforcement need consideration for successful wider implementation. Further qualitative evaluations, triangulation of research and evaluations across interventions should be used to provide a greater understanding of unresolved issues. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

This research used a cutting-edge approach to implementation science innovation by demonstrating a successful and practical application of the Behaviour Change Wheel framework and Intervention Taxonomy in the complex setting of community pharmacy, where heterogeneous studies and small effects from interventions provide little information on how to best improve practice.

The method for this research was strengthened by the involvement of two researchers in the coding of the behavioural components in all steps.

A limitation of the research was that the choice of tools used in the evaluation might not have been optimal for deciphering the intervention. There are many evaluation tools in the literature and other tools may have yielded different information or results more useful to informing future implementation.

There are inherent disadvantages in methodology involving post-hoc evaluation. There was also a lack of participant feedback and perception data that could have been useful in retrospective application. For instance, to understand the dose of the intervention achieved with the altered implementation strategy.

Background

Evaluation of interventions

Evaluation is a crucial component of implementation science. Evaluations may look at process, outcome, impact and economics of an intervention, and ideally the choice of evaluation method should suit the purpose of the evaluation.1 A variety of tools including frameworks, taxonomies and models have been developed to assist in the evaluation of interventions, all with their own strengths and limitations.2 3 There is no single ideal evaluation tool or method and more work is needed to understand the comparative benefits of the many tools already developed.3 Thorough evaluation is an ongoing process and multiple analyses are often required to inform key decisionmakers and policy directives.1 2

Process evaluation considers all aspects of delivering an intervention; this encompasses the concepts of logistics and fidelity. Logistics refers to the detailed organisation and implementation of a complex operation.4 Logistics in implementation science considers the ability to operationalise the research: how to plan and manage the intervention including resources and personnel. Fidelity is defined as the extent to which interventions are delivered as planned.5 When an intervention is not delivered as intended (eg, low participant responsiveness, low intervention dose), then the effect size and statistical power of the research can be diminished.5 Analogous to fidelity is the concept of adaptability. If an intervention has adaptive elements, it can be modified without compromising the integrity or results of the intervention.5

One of the many frameworks identified in a scoping review of classification schemes (including taxonomies, lists and frameworks)3 was the Intervention Taxonomy (ITAX).6 The taxonomy outlines the delivery characteristics and goals of an intervention3 6 and is relevant to the concepts of logistics, fidelity and understanding the implementation process. ITAX has the ability to characterise complex, multicomponent behavioural studies.6 The taxonomy has previously demonstrated its validity and acceptability; it was developed using literature review as the methodological basis and has been peer reviewed and pilot tested.3 Although it is a relatively simple to use checklist, it is comprehensive and focuses on elements that might be applicable to understanding outcomes.6 The aim of creating the taxonomy was to improve intervention design and execution and allow for comparison across studies.6 It was used in this way by the Resources Enhancing Alzheimer's Caregiver Health consortium to examine multiple intervention studies.7 These characteristics make ITAX an appropriate tool for process evaluation in post hoc analysis, rather than simply a checklist for use at the intervention design stage. The taxonomy provides greater insight than just the intervention process. In detailing goals of an intervention, it distinguishes between the strategies used and also considers the mechanisms of action or underlying rationale of the intervention.6

An understanding of these concepts is important in deciphering outcomes. Were outcome failures attributable to poor implementation process or to a failure in the theory of intervention design, be it implicit (common-sense approach) or explicit (formal theory-based approach)?8

Interventions in healthcare have traditionally been designed using a pragmatic approach. However, the healthcare system is complex and interventions to improve practice tend to be equally complex.9 They often achieve only variable or modest results.10–12 Furthermore, even when interventions are successful, they can be difficult to document, sustain, reproduce in new settings and implement on a larger scale.10 13 There are concerns that common-sense interventions have poorly developed rationales for achieving outcomes.14 These challenges have led to increasing support for the use of a systematic approach and behavioural theory in the design of clinical guideline implementation interventions.15–18 However, not all scientists believe that the complexities of a theory-based approach are superior to a pragmatic, logical, ‘common sense’ approach, based on empiric evidence.19 20 Their argument is that all interventions are based on ‘theory’ but some are implicit, informal and use a common-sense rationale, while others are formalised.14 This formalised theoretical approach does not necessarily improve outcomes.14 20

In advocating the use of theory in intervention design, the problem becomes a question of which theory to use. The antagonists would argue that this is one of the key flaws and that many theories overlap or contradict each other.19 The Behaviour Change Wheel (BCW) addresses this problem by synthesising 19 behaviour change frameworks identified in the literature into one comprehensive framework.21 Central to the BCW framework is the Capability, Opportunity and Motivation-Behaviour (COM-B) model which analyses behaviour change in terms of capability, opportunity and motivation.21 Complementing the BCW framework is the Behaviour Change Techniques Taxonomy (BCTTv1).21 The taxonomy describes 93 specific behaviour change techniques and allows for standardised reporting of interventions.21 These tools are novel and their usefulness has not been fully established.3 18 Proponents of a ‘common sense’ approach believe that until the use of theory has been demonstrated to be superior, it should remain a personal choice whether researchers adopt this method or not.19

An alternative, newer application of the use of theory is to deconstruct and retrospectively analyse common-sense interventions to evaluate and improve on preliminary research.7 18 22 This provides an understanding of the theoretical underpinning of the intervention, described in terms of the techniques used to change behaviour. It can be a useful method in determining whether poor outcomes of a common-sense intervention were due to a deficiency in the rationale. It can also assist in determining unnecessary elements that add to the cost or complexity of an intervention.

A community pharmacy intervention to implement asthma guidelines

Guidelines specific to community pharmacists are relatively new but under increasing development as pharmacists expand their clinical role in this setting.12 However, there is very little understanding of how the existing research on implementation science relates to community pharmacy, with most studies undertaken in hospital settings or in general medical practice.11 23 A systematic review of clinical guideline implementation to community pharmacy has determined that interventions in this setting have only yielded mixed and moderately effective results until now.12 Despite many in the scientific community now advocating for the use of theoretical frameworks, very few studies in this setting are grounded by the use of theory, and thus there is little evidence of the benefit of this approach.12 Also, no studies have used the BCW framework in community pharmacy research at this point in time.

In 2011, new guidelines for the provision of a Pharmacist Only medicine: short-acting β agonists (SABA guidelines) were endorsed by stakeholders and distributed to pharmacists by the Pharmaceutical Society of Australia and The National Asthma Council of Australia.24 SABAs are inhaled ‘reliever’ medications used to alleviate the symptoms of asthma, such as wheezing and breathlessness. However, inappropriate over-reliance on SABAs can put patients at risk of serious asthma exacerbations and even death.25 26 In Australia, legislation allows patients to purchase SABAs under the supervision of a pharmacist, without a prescription and without necessarily seeing a doctor. Thus, compliance with these guidelines and the role of community pharmacists in Australia is particularly significant. They may be the only health professional with a chance to intervene when patients inappropriately self-manage their asthma with non-prescription ‘reliever’ medications. Despite this crucial role, research indicates that there are many barriers to SABA guideline-based practice and optimal asthma management.27 28 This practice deficit was addressed by a common-sense intervention in 2013. The intervention aimed to formalise the role of pharmacy assistants, improve internal referral from pharmacy assistants to pharmacists and improve medical referral of patients with poorly controlled asthma, in the supply of non-prescription SABAs.29 The results of the intervention were positive but variable and require consideration on how they can be improved before wider implementation is appropriate.29

The aim of this research was to evaluate a common-sense intervention to implement asthma guidelines in the community pharmacy setting.

The objectives were to:

Perform a process evaluation using a taxonomy to evaluate the mechanisms for producing outcomes, fidelity and logistics

Deconstruct the common-sense intervention into behavioural change techniques to evaluate the implicit theory (rationale) of the intervention

Comment on the ability of these methods to refine a common-sense intervention to improve outcomes and improve suitability for wider implementation.

Methods

Evaluation tools

Triangulation methods involving two different approaches, using formalised tools, were chosen for evaluation of the common-sense SABA guideline implementation intervention designed by Watkins et al:29

The ITAX was retrospectively applied to evaluate the mechanism (how the intervention was designed to achieve outcomes), fidelity (how well the intervention was delivered as planned) and logistics (how the intervention was delivered).6

The theoretical rationale of the common-sense intervention was examined through retrospective application of the BCW framework, the COM-B model and BCTTv1 described in the work by Michie et al.21 30

Generally, these taxonomies, frameworks and models are used prospectively to ensure that all the important elements are considered in the intervention design process. The novel retrospective use of these tools involved the same process by considering each component of the SABA guideline intervention. Authors were required to consider the characteristics of the SABA guideline intervention in terms of the terminology and elements detailed in the checklists and worksheets relevant for each tool. The outcomes resulting from the retrospective application of the tools included categories consisting of dimensions (ITAX) and behaviour change techniques (BCW, COM-B & BCTTv1).

Retrospective application of ITAX

Application of ITAX involved completion of a taxonomy checklist6 to detail the SABA guideline intervention. The taxonomy considers various dimensions of an intervention organised into two broad categories: delivery characteristics and intervention characteristics. The dimensions were used to consider: the mechanism, fidelity and logistics of the SABA intervention, with a view to investigate possible future improvements. The checklist was completed by the primary author (KW) and checked by a second author (CS). Consensus was reached by discussion.

Retrospective application of the BCW framework, the COM-B model and BCTTv1

Application of the BCW framework is outlined in a guidebook that contains a series of worksheets based on eight steps.21 The process involved three stages encompassing the eight distinct steps (figure 1).21 Stage 1 included steps 1–4 in the guidebook and involved understanding the behaviours to be influenced. Retrospective application required identifying and analysing the specific behaviours that needed to change to enhance SABA guideline compliance and facilitate internal referral of patients from pharmacy assistants to pharmacists. Stage 2 included steps 5 and 6 from the guidebook and looked at the possible options for intervention. Stage 3 included steps 7 and 8 from the guidebook and decided on the content and implementation possibilities. Undertaking these steps provided an understanding of whether the chosen intervention was sensible and feasible to address the relevant behaviours identified and whether the implementation strategy chosen was reasonable with a sound basis for success.

Figure 1

Application of a theoretical framework to intervention design: THE BCW and COM-B system. BCW, Behaviour Change Wheel.

The COM-B model21 is used to assess capability, opportunity and motivation (figure 2). This model was applied as part of step 4. After identifying a number of behaviours that would improve SABA guideline compliance (steps 1–3), COM-B was applied to each one as part of a behavioural analysis, to determine what needed to change for the behaviours to be supported.

Figure 2

COM-B Model. COM-B, Capability, Opportunity and Motivation-Behaviour.21

Step 5 identifies intervention functions, which are the broad categories of methods designed to address behaviour change (eg, education, persuasion). Retrospective application involved looking at each element of the SABA intervention to ascertain its function in changing behaviour. The Affordability, Practicability, Effectiveness and cost-effectiveness, Acceptability, Side-effects/safety and Equity (APEASE) criteria21 were considered at this stage to check the utility of each function. The APEASE criteria were also used at steps 6, 7 and 8, which considered utility of policy categories, behaviour change techniques and the delivery mode for the intervention.

All worksheets were completed by the primary author (KW) and checked by a second author (LS) with previous experience using the tools. Disagreements in coding were resolved by discussion and consensus.

Results

The ITAX Checklist was populated with information regarding the common-sense SABA guideline intervention. A summary of the deconstruction into categories and dimensions is shown in table 1.

Table 1

SABA guidelines implementation: summary of intervention using ITAX

Dimension	Definition	Options checklist	Intervention logistics and characteristics of the SABA implementation
Delivery characteristics
Mode	Method of contact between interventionist and participant	Face to face (individual or group) Telephone (individual or group) Internet (individual or group) Video/CD instruction Telephone contact with computer Mailing of written material Personal digital assistant, mobile phone	Face to face (workshop or academic detailing visit) Video—motivational videos
Materials	Materials used in the delivery of the intervention	Manuals/workbooks Information sheets/checklists Pamphlets Videotapes Audiotapes CDs/DVDs Assistive devices Internet	Information sheets/checklist—Guidelines from PSWA, Asthma Medication Request Checklist (developed specifically for the research) Pamphlets—From NAC and AFWA—general information on asthma DVD—reference materials including two videos Internet—reference materials including two videos
Location	Where the intervention is delivered	Participant's home Classroom Healthcare provider's office Hospital, clinic, operating room Work site Community centre Nursing home Group residence facility Research facility	Workplace (community pharmacy)
Schedule	Duration and intensity of intervention	Overall duration of the intervention Number of sessions Minutes of contact per session Distribution of sessions over time	1 workshop of approx 1.5 hours or academic detailing visit about 15 min Reinforcement via resources provided
Scripting	Level of detail guiding interaction between the interventionist and the participant	Exact script/protocol provided Specific language provided with elaboration allowed/not allowed Goals/tasks specified but no further scripting General guidelines provided	Specific language provided via power point with elaboration allowed in interactive discussion—all workshops undertaken by one interventionist to maintain consistency of message
Sensitivity to participant characteristics	Extent to which participant background, experience and abilities are incorporated in the delivery of intervention	Intervention materials and delivery in language preferred by participant Materials written for specific reading or health literacy level Visual supplements, augmentative communication devices for hearing impaired Oral supplements and visual enhancements for vision impaired	Intervention materials and delivery in language preferred by participant—recognition of the level of understanding and perspective of both non-professional and professional staff
Interventionist characteristics	Qualifications and training, concordance with participant characteristics	Required disciplinary/professional expertise for interventionists Licensing/certification requirements Type and quantity of training provided Proficiency tests passed Race/ethnicity/age/gender matching of interventionist to participant Intervention staff recruited from participant community Interventionist knowledgeable of cultural views and values of participants	Intervention staff* recruited from participant community Training for academic detailers–one-on-one training of 1 hour length Workshops undertaken by trainer of academic detailers Interventionist knowledgeable of cultural views and values of participants
Adaptability	Extent to which intervention can be modified. What can be modified? On what basis are modifications made? When in the course of the study can modifications be made?	What: Number/schedule/duration of sessions Location Mode of delivery Content/target Dosage On what basis: Participant assessment Participant progress Spontaneous request Secular event Clinical judgement Checklist/laboratory test results, performance outcomes When: Intake Baseline Specified intervals during intervention	What: Could increase number of sessions if a pharmacy could not get all staff to attend the one session Could change the location to training room of a professional organisation Could change mode of delivery—changes were made to incorporate academic detailing. Could also use a large multipharmacy lecture. On what basis: Spontaneous request (eg, participant request to hold multiple workshops or change location) Based on participant progress—low recruitment numbers for workshops required adapting the intervention to academic detailing When: At intake (being adaptable during recruitment may increase participation) Specified intervals—throughout the recruitment adapting to recruitment numbers required a change in delivery mode
Treatment implementation	Treatment Delivery: Documentation of interventionist compliance to intended treatment and modificationsTreatment Receipt: Extent to which processes are implemented by participant and/or goals are metTreatment Enactment: Extent to which knowledge and skills acquired during treatment are applied in real-world settings outside of treatment	Number and duration of sessions Content delivered Knowledge, skills, motivation, self-efficacy, social support/integration, changes in pathophysiology assessed in participant Direct observation, self-report, observer report of participant	Documentation of number of sessions and duration—25 workshops and 162 academic detailing visits The content was delivered as intended (via workshop) to 13.4% of participants. It was delivered in a modified version (academic detailing) to 86.6% Treatment receipt (knowledge, skills, motivation, self-efficacy, etc) was not specifically assessed, although the format which was interactive provided an opportunity for participants to clarify the information Treatment enactment was assessed by way of direct observation
Content and goals of intervention (intervention characteristics)
Treatment content strategies	Specific strategies aimed at improving outcomes	Provision of feedback to participant through tracking and monitoring Provision of information Behavioural incentives/reinforcements Didactic instruction Skill-building techniques Problem-solving techniques Stress-management techniques Facilitation of social support Biological interventions (surgery, medications, radiation) Structure/process modifications (eg, staffing, scheduling, communications)	Formal feedback was not provided, although participants were encouraged to make contact: with any problems that required advice/troubleshooting, with any success stories Provision of information—website, DVD and paper resources were provided to reinforce messages Behavioural incentives were given in the form of information about how to avoid conflict and access remuneration pathways Some didactic instruction was used to initiate conversation Skill-building techniques were used to encourage more effective patient engagement and communication Problem-solving techniques were given in the form of a checklist to facilitate the collection of patient information and a video role play Stress-management technique used was the empathetic approach to the barriers faced in being guideline compliant Structure/process modifications were offered in the form of how to manage the workflows to achieve guideline compliance.
Mechanisms of action	Key processes, goals or mediators of desired treatment outcomes	Ability to assess risks/goals Knowledge Behavioural skills Problem-solving skills Motivation Self-efficacy Social support Social engagement Environmental motivation Change in policies/regulations Biological pathways	Knowledge—of guidelines, legal requirements Behavioural skills—how to engage and communicate more effectively with patients. Problem-solving skills—how to engage with reluctant patients Motivation—demonstration that patients may not always recognise the need for help but do need it. Information about how to be remunerated for service provision Self-efficacy—checklist improves belief and ability to collect appropriate patient assessment information

In the context of the SABA guideline intervention, the following definitions apply:

Interventionist/intervention staff: These were the personnel who conducted the intervention. This included researchers who conducted the workshops and/or academic detailing visits.

Participants: These were the pharmacy staff who received the intervention including pharmacists and non-professional staff (pharmacy assistants).

AFWA, Asthma Foundation of Western Australia; ITAX, Intervention Taxonomy; NAC, National Asthma Council of Australia; PSWA, Pharmaceutical Society of Western Australia; SABA, short-acting β agonists.

Mechanism of the SABA intervention

The mechanism of the SABA intervention was detailed in the ‘intervention characteristics’ category of the ITAX checklist, which included the dimensions: treatment content strategies and mechanism of action. Retrospective application of the ITAX described mechanism in general terms such as: knowledge, behavioural skills, problem-solving skills, motivation and self-efficacy. The retrospective application of the BCW framework also considered mechanism but described in terms of behaviour change techniques. Both tools demonstrated that there was a clear mechanism and sound rationale for the SABA intervention.

Fidelity of the SABA intervention

Dimensions relevant to the fidelity of the SABA intervention included: adaptability, schedule, scripting, treatment implementation, treatment content strategies, interventionist characteristics and sensitivity to participant characteristics. Of interest in considering the fidelity of the SABA intervention was the recognition that the intervention had been substantially adapted. The content was delivered as intended (as a workshop) to 13.4% of participants. While ITAX considered the level of adaptation, it did not measure the effectiveness of the adaptation.

Logistics of the SABA intervention

The logistics of the SABA intervention were detailed in the ‘intervention delivery’ category of the ITAX checklist. These included dimensions: mode, materials, location, schedule, scripting and treatment implementation. All the logistical elements were considered in the SABA intervention design but the duration and intensity (schedule) of the intervention was minimal. The intervention consisted of one workshop of ∼1.5 hours or a detailing visit of 15 min. Resources were provided to reinforce messages from the intervention.

All components of the common-sense SABA guidelines intervention could be categorised according to their effect on behaviour change, using the BCTTv1 by following the eight steps of the BCW framework and applying the COM-B model.21 A summary of the deconstruction into behavioural change techniques is shown in table 2. The stages and steps undertaken in the identification of the behaviour changes techniques were as follows:

Table 2

Asthma intervention summary of content using the BCW, BCT and COM-B

Behavioural Intervention: small group workshop in individual community pharmacies
BCT	Functions	Text description	Capability		Opportunity		Motivation
			Physical	Psychological	Social	Physical	Reflective	Automatic
Information about social and environmental consequences	EducationPersuasion	Information was given about asthma management and current information about the gaps in practice. Explanations were provided about why patients may not recognise the need for assistance in asthma management.Information was also provided about legislative requirements and guidelines for practice		✓			✓	✓
Information about health consequences	EducationPersuasion	Information and statistics about the serious consequences of asthma including a patient's story about a near-death experience		✓			✓	✓
Prompts/cues	EducationEnvironmental restructuring	A checklist was introduced as a way of asking appropriate questions to support internal referral and pharmacist decision-making based on the guidelines	✓	✓	✓	✓	✓	✓
Self-monitoring of behaviour	EducationIncentivisation Training	The use of a checklist helped to provide a quantifiable measure of service provision. Encouragement was given that patients may still be resistant at first but that the process could become habitual	✓	✓	✓	✓	✓	✓
Verbal persuasion about capability	Persuasion	Pharmacy assistants were encouraged that patients with asthma would engage if approached in the right way. The difficulties were acknowledged and then strategies were provided to enhance communication					✓	✓
Identity associated with changed behaviour	Persuasion	Pharmacy assistants were encouraged about the importance of their role in the process of managing patients with asthma and given clear guidance on what their role was in the context of the legislative requirements.					✓	✓
Incentive	Incentivisation	Information was provided about how to access remuneration for professional services resulting from guideline-based practice					✓	✓
Demonstration of the behaviour	TrainingModelling	A role play video was used to demonstrate difficulties with patient engagement and then a discussion was based on the videoAnother patient story video was used to demonstrate the patient perspective to enhance communication and motivation	✓	✓	✓	✓	✓	✓
Instruction on how to perform a behaviour	TrainingModelling	A checklist was introduced as a way of collecting appropriate information and as a way of facilitating internal referral of patients within the pharmacy	✓	✓	✓	✓	✓	✓

BCT, Behaviour Change Techniques; BCW, Behaviour Change Wheel; COM-B, Capability, Opportunity and Motivation-Behaviour.

Stage 1: Understand the behaviour

The defined problem behaviours were related to the provision of non-prescription asthma reliever medications in community pharmacies to patients in accordance with the SABA guidelines and legislation.

Step 1: Define the problem in behavioural terms

Step 2: Select the target behaviours

A total of 28 potential target behaviours were identified. These were then analysed and prioritised according to: impact on desired outcome, likelihood of being able to change the behaviour, likelihood that the behaviour will have a positive or negative impact on other related behaviours (spillover score), how easy it will be to measure the behaviour (measurement score). Target behaviours were rated as very promising (VP), promising (P), unpromising but worth considering (UP) or unacceptable (U). The SABA intervention aimed to target behaviours related to pharmacy assistants and these were determined to be very promising.

Step 3: Specify the target behaviours

The specific target behaviours involving pharmacy assistants included:

Asking patients appropriate questions

Referring patients to the pharmacist to meet legislative requirements (internal referral)

Managing the workload/workflow of the pharmacists to facilitate pharmacist involvement

Step 4: Identify what needs to change

Application of the COM-B model for each of the three target behaviours (above) identified that physical capability, psychological capability, physical opportunity, social opportunity, reflective motivation and automatic motivation all needed to be addressed for all the target behaviours to occur.

Stage 2: Identify intervention options

Applying the APEASE criteria resulted in selection of the following intervention functions appropriate to the aim: education; persuasion; incentivisation; training; environmental restructuring; modelling.

Step 5: Identify intervention functions

Step 6: Identify policy categories

Relevant policy categories identified to support target behaviours included: communication/marketing, guidelines and fiscal measures.

Stage 3: Identify content and implementation options

Step 7: Identify behaviour change techniques

Using the identified intervention functions, the active components (Behaviour Change Techniques—BCTs) were determined using the taxonomy (BCTTv1) and included:

Information about social and environmental consequences

Information about health consequences

Prompts/cues

Self-monitoring of behaviour

Verbal persuasion about capability

Identity associated with changed behaviour

Incentive

Demonstration of the behaviour

Instruction on how to perform a behaviour

Step 8: Identify mode of delivery

Options for delivery of the intervention included face-to-face delivery either at an individual or group level and at a distance using phones and computers. Population level interventions using broadcast, outdoor, print and digital media were considered cost prohibitive.

Discussion

This evaluation and deconstruction resulted in a greater understanding of the complex elements of a behaviour change intervention to implement clinical guidelines for asthma management, in the community pharmacy setting. It demonstrated that the common-sense intervention was comprehensive and comparable to an approach using theory and a logistics taxonomy. This supports the notion that the use of taxonomies, frameworks and a theory-based approach in intervention design is not necessarily going to make interventions more successful.21

However, the evaluation involving a retrospective application of ITAX did highlight some issues that may have influenced the success of the intervention. The intervention design process was based on using workshops as the implementation strategy. It became clear that significant adaptation had occurred which may have affected the fidelity of the original team-based approach to the intervention. The theoretical basis for the intervention might have been compromised in the adaptation, which replaced workshops with academic detailing. Academic detailing involved delivery of the information to an individual and required the individual to disseminate the information, as well as galvanising and motivating the ‘pharmacy team’ to change behaviour. This is quite a different prospect to the researcher interacting and tailoring information to the ‘pharmacy team’ as a whole, in a workshop situation. What was unclear from the use of the taxonomy was to what extent the adaptation had affected outcomes. Further process evaluations, using qualitative research methods, are required to understand to what extent fidelity had been maintained by the change in intervention delivery and what improvement in the outcomes could be expected, if more workshops were delivered.

The necessity to adapt the intervention was related to logistical issues of recruitment and reach29 31 and these barriers need to be understood before considering the intervention for wider health system implementation. Again, further qualitative research could potentially provide insight into the logistical barriers encountered.31 Nevertheless, despite the significant adaptation, positive outcomes were achieved.29 The intervention resulted in increased medical referral of patients with poorly controlled asthma as per the SABA guidelines. This leads to the conjecture that if the barriers to participation in small group education in community pharmacy could be overcome, this may prove to be a successful implementation strategy for this setting. There is some evidence in the literature to support this hypothesis. Schneider et al32 used intern pharmacists to deliver asthma interventions in the workplace, as part of their training. Their interventions included delivery of an educational session for pharmacy staff and a health promotional activity directed at patients.32 The results were an impressive doubling of the rate of medical referral.32 Similarly, Kritikos et al33 conducted a pilot study, using small group asthma education sessions in the community pharmacy setting, which resulted in improvements in clinical and humanistic outcomes.

The deconstruction process into behaviour change techniques demonstrated that there was a sound rationale for all the components of the common-sense intervention to achieve SABA guideline compliant practice. This is important, as a criticism of complex common-sense interventions is that there can be a tendency to simply add elements without necessarily having a clear purpose or enhancing effectiveness.34 The fact that all the elements of this common-sense intervention had a specific purpose indicates that explicit theory is not essential in the intervention design process and may not be the panacea to implementation science it claims to be. It is possible that the use of formalised theory just adds unnecessary complexity and distracts from the ultimate goal of improving health outcomes. Common-sense interventions, designed by researchers who understand the participants, their behaviours and the research setting, can be just as viable a method. Perhaps it is the preliminary research and formative evaluations that provide an entrenched understanding of the issues that are most crucial in intervention design. Supporting this view is the research by Presseau et al22 who undertook a similar deconstruction process to code a random sample of 23 trials of diabetes implementation interventions. The majority of the implementation interventions in the study targeted evidence-based care processes and outcomes.22 When deconstructed, they were found to influence multiple behaviours, in a similar way to the common-sense SABA intervention.22 The methodology involved initially coding the intervention strategies in terms of The Effective Practice and Organisation of Care taxonomy and then using the BCTTv1 to detail the ‘active elements’.22 Presseau et al22 found that many of the interventions coded had good scope, detail and rigour, supporting the value of well-thoughtout common-sense interventions. This is in contrast to other literature that criticise the poor methodology and reporting of intervention studies.9 35 36

Regardless of the benefits or not of a theory-based approach, what was evident from the deconstruction process was that theory alone is not enough in intervention design or understanding outcomes. The logistics and fidelity of an intervention are equally important considerations and need to be given more attention. Logistics can be challenging in naturalistic settings. Therefore, it may be advantageous to use a more structured approach when planning interventions. One that considers content as well as delivery mode and resources. ITAX or similar logistics frameworks and taxonomies remain vital for process evaluation. Apart from the significant adaptation required, other issues highlighted by deconstruction using ITAX included the duration and intensity of the intervention. Only one session was held to convey the information of the SABA guideline implementation intervention, and reinforcement was by way of resources but no further contact with researchers. The relatively short duration and intensity were appropriate for the exploratory nature of the research, but demonstrate that implementation on a wider scale requires more resource allocation to these elements. It also points to the importance of pilot/feasibility/exploratory studies before large-scale health system implementation.37 What was not clear was how long and how intense the reinforcement needed to be to substantially improve outcomes or ensure effectiveness of wider implementation.

Other issues the evaluation process did not address were the concepts of context, climate and sustainability. Further qualitative evaluations are needed to consider these issues, to understand outcomes and to ensure the success of wider implementation. The taxonomies do not replace the requirement for evaluations based on participant and stakeholder feedback. At present, these concepts are hard to define and measure, but are highly relevant to implementation effectiveness.36 38–40 Context considers the physical and social environment39 40 and relates to the generalisability of the research findings. However, the important elements that constitute ‘context’ are yet to be fully elucidated, making assessment challenging.39 40 The implementation climate is the extent to which use of a specific innovation is rewarded, supported and expected within an organisation.38 The climate could vary between individual community pharmacies and its effect on overall outcomes could be significant. Thus, it could also impact on results of a wider implementation. Sustainability refers to the ability to maintain intervention effects.36 This becomes an important consideration in the allocation of limited resources available for health system implementation.36 These unresolved issues indicate: the complexity of implementation science; the difficulty in understanding all the factors affecting intervention outcomes; and the difficulty in predicting the effectiveness of interventions chosen for health system implementation.

In using two different approaches in this evaluation, there was some overlap between the information gleaned. Both taxonomies examined the ‘rationale’ of the SABA intervention, but in different ways. ITAX looked at the mechanisms for achieving outcomes, whereas the deconstruction process resulted in a list of behaviour change techniques used in achieving outcomes. Neither of these approaches provided information about which elements were important or had the greatest impact on outcomes. This duplication of information, using different terminology to essentially describe the same element, highlights an emerging issue in implementation science. The growing interest in the use of taxonomies, frameworks and theories has seen an explosion in their development and use, creating some confusion in how to choose and use tools appropriately.14 Consolidation of the available tools and consensus about their application are required to maximise benefits of a formalised approach to research.

Despite the unresolved issues, there are several potential advantages to undertaking the evaluations completed in this paper. The evaluation of the SABA guideline intervention indicated a: well-thought-out mechanism and rationale for achieving outcomes, high degree of adaptation and short duration and intensity of the intervention. However, it did not provide a comprehensive solution to improving outcomes or understanding the suitability of the intervention for wider implementation. The value of retrospective deconstruction may be in using the methodology across a number of studies for comparative purposes. There are a few examples in the literature, but this approach is yet to be rigorously tested.7 22 Deconstruction into individual elements such as BCTs and using a common language (taxonomy) has the potential to simplify the comparison of studies, improve replication and link BCTs to outcomes.7 22 This process may be especially relevant in the community pharmacy setting, due to the heterogeneity of research involving implementation of clinical guidelines.12 36 However, successful deconstruction is dependent on the quality of reporting of studies. Without effective descriptions, interventions cannot be replicated, evaluated and compared and researchers cannot build on research findings.35 This is where taxonomies, frameworks and checklists do become important: in the standardisation of reporting. There are numerous examples in the literature of tools for reporting interventions, including the tools used in this evaluation and their use is becoming more widespread.3

While formalised approaches to implementation interventions, either at the design stage or in a subsequent evaluation, have benefits, they are not the complete answer. Implementation science is complex and there will always be intangible human influences that confound deciphering interventions and understanding outcomes. Generally, triangulation in evaluation looks at a single phenomenon but uses multiple data sources, observers, methods or theories.41 The objectives are twofold: to validate findings and to uncover new information to provide a greater understanding of a complex problem. Evaluations can use both quantitative and qualitative methods. Employment of both methods can be helpful in addressing research questions in a more comprehensive way.41 Triangulation research methods are required to build up a picture of what is required for successful implementation in any single context. Multiple evaluations are required and more emphasis on evaluation of interventions is needed in implementation science.

Conclusion

It was feasible to conduct a process evaluation and deconstruction based on theory to appraise an existing common-sense, clinical guidelines implementation intervention in community pharmacy. The deconstruction of the intervention, into behaviour change techniques, demonstrated that a common-sense approach to intervention design could be equivalent to a theoretical approach. The common-sense intervention was comprehensive and had a sound rationale and theoretical underpinning. However, the deconstruction of a single intervention did not provide insight into how to improve outcomes. The application of ITAX highlighted issues related to logistics, adaptability and fidelity but additional evaluations of context, climate and sustainability are required. Duration, intensity and reinforcement of the intervention need consideration for successful wider implementation. Small-group workshops may prove to be a useful implementation strategy in community pharmacy if logistical issues can be overcome and less adaptation occurs. The assessments completed and the frameworks and taxonomies used provided some answers to improving implementation of guidelines, but further qualitative evaluations, triangulation of research and evaluations across interventions are required.