Using an online nominal group technique to determine key implementation factors for COVID-19 vaccination programmes in community pharmacies.

Background One vital strategy to fight the COVID-19 pandemic is the rapid roll-out of vaccination programmes. In a number of countries pharmacists are joining the vaccination programme workforce, including plans to involve community pharmacies. Objectives (1) to determine key implementation factors for rapid roll-out of COVID-19 vaccination programmes in European community pharmacies and (2) to trial an online nominal group technique to generate ideas and reach consensus on the first aim. Setting In February 2021, during a workshop at the 12th Working Conference of the Pharmaceutical Care Network Europe. Method An online nominal group technique workshop over 10 hours was conducted. Identified implementation factors were mapped to the Framework for the Implementation of Services in Pharmacy and assessed in terms of importance and changeability. Main outcome measure Consensus of key implementation factors. Results In total, 85 implementation factors were identified. The top 3 factors were the same for both criteria: "(Regular) Staff training on correct vaccination procedure/to perform hands-on injection technique"; "Training in basic life support/first aid (Cardiopulmonary resuscitation)", and; "Definition of process, roles and responsibilities in the team". Conclusion An online nominal group technique enabled international collaboration to gather diverse perspectives on the implementation of COVID-19 vaccination programmes in a time and cost-efficient manner. "Staff training on correct vaccination procedure", "training in basic life support" and "definition of process, roles and responsibilities in the team" were deemed the most important and changeable implementation factors. Online nominal group technique may be a suitable method for other implementation problems.

Impacts on practice

With appropriate training and support, community pharmacists can deliver critical health services to the public and communities they serve, including mass vaccinations in primary care.

As the network of community pharmacies is already embedded within primary health care systems, this resource could be used for the rapid roll out of public health services, including mass vaccinations.

Nominal group technique is a suitable method for generating ideas and consensus in a virtual format.

Introduction

SARS -CoV-2 (COVID-19) emerged in November 2019 and was declared a global health emergency and pandemic by the World Health Organization (WHO) in March 2020 [1, 2]. The virus has since spread across parts of the world at an alarming rate and COVID-19 infections are causing significant morbidity and mortality. Many countries are experiencing second and third waves of COVID-19 [3]. Lockdown measures and mass vaccinations can help reduce viral transmission [3]. However, the impact of mass vaccination will depend on the speed and coverage of its roll-out [4].

Utilising existing primary health care, including pharmacists and community pharmacies, is one component of government policies to expedite mass vaccination programmes [5, 6]. The involvement of trained pharmacy staff delivering vaccinations in community pharmacies has shown to have a positive impact on public vaccination coverage, for example increasing the reach of annual influenza vaccination programmes [7-9]. One advantage of including community pharmacies in vaccination programmes is convenient access to the public due to their geographical spread [5, 6]. As such, engaging community pharmacies and pharmacists into the strategy to provide COVID-19 vaccinations has already been initiated or planned in some countries, including Canada, USA and some European countries including, Northern Ireland, Italy, Switzerland, Portugal, Germany, Italy and the UK [10-22]. Community pharmacies’ involvement varies from counselling and pharmacovigilance to logistic tasks and actual administration of vaccines within the pharmacy [22]. To ensure rapid roll-out of COVID-19 vaccination programmes across European community pharmacies, both in terms of widespread uptake by pharmacists and the population, consideration should be given to the implementation factors which may facilitate or hamper the endeavour.

Identifying potential implementation factors during programme development and prior to roll-out can assist the development of tailored implementation strategies, thus reducing the risk of poor uptake and subsequent low reach to the target population [23]. Implementation science bridges the gap between innovation development and evidence gathering (in this case COVID-19 vaccination programmes) and real-world practice [24, 25]. Mapping implementation factors to constructs in an established implementation framework allows better understanding of the different influences and how they may be addressed [26].

In February 2021, at the 12th Working Conference of the Pharmaceutical Care Network Europe (PCNE), a workshop was held to identify factors potentially influencing the implementation of COVID-19 vaccination programmes in European community pharmacies. The PCNE is an international pharmacist society which focusses on the development of pharmaceutical care in European countries. PCNE’s main role involves supporting pharmaceutical care and pharmacy related outcomes research including facilitation of multinational projects [27]. The workshop used an adapted online format of the nominal group technique (NGT). NGT is a highly structured consensus method that is widely used in health services research [28]. The technique offers participants the opportunity to discuss ideas freely and to voice their opinion without judgement [29].

Aim of the study

This study had two aims (1) to determine key implementation factors for rapid roll-out of COVID-19 vaccination programmes in European community pharmacies and (2) to trial an online approach of the nominal group technique.

Ethics approval

This study was submitted to Ethics Committee Northwest and Central Switzerland where it was deemed exempt from Ethics’ approval under Swiss law because it “does not concern human diseases or structure and function of the human body”. (Req-2021-00,147, 10.2.2021).

Method

Participants

Participants signed-up for the NGT implementation workshop after registration for the PCNE-Working Conference. The workshop group comprised 23 pharmacists from 11 countries, working in a range of settings, with varying levels of clinical, research and implementation expertise (Table 1). The workshop was facilitated by 3 experienced researchers (TFC, JCM, EP).

Table 1

Participant demographics

Age, mean (SD), range years	38.0 (9.6), 25–63
Professional experience, mean (SD), range years	11.6 (7.6), 1–30
Gender, n (%)
Female	17 (77.3)
Male	5 (22.7)
Country of residence, n (%)*
Australia	1 (4.5)
Belgium	2 (9.1)
Denmark	2 (9.1)
Germany	4 (18.2)
Norway	1 (4.5)
Slovenia	1 (4.5)
Serbia	3 (13.6)
Switzerland	6 (27.3)
The Netherlands	1 (4.5)
United Kingdom	1 (4.5)
Professional expertise, n (%)*
Academia	13 (59.1)
Community pharmacy	3 (13.6)
Academia and Community pharmacy	3 (13.6)
Hospital pharmacy	2 (9.1)
Professional organisation (e.g. pharmacy chamber, national pharmacy board)	1 (4.5)
Prior experience with NGT, n (%)
Used it before	2 (9.1)
Read about it	7 (31.8)
No experience	13 (59.1)
Familiarity with implementations methods and strategies, n (%)
Worked in this field	9 (40.9)
Read about it	8 (36.4)
Not familiar	5 (22.7)
Is vaccination in community pharmacies legal (in participants’ country of residence)?, n (%)*
Yes	10 (45.5)
No	6 (27.3)
Currently under development	6 (27.3)
Have you ever been vaccinated in a community pharmacy? n (%)
Yes	7 (31.8)
No	15 (68.2)
Have you ever vaccinated a patient?, n (%)
Yes	3 (13.6)
No	19 (86.4)

*percentages do not add up to 100% due to rounding

Online nominal group technique (NGT)

The NGT was used as an idea generation and consensus method. The NGT has several advantages including time efficiency, cost effectiveness and adaptability. Furthermore, NGT equally considers views of all participants, avoiding one individual dominating the group, and has demonstrated validity [30].

The original NGT consists of 4 steps, namely (1) Silent generation of ideas in writing, (2) Round-robin feedback from group members to record each idea (3) Discussion of each recorded idea for clarification and evaluation, (4) Individual voting on priority ideas [31].

The NGT has been adapted and expanded including for priority setting of evidence-practice gaps in implementation science [30]. Further adaption to the NGT occurred for this study. Due to the COVID-19 pandemic the conference, including the workshops, were conducted in an online format via Zoom® software. This study’s adaptations to the NGT process included the use of Zoom® break-out rooms for the parallel meetings of smaller groups and the use of excel-sheets to record ideas instead of using sticky notes on a flip chart. The excel sheets were prepared by the workshop organisers beforehand (JCM, TC) allowing intuitive use by the participants and providing a standardised format for combining individual sheets into one master sheet. Conducting the robin-rounds, in which participants speak in turn, were facilitated by the online nature of the sessions as the technique inherently impedes superimposed speaking. The workshop time frame was for 5 two-hour sessions (10 h in total), over 3 days (Fig. 1).

Fig. 1

Flow chart of the conducted online NGT

At the opening workshop session (2 h) the participants received an introduction to implementation science and practice. This introduction focused on the Generic Implementation Framework (GIF) and Framework for the Implementation of Services in pHarmacy (FISpH) [32, 33]. Background literature regarding COVID-19 vaccines and NGT were also presented. Following the general introduction participants were purposively assigned based on their knowledge and experience with implementation science and/or vaccination in community pharmacies into 4 smaller heterogeneous groups, consisting of 5–6 participants each, in accordance with a team-based learning approach [34]. This manuscript’s authors were full participants while taking field notes about their observations, additionally the facilitators took notes during the large group discussions. At the end of the opening session a preliminary research question was presented, which was discussed and refined by the whole group. The final research question was: What are the factors required for the implementation of a rapid roll out of a COVID-19 vaccination strategy during a community pharmacy delivery phase in Europe? Three assumptions were made: (1) vaccination in community pharmacies is legally allowed, (2) vaccine supply is ensured and (3) vaccines require normal cool storage (2–8° Celsius).

The next 4 workshop sessions (8 h in total) were used to conduct the steps of the NGT in an online format (Fig. 1). During the first step (“Generating implementation factors”), participants individually brainstormed the factors that may influence the implementation process of a COVID-19 vaccination program in community pharmacies in Europe. Participants noted their ideas individually on excel sheets and sent them via email to the respective small group facilitator. The aim of this step was to generate as many ideas as possible thinking across the phases of the implementation process.

The 2nd step (“Recording implementation factors”) included round robin sessions within participants’ four smaller breakout groups to concisely record each factor. Factors were required to be independent of each other without multiple clauses. Groups listed their implementation factors in a spreadsheet (MS Excel) for collation by the facilitators.

The 3rd step of the online NGT (“Clarifying and distilling implementation factors”) involved the participants and their group facilitators within the 4 smaller separate groups clarifying any factors for which they were unsure of the meaning or whether they constituted any duplicates. At this step considerable discussion about meaning and context of the named factors happened within the small groups which were captured in notes by the manuscripts’ authors. The 4 excel sheets of the small groups were sent to the workshop moderator (EP) who distilled statements into individual factors and removed duplicates after double-checking with participants if necessary. A final version of the list was provided to all participants.

At the 4th step (“Individuals rate the importance and changeability of factors”) participants were asked to rate the relative importance and changeability of each factor of the compiled list using a Likert-type scale from 1 to 5, with 5 being the most important or changeable. This step was used to ensure that each participant had read and considered each implementation factor in detail prior to group discussion and consensus.

During step 5 of the online NGT (“Prioritisation of implementation factors”), each participant prioritised 5 implementation factors for the criteria “importance” and “changeability”.

Step 6 (“Rating group discussion”) began with discussions in smaller groups about each participant’s individual prioritised implementation factors. Participants shared their ratings, speaking in turn to list their top 5 for each criterion (importance and changeability) and provide any clarification for their choices.

During step 7 (“Reaching group consensus”), the breakout groups re-convened as one large group to review and discuss the implementation factor ratings, resolve any differences and reach consensus about the five most important and most changeable implementation factors.

At the end of the workshop all participants provided verbal consent to receive an invitation by email to an online survey to collect data about demographics and professional experience. The survey contained items related to age, gender, professional experience, country of residence, prior expertise with NGT/implementation science and vaccination in community pharmacy. The survey was designed as a google form and a link was emailed to all participants who had indicated that they were happy to complete the survey which was completely voluntary.

Post-conference data analysis:

The Framework for the Implementation of Services in pHarmacy (FISpH) was used to code the identified implementation factors [33]. The FISpH was deemed the most appropriate framework because it has been developed for community pharmacies and is based on pharmacy practice research. This framework encompasses 2 established implementation science frameworks namely the Consolidated Framework for Implementation Research (CFIR) and the Theoretical Domains Framework (TDF) that were specifically adapted for pharmacy practice [35, 36]. All identified implementation factors were mapped independently by 2 authors (DM and AI) whilst striving to avoid double-coding. These factors were considered within a European context, where in many countries activities like training and marketing are provided at the system level. Any discrepancies in coding were resolved by thorough discussion. A 3rd coder (JCM) checked for consistency.

The final scoring results for each factor on importance and changeability given by the small groups were presented in Tables 3 and 4.

Table 3

Implementation factors ranked by importance

Implementation factors: Importance	FISpH constructs	Group
		A	B	C	D
(Regular) Staff training on correct vaccination procedure/to perform hands-on injection technique*	External support and/or assistance	5	5	5	5
Training in basic life support/first aid (Cardiopulmonary resuscitation) *	External support and/or assistance	4			4
Definition of process, roles and responsibilities in the team*	Teamwork			4	2
Room/Space for vaccination (external flat/containers/office spaces/hotels)*	Layout and workflow	3	2
Pharmacy workforce	Staff		4
Information on vaccination status: Vaccination Passport (or access to online tool of health care system)	External support and/or assistance				3
Appointment system (online booking of vaccination time slots for citizens)	External support and/or assistance			3
(Regular) Staff training on correct patient risk assessment for vaccination*	External support and/or assistance		3
Staff, including technicians, nurses, pharmacists, students (medicine, pharmacy)	Staff	2
Hygiene concept (general for the vaccination site; separation from infected people)*	Layout and workflow			2
Patient acceptance for vaccination (local population beliefs)	Community’s perception about innovation and organisation			1
(Regular) Staff training on correct monitoring procedures after vaccination	External support and/or assistance		1
Remuneration to set-up/prepare for service implementation	Remuneration	1
Inclusion and exclusion criteria for vaccination (risk factors)—check list and priority lists	Quality assurance system				1

5 indicates highest, 1 indicates lowest importance; implementation factors without a ranking number were considered less important by the respective groups

*Indicates factors that were rated as high importance and changeability

Table 4

Implementation factors ranked by changeability

Implementation factors: Changeability	FISpH constructs	Group
		A	B	C	D
(Regular) Staff training on correct vaccination procedure/to perform hands-on injection technique*	External support and/or assistance	5	3	5	5
Training in basic life support/first aid (Cardiopulmonary resuscitation) *	External support and/or assistance	3			4
Definition of process, roles and responsibilities in the team*	Teamwork	1		2	2
Provide information/leaflets to patients about possible ADR and instructions what/where to report	Design quality & packaging	2		3
Hygiene concept (general for the vaccination site; separation from infected people)*	Layout & workflow		5
Patient awareness of the service	Community’s perception about innovation and organisation	4
FAQ sheet/database for pharmacy staff	Access to knowledge and information			4
(Regular) Staff training on correct patient risk assessment for vaccination*	External support and/or assistance		4
Expertise in recognised inclusion and exclusion criteria for vaccination (risk factors)—check list	Technical skills (experience, capacity & competence)				3
Software to register vaccination to link to national surveillance system	Available resources and training		2
Follow up of patients for 2nd dose	Data management system			1
Storage space (fridge), open vial	Available resources and training				1
Room/Space for vaccination (external flat/containers/office spaces/hotels)*	Layout & workflow		1

5 indicates highest, 1 indicates lowest changeability. Implementation factors without a ranking number were considered less changeable by the respective groups

*Indicates factors that were rated as high importance and changeability

Demographic data of participants were presented using descriptive statistics (frequencies, mean, standard deviation, and range). Microsoft Office Excel 10 and Statistical Package for Social Sciences (SPSS) (SPSS v.20; SPSS Inc, Chicago, IL, USA) were used for data analysis.

Results

Study participants

23 participants from 11 different countries with expertise in pharmacy (research and/or clinical practice) signed up for the online workshop held at the 12th PCNE-Working Conference. All participants were purposively divided in 4 groups (with 5–6 participants per group respectively). Participants were free to fill in the online survey on basic characteristics and all but one did so. The basic characteristics of these 22 participants are presented in Table 1.

Adaptations of NGT to the online format

Meeting online via Zoom® software as one large group, as well as dividing into smaller groups in breakout rooms, progressed smoothly without technical incident. The completion and collation of the prepared excel sheets worked well. However, additional time for facilitators to compare and combine sheets was not feasible due to differing time zones which led to perceived time constraints. The online nature of discussions enhanced sharing of ideas as it impeded participants speaking over each other and also facilitated a smooth flow through the NGT process as participants were moved by the moderator in and out of the break-out rooms.

Identified implementation factors

Individual and group brainstorming yielded 160 implementation factors. After removal of duplicates 85 implementation factors remained. Post-workshop, using framework analysis, the identified implementation factors were mapped to the FISpH domains “innovation (vaccination programme)”, “external system”, “local setting”, “organisation (pharmacy)”, and “individual (pharmacist or other pharmacy staff)” factors, as shown in Table 2 [33].

Table 2

Implementation factors mapped to the Framework for Implementing Services in pHarmacy (FISpH) [33]

FISpH construct*	Implementation factors
Innovation (vaccination programme) factors
Implementation complexity	Preparation of the single shots
Design quality and packaging	Supporting materials from manufacturers (leaflets)
	Provide information/leaflets to patients about possible ADR and instructions what/where to report
Nature of innovation	Vaccine prescription
	Staff reorganisation e.g. hiring more staff
External system
Remuneration	Remuneration to set-up/prepare for service implementation
Healthcare budget and contracts	Remuneration for the service (by government, patients or other sources/third party payers)
Interprofessional networks and communication	Inter professional recognition (system level recognition and acceptance by other professionals)
Stakeholder buy-in	Support of the national pharmacy organisation(s)
	Acknowledgement of the pharmacists a health care provider from other healthcare providers who may also participate in vaccination
External support and/or assistance	Advertisement/public announcements/recommendation on vaccination in pharmacies
	Guidelines/SOPs from national pharmacy organisation
	Strategy in case of an emergency
	Clarification on insurance for staff
	Telephone helpline for patients (nationwide)
	Information on vaccination strategy (voucher/sheet/appointment card)
	Information on vaccination status/vaccination passport (access to online patient record/tool of health care system)
	(Regular) Staff training on correct patient risk assessment for vaccination
	Appointment system (online booking of vaccination time slots for citizens)
	Administrative support to identify patients belonging to a certain priority group
	Information and measures for a hygiene concept (guidelines)
	(Regular) Staff training on correct vaccination procedure/to perform hands-on injection technique
	(Regular) Staff training on correct monitoring procedures after vaccination
	Training in vaccine centres prior to local pharmacy vaccination
	Training in handling side effects
	Training in basic life support/first aid (Cardiopulmonary resuscitation)
	Material for vaccination (needles)
Local Setting
Intraprofessional network and communication	Collaboration on the task within smaller pharmacies
Interprofessional network and communication	Availability to communicate with other healthcare (GPs) in situations where it is necessary
	Referral from other health care professionals (local)
	Support from other health care professionals
Community’s perception about innovation and organisation	Clear arrangements in the community (clear direction in the community)
	Patient awareness of the service
	Patient acceptance for vaccination (local population beliefs)
	Patients' trust for service in the pharmacy
Relationship with patients and community	The pharmacy is easily accessible by the patient/ Community pharmacy location in terms of accessibility (physical location)
Demand	Local demand
	Active recruitment of patients
Patient needs and resources	Community pharmacist timings need to be accounted especially and should be accessible beyond school and office hours
	Patients' satisfaction with the service
	Relatively short waiting time (patient perspective of getting in CP)
Patient needs and resources/Organisational factor: Data management system	Documentation on vaccination/reminder for patient
Peer pressure	External pressure for broad provision of the vaccination
Organisation (pharmacy) Factors
Structural characteristics	Waiting facilities need to be considered especially when administering to older people
Staff	Number of pharmacists and technicians (and enough skilled and trained staff)
	Staff, including technicians, nurses, pharmacists, students (medicine, pharmacy)
	Limited pharmacy workforce
Layout and workflow	Room/Space for vaccination (external flat/containers/office spaces/hotels)
	Community pharmacy infrastructure should support administering in private consultation room/appropriate space within the pharmacy to perform vaccination
	Availability of the waiting area/observation area/space (inside)
	Provision of social distancing measures (inside)
	Hygiene concept (general for the vaccination site; separation from infected people)
	Preparation area (laboratory)- workflow (inside)
	Waste removal
Teamwork	Definition of process, roles and responsibilities in the team
Readiness for implementation	Rearrange internal workflow (compatibility of intervention to existing systems)-
Leadership engagement	Head of pharmacy/pharmacy manager acceptance
Available resources and training	Availability of internet/telephone/documentation to report the adverse drug reactions
	Long opening hours in pharmacies
	Software to register vaccination to link to national surveillance system
	Correct storage of vaccines at pharmacy (stock)
	Storage space and equipment (fridge), for open vial
	Material for hygiene/safety
	Material for emergencies
	(Lack of) Time
Access to knowledge and information	Availability of software/literature for drug-drug interactions
	FAQ sheet/database for pharmacy staff
Data management system	Access to central database to view and record patient and intervention data/availability of the medication patients records in order to review the health condition of the patients and history of previous illness (including COVID)
	Follow up of patients for second dose
Quality assurance system	Inclusion and exclusion criteria for vaccination (risk factors) (check list and priority lists)
	Collection of additional data (drug information) post marketing
Environmental stressors	Workload
Organisational support and assistance and Access to knowledge and information	Clear information at the pharmacy on when, how and why to be vaccinated
Organisational support and assistance	Social media
Experience	Previous community pharmacy experience with vaccination (i.e. flu)
Individual (pharmacist and other pharmacy staff) factors
General knowledge	Knowledge about patient target group
Knowledge about the innovation	Pharmacy staff knowledge (pharmacists and technicians) education about immunization with different types of vaccines (indications, contraindications, adverse drug reactions)
Beliefs about the innovation	Concerns about adverse events by pharmacy staff
Technical skills (experience, capacity and competence)	Staff identifying eligible patients
	Expertise in recognized inclusion and exclusion criteria for vaccination (risk factors)/check list
Interpersonal skills (experience, capacity and competence)	Patient counselling about vaccination
	Community pharmacists’ patient dealing skills (social skills) in order to help minimize vaccine/injection fear/reluctance
	Communications skills of pharmacists
Values and motivation	Staff acceptance/willingness
	Pharmacist motivation to contribute to control the Covid –19 pandemic

*FISpH Framework for the Implementation of Services in pHarmacy [33]

Qualitative findings

The different background of participants sparked substantial discussion regarding the best way to describe the factors and whether a named factor, for example “vaccine prescription”, should be included. It was decided to include any potential factor and let the prioritisation activities decide about its importance. Discussion about the preferred wording, so that it was precise and clear but also broad enough to allow for international differences, led to wordy factors such as “information on vaccination status/vaccination passport (access to online patient record/tool of health care system)”. We showcase the group discussions using examples from three domains.

External system

In general participants viewed pharmacists’ training on correct vaccination technique as highly important. While some participants argued that training sessions would be organised by a pharmacy or pharmacy’s headquarters and were therefore an organisational factor, most participants agreed that the training would be delivered by an external entity, by doctors, a specialised training company, or the national pharmacists’ training centre in their respective countries. To avoid compromising any existing medication services, participants also discussed that minimum standardised requirements needed to be developed before authorising any community pharmacy for COVID-19 vaccination.

Local setting

Participants shared that the overall acceptance and perception of receiving COVID-19 vaccine in community pharmacies might be influenced from previous encounters of receiving flu vaccines or other patient services in their local community pharmacies and thus act as a facilitator towards acceptance of the service. However, other participants thought that prior experiences may act as a barrier if people were not satisfied in the past. Further discussion added that if social desirability to receive the vaccine could be inspired, people might enjoy the flexible time schedule community pharmacies would be able to offer as compared to vaccination centres and primary care facilities and people might prefer getting their vaccine when they filled their repeat prescriptions and save another trip.

Organisation (pharmacy) factors

Participants discussed the impact of opening and closing hours of community pharmacies, location, and in particular pharmacy layouts. Some stressed that additional space would be needed to ensure social distancing and to separate potentially infected patients coming for medicines from those coming to be vaccinated. One participant suggested external sites could be employed, such as unused offices or hotel rooms, for the vaccination service while others opposed that working as pharmacist off premises would not be allowed in their country. As with other factors it was agreed to list the factor and then let the voting decide on its importance and changeability. Several participants emphasised that offering the vaccination service would entail new roles and responsibilities within the pharmacy team, with the need to communicate this clearly beforehand to ensure a smooth workflow. Many participants highlighted that the pre-existing shortage of skilled and trained staff would be a major barrier to the implementation of the vaccination service, but participants also concurred that pharmacists showed a high motivation to fight the pandemic and would therefore accept a higher workload.

Quantitative findings

Implementation factors pertaining to the organisational/pharmacy level were most frequent (n = 32), followed by implementation factors relating to the wider external setting (n = 22), the local setting (n = 16), characteristics of individuals involved in the implementation of the service (n = 10) and finally factors relating to the intervention characteristics [the vaccination programme] (n = 5) [see Table 2].

Table 3 lists 14 implementation factors that were deemed important, and Table 4 lists 13 implementation factors deemed changeable by the 4 groups. 6 implementation factors were ranked as important and as changeable. The top 3 factors in terms of importance and changeability were the same for both criteria. These were: (1) (Regular) Staff training on correct vaccination procedure/to perform hands-on injection technique; (2) Training in basic life support/first aid (Cardiopulmonary resuscitation), and; (3) Definition of process, roles and responsibilities in the team. In addition, (4) Hygiene concept (general for the vaccination site; separation from infected people); (5) (Regular) Staff training on correct patient risk assessment for vaccination and; (6) Room/Space for vaccination (external flat/containers/office spaces/hotels) were considered both as important and changeable.

6 of the 14 implementation factors rated in the participants top 5 factors for importance were related to “External support and/or assistance”. “Remuneration” was an additional implementation factor rated as important at the external system level. 6 implementation factors related to pharmacy/organisational level, with the final factor being at the local setting level related to the community/patients’ perception or acceptance of the vaccination programme. On the contrary, in terms of changeability only 3 implementation factors were related to the external system (all coded as “external support and/or assistance “), while 7 were at the pharmacy/organisational level (coded within “available resources and training”, “data management system”, “teamwork”, “layout & workflow”), and one factor related to the innovation (“Design quality and packaging”), the individual/staff level (“technical skills” (experience, capacity & competence)) and local setting (community/patients’ perception or acceptance of the vaccination programme) respectively (Table 5).

Table 5

Spread of implementation factors across FISpH domains

Rating criterion	Important	Changeable
Innovation (vaccination programme)	–	1
Individual (staff)	–	1
Organisation (pharmacy)	6	7
Local Setting	1	1
External System	7	3

Discussion

Several implementation factors must be considered to enable successful implementation of COVID-19 vaccination programmes in community pharmacy. These factors can help decision makers to decide and prioritise implementation strategies during the implementation process. Factors can present as barriers or facilitators depending upon the country’s unique health care systems. As such, it is essential each country contextualises the implementation factors for its own setting to develop evidence-based implementation strategies tailored to their unique circumstances [24].

Key implementation factors identified by the 23 workshop participants were related to staff training in vaccination, staff training in basic life support (in case of adverse events), and workflow of the pharmacy. These findings align with a review by Shoemaker et al. who highlighted adequate training as well as compatibility with existing workflow as key facilitators for successful implementation of a service in community pharmacy [37].

As pharmacists are not currently providing vaccines across all European countries, training to ensure pharmacy staff have sufficient capability and capacity is necessary [22]. Trainings could be performed combining online material and demonstration videos along with face-to-face practicum on the application procedure [38]. Pharmacy staff must also be prepared to deal with anaphylactic events due to injection reactions and thus ensure patient safety [13, 38, 39]. In this study “staff training” was identified as the most important and changeable implementation factor. It is interesting to note, “Staff training” can be considered as either an organisational or internal factor, if such training would be delivered within a pharmacy or pharmacy chain, or a system or external factor, if a professional organisation or national body were to develop and deliver such training [40]. Implementation science acknowledges the fluidity of constructs across contexts and that the interface between inner and outer setting is dynamic [41].

Besides the practical skills and knowledge, it is essential the vaccination service is integrated into the workflow of the pharmacy including defining roles and responsibilities within the team. As pharmacies implement COVID-19 vaccination programmes, there will be additional workload including logistics, storage, and appointment management [40]. The FIP has developed a guidance document detailing these aspects with regard to vaccination programmes in community pharmacy in general [42].

This study highlights implementation factors, which require corresponding implementation strategies in European countries. However, the NGT process and findings may be useful for other countries outside Europe. The identified and mapped implementation factors give policy makers a cohesive direction where participants have scientifically ranked factors which were deemed most important and changeable, and which need to be addressed in order to facilitate implementation. This study provides a list of potential implementation factors, which may help to develop, improve, and revisit strategies for new and existing COVID-19 vaccination programmes. It may also assist in post-evaluation implementation research of COVID-19 vaccination programmes in countries where the vaccine(s) have already been rolled out in community pharmacies. As common with implementation science research, any factor (whether a barrier or facilitator) needs to be contextualised and subsequently appropriate strategies can be developed.

Adaptation of an online NGT process

While NGT has been employed for a range of health care questions it is usually associated with face-to face meetings [43-45]. However, an online approach to NGT has been shown to reach the same quality of outcomes in other disciplines [46]. To our knowledge, this is the first study to conduct a NGT online focusing on a health-related research question. The online NGT facilitated the collaboration of international participants during a global pandemic and enabled consensus on the key implementation factors. Video conferencing together with expert facilitation meant communication and discussion flowed easily. During the online NGT, all participants had a chance to express their opinion equally and therefore objective conclusions emerged. Group consideration of individual ideas made this method transparent. To standardise the process, the NGT steps as outlined were followed with spreadsheets documenting the progress. In addition, the online process was both time and cost efficient as well as environment friendly as participants were not required to travel which is particularly advantageous and pronounced when international collaboration is desired [30].

Strengths and limitations

A strength of this study was the diverse participant group. The workshop was attended by participants from 11 countries, with backgrounds across academia, a range of clinical practices, and professional organisations, which provided a rich distribution of perspectives and factors. The diverse group could also be seen as a limitation as implementation factors are context dependent, particularly their importance and changeability. In addition, as discussed, factors may be situated and addressed by different levels of the system depending on the context.

A further strength and limitation of the study, and NGT in general, is the necessity for the research question to be specific and narrow to produce meaningful results [47]. Hence, the research question focused only on vaccines stored between 2 and 8 °C, programmes that take place in community pharmacies in Europe and assumes countries have the legislation in place for vaccination programmes to occur in community pharmacies. To be inclusive of Low- and Middle-Income Countries would have resulted in a list of implementation factors that would have been too broad to be useful. We recommend the process is repeated for these settings.

Due to time constraints and time zone differences, only brief periods of time were possible for collation, clarification, removal of duplicate items, and prospective classification of items under an implementation framework. Additional time to ensure consistent wording of implementation factors would have been beneficial. Furthermore, due to time, final group consensus on the 5 most important and changeable factors was not reached by discussion, but rather was appraised by the 4 sub-group ratings. A factor was considered as overall important or changeable if either a high rate was given by one group, which depicts the importance of the factor for that particular group, or if multiple groups selected the factor [48].

Finally, when the authors were mapping the implementation factors to the respective constructs in the FISpH it became clear that due to the European context of this project more differentiation in the external system of FISpH would be useful. The FISpH was originally developed using practice research in Australia where external influences on service implementation in community pharmacies are less dominant. For example, it would be valuable if the construct “external support and/or assistance” was subdivided into “regional pharmacy chambers”, “national pharmacy board” and “owners’ associations”.

Conclusion

An online NGT process enabled international collaboration in gathering diverse perspectives on the implementation of COVID-19 vaccination programmes in European community pharmacies in a time and cost-efficient manner. “Staff training on correct vaccination procedure”, “training in basic life support” and “definition of process, roles and responsibilities in the team” were deemed the three most important and changeable implementation factors. Conducting NGT online was successfully trialled as a method which was deemed effective in identifying and prioritising implementation factors. The adaptation to online delivery of NGT occurred due to the pandemic. However, the authors believe online NGT could be a suitable method for other research problems, where it is difficult, time intensive and costly to gather participants from diverse geographical locations.