Virtual accreditation visits for pharmacy programs in light of the COVID-19 pandemic: Team members' perspective.

PURPOSE: This wisdom of experience commentary, from peer academic reviewers serving on accreditation teams, will discuss benefits and challenges of international and national virtual accreditation visits (VAVs) using a "What? So What? Now What?" reflective model. DESCRIPTION: Onsite accreditation reviews for health professional education programs require investments in time, effort, and money to maintain program alignment with accreditation standards and continuously generate quality practitioners. When COVID-19 entered the accreditation world, reviewers had to pivot modalities to a VAV format. ANALYSIS/
INTERPRETATION: Adaptation and expectations of VAVs present several challenges. Barriers and advantages will be discussed as well as implications for the future. While medical and pharmacy education standardization has long been established, the authors propose national and international accrediting bodies will utilize the ingenuity of emergency COVID-19-driven onsite accreditation alternatives to develop protocols for novel accreditation methodology.
CONCLUSIONS: Whether the continued mutation of COVID-19 prevents the return to previous accreditation visits or not, the experiences gained from the emergency-driven VAV, can inform and enrich accrediting bodies knowledge, theories, and practices of future VAVs. IMPLICATIONS: Higher-education institutions, accreditation bodies, and government entities will use experiences during COVID-19 to transform and improve academic requirements and future practices. Even if there is a full return to onsite reviews, such guidelines or improved versions of them can be applied to situations where immobility or restricted mobility is an issue, such as in illness, pregnancy, travel, war, etc. It is crucial for educators and accrediting bodies to evolve as we navigate these unprecedented times.

Purpose

Accreditation agencies are essential for continued quality assurance of educational programs.1, 2 The amount of time and manpower to evaluate health professional programs can be both laborious and costly. Nonetheless, education standardization is necessary to ensure achievement of learning outcomes, the success and longevity of educational programs and the institutions offering them, and the wellbeing of subsequent patients which benefit from competent health professionals. The COVID-19 pandemic forced many health professions education providers, including pharmacy, to take their learning and teaching activities online, and we believe this same virtual transition model may be a windfall to reduce cost and time burdens on accreditation agencies while maintaining high standards of qualifications.

Through experiences with both national and international virtual accreditation visits during the COVID-19 pandemic, we believe a model shift from onsite to virtual accreditation visits (VAVs) may become the new “normal” and a benefit to all of those involved. This wisdom of experience article aims to provide personal observations, ideas, and commentary from a group of United States and international accreditation team members to address virtual quality assurance issues and challenges in a post-pandemic world. Using the “What? So What? Now What?” reflective model this manuscript will also discuss positive and negative effects that reviewers face, changes to their practices, perceptions, and potential opportunities.

Description: what?

During 2020, many schools and colleges of pharmacy around the world were asked to cease face-to-face campus instruction and immediately withdraw students from experiential training sites and hospitals due to the COVID-19 pandemic. Immediately, there was an urgency-driven conversion of pharmacy curricula to an online distance learning format. To maintain the quality of education and to achieve the required outcomes, educators and preceptors were called upon to become innovative and creative. Similarly, during the COVID-19 pandemic, accrediting bodies lost the ability to conduct onsite accreditation reviews due to rapid institutional closures, travel restrictions, and social distancing mandates. The initial response by some regulators was to postpone the scheduled accreditation site visits to later dates. Very few responded by quickly shifting to VAVs as they believed that pandemic-imposed restrictions would be short lived. It was not until later in the process that accrediting bodies started to slowly embrace VAVs.

The difference between an onsite visit and VAVs in theory seems simple enough. Instead of packing, traveling to a destination, spending the night in a hotel, and reporting each day to the college undergoing a review, one simply awakes from their own bed, reports to a room in their home, and signs on to their computer or laptop. As we all learned from 2020, working from home in your sweatpants is a perk for many. However, VAVs are not without challenges. The Table outlines six different observations experienced by team members and indicates the arbitrary relevance for each condition, national or international. These include observations relating to travel, time zone differences, computer technology, communication/cultural immersion, virtual site walk throughs, and variability among reviews.

The team members felt one of the most important differences with VAV when compared to onsite visits centered on technology and connectivity. This is common with VAVs involving reviewers form different countries or sites in different countries than the accreditation team, where differences in computer communication software platforms (Zoom [Zoom Video Communications], Skype [Skype Technologies], Go-to-Meeting [LogMeIn], Microsoft Teams [Microsoft, Corp.], etc.), connection stability and speed, devices, and time zone make the process cumbersome. For example, for one international site visit in the United Arab Emirates (UAE) the nine-hour time zone difference required a domestic wake alarm for two o'clock in the morning to attend simultaneous meetings. Yet for some, this might be a small price to pay for comfort and to sleep in your own bed.

Another challenge faced with VAVs is meeting people online is simply not the same as in person. Soft skills and body language interpretation are often blunted on a screen. When participating in question/answer sessions, mute buttons, hand raising toggles, and chat box windows are not the same and introduce barriers in communication. Another difference experienced by the team was the inability to physically walkthrough the college site under review. The VAV team had to solely rely on what was presented to the team through videos. The dimensional loss from three to two on a computer screen has potential to lose important details and gloss over areas not wanting to be highlighted by the schools. This may or may not paint an accurate picture of the school under review.

Nonetheless, VAVs continue to be an option for accreditation reviews and with the continued mutation of COVID-19, it is unlikely we will completely return to the way accreditation onsite visits were conducted in the past. Using experiences gained from this emergency-driven online VAV, we can develop our knowledge and theories of future VAVs.

Analysis/interpretation: so what?

After participation in both national and international VAVs over the past year, our team identified several challenges for consideration. For the most part, the challenges associated between national and international VAVs are similar but with slightly different levels of importance as experienced by the team. (See Table 1 ) These differences included travel, time zone, and communication/cultural immersion.

Table 1

Observations from team members on virtual accreditation visits.

What? (descriptive level from observations)	Nationallya experienced	Internationallya experienced	So What? (assumptions, conclusions, beliefs)	Now What? (actions)
TRAVELReduced expenses for travel and board of the team.	X	XX	VAVs cost less money and may be an economical choice for international or interim accreditation visits.	Additional funds can be used to develop other services or incentives to benefit quality of team members.
BIOLOGICAL CLOCKTime zone differences.	X	XX	Potentially affect team members' cognition and level of participation.	Select reviewers within a certain distance of site to ensure similar time zones.
COMPUTER TECHNOLOGYTechnology for communication is dependent upon individual accreditation team member's resources.	X	X	Variances in software may be outdated or require additional support. Delays in team members' participation due to technology issues may affect quality of review.	Additional funds from minimized travel may be used to ensure equitable technology resources.
COMMUNICATION and CULTURAL IMMERSIONAccreditation team members meet team and site members virtually.	X	XX	Relationship building and communication during VAVs may be hindered.	Develop virtual forums, games, or meet and greet sessions to promote interpersonal dynamics.
VIRTUAL SITE WALK THROUGHAccreditation team members review site virtually through videos.	X	X	Professionally developed videos may not accurately reflect reality of site and pose ethical considerations.	Propose VAVs include a live component walkthrough with time for Q/A to ensure accuracy and accountability for virtual tours/videos.
VARIABILITYVariability in visit format, onsite, fully virtual, or hybrid.	X	X	Quality assurance standards are similar, but the effectiveness of their verification might be negatively affected by the visit format. Virtual visits might be inadvertently biased toward institutions with more robust electronic learning infrastructure making them more comfortable engaging in a virtual visit and enhances their ability to provide evidence electronically.	Alert programs and institutions to the importance of having all documentation electronically available, accessible, and securely sharable. Provide awareness/training activities on who will conduct VAVs and expectations.

Q/A = question and answer; VAV = virtual accreditation visit.

X indicates the arbitrary importance experiences in setting by team members conducting either a national or international accreditation site review.

With the lack of travel, one barrier to consider is the lack of accompanying acclimation to time zone differences, especially for international accreditation reviews. This requires reviewers to engage in meetings outside the normal working hours or inconvenient times. This could potentially affect team members cognition and level of participation due to sleep deprivation. Additionally, team members may still try and conduct regular work during the business day and not be fully engaged in their accreditation review duties.

Other challenges are the inability to inspect the physical facilities and the lack of face-to-face social interaction between the stakeholders and the visiting accreditation team members. In the absence of onsite presence, team members will not be able to experience the culture of the school and/or country. This is especially true for international visits. As such, team members might need to undergo psychological adaptation in terms of VAVs. This includes any apprehension of adequacy of interpersonal interaction provided by virtual meetings, confidentiality of the process, and willingness to give more control over to institutions for visit proceedings. During traditional on-site visits, interpersonal skills are not only important for the reviewers' and institution members' interactions, but also for the deliberation and discussions among members of the visit team itself. Another similar issue with some members of national and international VAV teams is the notion that virtual meetings might not be as engaging as onsite meetings. This requires awareness and training within the accreditation bodies, among accreditation teams, and in the academic community. Other areas concerning accreditation teams, albeit to variable extents is the evaluation of programs that quickly adapted to an online distance learning (ODL) model. As faculty had to quickly pivot from live to virtual education due to COVID-19, the site team may need to adjust their expectations of the site, faculty workload, and faculty development as this transition greatly affects a multitude of factors.

It is undisputed that ODL directly affects course design, content, delivery, and student assessment. Effects also extend to other pharmacy accreditation standards such as faculty online skills, online training, professional development programs, and teaching load calculations. Accreditation teams are mainly concerned with the quality assurance of education. However, in a remote learning environment, accreditation teams are concerned with student services and support, students' orientation programs, training, and equitable access to technology and support. Of paramount importance is mental and social support for students and faculty in online learning environments. ODL also broadens the definition of the academic campus, resulting in the need to enhance learning resources and information technology support to include content creation facilities and tools, online communication and instruction platforms, and online assessment procedures. ODL also expands the definition of education crisis management. All these aspects might be quite challenging to accreditation reviewers. More specific to pharmacy, accreditation teams may need to require the addition of tele-health, drive-thru vaccinations/virus testing, and tele-pharmacy to clinical rotation(s) or ask for a standalone tele-pharmacy rotation in advanced pharmacy practice experiences to accommodate emerging health care practices that have gained momentum as result of COVID-19 and are becoming a part of the new normal. ,

Accreditation teams had to adapt to the distance nature of VAVs and were required to review programs that quickly put in place nontraditional and online equivalents of normal in-person instruction to address the COVID-19 restrictions. As the primary responsibility for evaluating the quality of academic programs lies with the accrediting organization, the effectiveness of VAVs could be hard to predict and the efficacy of evaluations and reports from the VAV teams in comparison to previously assessed curricula may be difficult to reconcile. Moving forward, this is something accrediting bodies must consider.

Conclusions: now what?

Through our experiences with VAVs, we question issues central to the educational quality assurance process: the readiness of virtual teams to participate in a non-traditional review setting (VAVs), the accreditation review of programs that have significant virtual components due to the rapid conversion from the COVID-19 pandemic, and the availability of robust standards to rely on in the era of digital education. To facilitate the process of a streamlined quality assurance program for a post-COVID-19 world, accreditation bodies and team members around the globe should share their experiences and best practices during this pandemic, especially those who have integrated ODL in its accreditation standards and have already transitioned to VAVs or blended reviews.

When designing and preparing teams for VAVs in the future, we have identified key areas for consideration (Table 1). VAVs have advantages and present potential opportunities as they would have a reduced cost benefit since they require no travel or hotel accommodations. Additionally, a blended or hybrid VAV format could allow institutions and accrediting bodies to organize shorter onsite visits that would increase the efficiency of the accreditation process. Also, they would allow flexibility in choosing the reviewers from any country without travel and visa limitations. VAVs can potentially provide extended coverages to places and times where and when travel is not possible. Additionally, expenses saved from travel have the potential to fund other worthy causes which might have been put on hold due to tighter budgets. These funds could be used for virtual accreditation training modules or used to ensure equitable technology resources by purchasing reliable technology equipment for team members. VAVs also open the opportunities for new work dynamics as the experience gained from organizing and engaging in VAVs might present practices that can be adopted as norms in the future. Other areas for consideration include development of virtual interactive sessions to facilitate interpersonal dynamics and comradery among reviewers. When considering the site facilities, another consideration would be the inclusion of live questions/answers facilities tours be used ensure accountability among schools. Finally, one question remains, how and/or should we compare previous onsite visits to VAVs or a hybrid VAV?

It is our hope that these observations may facilitate conversations with accrediting bodies to permanently add or include a hybrid version of VAVs to their armamentarium of tools for accreditation visits. However, an alternative to VAVs can be to simply continue postponing all visits until circumstances allow onsite visits. Accreditation bodies could defer visits for currently accredited programs with minor or no changes to their curricula, and instead arrange for virtual review of initial accreditations or programs with major curricular changes. With the ease of pandemic restrictions, a blended or hybrid model can be adopted, with some reviewers on site and others working remotely, or a hybrid model, with some tasks done online and other tasks done onsite. In a blended setup, members of the accreditation teams participating from distance will feel more assured as they rely on those colleagues who are onsite for input, clarifications, and feedback. In a blended or hybrid model, accreditation teams would review the self-study and enclosed documents from their desks then could hold virtual meetings with the institution and each other to resolve ambiguities and gain clarifications efficiently and effectively.

Even after considering perspectives from team members who participated in VAVs, an underlying question remains, how much of the activities of accreditation visits need to focus on the quality assurance of the ODL mode of delivery, which was overwhelmingly adopted by all pharmacy programs? Also, do the accreditation teams need to consider it a temporary shift in focus, or a more transformational change in instructions? A follow-up question then emerges: do accrediting bodies have sufficient coverage of ODL in their standards and quality criteria for review teams to rely on for their evaluation, and do reviewers have sufficient training to evaluate the ODL mode of delivery? The answers to these questions were variable and depended heavily on the set of the quality assurance standards used. For example, the standards of the Commission for Academic Accreditation of the UAE include a specific annex for e-learning which has been used by accreditation teams to audit the readiness of higher education institutions for ODL delivery and guided institutions in their transition into ODL and efficacy demonstration. Other accrediting bodies considered ODL just one of the instruction methods used in the programs' delivery and relied on institutions to provide evidence for its effectiveness and comparability to face-to-face instruction.

Nonetheless, no matter what form accreditation visits assume in a post-COVID world, that format should be able to verify the standards and assess the effectiveness of the program to the same level of the fully onsite review. It is clear as we march later into the pandemic, that there is a global trend to transform education into hybrid-learning models that mix face-to-face instruction and training with ODL activities.

Implications

Higher education institutions, accreditation bodies, and government entities will use their experience during COVID-19 to transform and improve their future practices. Although the current situation is dynamic and unpredictable, by making the most of the current social distancing mandates, new guidelines should be placed to better suit VAVs. Even if there is a full return to in-person education, these guidelines will be present and improved to serve those who cannot attend in-person due to illness, pregnancy, travel, convenience, war, etc. Striving for better online education practices is the intention of all involved in professional education and accreditation. Another intention is to utilize better methods to educate students online while maintaining the quality provided by face-to-face education. The COVID-19 pandemic has transformed people's views on life and reminded us that life is ever changing. The same transformation is true for health care and professional education. It is crucial for educators and accreditation bodies to evolve as we enter these unprecedented times.

Declaration of Competing Interest

None.

Disclosure(s)

Abeer M Al-Ghananeem (Accreditation Council for Pharmacy Education (ACPE) International Commissioner) and Amjad M Qandil (Commission for Academic Accreditation (CAA) Commissioner) would like to disclose that these are their personal opinions and they do not necessarily reflect the views of ACPE and CAA.

Author statement

All authors certify that they have participated sufficiently in the manuscript, including participation in the concept, design, analysis, writing, and revision of the manuscript.

Authors would like to express their acknowledgments and special thanks of gratitude to Mike Rouse (Director, International Services, Accreditation Council for Pharmacy Education (ACPE) for his review and constructive comments.