Get waivered remote: Nationwide, remote DEA-x waiver course in response to COVID-19.

PROBLEM: As of 2020, less than 5% of physicians in the United States have a drug enforcement administration-X waiver to prescribe buprenorphine. The coronavirus-2019 pandemic restricted in-person gatherings, including traditional drug enforcement administration-X waiver courses. As a result, in-person conferences have needed to adopt remote formats. Many programs identified a gap between educational delivery and the faculty skills required to deliver content remotely. APPROACH: To address the need for high-quality remote learning, Get Waivered designed and implemented a novel experience for clinicians, called Get Waivered Remote. An educational session was live-streamed via Zoom™. To foster interactivity, like in-person didactic conferences, participants were polled to facilitate discussion among presenters, learners, and facilitators during the broadcast. OUTCOMES: The RE-AIM framework was used for evaluation. Our program had a Reach encompassing 814 users that participated during the live-streamed event; Effectiveness with 73.79% reporting being somewhat familiar or very familiar with the practice of opioid dependency treatment with approved buprenorphine medications; Adoption with 95.15% reporting a favorable experience and 92.23% reporting it was similar or more enjoyable than their usual teaching; Implementation with 450 messages sent by 281 users to engage with presenters and other learners via Zoom chat in real time. NEXT STEPS: Get Waivered Remote provides a proof-of-concept that a broadcast with a concurrent, interactive remote learning platform is feasible, low cost, and simple to execute. Further study is required to assess the ability of our group to maintain this innovation and also to measure its impact on the treatment of opioid use disorder.

Background

Problem

Buprenorphine was approved by the food and drug administration for the treatment of opioid dependance on October 8, 2002, but as recently as 2016 only 10% of patients who needed this treatment had access to medication. One of the challenges patients face in gaining access is the limited number of providers who are able to prescribe buprenorphine. The Drug Addiction Treatment Act of 2000 (DATA 2000) requires physicians to complete an 8 h training course and receive a waiver, often known as a Drug Enforcement Administration (DEA) X waiver. The Comprehensive Addiction and Recovery Act of 2016 expanded the waiver process to nurse practitioners and physician assistants after 24  h of training. However, recent estimates indicate that less than 5% of physicians in the United States had obtained an X waiver to prescribe buprenorphine in early 2020, creating a significant barrier to care for patients with Opioid Use Disorder (OUD).

On January 14th, 2021, in an effort to ease federal requirements to grant more physicians the authority to prescribe buprenorphine, the U.S. Department of Health & Human Services expanded access to medication-assisted treatment (MAT) by exempting physicians that possess DEA registrations from the 8 h training requirement needed to obtain an X waiver. However, deemed published prematurely, these notable guidelines were nullified by the newly inaugurated Biden administration several days later, essentially reverting to the original DATA 2000 requirements.

Furthermore, these unsuccessful legislative changes, compounded by a coronavirus (COVID)-2019, have exacerbated the barriers to care faced by patients with OUD, introducing new challenges to providing patient care, clinician training, and the accessibility and delivery of evidence-based care. Importantly, measures to suppress the spread of the virus, through social distancing, broad limits on in-person gatherings, and self-isolation, create distinct barriers to traditional forms of medical education, including X waiver training courses. As a result, COVID-19 has required, and produced, dramatic innovations in online educational resources.

COVID-19 and the opioid epidemic

COVID-19 has had a disproportionate impact on the already vulnerable patients who struggle with opioid addiction. Patients with OUD are at a higher risk of the negative sequelae associated with COVID-19.[3,4] Protective measures, like social distancing and self-isolation, both exacerbate the drivers of addiction, namely anxiety and isolation, increasing the risk of relapse, while simultaneously decreasing access to treatment. Outpatient clinics and other treatment programs have struggled to provide care in accordance with social distancing guidelines. Ensuring uninterrupted access to evidence-based MAT, with agents such as methadone and buprenorphine, is integral to counteract these additional challenges.

Training related to both the prevention and treatment of opioid overdose with the use of evidence-based prescribing practices leads to improvements in clinician preparedness to address OUD.[6,7] To treat OUD with buprenorphine, clinicians must undergo additional training to receive an “X” waiver from the DEA, yet despite the documented efficacy of MAT, less than five percent of physicians in the United States had obtained an “X” waiver certifying them to prescribe buprenorphine prior to the emergence of COVID-19. Physicians obtain this waiver by completing an 8-h course while nurse practitioners and physician assistants are required to complete a 24-h course. These sessions are generally presented to live clinician audiences in group settings of ten to twenty providers. However, the recently implemented social distancing measures prompted the cessation of in-person training sessions, leading to fewer opportunities to obtain the DEA-X certification.

While successful virtual conferences have been previously described as an effective solution for other educational venues, there has been no large-scale approach to replicate the in-person “X” waiver course using virtual platforms. This paper describes the pilot of Get Waivered (GW) Remote, an interactive educational conference designed and implemented by GW.

Objectives

We aim to describe the implementation of the GW Remote Course and use the reach effectiveness adaptation implementation maintenance (RE-AIM) framework to evaluate our program outcomes to suggest potential opportunities for future improvement.

Approach

The GW team delivered and hosted a nationwide, remote DEA-X waiver training course that would serve to get clinicians their DEA-X waiver using a live, synchronized, and interactive digital platform. We aimed to mitigate the challenges created during the COVID-19 pandemic which hindered clinician education including social distancing requirements that limited in-person gatherings.

Our previous work focused on the three key behavioral patterns that were found to limit the number of clinicians who completed the “X” waiver process. These barriers were identified as: (1) absence of a social norm; (2) hassle bias in obtaining the waiver; and (3) a lack of salience in treating OUD. We tailored the course around these three elements to maximize the user experience.

First, our course sought to bolster social norms around treating OUD using evidence-based practices. On our website, we displayed the session's enrollment numbers and provided real time enrollment data to individuals seeking to enroll. We hoped that interested parties would see that hundreds of their colleagues had already enrolled, increasing their motivation to enroll as well. Previous research has found that faculty and residents were often unclear about their institution's stance on the treatment of OUD which adds to an absence of social norms; therefore, our course was jointly sponsored by the American College of Emergency Physicians, the largest society for emergency medicine (EM) physicians, effectively demonstrating new, pro-MAT social norms at the specialty level.

Second, the GW Remote course sought to decrease the level of hassle bias experienced in obtaining the DEA-X waiver. Hassle bias describes the obstacles that can make a task more difficult, thereby decreasing the likelihood of completing the task. This occurs when individuals delay completing a task or by reducing follow-through. By introducing a remote, digital course focused on maximizing ease of access, the session aimed to decrease this hassle bias. This was accomplished in two ways. First, the traditional course requires that clinicians drive to a central location where the course is held. GW Remote offered clinicians an opportunity to access the training in a more convenient fashion, via an online platform, reducing the hidden financial and time costs associated with traveling to and from the course training location. Second, by laying out an easy-to-follow process to waiver training completion on a user-friendly, navigable website, and presenting information about how to sign up for the waiver process in a streamlined fashion, we aimed to address the administrative complexity associated with completing the waiver process. After completing the free online course, participants were educated on how to submit their certificate of completion to substance abuse and mental health services administration (SAMHSA) to receive the waiver.

Third, our program attempted to mitigate the lack of salience in treating OUD, which also acts as a barrier to obtaining the waiver. Salience refers to the idea that individuals focus on pieces of information that are more prominent to them and ignore those that are less so. In doing so, a bias is created in engaging in behaviors that are more striking and perceptible. This may not be so apparent in treating OUD because the long-term benefits and outcomes in treating these types of patients are often hidden and at times not readily apparent to the clinician who experienced the initial encounter. To counteract these biases, the recruitment materials used to promote the GW Remote course highlighted the urgency of treating OUD in the context of COVID-19, linking waiver training to a highly salient issue among Emergency Medicine physicians in May 2020. Further, the training was delivered by DEA waivered speakers that introduced clinical vignettes containing real-life patient situations highlighting stories of recovery.

Course content and its method of delivery were developed by Providers Clinical Support System. Presenters had experience with DEA-X waiver courses previously delivered in-person. Clinicians (faculty, residents, nurse practitioners, and physician assistants) and medical students and nurse practitioner (NP)/physician assistant (PA)s in training were eligible to register for the course. The didactics were delivered via the Zoom platform on May 20th, 2020 from 10 AM to 6 PM Eastern Standard Time, totaling the required eight hours. During this time, participants studied a series of modules covering all aspects of treating OUD patients with buprenorphine. The course allowed for participants to interact in a variety of ways, both within the zoom platform and on external social media platforms (Figure 1). In addition to the core presentation, the digital, remote platform allowed individuals to ask questions which were then answered in real time by additional staff with clinical and administrative experience with both completing the waiver process and prescribing buprenorphine. To measure the engagement of the participants, poll questions were sent to all participants during the course to measure learning and retention. Additionally, the event was made free for participants through sponsorships from the foundation for opioid response efforts (FORE) foundation and by a grant from SAMHSA.

Figure 1.

Get Waivered Remote format and delivery.

Outcomes

Methods

We utilized the RE-AIM model to evaluate our training session's outcomes. The five RE-AIM dimensions include the following, Reach, Efficacy, Adoption, Implementation, and Maintenance. These categories are examined at both the level of the individual and the organization to assess the delivery, reception, and overall efficacy of public health interventions. According to Glasgow et al., “reach” is a measure of individual participation; more specifically, it refers to the percentage of individuals in a defined population who are affected by a new program or policy, broken down by demographic categories. “Effectiveness” measures outcomes of the program in terms of end results, in this case, completing the GW Remote course. “Adoption” is the proportion of individuals or organizational settings that affect change to adopt a given policy or program and corresponds to predicting future behavior. “Implementation” refers to the extent to which the program is delivered as it was originally intended. Finally, “maintenance” is assessed at an organizational and community level to evaluate the extent to which the policy becomes routine and part of regular practice. Below, we report results for the first four of the elements of the RE-AIM model. Appendix 1 contains a matrix of our data sources and analyses and Appendix 3 addresses the survey development and deployment process.

Results

Reach

All physicians, nurse practitioners, physician assistants, and medical students were invited to participate. In total, we received 1179 enrollments during the enrollment period. A total of 814 participants joined the Zoom meeting delivered on May 20th, 2020, completed a precourse survey, and participated in the first GW Remote pilot. Of those who participated, 40.79% (n = 332/814) were physicians, 18.06% (n = 147/814) were residents, 3.44% (n = 28/814) were fellows, 5.90% (n = 48/814) were nurse practitioners, 14.13% (n = 115/814) were physician assistants, and 17.69% (144/814) were medical students. 59.46% (n = 484/814) participants learned about Get Waviered Remote via email, 15.97% (n = 130/814) from a text from a friend, 7.13% (n = 58/814) from the GW Website, 6.51% (n = 53/814) from Facebook, 6.14% (n = 50/814) from Instagram, and 4.79% (n = 39/814) from Twitter. Postcourse survey revealed that 55.34% of participants were in or entering Emergency Medicine as a specialty (n = 57/103). Between August 6, 2019–June 6, 2020, the GW Remote website's Google Analytics revealed: (1) Users: There were a total of 8919 users (Tables 1 and 2), with session durations of 1:32 and 1.73 page views, on average. Of users with identified genders, 65% were male and 35% were female. (2) Site Content: Of a total of 18,164 page views, 5711 (31%) were on the Enrollment page and 2637 (15%) were on the Home page. Of the 5711 sessions beginning on the Enrollment page, 4437 (78%) exited (including to the registration page) and 1274 (22%) continued to other pages. (3) Sources: 70% of traffic to the site was Direct (visitors who arrive directly by typing the uniform resource locator into the address bar, clicking a bookmark, or clicking a link in an email, text message, or chat; can be a strong indicator of brand strength), 13% was Social (e.g. Facebook, Twitter, Instagram, etc); 13% was Organic (arrive via a search results page, e.g. Google, Bing, etc., and can be a strong indicator of strong content or Search Engine Optimization (SEO); and 4% was Referral traffic (arrive at the site via another site). Engagement for Organic and Referral traffic was 2.3 and 2.8 pages per session, and 2:17 and 4:37 min per session, on average, respectively. In comparison, users from Direct and Social sources visited 1.6 and 1.6 pages per session, and spent 1:16 and 0:52 min per session on average, respectively.

Table 1.

Top 10 US cities of those visiting the get Waivered remote website.

New York	296 (3.56%)
Ashburn	278 (3.34%)
(not set)	271 (3.26%)
Boston	218 (2.62%)
Los Angeles	168 (2.02%)
Chicago	164 (1.97%)
San Francisco	149 (1.79%)
San Diego	114 (1.37%)
Des Moines	101 (1.22%)
San Antonio	97 (1.17%)

Table 2.

Top 10 countries of those visiting the get Waivered remote website.

United States	7643 (95.24%)
Canada	42 (0.52%)
India	36 (0.45%)
China	33 (0.41%)
France	33 (0.41%)
Germany	24 (0.30%)
United Kingdom	20 (0.25%)
Ukraine	17 (0.21%)
Philippines	12 (0.15%)
(not set)	12 (0.15%)

Effectiveness

Out of the original number of participants that enrolled, 69.04% (814/1179 participants) actually joined the Zoom course delivered live on May 20th, 2020 at 8 AM EST Out of the 814 providers that participated in the course, 98% (798/814) successfully completed all modules, readings, questions, and tasks during the 8-h didactic instruction to receive a certificate of completion. The precourse survey of participant familiarity with the practice of opioid dependency treatment with approved buprenorphine medications revealed 9.71% not at all familiar (n = 10/103), 25.24% not so familiar (n = 26/103), 42.72% somewhat familiar (n = 44/103), 16.50% very familiar (n = 17/103), and 5.83% extremely familiar (n = 6/103). Post survey of participant familiarity with the practice of opioid dependency treatment with approved buprenorphine medications revealed 0% not at all familiar (n = 0/103), 1.94% not so familiar (n = 2/103), 24.27% somewhat familiar (n = 25/103), 50.49% very familiar (n = 52/103), and 23.30% extremely familiar (n = 24/103).

Adoption

We surveyed participants about their experience with GW Remote using a postcourse survey, with a 12.65% response rate (n = 103/814 participants). 95.15% rated their overall experience about what they had expected or better (n = 98/103). When presented with a likert scale and asked how likely one would be to recommend the remote course to a friend or colleague, 88.35% (n = 91/103) rated a score of 7 or greater. When compared to standard in-person courses, 92.23% of participants (n = 95/103) felt that course delivery via the digital remote platform was equivalent to or better than their usual academic proceedings.

Implementation

When assessing Q&A comment features, we found that 14.67% of total comments (n = 66/450) were related to a technological issue. 34.52% (n = 281/814) of participants interacted with others enrolled through the chat function or Q&A, sending a total of 450 messages over the course of the 8 h of the course (Tables 3 and 4).

Table 3.

Question-and-answer data poster's intention.

Poster's intention	Number of posts	% of total (n/total)
Seeking information	241	53.32 (241/450)
Seeking discussion	87	19.25 (87/450)
Nonquestion	112	24.78 (112/450)
Answering a question	7	1.55 (7/450)
Furthering discussion	3	0.66 (3/450)
Total	450	100

Table 4.

Question-and-answer data comment features.

Comment features	Number of posts	% of total (n/total)
Providing factual information	6	1.33 (6/450)
Providing personal experiences	12	2.67 (12/450)
Making an inquiry—administrative questions about the Waiver process	56	12.44 (56/450)
Making an inquiry—course content	152	33.78 (152/450)
Making an inquiry—technologically related	66	14.67 (66/450)
Making an inquiry—attendance/course credit	80	17.78 (80/450)
Providing opinions	23	5.11 (23/450)
Requesting resources	34	7.56 (34/450)
Off topic	21	4.67 (21/450)
Total	450	100

In summary

The GW Remote course was an effective solution for delivering live, real time educational content while limited by in-person gathering restrictions due to COVID-19. The delivery platform also allowed for collaborative interaction among speakers, moderators, and attendees, something rated positively by participants. Our digital conference implementation and delivery style may provide a framework for future medical education conferences set to impact public health policy across the nation.

Discussion

GW is an initiative that was founded with the goal to expand the waiver network by encouraging and assisting clinicians to get their DEA-X waiver. The GW team includes highly motivated individuals ranging from board-certified physicians with tremendous experience in emergency medicine to aspiring physicians of various backgrounds looking to better understand and act on the opioid crisis.

The GW Remote X waiver training course is a novel implementation of the classroom-based DEA-X waiver training course and the largest known course at the time of this manuscript. It has the potential to scale the DEA-X waiver course to a national level among healthcare providers in the setting of COVID-19 and beyond. It was seen as a convenient option for providers interested in completing their X waiver training, successfully reducing the hassle bias associated with low completion rates.

Further, the postsurvey results showed that remote courses can be effective educational opportunities for medical professionals, comparable to or even better than, in-person courses. This indicates that training opportunities can be made more convenient to providers, thereby increasing the number of providers who complete additional training, without reducing the quality of the course.

Results of the comment analysis suggest that most participants that submitted comments sought to obtain information with the intention of receiving a real time response from course moderators or facilitators rather than from other participants. Among this predominantly unidirectional comment thread, the most frequent poster intentions were contributions either seeking information or nonquestion comments.

It may be worth further investigating why there was not much bidirectional conversation among the course's participants. Furthermore, the implications of more or less bidirectional interaction in real time comments are not known. Therefore, in future studies, surveying the level of participation with others via questions, comments, and answers may be important in gauging the overall user experience and success of course delivery.

Most participants posted questions about course content, receiving course credit, and others submitted concerns primarily related to the remote learning technology. Potential reasons for the last two topics include heterogeneity in user technological ability, connectivity issues, uncertainty on the impact of switching devices, and lack of periodic attendance checks. Clearer communication on record-keeping, particularly that the webinar platform automatically records user participation would be helpful in the future.

Finally, our framework can be extrapolated to other campaigns and other new educational experiences that may allow for audiences to have the ability to ask questions in real time to concept experts. This application will allow those to fulfill learning collaboratives organized by authorities from other institutions. Lastly, the final component of the initiative takes a proactive step towards finding new organizations and members to adopt a protocol for more expansive coverage.

Next steps

Overall, of the 8919 users who visited the site, representing a 13% conversion rate. It could be worth investigating why there were nearly double the number of male users compared to female users. It may be worth focusing development efforts on the Enrollment and Home pages, which together receive nearly half of all traffic. More than one-fifth of all users who visit the Enrollment page continue to other pages on the site, which may indicate that users want more information before registering. Though the top sources of traffic were Direct and Social, which would encompass GW's email and Instagram/Twitter campaigns, higher engagement came from Organic and Referral traffic, which indicates a stronger ROI from content developments, SEO, and public relations.

The final dimension of the RE-AIM model is to examine maintenance. According to Glasgow et al. maintenance is evaluated on both an individual and organizational-community level. The measurement examines the maintenance of behavioral change, and in our case, this includes the number of clinicians utilizing the evidence-based approach of prescribing buprenorphine in the treatment of OUD. Maintenance also measures the extent to which the implementation of the policy becomes stable overtime. Given the nature of this dimension's evaluation, this assessment will require more time for adequate evaluation. We plan to follow up with participants in the future to survey the impact the waiver course had on their prescribing practices, which will allow us to assess the extent to which a remote waiver course helped overcome the obstacles clinicians face in providing addiction treatment during the COVID-19 pandemic. Given widespread OUD, real time, digitally delivered, interactive medical education courses such as GW Remote have the potential to positively impact the treatment of patients nationwide.

Table 5.

Contains a matrix of all the data sources and our analysis approach for each.

Platform	Types of data	Analysis
Google analytics	Categorical and numerical dataGeographical demographic, gender, and traffic source estimation data	Qualitative and quantitative analysis of descriptive statistics within Google analytics results, including the following metrics:
		1. Average concurrent users over time 2. Descriptive analysis for a number of page views, clicks, and users. 3. Traffic sources metrics from Google dashboard 4. Estimated geographical and gender percentage metric from Google dashboard
Zoom	Categorical data Poster intention Comment Features	Qualitative, nominal data—simple thematic analysis, assigning groupings to comments by common theme
Precourse survey	Categorical and numerical data Those participating in the live Get Waivered Remote event Open-ended questions	Qualitative data—ordinal and nominal—simple categorical analyses made
Postcourse survey	Categorical and numerical data Overall experience post event recorded Open-ended questions	Qualitative data—ordinal and nominal—simple categorical analyses made