COVID-19-related adaptations to the implementation and evaluation of a clinic-based intervention designed to improve opioid safety.

The United States faces an opioid crisis with an unprecedented and increasing death rate from opioid overdose. Successfully reducing the rates of opioid use disorder (OUD) and overdose will require the engagement of frontline clinicians to prescribe opioids more safely and to build their capacity to treat patients with OUD using evidence-based approaches. The COVID-19 pandemic has created significant challenges for patients, clinicians and health systems and has been associated with increasing risks of overdoses and deaths. Herein, we review a multidisciplinary project designed to implement and evaluate clinic-based interventions in Oregon, USA, to improve pain management, opioid prescribing and treatment of OUD. The intervention, called Improving PaIn aNd OPiOId MaNagemenT in Primary Care (PINPOINT), combines practice facilitation, academic detailing and education through the Oregon ECHO Network. Implementation of PINPOINT has occurred across the Oregon Rural Practice-based Research Network and has involved 49 clinic sites to date. To evaluate the impact of the intervention, the research team created the Provider Results of Opioid Management and Prescribing Training (PROMPT), a dataset that links information from the state prescription drug monitoring program, all-payer claims database, emergency medical services, vital records and substance use disorder treatment system. The PROMPT dataset will allow evaluation of the impact of the intervention at both the clinician and clinic levels. Due to the constraints of the COVID-19 pandemic, elements of both implementation and evaluation required significant adaptations to continue to meet the original project goals.

Background

The United States faces an opioid crisis with an unprecedented and increasing death rate from opioid overdose. Although the use and misuse of opioids have occurred for centuries, changes over the past 20 years have contributed to dramatic increases in the morbidity and mortality associated with these agents.1 The current crisis began with the use and overuse of prescription opioids for pain, followed by increases in heroin use and associated mortality and, most recently, a shift to the use of synthetic opioids such as fentanyl.2,3 At present, all three contribute to the crisis.4–6 Clinicians, public health officials and policy-makers have implemented a wide range of interventions to address the crisis but evidence of the effectiveness of these approaches remains limited.

In primary care clinics, increases in opioid prescribing started around 2006 and have been linked to a focus on the more effective management of pain, which had been historically undertreated.7–10 Guidelines at the time emphasized the use of long-acting opioids,11,12 marketing campaigns promoted these agents as safe options11,13 and associated incentives14 encouraged medication use. As a result, prescribers started many patients on long-acting opioids and increased doses frequently and aggressively. The harms of this approach are now evident and continue to lead to significant challenges for primary care clinicians in particular. Pain management makes up a tiny fraction of clinical educational curricula, and education on the diagnosis and treatment of opioid use disorder (OUD) is lacking.15,16 For clinicians, partnering with patients to realign existing pain treatments to match the best evidence is time consuming, often unsatisfying and contributes to clinician burnout.17 For patients, chronic pain and OUD carry stigma, which may act as a barrier for both clinicians and patients in engaging with effective treatments.18

Interventions to help clinicians enact beneficial practice changes to reduce the harms from opioid overprescribing and decrease overdose risk require the direct engagement of these barriers. The Centers for Disease Control and Prevention (CDC) initiated the Overdose Data to Action (OD2A) program to support state-level efforts to address the ongoing crisis. In Oregon, the Oregon Health Authority (OHA) led a collaboration of multiple groups to implement an intervention called Improving PaIn aNd OPiOId MaNagemenT in Primary Care (PINPOINT). PINPOINT combines evidence-based approaches to improving clinical care and patient outcomes by connecting directly with clinicians and clinics to identify how specific barriers arise in context and identify pragmatic solutions that can be successfully implemented. Paired with the PINPOINT intervention, an affiliated research team is developing the Provider Results of Opioid Management and Prescribing Training (PROMPT) database to allow for robust evaluation of the intervention’s impact.

The first enrolled clinics were slated to start the PINPOINT intervention in May 2020 but the COVID-19 pandemic created delays and forced adaptation of the implementation plans.

Due to the pandemic, only a subset of the planned clinics began participation in the intervention in May 2020 and many clinics opted to defer participation for one or more later cycles of enrolment. To accommodate this, additional quarterly enrolment cycles were added, now planned for completion in November 2021. Clinics that chose to participate in the intervention during the pandemic were doing so in a changed environment of care delivery with new workflows, adjusted staffing and constantly changing regulations. In parallel, some of the public health entities providing data elements for the PROMPT dataset had personnel reassigned to do pandemic-related work, creating delays in the evaluation plans. In this review, we will describe the PINPOINT and PROMPT interventions and the adaptations needed due to the pandemic.

Structure of the PINPOINT intervention

Funded under the CDC Cooperative Agreement OD2A, the PINPOINT intervention builds on existing public–private partnerships and health system transformation efforts to implement and sustain drug overdose prevention strategies that reflect evidence-based best practices. The OHA – recipient of Oregon’s OD2A funding – contracted with the Oregon Rural Practice-based Research Network (ORPRN) to deliver the intervention.

Established in 2002, ORPRN is a network19 of over 400 primary care clinics that are dedicated to community-based research. ORPRN research is multidisciplinary and the network specializes in intervention studies that translate research into practice, including clinical and practice change research, patient safety research, and randomized controlled trials. ORPRN is also home to the Oregon ECHO Network, a state-level collaboration created in 2017 to enable primary care, payer, and stakeholder engagement in Project ECHO20 telehealth and training.

ORPRN staff members participating in the PINPOINT intervention include the network director, a practicing rural physician-scientist, a project manager, and six practice facilitators responsible for quality improvement (QI) and research implementation in the network’s clinics. Practice facilitators are trained individuals who use a range of organizational development, project management, QI, and practice improvement approaches to build the internal capacity of a practice over time and support it in reaching incremental and transformative improvement goals.21,22 The practice facilitators are regionally based across Oregon and routinely travel to ORPRN member practices and support matches between research topics and clinic and community interests and needs.

The implementation activities of PINPOINT are offered to each participating clinic, allowing the clinic to select activities that best fit their local needs. The activities are as follows:

Regional trainings – the ORPRN team introduces chronic pain management and opioid prescribing and quality improvement basics and assists participants in brainstorming potential targets for practice change.

Practice facilitation – ORPRN practice facilitators support clinics through evidence-based practice change to improve pain management and opioid prescribing (such as The Six Building Blocks of Opioid Prescribing23,24 framework and Oregon ECHO network educational opportunities).20

Academic detailing – PINPOINT staff engage directly with participating clinicians in outreach education sessions, targeting clinician-focused improvement activities.

Learning collaboratives – the ORPRN team assists in linking clinics to exemplar clinics to learn from each other.

For each enrolled clinic, the project commences with a baseline survey that asks clinic providers and staff questions about behaviours and attitudes about opioid therapy for the treatment of chronic pain, knowledge and attitudes about collaborative opioid management and opioid use disorder, familiarity with and behaviour related to opioid prescribing best practices, and opioid-related policies and procedures. Upon enrolment, clinic personnel also participate in a hands-on multidisciplinary regional training aimed at reviewing best practices in chronic pain and opioid management and serving as a primer on clinic-based QI methods. At the regional training, clinic staff use ‘Six Building Blocks’ materials to reflect on their clinic’s current resources and skills, and they work toward mapping an improvement project to address the aspects of chronic pain and opioid therapy that are most needed in their clinic community. After these core activities, each clinic has the opportunity to receive 12 months of longitudinal practice facilitation and/or complementary learning activities. These activities are optional and participation is at the discretion of each clinic. A follow-up survey, which replicates the baseline survey plus five questions about satisfaction with PINPOINT, is sent 4 weeks after the end of the intervention. The intervention was scheduled to begin in May 2020 and clinic-level interventions were planned to enrol 60 clinics in four waves, starting in May, August and November 2020 and February 2021 with clinics self-selecting into each intervention wave.

COVID-19 adaptations

The COVID-19 pandemic created significant delays and a few major adjustments in the implementation of the PINPOINT intervention. Just as the intervention was slated to begin, so began the impacts of the pandemic and primary care clinics experienced dramatic changes in operations. Across the board, ORPRN clinics reduced in-person activity; some were able to rapidly shift to telehealth whilst others were left with major staffing cuts and reduced capacity. For clinics still operating in person, staff roles were often different with new and evolving workflows for all clinical processes and substantial uncertainty about organizational and financial stability. Because of this, many clinics requested deferment of their participation in PINPOINT and some decided not to participate. Few clinics began participating in the project’s first wave – May 2020 – and additional waves of participation (August 2021 and November 2021) were added to accommodate deferred entry into the project.

As of June 2021, 49 clinics (of the planned 60) were enrolled in the PINPOINT project and efforts are ongoing to meet the original target. Participating clinics include both family medicine and internal medicine clinics, of varying sizes, from across the state of Oregon. PINPOINT clinics are independent private practices, clinic-members of larger health systems as well as Federally Qualified Health Centers, Rural Health Centers and other community health centres. Amongst participating clinics, many clinics have opted to limit their participation in available learning activities, citing stresses of the pandemic (particularly as it pertains to staff availability) as the reason. Baseline survey data have been collected from 143 respondents. As almost half of the 49 clinics enrolled or deferred to the February 2021 implementation group, additional baseline surveys will be collected during 2021. Follow-up surveys will be collected from all clinic sites as the PINPOINT intervention is completed.

In addition to the clinic impacts, the pandemic has also affected the operations of the study team. All activities (including the regional trainings, academic detailing and practice facilitation) have occurred via video conference technology and practice facilitators were unable to visit clinics for direct observation and in-person support. The transition to a virtual platform for the intervention led to time and cost savings for the intervention team – no longer having to travel long distances across the state to meet with stakeholders. Even so, many barriers overshadowed these benefits. Many clinics were unfamiliar with the technology and did not have the needed equipment for video conferencing, particularly in large groups. Practice facilitators also reported more difficulty connecting with clinic staff, both because of the social distance created by virtual visits and the inability to drop by the clinics informally and get acquainted with the staff and setting more deeply. As pandemic restrictions ease, the PINPOINT team may have the opportunity to resume in-person support as initially planned; otherwise, the analysis can be adjusted for periods when the direct observation was not possible.

Development of PROMPT dataset

The research team hypothesized that the PINPOINT intervention would result in beneficial changes to clinician opioid prescribing practices, increase referrals to OUD treatment, and decrease opioid-related harms related to overprescribing. Unfortunately, a single comprehensive data source that would allow for the assessment of these disparate clinical outcomes does not exist and conducting primary data collection to assess these outcomes would be extremely difficult, if not impossible. However, the data needed to evaluate clinically relevant outcomes do exist in several administrative datasets, with different data stewards, located across several departments of the state health authority and a non-profit healthcare data organization. To evaluate the PINPOINT intervention, the research team created a strategy to access and combine these datasets.

The evaluation centres on the construction of the PROMPT database, a large linked administrative dataset that combines data from Oregon’s voluntary All Payer All Claims Dataset (APCD), Prescription Drug Monitoring Program (PDMP), Vital Records (death), Hospital Discharge Data (HDD), Measures and Outcomes Tracking System (MOTS), Emergency Medical Services (EMS),25 and PINPOINT intervention data collected from providers and clinics. This longitudinal, provider-level dataset will allow the research team to examine the impact of the PINPOINT intervention on clinician panel metrics assessing risky opioid prescribing, referrals to OUD treatment, and fatal and non-fatal opioid overdose. Table 1 outlines the primary panel metrics of interest and the data sources from which they are derived. Numerators are defined monthly and panel denominators are calculated using a rolling 2-year attribution period that updates quarterly. Once complete, the PROMPT database will allow the study team to compare monthly trends in clinician panel metrics before, during and after PINPOINT for intervention clinicians relative to similar clinicians working at practices that did not receive the intervention.

Table 1

Primary outcome measures to be calculated from PROMPT.

Description	Data source1
Proportion of provider’s patient panel filling opioid prescription (all, including tramadol)	PDMP
Proportion of provider’s patient panel with long-term high-dose opioid prescription	PDMP
Proportion of provider’s patient panel with high-dose opioid prescription (any fill)	PDMP
Proportion of provider’s patient panel with multiple prescribers	PDMP
Proportion of provider’s patient panel with multiple prescribers and multiple pharmacies	PDMP
Proportion of provider’s patient panel with coprescribed opioid/benzos	PDMP
Proportion of provider’s patient panel filling suboxone prescription (both new/existing)	PDMP
Proportion of provider’s patient panel with a new opioid prescription	PDMP
Proportion of provider’s patient panel engaged in treatment (any substance)	MOTS, APCD
Proportion of provider’s patient panel engaged in treatment (heroin or prescription opioid)	MOTS, APCD
Proportion of provider’s patient panel with overdose (hospitalization or ED visit, any opioid)	HDD, APCD
Proportion of provider’s patient panel with overdose (hospitalization or ED visit, non-heroin opioid)	HDD, APCD
Proportion of provider’s patient panel with overdose death (accidental + undetermined death, any opioid)	Vital Records (Death)
Proportion of provider’s patient panel with opioid overdose reversal based on naloxone administration (exploratory)	EMS

All outcome variables will be calculated monthly for each provider during the study period. Panel denominators are calculated using a rolling 2-year attribution period that updates quarterly.

APCD, all-payer claims database; ED, emergency department; EMS, emergency medical services; HDD, hospital discharge data; MOTS, Measures and Outcomes Tracking System; PDMP, Prescription Drug Monitoring Program.

The base dataset for PROMPT is the Oregon voluntary APCD managed by Comagine Health’s Oregon Data Collaborative. The APCD includes claims for all medical services and prescriptions paid for by participating health plans. The dataset includes 100% of the Medicaid population, 85% of the Medicare (Fee for Service and Advantage) population, 81% of the fully insured commercial population and 24% of the self-insured commercial population, representing approximately 80% of all insured Oregonians.

Patient attribution methodologies (which allow for the evaluation of clinician practice patterns) are run quarterly by the data vendor using proprietary algorithms. The primary care patient attribution algorithm assigns patients to a primary care provider (PCP) by identifying the PCP most commonly billed in professional and facility claims for primary care visits during a rolling 2-year period. Comagine Health also maintains a provider directory through validation and updates from clinics, requesting roster information from health systems, external sources and claims data. Provider attribution was supplemented with additional provider roster data collection from PINPOINT clinics.

The linkage methodology used for PROMPT was developed in collaboration with data stewards at the Oregon Data Collaborative and the OHA. The method was designed specifically to conform with state statutes protecting the confidentiality of provider drug prescribing and patient health information in research. Figure 1 outlines the steps in the linkage methodology. A firewall is maintained between the linkage analyst, who has the authorization to access APCD patient identifiers to conduct the linkages, and the research analyst, who receives only de-identified datasets.

Figure 1

PROMPT linkage methodology.

APCD, All Payer All Claims dataset; DEA, Drug Enforcement Agency; DOB, date of birth; IVPP, Injury and Violence Prevention Program; NPI, National Provider Identifier; PCP, primary care provider; PDMP, Prescription Drug Monitoring Program.

Although most linkages occur at the patient level, the PROMPT database is ultimately aggregated to the clinician level as the PINPOINT intervention being evaluated is clinician facing. The first step (Step 1a) in the creation of PROMPT requires identification of both the provider and patient study cohorts: providers at PINPOINT clinics, eligible control providers (in the Provider Lookup Table), and all patients in the APCD attributed to a provider in the study cohort (intervention and control) in any quarter during the study period (Patient-to-Provider Attribution Table). Step 1b involves the patient-level linkage of EMS, MOTS and Vital Records data, received from their respective data stewards, to the APCD, to create an Enhanced APCD. Patient-level linkages are conducted by the linkage analyst, using a probabilistic linkage algorithm to match patients by first name, last name, birth date and sex.26

Step 2 adds data from HDD to the Enhanced APCD created in Step 1b. Because the linkage analyst cannot receive patient-level identifiers in the HDD data, an OHA analyst and the linkage analyst conduct the linkage together. The OHA analyst always retains custody of HDD patient identifiers.

In Step 3, the linkage analyst transforms the Enhanced APCD into a minimally necessary dataset that contains monthly time-varying provider panel metrics and clinic-level information from all datasets linked in previous steps (PCP-level Enhanced APCD). The linkage analyst generates binary and categorical variables from the source variables that contain only the information necessary to conduct the study aims. No patient-level information is retained. During this step, the dataset is linked to the PINPOINT implementation data (collected at the provider/clinic level) to create the PCP-level Enhanced APCD w/PINPOINT Data file.

During Step 4, the OHA Injury and Violence Prevention Program (IVPP), which manages the PDMP, receives two cohort-defining files: the Provider Lookup Table (eligible intervention and control providers) and the Patient-to-Provider Attribution Table (patients attributed quarterly to eligible intervention and control providers). Within these tables, IVPP creates masked identifiers for all patients and providers. These tables replace the Provider NPIs in the PCP-level Enhanced APCD w/PINPOINT Data with masked provider identifiers. The IVPP analyst then uses the Prescription Drug Monitoring Program to identify filled prescriptions written by all providers in the cohort. All prescriber and patient identifiers in the PDMP dataset are replaced with the masked identifiers created previously by IVPP in the Provider Lookup Table and the Patient-to-Provider Attribution Table. For patients who received prescriptions from PROMPT study providers but did not have a masked identifier in the Patient-to-Provider Attribution Table, IVPP creates additional masked identifiers. During this step, all OHA source datasets held by the linkage analyst are destroyed to remove the possibility of relinkage. Outside of patient-to-provider attribution, these PDMP dispense records remain the only patient-level data in the PROMPT database post-linkage (after Step 5).

The final step (Step 5) involves the de-identification of the final tables in the newly created PROMPT database. The IVPP and linkage analysts destroy all patient, provider and pharmacist identifiers in accordance with Oregon statute and each undertakes a review to ensure no identifiers were erroneously retained. Only masked patient and provider identifiers remain in the final datasets retained by the study team and IVPP houses the Masterfile containing the only link between masked identifiers and actual identifiers. The final files delivered to the study team include the PROMPT Patient-to-Provider Attribution Table, containing only quarterly information on the patients attributed to providers for each quarter of the study period, the PROMPT PDMP Table, containing dispenses for all providers in the cohort, and the PROMPT Provider-level Table, which contains all information from source datasets (APCD, EMS, MOTS, Vital Records, HDD and PINPOINT implementation data) aggregated to the provider level prior to PDMP linkage.

Due to the COVID-19 pandemic, the PROMPT study design has undergone important changes to accommodate implementation decisions and delays in intervention enrolment.

Early in the project, shortly after the onset of COVID-19, delays in obtaining IRB approval and executing data use agreements were experienced. This was largely due to resource/staffing difficulties faced by the state at that time. Limited state resources have made it challenging to obtain all the datasets intended (e.g. EMS).

As additional PINPOINT waves were added, the evaluation timeline was adjusted to capture as many post-intervention time points as possible for those later waves. However, given a claims data lag of approximately 6–8 months, the length of the intervention (15 months) and the 3-year study period, it will not be possible to include postintervention data for all waves of the intervention in the main PROMPT analyses. Outcomes that can be evaluated at or near the time of enrolment, such as recruitment and fidelity, will be assessed for all participating sites and clinicians.

Analytic plan

The original analytic plan called for a stepped-wedge design. This is a crossover study design in which different clusters (in this case, the primary care practices) cross from the control to the intervention condition at regular time intervals.27 In light of the pandemic-related changes already described, the research team developed a modified plan that will rely on propensity score matching of clinicians to identify three controls for each clinician who engages with the PINPOINT intervention.28 Preliminary work conducted at the time of publication indicates that successful matching using these techniques will be feasible and allow for robust evaluation.

Discussion

Due to the complexity of the opioid crisis in the United States, this must be addressed with an interdisciplinary and collaborative approach from both an intervention and analytic point of view. The PINPOINT intervention incorporates diverse strategies to help clinics with quality improvement and reduce the harms from opioid overprescribing. A robust evaluation of the PINPOINT intervention will be conducted with the PROMPT database, which links multiple datasets whilst preserving patient confidentiality. Existing literature with similar approaches does not exist because of the complex nature of both the PINPOINT intervention and PROMPT evaluation.

Several adaptations to the study design and timeline have been made due to the COVID-19 pandemic. The pandemic has dramatically affected the healthcare system and the adaptations made to this project reflect those impacts. For example, pandemic-related challenges that have been seen across the national health system landscape, like clinic closures,29 were also seen in Oregon, the site of the PINPOINT intervention. Some of the intervention clinics began providing care via telehealth to reduce risk of infection for staff and patients, like many other clinics throughout the United States.30 Clinic closures, disrupted workflows and decreased staffing led to delays in clinics enrolling in the study and the inability for study staff to meet with clinic staff in person. The PROMPT database has been directly impacted by the intervention adaptations and changes to the evaluation have been made accordingly.

Additionally, the COVID-19 pandemic has created new challenges for individuals with OUD. Many local and state media sources have reported an increase in opioid-related overdoses throughout the United States31,32 but the lack of national reporting of mortality-related overdoses makes it difficult to confirm these data.29 Whether the increase in adverse outcomes associated with OUD is due to difficulty accessing care during the pandemic, worsening OUD and associated mental health conditions due to the stresses caused by the pandemic, or other factors has not been determined. Nonetheless, the COVID-19 pandemic has affected both those providing and those receiving treatment for OUD.

Conclusion

In 2022, the PINPOINT intervention will conclude whilst the research team will continue to complete and update the PROMPT database and provide a robust analysis in future publications. It is important to note that the PINPOINT intervention and PROMPT database may not be able to be replicated in all geographical locations. Because this project engages only one state, the results may not be generalizable to all other settings. In addition, Oregon has previously created infrastructure that made it possible to link the data sources described; other states may need to invest in developing similar infrastructure in order to apply the lessons from this work. Future publications will provide insights into the anticipated outcomes of these investments in addressing the opioid crisis.