Development of a Practice-based Community Outreach Intervention to Prevent Inequities in COVID-19 Vaccinations.

Despite disproportionately higher rates of morbidity and mortality from COVID-19 among Black and Hispanic adults in the United States, ethnoracial disparities in vaccination rates emerged rapidly. The objective of this quality improvement study was to rapidly develop and implement an equity-focused community outreach intervention that facilitated COVID-19 vaccine appointments. Using the Plan-Do-Study-Act model, this multipronged, primary care-based outreach intervention developed call/recall systems that addressed vaccine hesitancy and facilitated real-time vaccine scheduling. Through 5058 calls to 2794 patients, 1519 patients were successfully reached. Of the 750 patients eligible for vaccine scheduling, 129 (17.2%) had a vaccine appointment scheduled by the caller and 72 (9.6%) indicated a plan to self-schedule. Low confidence in the vaccine was the most cited reason for declining assistance with a vaccine appointment. Primary care practices may wish to consider introducing similar outreach interventions in the future to address ethnoracial inequities in vaccination distribution.

Introduction

Morbidity and mortality from COVID-19 has disproportionately impacted ethnoracial minority communities in the United States.[1] Vaccinations that effectively prevent transmission and death from COVID-19 were authorized for emergency use in December 2020 and became available shortly thereafter on January 12 in New York City for those ≥65 years old and shortly thereafter for younger adults with qualifying medical conditions.[2]

Despite this, COVID-19 vaccination rates among Black and Hispanic communities have been lower than among their White counterparts. As of August 2, 2021, 49% of White people had received at least one COVID-19 vaccine dose, compared with 38% of Black people and 43% of Hispanic people.[3] Several factors contributed to ethnoracial inequities, including early systemic challenges like required online registration and limited vaccination sites, along with other social challenges like historically driven mistrust in the health care system. In April 2021, eligible individuals who desired the vaccine reported several logistical barriers including being too busy, inability to get an appointment or take time off of work.[4] Others who were hesitant about receiving the vaccine cited interest in waiting until more people had gotten vaccinated, concerns about side effects or not enough available research.[4] When vaccine rollout began in New York City, vaccine scheduling relied on mobile and internet portals, a problem when over 40% of older adults, surveyed in 2019, do not have broadband or internet in their home with even lower rates among lower-income neighborhoods and minoritized communities.[5] Further, the first-come, first-serve model for scheduling of vaccines unfairly favored those with more resources including the time needed (sometimes hours) to repeat attempts to schedule an appointment.[5]

In February 2021, a month after the vaccine rollout in New York City to nonhealth care workers began, an internal quality assessment review conducted of 2 academic primary care sites in New York City found that only 14% of vaccine eligible patients had received at least one COVID-19 vaccine dose. While Black patients comprised 16% of vaccine eligible patients, only 10% were vaccinated. And while Hispanic patients comprised 13% of all vaccine eligible patients, only 7% of those were vaccinated. Because previous studies have shown that interventions that improve provider-patient communication increase vaccine uptake,[6,7] a Plan-Do-Study-Act model was used to develop and execute a multipronged outreach intervention targeting Black and Hispanic patients of those sites.

Although the literature describes many practice-based strategies that attempt to increase immunization rates,[8] few published studies have described the rapid development and implementation of an outreach intervention to increase uptake of a new vaccine with an equity-focused approach. The objective of this quality improvement study was to address this developing inequity by rapidly developing and implementing an equity-focused community outreach intervention to facilitate COVID-19 vaccine appointments for those who qualified.

Methods

Context

This quality improvement intervention was conducted at 2 academic primary care practices in New York City from March 15, 2021, to May 28, 2021, shortly after vaccines became available to nonhealth care workers. The intervention aimed to contact Black and Hispanic adults ≥65 years old, or ≥18 years old with qualifying chronic conditions, who had at least 1 visit at either practice within the past 3 years and who had not received or were not scheduled to receive a COVID-19 vaccine per the Electronic Health Record (EHR). This sample was selected based on information available within the EHR including race and ethnicity.

Intervention

The authors used the Plan-Do-Study-Act (PDSA) model as a quality improvement framework for developing, testing, and implementing changes to increase vaccine uptake. This PDSA model consists of a 4 step quality improvement cycle in which an intervention is developed (Plan), executed at a small scale (Do), evaluated for improvement (Study), and then implemented in its refined form (Act).[9] This model has been used in the past to implement other vaccine-related quality improvement initiatives.[10] Using biweekly PDSA cycles, the authors developed and implemented a multipronged outreach intervention consistent with the Health Belief Model,[11] a conceptual model that often is used in health services research to understand public and patient engagement in specific health behaviors. This model includes 4 main constructs which predict health behavior: (1) perceived threat from a health condition (composed of perceived susceptibility and perceived severity); (2) likelihood of enacting the health behavior (composed of perceived benefits and perceived barriers); (3) self-efficacy (one’s belief in the ability to actually perform the change); and (4) cues to action (motivating factors). The multipronged intervention consisted of (1) a mailed letter from the patient’s primary care physician endorsing the COVID-19 vaccine, (2) a phone call/recall system to remind patients about their eligibility to receive the vaccine, (3) discussions with a vaccine ambassador to address any questions or hesitancies regarding the vaccines, and (4) real-time facilitation with scheduling a vaccine appointment and information about where in the community to receive or schedule a vaccine.

The first component of the intervention included a letter from the patient’s primary care physician briefly describing the importance of receiving a COVID-19 vaccine. Prior evidence suggests that adults highly trust their own physicians for health information and that they are more likely to trust physicians as their primary source of health information rather than other sources.[12] The letter reiterated that the vaccines were safe and effective against COVID-19, that they protected individuals, their families, and their communities, and that getting vaccinated was pivotal to ending social restrictions placed during the pandemic. The letter also offered resources for learning about the vaccine and where to receive one. Finally, the letter invited patients to call a provided number with any questions about the vaccine.

The second component was a phone call/recall system to remind patients about their eligibility to receive the vaccine. Strong evidence suggests that patient reminder/recall systems improve access to vaccinations.[13] The patient population in these practices are not robustly enrolled in mobile texting, thus mobile texting was not used in this intervention. Contact information was collected from the EHR. A list of eligible patients from both primary care sites was created. Volunteer callers were recruited from several programs including community health workers, nursing students, medical students, and other volunteers from several institutional centers. Each caller was assigned a weekly set of 25–50 patients to call. Callers were instructed to attempt a maximum of 3 calls per patient; the patient was considered unreachable if they did not answer by the third call. To protect patient privacy, callers were instructed not to leave voicemails. Finally, the list of eligible patients was updated biweekly using EHR data to exclude patients that had already received a call or been vaccinated.

Based on a call/recall script for a prior hospital-wide influenza vaccination campaign, a semistructured script was developed for the callers. The goals of the script were to (1) confirm whether patients had already received a COVID-19 vaccine, (2) remind patients about their eligibility for the vaccine, (3) endorse confidence in the vaccine, (4) identify interest in the vaccine, addressing any questions or hesitancies with skills gained from a vaccine ambassador program, and (5) facilitate scheduling a vaccine appointment.

The vaccine ambassador program is a 2-hour lecture session developed at the academic institution by leaders in community engagement and experts in primary care and infectious diseases. The 2-hour session was designed to train health care providers, community health workers, medical students, and the lay public to discuss COVID-19 vaccination with community members including information about the public health importance of vaccines, the process for developing the vaccines during the pandemic, and the details about the specific vaccines authorized for emergency use. The session also provided evidence-based approaches for discussing the vaccine with individuals who may be hesitant to receive the vaccine. The sessions were recorded and made available publicly.[14] Leveraging the skills gained from this program, callers were encouraged to engage in conversations with patients, answer any questions and address any hesitancies regarding the vaccines.

Finally, real-time facilitation with scheduling a vaccine appointment and information about where in the community to receive or schedule a vaccine was based on resources that were available, including direct access to an institutional vaccination site, as well as resources available through the city or state.

Measures

Information about calls was collected using a REDCap survey that reflected the caller script. Self-reported data about vaccination status were recorded at the beginning of the call, interest in receiving the vaccine, hesitations regarding the vaccine, barriers to receiving the vaccine, and whether a vaccination appointment had been facilitated. Reasons for vaccine hesitancy were categorized using the 5C scale. This scale is based on established models of vaccine acceptance and hesitancy and describes 5 constructs that explain vaccine-related health behaviors: (1) Confidence in vaccine safety and efficacy; (2) Complacency, where perceived risks of vaccine-preventable diseases are low and a vaccine is not deemed necessary; (3) Constraints refers to availability, affordability, and accessibility, including limitations in language and health literacy; (4) Calculation points to individuals’ engagement in information searching in pro- and contravaccination resources; and (5) Collective Responsibility is the willingness to protect others by one’s own vaccination by means of herd immunity.[15]

Analysis

The number of calls made, patients reached, and vaccinations scheduled were analyzed weekly to identify when to cease the intervention. Based on consensus, the intervention was stopped when <5% of the eligible patients that were reached were scheduled for the vaccine.

Biweekly focus groups were held with callers and leadership to identify challenges with implementation of the intervention. Reflecting the iterative process of quality improvement interventions, focus groups consisted of brief reports of the number of calls made during the intervention and were followed by open format discussions on the progress, challenges, barriers, and facilitators discovered during execution of the calls and reporting through REDCap. Feedback from these discussions was identified by a note taker, summarized at the end of the meeting, and subsequently addressed with leadership.

Standards for QUality Improvement Reporting Excellence (SQUIRE) guidelines were followed to report this quality improvement intervention.[16]

Ethical Considerations

The project was reviewed and approved by the Institutional Review Board as a quality improvement project.

Results

Modifications to the Intervention

A summary of the PDSA cycle can be found in Table .

A primary area of feedback initially included addressing perceived inconsistencies and subjectivity in the caller script. One particular area of focus was the complexity of gauging patient interest in a vaccine at the beginning of the call. While patients often voiced interest in receiving the vaccine, they later revealed hesitancy when presented with the option of scheduling the appointment. Similarly, categories of hesitancies did not fit entirely into the initial 5 categories offered, with many patients expressing they thought they had a contraindication to vaccine receipt or wanted consent or discussion directly from their doctors before scheduling the appointment. Consensus was reached on how to address these changes, including through further training on reporting findings from the call and adapting the survey to include 2 novel and additional C’s: Consent (from their physician) and Contraindication (perceived or real) to pragmatic responses received from patients.

A secondary area of feedback was the inaccuracy of patient eligibility. A substantial portion of the initial patients that were called reported already having received the vaccine despite no record of this in the EHR. This was due to multiple places in the EHR to log patient vaccination status as well as delays in the migration of vaccination data into the EHR from other databases. This was discussed with EHR management for identification of the most accurate sources and biweekly patient list updates. A larger contributor to the discordance in the sample was the rapid rate at which patients were receiving the vaccine, and the lack of infrastructure for communicating this information to the primary care physician’s office. In response, calls were considered a temporary but useful method for updating the vaccination status of patients.

Finally, the rapidly changing landscape for vaccine rollout in New York City provided several challenges. The city and state COVID-19 vaccination eligibility criteria for adults changed nearly every 2 weeks, as new age groups and people with certain medical conditions became eligible. Given the number of individuals eligible based on the criteria at the initial development of the intervention, and the weekly rate at which calls were made, a decision was made not to expand the patient list, and the initial sample was retained during the entirety of the intervention. Moreover, resources and locations for obtaining a vaccine evolved rapidly, including the opening and closing of vaccination sites as well as changing access to appointments. The script was updated frequently to reflect the availability of resources. Notably, the single vaccination site to which callers could initially directly facilitate making an appointment was located approximately 7 miles from the main primary care practice locations and callers reported that patients declined direct appointment scheduling due to this distance. To help overcome this restriction, 4 additional vaccines sites located in underserved communities throughout New York City were eventually added for which callers were able to make direct appointments for patients.

Process Measures and Outcomes

From March 15, 2021, to May 28, 2021, 38 callers were recruited and 5058 total calls to 2794 unique patients were made as part of this outreach intervention; 1519 patients were successfully reached, 746 (49.1%) of whom had already scheduled or received a vaccine, and 23 of whom were ineligible for the vaccine (actively infected or deceased at the time of the call).

Of the 750 patients who were reached and eligible for the intervention, the majority were female (73%), with a median age of 57, and 97% (699) identified as either Black or Hispanic. At the beginning of the call, 229 (30.5%) reported that they would get the vaccine if it were available to them, 294 (39.2%) reported they would not get the vaccine, 168 (22.4%) reported being unsure of whether they would want to receive it, and 59 (7.9%) did not respond.

Among those who reported they would get the vaccine if available to them (n = 229), the most common reason for not scheduling a vaccine appointment yet was not knowing how to book an appointment (40.2%). Among those who reported at the beginning of the call that they would not want to get vaccinated if the vaccine was available to them (n = 294), “low confidence in the vaccine” (68.3%) and “still contemplating whether to receive the vaccine” (12.9%) were the 2 reasons most cited for not wanting to get it (Table ). Notably, 15.3% of those patients cited a reason not included among the hesitancy constructs utilized in the adapted 5C scale. Among those who reported at the beginning of the call that they were unsure if they would want to get vaccinated if the vaccine was available to them (n = 168), “low confidence in the vaccine” (51.7%) and “still contemplating whether to receive the vaccine” (37.5%) were also the 2 reasons most cited for their decision, and 14.9% of them cited a reason not included among the hesitancy constructs utilized in the expanded 5C scale (Table ).

Despite the interest initially reported at the beginning of the call, of the 750 patients who were reached and eligible for the intervention, only 129 (17.2%) had a vaccine scheduled by the caller, 72 (9.6%) planned to self-schedule a vaccine and were instructed on how to do so by the callers, and 549 (73.2%) declined assistance with vaccine scheduling. The weekly proportion of eligible patients scheduled for vaccine decreased over time, starting at 30% and ending at 0% in the 11th week of the intervention, at which point the intervention was stopped (Figure ).

Percent of eligible patients scheduled for a vaccination, by week of intervention. Time in which each Plan-Do-Study-Act (PDSA) cycle was completed is represented by the vertical dotted lines.

Of the 129 patients who were scheduled to receive a vaccine by this intervention, by June 10, 2021 (2 weeks after the intervention was complete), 83 patients (64.3%) had received their first dose. Of the 72 patients who were contacted and wished to self-schedule their vaccine, only 25 patients (35%) received the first dose. This vaccination rate was similar to the rate of vaccination among those who did not answer a call (33%). Finally, 13% of people who declined the vaccine, ended up getting vaccinated.

Discussion

To the knowledge of these authors, this is the first single-center quality improvement study demonstrating the feasibility and shortcomings of implementing an outreach intervention aimed at increasing COVID-19 vaccination rates in Black and Hispanic older adults in New York City. This equity-focused program was successfully implemented and aided in scheduling 129 patients for the vaccine. Although only 64% of these patients received their first dose by June 10, 2021, the intervention resulted in almost twice the rate of vaccinations as those who did not answer a call or received a call without scheduling.

This study supports previous literature that has indicated that patient reminder and recall interventions through telephone or mail, increases immunization rates in primary care settings.[13] However, these results suggest that this approach can require a remarkable amount of resources, and that even with this outreach, a large proportion of patients declined the vaccine. Despite 5058 calls being made, only 750 of the patients called were ultimately eligible to have a vaccination scheduled, and only 129 of those patients were actually scheduled for a vaccination appointment. This does not preclude the development of similar initiatives in the future but provides insight into the resources required to accomplish such an initiative and the ways that they could be improved to increase yield. Importantly, almost half of the patients who were successfully reached had already scheduled or received a vaccine. This demonstrates the difficulties in relying on EHR data to track vaccination rates given the lack of interoperability. Future interactions of similar outreach initiatives may aim to work closely with local health departments to develop more integrated vaccine record systems. Until data sharing is more expansive, future interventions may aim to remind patients to update their vaccination status with their primary care doctors.

Among patients called during this intervention, the most commonly cited reason for declining a vaccination appointment was low confidence in the vaccine. Although the pandemic has disproportionately affected ethnoracial minority communities, survey studies have found that people who identify as Hispanic or Black reported less interest in getting the vaccine.[17-19] Among these populations, the most frequent concerns for deciding not to get the vaccine were similar to what was seen in the current study including concern for long-term side effects, concern for adverse reactions, and lack of knowledge about the vaccine.[18] Previous literature has shown that ethnoracial minorities have higher rates of distrust in the health care system in comparison to their White counterparts.[20] This distrust is often rooted in past exploitation of these populations by biomedical researchers.[20] Unfortunately, this distrust has also been linked to poorer health outcomes.[21] Future efforts should focus on more rigorous vaccine ambassador training along with further research and quality improvement initiatives aimed at addressing distrust, particularly in minoritized communities. Health care institutions may consider investing in longitudinal community collaborations that improve communication, engagement, and trustworthiness.

However, the disproportionately low rate of vaccination among Black and Hispanic adults in the United States cannot be fully attributed to vaccine hesitancy. Several structural factors, including inadequate access to vaccine distribution clinics in underserved areas, digital inequalities, and competition for limited vaccine appointments are also contributors. This quality improvement intervention aimed to address these barriers by facilitating access to vaccine appointments. Of the 229 patients who reported interest in receiving a vaccine at the beginning of the call, 40.2% stated they had not scheduled a vaccination appointment because they did not know how. This intervention initially facilitated scheduling of appointments to only a single vaccination site but progressively gained access to more sites. Future interventions should aim to work with community partners early and overcome a variety of logistics barriers that patients may face. As of August 2, 2021, the United States Center for Disease Control reports that vaccinations are reaching larger shares of Hispanic, Asian, and Black population compared with overall vaccinations, suggesting a narrowing of racial gaps in vaccinations at the national level.[3]

PDSA cycles can be an important tool when quick responses to rapidly changing circumstances are required.[22] Given the unpredictable nature of the pandemic and the ongoing policy changes occurring at the institutional, city, state, and national levels, the PDSA process allowed for rapid implementation of this quality improvement intervention and optimization of that intervention based on consistent feedback. These PDSA cycles allowed the intervention to be rapidly adapted to offer accurate and updated resources despite the frequent addition of New York State and City vaccine distribution sites. Moreover, through the PDSA cycles, limitations in the data acquisition process were identified, not only as it related to the vaccination status of the original sample, but also as related to streamlining the caller script and recording reasons for declining the vaccine.

Strengths and Limitations

This quality improvement study had several strengths including the rapid implementation of a framework-based intervention in a unique context using an established quality improvement method. However, several remarkable limitations should be noted. There were several challenges in obtaining accurate data regarding the vaccination rates of the sample, largely due to the rapidly changing availability of vaccination sites and lack of a unified infrastructure for recording and reporting a person’s vaccination status. Moreover, this intervention aimed to facilitate scheduling vaccination appointments, but these may not have resulted in actual vaccinations. Finally, the sample was identified based on ethnicity and race data as available within the EHR, where 15.5% of patients do not have complete ethnoracial data (marked as unknown or declined). Furthermore, confirmation of self-identified race and ethnicity was not established. Future efforts to provide equitable care in primary care practices may be optimized by collecting more accurate and complete race and ethnicity data.

Conclusions

An equity-focused outreach program that aimed to facilitate scheduling COVID-19 vaccination appointments for eligible adult patients of outpatient primary care practices was rapidly developed and implemented. A substantial proportion of the patients that were called had already scheduled or received the vaccine despite no report of this documented in their medical records; nonetheless, this multipronged quality improvement intervention directly scheduled vaccination for 129 patients. Other primary care practices may wish to consider introducing similar outreach interventions in the future to address ethnoracial inequities in vaccination distribution.

Acknowledgments

The authors would like to thank the volunteer callers including the community health workers from the NewYork-Presbyterian Hospital, nursing students from Hunter School of Nursing, medical students from Weill Cornell Medicine, and other volunteers from the Clinical and Translational Science Center (CTSC) and the Division of General Internal Medicine (DGIM). They would also like to acknowledge the leadership in the New York Presbyterian Analytics and Enterprise Data Warehouse and in the Division of Community and Population Health at the New York Presbyterian Hospital. Finally, they would also like to thank Susana Morales, MD, for her leadership in the Vaccine Ambassador Training.

Conflict of interest

The Authors declare that there are no conflicts of interest.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. CJG was supported by the Weill Cornell Diversity Center of Excellence (grant D34HP31879 from the Health Resources and Services Administration). Research reported in this publication was supported by the National Center For Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR002384 for use of REDCap and CTSC staff.

Table 1.

Results From the Biweekly PDSA Cycles Utilized to Implement a Practice-Based Community Outreach Intervention to Prevent Inequities in COVID-19 Vaccinations

Plan	Do	Study	Act
Cycle 1
Develop a multipronged intervention to facilitate vaccine scheduling for Black and Hispanic patients of an academic outpatient practice.	Deliver letters from PMD endorsing the vaccine.Recruit, train and assign callers for a call/recall intervention.Design a caller script that addressed vaccine hesitancy and facilitated vaccine scheduling.	Volunteer callers were recruited through various institutional and community partnerships.Callers felt that the caller script was at times ambiguous or inconsistent with some patient responses.	The caller script was modified with a focus on clarity and thoroughness.
Cycle 2
Utilize an adapted caller script in the call/recall system.	Introduced the adapted caller script, highlighting the changes made.Further trained callers on how to clarify ambiguous patient responses.	Adapted script was clearer and more consistent with patient responses.Callers noted a substantial proportion of patients called had already been vaccinated; inconsistently reflected in the EHR.	Re-evaluate various EHR sources documenting vaccination status.Reached consensus with EHR management regarding the most accurate and updated source of vaccination history.
Cycle 3
Maintain an accurately updated list of unvaccinated patients to be assigned for the call/recall intervention.	Update the patient sample biweekly using an accurate EHR source.Leverage calls to already-vaccinated patients to update and correct the EHR.	Fewer ineligible patients were being called.Available vaccination sites often too far. Several vaccination sites were closing, migrating or newly opening.	Updated the vaccination sites weekly and obtained access to facilitate vaccination appointments at more sites across a larger geographic area.

Abbreviations: EHR, electronic health records; PDSA, Plan-Do-Study-Act; PMD, primary medical doctor.

Table 2.

Specific Concerns for Being Unsure or Declining the COVID-19 Vaccine at the Beginning of the Call

	UnsureN = 168	DeclinedN = 294
Confidence[a]	51.7%	68.3%
Constraints[b]	6.0%	2.4%
Complacent[c]	1.8%	3.1%
Calculating[d]	37.5%	12.9%
Collective[e]	0.6%	2.0%
Contraindicated[f]	7.1%	11.0%
Consent[g]	20.8%	4.1%
Other	15.3%	14.9%

The caller script categorized reasons for vaccine hesitation using an adapted 5C scale.

Confidence in vaccine safety and efficacy.

Complacency, where perceived risks of vaccine-preventable diseases are low and a vaccine is not deemed necessary.

Constraints refers to availability, affordability and accessibility, including limitations in language and health literacy.

Calculation points to individuals’ engagement in information searching in pro- and contravaccination resources

Collective Responsibility is the willingness to protect others by one’s own vaccination by means of herd immunity. The additional 2 Cs reflect additional constructs that emerged from the initial calls.

Contraindicated refers to the belief that one does not qualify for the vaccine for other health reasons.

Consent is a desire for direct consultation and permission from their doctor regarding the vaccine.