Use of a Best Practice Alert (BPA) to Increase Diversity Within a US-Based Autism Research Cohort.

We evaluated the success of a best practice alert (BPA) in recruiting underrepresented families into an autism spectrum disorder research cohort by comparing BPA-response outcomes (Interested, Declined, Enrolled, Dismissed) in pediatric primary care practices (TCPs) serving diverse communities with those of subspecialty clinics. Compared to subspecialty clinics, TCPs had higher proportions of Interested responses for patients with private insurance (60.9% vs. 46.2%), Dismissed responses for patients with public insurance (30.1% vs. 20.0%), and Interested responses for non-white patients (47.7% vs. 33.3%). A targeted BPA can help researchers access more diverse groups and improve equitable representation. However, select groups more often had their alert dismissed, suggesting possible selection bias among some pediatricians regarding who should receive information about study opportunities.

Introduction

Underrepresentation of racially and ethnically diverse populations in clinical research limits our ability to generalize results for treatment intervention, which contributes to healthcare inequities (Haley et al., 2017). Lack of representation is driven by challenges with investigators’ recruitment of diverse samples, as well as patients’ distrust of the medical and scientific communities (Zamora et al., 2016). Studies examining recruitment strategies report a need for research teams to increase their time spent in active recruitment, coordination of logistics, and community involvement, yet time constraints, competing study demands, and insufficient resources pose challenges to investing in these efforts (Amorrortu et al., 2018; Zamora et al., 2016). Researchers are in desperate need of efficient and practical tools that enable them to connect with underserved populations.

Studies have shown that having healthcare providers introduce a study to their patients during their clinical visits is an effective recruitment strategy and can be helpful in engaging diverse participants (Chhatre et al., 2018; Tilley et al., 2017). However, initiating and maintaining provider cooperation in this setting is challenging (Amorrortu et al., 2018). Electronic Health Records (EHRs) have been shown to facilitate outreach to historically marginalized populations but traditionally involve passive methods that do not lend to effective follow-up and enrollment efforts (Devoe et al., 2019; Lai & Afseth, 2019). Tools that can increase provider engagement with research referrals at the point of care for large numbers of patients, as well as best-practice recommendations for successful implementation of these strategies, are essential.

We recently demonstrated the efficacy of an EHR-based best practice alert (BPA) in engaging subspecialty providers in research recruitment for large numbers of participants at the point-of-care (see Simon et al., under review). However, many families of color and/or families with lower levels of socioeconomic status (SES) experience challenges that make it difficult for them to access subspecialty services (e.g., language barriers, difficulties completing referral paperwork, lack of transportation (Haley et al., 2017; Zamora et al., 2016)); subsequently, they are not exposed to clinical-research opportunities offered in those settings. In these cases, it is possible that primary care providers (PCPs) may serve as the only community resource to promote the benefits of research engagement within the healthcare system.

Objective

The current study focused on how a BPA can be used to overcome recruitment barriers that contribute to racial/ethnic and socioeconomic disparities in clinical research. Specifically, we evaluated the success of a BPA in recruiting underrepresented families into a research cohort of individuals with autism spectrum disorder (ASD) by implementing it in primary care practices (PCPs) that serve racially, ethnically, and socioeconomically diverse communities. Our primary objectives were to compare (a) patient-demographic characteristics and responsiveness to the BPA between practice types (specialist providers versus PCP providers) and (b) study interest/dismissal within practice types by patient-demographic characteristics.

Methods

Texas Children’s Pediatrics Selection

Texas Children’s Pediatrics (TCP) is a network of 52 primary care pediatric clinics that is linked with Texas Children’s Hospital and serves approximately 23.0% of the greater Houston pediatric population. We initially examined characteristics of the patient community at each TCP practice and approached five that served relatively high proportions of both children with ASD and families of color. All practices were willing to participate in recruitment efforts for this pilot project. The number of providers at each practice ranged from 6 to 11. A map showing the distribution of the five TCP sites across greater Houston and in relation to the multidisciplinary Texas Children’s Hospital autism center can be found in Fig. 1. Prior to the launch of the BPA, providers and practice managers at each TCP site were educated on how to introduce the study and respond to the alert, as well as on the importance of sample diversity in research studies.

Fig. 1

Map of TCP locations

Specialty Clinic Selection

As noted earlier, we previously implemented a BPA for recruitment of patients with ASD and their family members into the Simons Foundation Powering Autism Research for Knowledge (SPARK) study (Feliciano et al., 2018). For this initial effort, 22 subspecialty clinic locations under Psychology, Neurology, Psychiatry, and Developmental-Behavioral Pediatrics had been selected to receive the BPA because these sections served the largest numbers of children with ASD across Texas Children’s Hospital (Simon et al., under review). BPA data from these subspecialty practices were used as a baseline for comparison with the TCP practices in the current study. As with the TCP sites, providers in each of these subspecialty practices were educated about the study and how to respond to the alert prior to launch of the BPA. However, they did not receive information on the importance of sample diversity in research studies.

Best Practice Alert (BPA)

A complete description of the BPA and its development is provided in Simon et al. (under review). Briefly, upon opening the patient’s chart in the EPIC EHR, the provider sees a message that succinctly describes the SPARK study and alerts him/her that the current patient is potentially eligible (see Fig. 2). The provider then has the option of reporting the family’s level of interest in the study by selecting either Interested (the family would like more information/to be contacted by the study team), Declined (the family is not interested/does not want to be contacted), or Enrolled (the family indicates they are already participating in the study). Alternatively, providers can dismiss the alert if it is not an appropriate time to discuss the study; the BPA will fire again the next time the chart is opened. Trigger criteria were put into place to prevent duplicate referrals of the same patient over time. When a provider records a response, an EHR-inbox message with this information is sent to the research team. The team then contacts all Interested and Enrolled families by email and telephone to send them information about the study and offer enrollment assistance. Figures 2 and 3 in Simon et al. (under review) outline both the referral process and the follow-up schedule for contacting these families.

Fig. 2

Screenshot of the Best Practice Alert (Simon et al., under review)

Data Analysis

Data on the total number of patients triggered and their responses to the alert, as well as practice location, provider participation (i.e., responding to versus dismissing the alert), and patient characteristics were extracted from the EPIC EHR after the first eight months the BPA was active in the TCP practices (10/9/19–6/9/20). Descriptive statistics were used to calculate the frequencies of response types by practice group (i.e., TCP versus subspecialty). Chi-square tests of homogeneity were used to compare demographic information and proportions of Declined, Dismissed, Enrolled, and Interested responses to the BPA between patients in the TCP versus subspecialty groups. Within practice types, chi-square tests of homogeneity were also used to compare proportions of Interested and Dismissed responses to the BPA by race (White vs. families of color), ethnicity (Hispanic vs. non-Hispanic), and payor type (private insurance vs. public insurance; public insurance plans included Medicaid, CHIP, STAR, and other government-subsidized Health Plans).

Results

Sample Characteristics

A total of 1388 patients triggered the BPA during the 8-month surveillance window, with 466 coming from the 5 TCP practices and 922 coming from the 22 subspecialty clinics. Most patients in both groups were reportedly white and non-Hispanic; however, in comparing the TCP and subspecialty groups, there were significantly (a) larger proportions of Black/African American (133 [28.5%] vs. 169 [18.3%]) and Asian patients (66 [14.2%] vs. 50 [5.4%]) and (b) smaller proportions of White patients (211 [45.3%] vs. 535 [58.0%]) in the TCP group, x2(6) = 66.911, p < 0.001. TCP practices also had a higher proportion of non-Hispanic families (299 [64.2%] vs. 531 [57.6%]) and a lower proportion of families with unknown ethnicity (36 [7.7%] vs. 115 [12.5%]), x2(2) = 8.999, p = 0.011. With regard to payor type, the TCP group had a slightly higher proportion of families with private insurance (235 [50.4%] vs. 433 [47.0%]) and a lower proportion who were self-pay (45 [9.7%] vs. 114 [12.4%]), x2(3) = 11.964, p = 0.008. Complete results can be seen in Table 1. For comparative purposes, information on the racial/ethnic breakdown of the Houston population is provided in Table 2.

Table 1

Demographic characteristics of patients who triggered the BPA (N = 1388) by practice group (TCP vs. subspecialty)

Demographic variable	TCP (n = 466) n (%)	Subspecialty (n = 922) n (%)	x2(df)
Race
American Indian/Alaskan Native	2 (0.4%)	9 (1.0%)	66.911(6)**
Asian	66 (14.2%)	50 (5.4%)
Black/African American	133 (28.5%)	169 (18.3%)
Native Hawaiian/Other Pacific Islander	1 (0.2%)	1 (0.1%)
Two or More Races	1 (0.2%)	20 (2.2%)
Unknown	52 (11.2%)	138 (15.0%)
White	211 (45.3%)	535 (58.0%)
Ethnicity
Hispanic	131 (28.1%)	276 (29.9%)	8.999(2)*
Non-Hispanic	299 (64.2%)	531 (57.6%)
Unknown	36 (7.7%)	115 (12.5%)
Payor Type
Public	186 (39.9%)	357 (38.7%)	11.964(3)*
Private	235 (50.4%)	433 (46.9%)
Self-Pay	45 (9.7%)	114 (12.4%)
Unknown	0 (0.0%)	18 (2.0%)
Response to BPA
Declined	94 (20.2%)	135 (14.6%)	18.325(3)**
Dismissed	126 (27.0%)	343 (37.2%)
Enrolled	6 (1.3%)	5 (0.5%)
Interested	240 (51.5%)	439 (47.6%)

*p ≤ .01; **p < .001

Table 2

Racial/ethnic representation across the U.S., Texas, Houston

	U.S (%)	Texas (%)	Houston (%)
Race or ethnic group
American Indian/Alaskan Native	1.3	1.0	0.3
Asian	5.9	5.2	6.8
Black/African American	13.4	12.9	22.6
Native Hawaiian/Other Pacific Islander	0.2	0.1	0.1
Two or more races	2.8	2.1	2.2
White	76.3	78.7	57.0
Hispanic or latino	18.5	39.7	45.0
Non-Hispanic or non-latino	60.1	41.2	24.4

BPA-Response Types

Responses to the BPA varied significantly between practice types. Subspecialty providers dismissed the alert more frequently than TCP providers (343 [37.2%] vs. 126 [27.0%]), while TCP providers had higher proportions of both Interested (240 [51.5%] vs. 439 [47.6%]) and Declined responses than the subspecialists (94 [20.2%] vs. 135 [14.6%]), x2(3) = 18.325, p < 0.001. Frequencies of BPA responses by practice group can be seen in Table 1.

Interested and Dismissed Responses by Patient Demographic Factors

To evaluate responses to the BPA in relation to patient demographic factors, we compared rates of both Interested and Dismissed responses, first within practice types and then between practice types. Because of small sample sizes across many racial groups, race data were recoded for these analyses to dichotomize patients as white or individuals of color, with those of unknown race or ethnicity excluded. Similarly, payor-type data were organized to exclude those with self-pay or unknown payor status to afford the clearest comparisons. Within the subspecialty clinics, there were no differences in the proportions of either Interested or Dismissed responses between racial, ethnic, or payor-type groups. However, within the TCP practices, a significantly higher proportion of Hispanic patients had Dismissed responses compared to non-Hispanic patients (47 [35.9%] vs. 65 [21.7%]), x2(1) = 9.453, p = 0.002. The TCP practices also had significantly higher proportions of Interested responses for patients with private insurance (143 [60.9%] vs. 86 [46.2%]), x2(1) = 8.939, p = 0.003, and Dismissed responses for patients with public insurance (56 [30.1%] vs. 47 [20.0%]), x2(1) = 5.740, p = 0.017. Complete results can be found in Table 3.

Table 3

Patient interest and provider dismissals by race, ethnicity, and payor type within practice groups (TCP and subspecialty)

Demographic variable	TCP (n = 466) n (%)				Subspecialty (n = 922) n (%)
	Interested, n (%)	x2 (df)	Dismissed, n (%)	x2 (df)	Interested, n (%)	x2(df)	Dismissed, n (%)	x2 (df)
Racea
White	112/211 (53.1%)	0.276 (1)	63/211 (29.9%)	3.071 (1)	240/535 (44.9%)	0.760 (1)	200/535 (37.4%)	0.179 (1)
Non-White	102/202 (50.5%)		45/202 (22.3%)		120/249 (48.2%)		97/249 (39.0%)
Ethnicityb
Hispanic	65/131 (49.6%)	0.305 (1)	47/131 (35.9%)	9.453 (1)‡	138/276 (50.0%)	3.272 (1)	100/276 (36.2%)	0.942 (1)
Non-Hispanic	157/299 (52.5%)		65/299 (21.7%)		230/531 (43.3%)		211/531 (39.7%)
Payor typec
Public	86/186 (46.2%)	8.939 (1)‡	56 (30.1%)	5.740 (2)*	179/357 (50.1%)	1.085 (1)	131 (36.7%)	0.020 (1)
Private	143/235 (60.9%)		47 (20.0%)		201/433 (46.4%)		161 (37.2%)

*p < 0.05; ‡p < 0.01

aTCP n = 413, Subspecialty n = 784

bTCP n = 430, Subspecialty n = 807

cTCP n = 421, Subspecialty n = 790

Analyses comparing rates of Interested and Dismissed responses by demographic characteristics between practice types revealed a higher proportion of Interested patients of color in the five TCP practices compared to the 22 subspecialty practices (102 [47.7%] vs. 120 [33.3%]), x2(3) = 34.879, p < .001. A higher proportion of Dismissed responses was also observed for patients of color in the TCP practices versus subspecialty practices (45 [41.7%] vs. 97 [32.7%]), x2(3) = 37.316, p < .001. There were no differences in the proportions of Interested responses for Hispanic versus non-Hispanic patients between practice types; however, there was a higher proportion of Dismissed responses for Hispanic patients in the TCP practices compared to the subspecialty practices (47 [42.0%] vs. 100 [32.2%]), x2(3) = 11.804, p < .01. With regard to payor type, the proportion of Interested responses for patients with public insurance was significantly lower in the TCP practices than in the subspecialty practices (86 [37.6%] vs. 179 [47.1]), x2(3) = 10.112, p < .05. There were no differences in the proportions of Dismissed responses for patients with public versus private insurance between practice types. Complete results can be found in Table 4.

Table 4

Patient interest and provider dismissals by race, ethnicity, and payor type between practice groups (TCP and subspecialty)

Demographic variable	Interested (n = 679) n (%)			Dismissed (n = 469) n (%)
	Specialty, n (%)	TCP, n (%)	x2(df)	Specialty, n (%)	TCP, n (%)	x2(df)
Racea
White	240/360 (66.7%)	112/214 (52.3%)	34.879(3)‡	200/297 (67.3%)	63/108 (58.3%)	37.316(3)‡
Non-white	120/360 (33.3%)	102/214 (47.7%)		97/297 (32.7%)	45/108 (41.7%)
Ethnicityb
Hispanic	138/368 (37.5%)	65/222 (29.3%)	5.401(3)	100/311 (32.2%)	47/112 (42.0%)	11.804(3)‡
Non-Hispanic	230/368 (62.5%)	157/222 (70.7%)		211/311 (67.8%)	65/112 (58.0%)
Payor typec
Public	179/380 (47.1%)	86/229 (37.6%)	10.112(3)*	131/292 (44.9%)	56/103 (54.4%)	5.856(3)
Private	201/389 (52.9%)	143/229 (62.4%)		161/292 (55.1%)	47/103 (45.6%)

*p < 0.05; ‡p < 0.01

aInterested n = 574, Dismissed n = 405

bInterested n = 590, Dismissed n = 423

cInterested n = 609, Dismissed n = 395

Discussion

Placing a research-recruitment BPA into strategically selected primary care settings (i.e., TCPs) successfully reached families of color who were managing ASD, particularly Black/African American and Asian families. Moreover, most of these families were interested in learning more about the SPARK study. Incredibly, just five TCP practices were able to identify nearly the same volume of Interested patients of color as all 22 subspecialty-practice locations combined (102 in TCP vs. 120 in subspecialty), despite the higher concentrations of patients with ASD at subspecialty practices. This suggests that many families of color are interested in clinical research opportunities and that their receptivity to information about research studies may be enhanced when delivered in community-based care settings. It is possible that even when seeking participants with complex diagnoses such as ASD, investigators may increase sample diversity by engaging primary care providers in medically underserved communities more so than condition-specific specialists. Relatedly, the TCP group had a significantly higher rate of Declined responses to the BPA compared to the subspecialty group. This could indicate that TCP providers are more regularly engaging in a conversation about the study with families (as opposed to dismissing the alert) in order to accurately record their wishes, which is helpful to (a) reduce unwanted contact from the study team and (b) funnel research-staff efforts toward other activities.

Although the TCP group had a lower proportion of Hispanic families compared to the subspecialty group, this finding was not surprising, as the practices in this pilot were chosen based on the racial and ASD-diagnostic makeup of their patient populations and not ethnicity. However, examining responses to the BPA within practices revealed that the alert was dismissed significantly more often for Hispanic patients in the TCP group compared to the subspecialty group. One eligibility requirement of SPARK is that families are fluent enough in English to understand the consent language, and for this reason, the BPA included a filter so that it would not fire for patients who required an interpreter. It is possible that some providers dismissed the BPA for Hispanic families if they were minimally fluent in English and if they had concerns about the family being able to fully understand what participation entailed. Relatedly, it is possible that more TCP providers exercised selection bias in the type of family who they perceived as “appropriate for” or “ready for” the study and that this led to partiality in the sharing of study information. Future studies of this nature may consider ways to distinguish responses reported by providers versus those reported by families directly.

An important observation was that subspecialty providers dismissed the BPA significantly more often than the TCP providers (37.2% vs. 27.0%). The BPA had already been active in the subspecialty practices for more than a year, so long-term engagement and responsiveness to the BPA (i.e., introducing the study and recording a response of Interested, Enrolled, Declined) could be a challenge. It is also possible that the subspecialists were more prone to alert fatigue because they saw many more children with ASD and received the alert more frequently. At the same time, subspecialty visits are often complex and time-intensive with many competing priorities, and providers may have been more likely to simply dismiss the alert to remove it from the screen if they felt there was not time to introduce the research study. Finally, the BPA fired when an ASD diagnosis was added to the patient’s chart, so some providers may have felt uncomfortable discussing research opportunities in the context of a new ASD diagnosis.

Dismissal patterns also revealed potential socioeconomic disparities within the TCP practices that were not observed in the subspecialty group. Patients with public insurance (e.g., Medicaid, government-subsidized Health Plans), which we used as a surrogate measure of SES, were more likely to have the alert dismissed than to have a response recorded. Similarly, when analyzing demographic factors by BPA-response types between practices, we saw in the TCP group (a) higher proportions of both Interested and Dismissed responses for patients of color, (b) a higher proportion of Dismissed responses for Hispanic patients, and (c) a lower proportion of Interested responses for patients with public insurance. While our aim was to see if we could increase the proportion of Interested responses for families of color when the BPA was placed strategically, we did not anticipate a higher rate of Dismissals in this group, as well. However, this could be explained by the fact that the TCPs served a higher proportion of families of color compared to the subspecialists, so higher proportions of both response types are to be expected. The additional findings of increased alert dismissals among Hispanic patients and fewer Interested responses for patients with public insurance could suggest that the TCP group exercised selection bias in who they perceived as “a good fit” or “likely to engage” with the study and that this led to inequities in offering study information. Alert fatigue and overall higher dismissal rates at subspecialty locations may have obscured effects of similar biases among subspecialist providers. Given that people are often unaware of their own biases, sharing these findings and offering provider education about the need for participation from racially, ethnically, and socioeconomically vulnerable groups to help answer questions critical to empowering these communities may help minimize potential biases in the distribution of research information to these patients at the point of care.

Limitations and Future Directions

Although we demonstrated that a targeted BPA can increase outreach about study opportunities to marginalized communities, some limitations should be noted. While the actual numbers of patients who triggered the BPA within the 8-month surveillance period were high (N = 1388), dividing this number into various subsamples for comparisons resulted in smaller groups that may have limited our power to detect further significant differences. For this reason, we were unable to make meaningful comparisons by individual races, and, instead, examined general differences between white patients and patients of color. It will be important to continue this line of research with larger samples to understand whether targeted BPA-based research recruitment at the point of care is more/less successful with certain groups, as well as how race and ethnicity interact with payor type to influence both provider responsiveness to the BPA and patient interest in research opportunities.

Additionally, while the current study underscores the success of a BPA in identifying underrepresented populations who are interested in research, we did not examine actual study enrollment and completion rates. The primary reason for this is that the surveillance period occurred during the emergence of the COVID-19 pandemic, which had widespread disruptive effects on people’s lives. Indeed, in the weeks and months following the onset of the pandemic, we observed reduced enrollment rates across recruitment efforts and slower study progression, particularly among people of color. Also, the majority of healthcare visits were converted to virtual formats, which may have impacted providers’ comfort with introducing study opportunities and/or the likelihood of families to enroll. Further research is needed to investigate differences in whether or not people of color recruited through primary care visits are as likely to enroll in studies as those recruited other ways (e.g., virtual visits, social media, community events), as well as methods for how to increase conversion rates from Interested patients to enrolled and completed participants.

Conclusions

Advancements in clinical care for underrepresented groups and the quest for health equity depend upon the ability of clinical researchers to obtain sufficiently large, representative study samples. EHR tools like BPAs can be used at the point of care to systematically improve awareness of clinical research opportunities among underrepresented groups. However, even with these tools, institutional biases can lead to exclusionary practices in participant selection, which may undermine equitable access to research. Investigating, quantifying, and addressing biases that influence provider practice patterns when sharing information about clinical research opportunities, as well as identifying ways to successfully engage interested families of color in the research process, is fundamental to overcoming healthcare disparities that are perpetuated by the ongoing exclusion of diverse populations.