Understanding User Acceptance of Clinical Decision Support Systems to Promote Increased Cancer Screening Rates in a Primary Care Practice.

OBJECTIVE: Clinical decision support systems (CDDSs) in the electronic medical record (EMR) have been implemented in primary care settings to identify patients due for cancer screening tests, while functioning as a real time reminder system. There is little known about primary care providers (PCPs) perspective or user acceptance of CDSS. The purpose of this study was to investigate primary care provider perceptions of utilizing CDSS alerts in the EMR to promote increased screening rates for breast cancer, cervical cancer, and colorectal cancer.
METHODS: An electronic survey was administered to PCPs in a Midwest Health Institution community internal medicine practice from September 25, 2019 through November 27, 2019.
RESULTS: Among 37 participants (9 NP/Pas and 28 MD/DOs), the NP/PA group was more likely to agree that alerts were helpful (50%; P-value = .0335) and the number of alerts (89%; P = .0227) in the EMR was appropriate. The NP/PA group also was more likely to find alerts straightforward to use (78%, P = .0239). Both groups agreed about feeling comfortable using the health maintenance alerts (MD/DO = 79%; NP/PA = 100%).
CONCLUSION: CDSSs can promote and facilitate ordering of cancer screening tests. The use of technology can promptly identify patients due for a test and act as a reminder to the PCP. PCPs identify these alerts to be a beneficial tool in the EMR when they do not interrupt workflow and provide value to patient care. More work is needed to identify factors that could optimize alerts to be even more helpful, particularly to MD/DO groups.

Introduction

Primary care has immense value on patient outcomes, quality, and decreasing costs. In addition to acute and chronic disease management, PCPs have important roles to recommend and order cancer screening tests. Lower cancer screening rates are correlated with shorter survival times and late stage diagnosis.[1,2] Consequently, delayed diagnosis can lead to poor patient outcomes, economic burden, and emotional insecurity.[3] As complexity of cancer screening evolves, PCPs face challenges in evaluating multiple screening options and appropriately following up on results and rescreening intervals.[4-7]

Cancer is the second leading cause of death in the United States (US).[8] Estimated new US cases for 2020 includes increasing breast (276 480 cases), colorectal (147 950 cases), and cervical (13 800 cases) cancer.[9-11] US death rates for these 3 cancer types are declining and may reflect screening increases.[1,9-12]

Cancer Screening Tests: Breast, Colorectal, Cervical

Although the screening rates for breast, colorectal, and cervical (72.8%, 66.8%, and 81.1%, respectively) cancer have demonstrated increasing trends, the national rates reported in 2018 are well below targets.[13] Evaluation for breast, colorectal, cervical cancer has been reported to be preventable through regular screening intervals.[12,14] Benefits of detecting cancer early, including treatment effectiveness and survival, outweigh the potential risks associated with screening.[13]

Breast cancer screening guidelines continue to vary across government and professional organizations. This could contribute to overall confusion of when to initiate mammography and screening frequency. Healthy People 2020 follows recommendations of the United States Preventive Services Task Force (USPSTF): biennial screening mammography for women 50 to 74 years.[15] USPSTF promotes individual discussion of mammography screening prior to age 50.[15] Women at average risk for breast cancer with dense breast tissue may also qualify for supplemental screening in addition to mammography.[16-18]

Colorectal cancer screening guidelines vary on method and interval of screening based upon testing. Recommendations for adults ages 50 to 75 is to complete one of the following stool-based or structural exams: colonoscopy, sigmoidoscopy, computed tomography (CT) colonography, fecal occult blood test (FOBT), fecal immunochemical test (FIT), or fecal DNA test.[19] Rescreening intervals are variable based on test, results, and should be individualized to the patient.[13]

Cervical cancer screening guidelines, reported by the USPSTF, recommends cervical cytology every 3 years for women ages 21 to 29.[20] In addition, the USPSTF recommends cervical cytology every 3 years, high risk HPV testing every 5 years, or cervical cytology with HPV co-testing every 5 years in women ages 30 to 65.[20] Other professional organizations, including the American College of Obstetricians and Gynecologists (ACOG),[21] the American Cancer Society (ACS), the American Society for Colposcopy and Cervical Pathology (ASCCP), and the American Society for Clinical Pathology,[22] prefer the screening method for women ages 30 to 65 as cervical cytology with HPV co-testing.

It has been previously reported that a barrier to cancer screening is lack of provider recommendation.[23-25] For example, reports show patients are directly influenced by physician recommendation to complete colorectal cancer screening.[26,27] Knowledge sharing can reduce barriers to cancer screening when patients understand importance.[28] Providers should also be aware of individuals at higher risk of developing cancer, such as family history or other health risk factors, and provide education on benefits from earlier and perhaps frequent screening intervals.

Clinical Decision Support Systems

Implementation of a systems strategy utilizing EMR alerts promotes and guides provider recommendations for cancer screening. Alerts prompting within the EMR are considered CDSS. CDSSs are multifaceted and incorporate individualized patient recommendations through information technology algorithms enhancing clinical decision-making skills.[29-32] CDSS alerts can also evaluate and improve metric performance.[33]

These algorithms can facilitate cancer screening recommendations based on a patient’s individualized health risk and comorbidities, to guide the provider on specific orders. In order to avoid workflow disruption, these tools are optimized when thoughtfully embedded directly into the EMR. Dynamic health care needs require PCPs to prioritize competing patient and clinic responsibilities, including recommendations for cancer screenings amidst managing other complex health conditions. CDSS tools provide a layer of patient safety to reduce medical errors and improve patient outcomes.[30,34-36]

Previous studies have presented benefits of CDSS alerts within the EMR to facilitate ordering of preventive care, including cancer screening procedures.[37,38] Benefits of CDSS include workflow efficiency, patient safety, cost effectiveness, and system replication.[39,40] However, few studies have demonstrated providers increasing their ordering behaviors for breast, cervical, and colon cancer screening tests directly through alert systems.[41-43] There is little known about provider perspective or user acceptance in relationship to CDSS use in primary care.[33,36,44,45]

The objective of this study was to investigate PCP perceptions of utilizing CDSS alerts in the EMR to promote increasing screening rates for breast cancer, cervical cancer, and colorectal cancer.

Methods

Setting

The study was a cross-sectional survey of PCPs assigned to care for patients within the Community Internal Medicine practice at a Midwest Health Institution in the United States. This practice operates at 4 different free-standing local clinics.

Variables

The survey examined provider demographics, perceptions, knowledge, and experiences related to general use of CDSS and to recommend cancer screening procedures in a primary care setting.

Data Collection

Surveys were emailed to a total of 73 study participants (11 NP/PAs and 62 MD/DOs). This included 9 NP/PAs and 28 MD/DOs. Using REDCap,[46] the initial survey was sent through email on September 25, 2019; the final and fourth reminder email with attached survey was sent to non-responders on November 27, 2019. Data collection was closed on December 31, 2019.

Participants

The medical staff surveyed included the following criteria: NPs, PAs, MDs, and DOs, whose assignment was within the Community Internal Medicine practice at a Midwest practice in the US.

Survey Development: The survey focused on knowledge and practices of a real-time clinical decision support tool, automatic cancer screening alerts within the electronic medical record. These questions were developed by the study team. Pilot testing of the survey was conducted with 3 clinicians to assess the acceptability, readability, and understandability of the survey. The pilot survey underwent 4 rounds of testing and refinement before finalization. The resulting one-time online survey took 5 minutes to complete.

The survey questions are found in Appendix Tables 1, 2 and 3. A number of the questions had branching logic and a majority of the questions had Likert scale responses which included responses such as “strongly agree,” “agree,” “disagree” and “strongly disagree”. The 5 overarching components of the survey were: (1) Demographics; (2) General uses of all Alerts; (3) Alert uses for Breast Cancer Screening; (4) Alert uses for Cervical Cancer Screening; (5) Alert uses for Colon Cancer Screening.

Appendix Table 1.

Employment Demographics by MD/DO and NP/PA.

	MD/DO (n = 28)	NP/PA (n = 9)
Age Range n, (%)
<30	0 (0%)	1 (11.1%)
30-39	8 (28.6%)	3 (33.3%)
40-49	9 (32.1%)	6 (55.6%)
50-59	3 (10.7%)	0 (0%)
60-69	7 (25.0%)	0 (0%)
≥70	1 (3.6%)	0 (0%)
Sex n, (%)
Male	13 (46.4%)	0 (0%)
Female	15 (53.6%)	9 (100%)
Race/ethnicity n, (%)
White, non-Hispanic	22 (78.6%)	9 (100%)
White, Hispanic	1 (3.6%)	0 (0%)
Asian	2 (7.1%)	0 (0%)
Other	2 (7.1%)	0 (0%)
Chose not to disclose	1 (3.6%)	0 (0%)
Current Employment Status n, (%)		*
Full time	20 (71.4%)	6 (75.0%)
Part time	7 (25.0%)	2 (25.0%)
Retired/emeritus	1 (3.6%)	0 (0%)
Time worked per week in direct patient care setting (1 = one half day in clinic)
Mean ± SD	5.8 ± 1.8	7.4 ± 2.7
Min, Max	2, 10	2, 10
Years of practice
Mean ± SD	16.3 ± 12.7	7.8 ± 3.7
Min, Max	0, 51	4, 15

1 missing value.

Appendix Table 2.

General Uses of All Alerts.

	MD/DO (n = 28)	NP/PA (n = 9)	Total (n = 37)	P-value
I frequently utilize the EMR alerts to order health maintenance procedures for which a patient is due, n (%)				1.0000[1]
Agree	27 (96.4%)	9 (100.0%)	36 (97.3%)
The alert reminds me about tasks that I would have otherwise forgotten, n (%)				1.0000[1]
Agree	26 (92.9%)	9 (100.0%)	35 (94.6%)
The location of the alert in the EMR affects my use of it, n (%)				.3727[1]
Agree	23 (82.1%)	6 (66.7%)	29 (78.4%)
I can do my job more efficiently as a result of the alert, n (%)				.1600[1]
Agree	21 (75.0%)	9 (100.0%)	30 (81.1%)
I am motivated to use the health maintenance alerts in the EMR, n (%)				.3067[1]
Agree	23 (82.1%)	9 (100.0%)	32 (86.5%)
The alert reminds me of current evidence based guideline recommendations, n (%)				.1589[1]
Agree	20 (71.4%)	9 (100.0%)	29 (78.4%)
The alert is accurately prompting in the EMR, n (%)				1.0000[1]
Agree	11 (39.3%)	4 (44.4%)	15 (40.5%)
I am uninterested in using the EMR alert to order a health maintenance test, n (%)				.4324[1]
Agree	1 (3.6%)	1 (11.1%)	2 (5.4%)
Alerts in the medical record are straight forward to use, n (%)				.0239[1]
Agree	9 (32.1%)	7 (77.8%)	16 (43.2%)
I am comfortable using the health maintenance alerts, n (%)				.3025[1]
Agree	22 (78.6%)	9 (100.0%)	31 (83.8%)
I do not know how to use the health maintenance alerts, n (%)				.3025[1]
Agree	6 (21.4%)	0 (0.0%)	6 (16.2%)
I would benefit from education about EMR alert use, n (%)				.4339[1]
Agree	12 (42.9%)	2 (22.2%)	14 (37.8%)
It would be helpful if there were more alerts in the EMR, n (%)				.0335[1]
Agree	3 (11.1%)	4 (50.0%)	7 (20.0%)
Missing	1	1	2
It would be helpful if there were fewer alerts in the EMR, n (%)				.2546[1]
Agree	19 (67.9%)	4 (44.4%)	23 (62.2%)
The number of alerts generated by the system is appropriate, n (%)				.0227[1]
Agree	12 (42.9%)	8 (88.9%)	20 (54.1%)
Completing the recommendations shown by the alert did not take too much time, n (%)				.2546[1]
Agree	9 (32.1%)	5 (55.6%)	14 (37.8%)
The alerts do not interrupt my usual workflow, n (%)				.2616[1]
Agree	15 (53.6%)	7 (77.8%)	22 (59.5%)
The use of clinical decision support system alerts is helpful, n (%)				.6563[1]
Agree	22 (78.6%)	8 (88.9%)	30 (81.1%)
The alert improves patient care, n (%)				.5536[1]
Agree	24 (85.7%)	9 (100.0%)	33 (89.2%)
The alert improves patient outcomes, n (%)				.0786[1]
Agree	19 (67.9%)	9 (100.0%)	28 (75.7%)
The alert enhances patient safety, n (%)				.2293[1]
Agree	18 (64.3%)	8 (88.9%)	26 (70.3%)

Fisher Exact P-value.

Appendix Table 3.

Percentages of MD/DO and NP/PA Agreeability to EMR Alert Usage for Breast, Cervical, and Colorectal Cancer.

	MD/DO (n = 28)	NP/PA (n = 9)	Total (n = 37)	P-value
Questions
I feel comfortable discussing recommended cancer screening guidelines with patients, n (%)
Breast	24 (88.9%)[2]	9 (100.0%)	33 (91.7%)[2]	.5576[1]
Cervical	28 (100.0%)	8 (100.0%)[2]	36 (100.0%)[2]
Colorectal	27 (100.0%)[2]	9 (100.0%)	36 (100.0%)[2]
The cancer screening Best Practice Advisory (BPA) alert guides my screening recommendation, n (%)
Breast	16 (61.5%)[3]	7 (77.8%)	23 (65.7%)[3]	.4496[1]
Cervical	20 (71.4%)	7 (77.8%)	27 (73.0%)	1.0000[1]
Colorectal	19 (70.4%)[2]	8 (88.9%)	27 (75.0%)[2]	.3963[1]
Limited appointment time prevents me from discussing cancer screening when it is not the primary reason for visit, n (%)
Breast	13 (46.4%)	6 (66.7%)	19 (51.4%)	.4470[1]
Cervical	17 (60.7%)	8 (88.9%)	25 (67.6%)	.2204[1]
Colorectal	14 (51.9%)[2]	6 (66.7%)	20 (55.6%)[2]	.7003[1]
I am able to identify patients due for cancer screening, n (%)
Breast	25 (89.3%)	9 (100.0%)	34 (91.9%)	.5622[1]
Cervical	25 (92.6%)[2]	9 (100.0%)	34 (94.4%)[2]	1.0000[1]
Colorectal	24 (88.9%)[2]	9 (100.0%)	33 (91.7%)[2]	.5576[1]
I have the resources to answer patient questions surrounding cancer screening, n (%)
Breast	23 (82.1%)	9 (100.0%)	32 (86.5%)	.3067[1]
Cervical	26 (92.9%)	9 (100.0%)	35 (94.6%)	1.0000[1]
Colorectal	26 (96.3%)[2]	9 (100.0%)	35 (97.2%)[2]	1.0000[1]
Although the patient is due for cancer screening, I do not discuss this when it was declined in the past, n (%)
Breast	4 (14.3%)	1 (11.1%)	5 (13.5%)	1.0000[1]
Cervical	3 (11.1%)[2]	2 (22.2%)	5 (13.9%)[2]	.5810[1]
Colorectal	3 (11.1%)[2]	2 (22.2%)	5 (13.9%)[2]	.5810[1]
Patient age influences the likelihood I order the screening, n (%)
Breast	21 (75.0%)	7 (77.8%)	28 (75.7%)	1.0000[1]
Cervical	16 (61.5%)[3]	7 (77.8%)	23 (65.7%)[3]	.4496[1]
Colorectal	16 (61.5%)[3]	7 (77.8%)	23 (65.7%)[3]	.4496[1]
Patient ethnicity influences the likelihood I order the screening, n (%)
Breast	2 (7.1%)	0 (0.0%)	2 (5.4%)	1.0000[1]
Cervical	3 (11.5%)[3]	0 (0.0%)	3 (8.6%)[3]	.5531[1]
Colorectal	3 (11.5%)[3]	0 (0.0%)	3 (8.6%)[3]	.5531[1]
I am less likely to order the screening when an interpreter is used during the visit, n (%)
Breast	6 (21.4%)	2 (22.2%)	8 (21.6%)	1.0000[1]
Cervical	5 (19.2%)[3]	2 (22.2%)	7 (20.0%)[3]	1.0000[1]
Colorectal	5 (18.5%)[2]	2 (22.2%)	7 (19.4%)[2]	1.0000[1]
The patient’s primary care provider should be responsible for ordering cancer screening tests, n (%)
Breast	15 (53.6%)	8 (88.9%)	23 (62.2%)	.1120[1]
Cervical	19 (70.4%)[2]	8 (88.9%)	27 (75.0%)[2]	.3963[1]
Colorectal	19 (70.4%)[2]	8 (88.9%)	27 (75.0%)[2]	.3963[1]
I regularly follow up with patients who have not completed their cancer screening test to understand why it was not carried out, n (%)
Breast	5 (18.5%)[2]	2 (22.2%)	7 (19.4%)[2]	1.0000[1]
Cervical	10 (37.0%)[2]	5 (55.6%)	15 (41.7%)[2]	.4427[1]
Colorectal	8 (29.6%)[2]	4 (44.4%)	12 (33.3%)[2]	.4428[1]
The patient has other health conditions requiring monitoring that are more important than cancer screening, n (%)
Breast	12 (44.4%)[2]	5 (55.6%)	17 (47.2%)[2]	.7060[1]
Cervical	11 (40.7%)[2]	4 (44.4%)	15 (41.7%)[2]	1.0000[1]
Colorectal	12 (44.4%)[2]	6 (66.7%)	18 (50.0%)[2]	.4430[1]
Questions specific to cervical cancer
I have skills necessary to perform pap smear examination in a clinic visit, n (%)	28 (100.0%)	9 (100.0%)	37 (100.0%)
Patients are willing to complete pap smear testing performed at the visit if it is recommended to them, n (%)	22 (78.6%)	7 (77.8%)	29 (78.4%)	1.0000[1]
Questions specific to colorectal cancer
Patients will follow through and complete colorectal cancer screening testing, n (%)	20 (74.1%)[2]	7 (77.8%)	27 (75.0%)[2]	1.0000[1]

Fisher Exact P-value

Missing, n = 1.

Missing, n = 2.

All potential participants were contacted via an email which informed them of the general purpose of the study, that the survey was voluntary, who to contact with questions or complaints, and that participation/nonparticipation did not jeopardize their care or employment at their institution. Participants were reassured of anonymity with survey participation. If they wished to participate, a link was provided at the end of the email, which led directly to the survey questions captured via REDCap™. All non-responders received up to 4 email reminders before all correspondence ceased.

Statistical Analysis

Baseline demographics and training background information provided by the responders are summarized for continuous variables using mean, standard deviation, min and max and for categorical variables using frequency percentages. These variables were calculated in total and by group: NP/PAs and MD/DOs. All questions assessing respondent attitudes and behaviors were assessed using a 4-point Likert scale (ranging from strongly agree to strongly disagree). For analysis purposes responses for “Agree” and “Strongly Agree” were combined, as were the responses of “Disagree” and “Strongly Disagree”. The resulting analysis variables had 2 levels (1=Agree, 0=Disagree). The individual questions were summarized using frequency percentages with Fisher’s Exact test to compare responses between groups. In all cases, a two-tailed P-value of less than .05 was considered significant. Statistical analysis was done using SAS statistical software, v 9.4[47]

Ethical Considerations

This study was determined to be exempt by the Mayo Clinic Institutional Review Board.

Results

In total, 37 participants (50.7%) completed and returned the survey. Of the surveys returned, 9 were from NP/PA and 28 were from MD/DO (Appendix Table 1). In the MD/DO group, most of the respondents were female (54%), white (79%), and full-time employees (72%) who had practiced an average of 16 years. The NP/PA group was all female and white, with most being full-time employees (75%) and having practiced an average of 8 years.

Figure 1 shows the survey questions where NP/PA and MD/DO significantly differed. Two of the questions were about the number of alerts. The MD/DO group were more likely to disagree with the statement that more alerts would be helpful (89%) compared to the NP/PA group where only half disagreed. When asked if they thought the number of alerts was appropriate, a majority of the NP/PA agreed (89%) while only 43% of MD/DO agreed. The other question which showed significant disagreement was if the participants thought that the alerts in the EMR were straightforward to use. 68% of the MD/DO disagreed but only 22% of the NP/PA disagreed.

Figure 1.

Significant differences in perceptions about EMR alerts between MD/DO and NP/PA groups. There were 3 survey questions related to questions surrounding general use of all alerts in the EMR with significant differences: alerts were straightforward to use (P = .0239), the number of alerts was appropriate (P = .0227), and more alerts would be helpful (P = .0335).

Both groups agreed that they felt comfortable using the health maintenance alerts (MD/DO = 79%, NP/PA = 100%).

Discussion

Value of CDSS

In this sample of PCPs, 97.3% reported frequent utilization of EMR alerts to order health maintenance including cancer screening tests and 94.6% reported interest in CDSS use. Other studies have reported low provider utilization of CDSS to address preventive care.[48-50] Of our survey responders, 94.6% agreed that the alert provided a reminder for tasks that may have otherwise been forgotten during the clinic visit as well as the alert promoted job efficiency (81.1%). The location of the alert in the EMR also affected the clinician’s use (78.4%). Despite these generally favorable results, 56.8% reported that the alerts were not straightforward to use and 80% believed it would not be helpful to have more alerts in the EMR. Although there was varying user acceptance, PCPs recognized that CDSS tools in the EMR provide value to patient care, which was similar to findings conducted in other studies.[50,51]

Promoting Screening

When other health conditions are competing with time spent on discussing recommended cancer screening tests, CDSS may be even more valuable in primary care. Although not statistically significant, our study found that approximately 40% to 50% of providers said the patient had other health conditions to monitor that took priority over breast (47.2%), cervical (41.7%), and colorectal (50%) screening. A previous study conducted in the Midwest reported only 53% of PCPs prioritizing cancer screening.[36] CDSS can provide guidance in ordering these tests known to be effective in preventing curable disease and facilitate an effective office visit.[52]

CDSS Design and Considerations

CDSSs should be designed to have logic implemented for appropriateness of when an alert should prompt in the EMR. For example, alerts for certain tests may prompt in an ambulatory setting including primary care and specialty clinics, but would not be appropriate when translated directly to an inpatient setting. In order to minimize alert fatigue, ensuring the appropriate audience to address the alert should be a factor taken into consideration by the development team.[53,54] CDSSs are real-time decision-making tools that leverage the EMR to make better-informed health decisions between the clinician and the patient. It has been reported that ordering cancer screening is not effective alone; rather communication between the PCP and patient at every visit is vital to promote test completion.[55] Thoughtful placement of EMR alerts also promotes flexibility, supports clinical workflow, and avoids a helpful tool from becoming burdensome.[40,45,53]

Causality of Employment Demographics in Participants

In this study, the NP/PA group had favorable responses to alerts being straightforward to use (77.8%), wanting more alerts (50%), and that the number of alerts generating is appropriate (88.9%) in comparison to their colleagues in the MD/DO group. These findings could be due to several factors seen in the demographic information obtained by the participants. The MD/DO group had more years of experience compared to the NP/PA group. The NP/PA group was younger in age and consisted of all white, non-Hispanic females. This finding suggests that providers younger in age and with less experience utilize EMR alerts more often, which is similar to previously reported data related to less experienced physicians adapting technology use into practice.[56-58] Healthcare training for this younger group likely included contact with multiple EMRs, leading to more technology exposure and consequently comfort with use. Advance practice providers (APPs) APPs may also have additional experience with CDSS if they previously held other healthcare roles using alert while rooming the patient (for example, nursing or urgent care setting).[53] It also may suggest that APPs use EMR alerts more than physicians. The NP/PA group also had more time worked per week in a direct patient care setting which may account for higher utilization having more exposure to seeing and using the alerts than the MD/DO group. There is multifaceted complexity of user acceptance of technology and influences of CDSS use in the EMR. User acceptance tends to be more favorable if the CDSS matches an individual’s decision-making process. Consequently, less favorable use may be driven by the unrevealing process of how output decisions are made causing uncertainty. Other researchers suggested user acceptance related to CDSSs could be achieved through end-user involvement in the design process and engagement[44]

Limitations

There were several limitations to this study. First, although the survey response rate was 50.7%, we do not know if participants who responded to the survey were more likely to be high utilizers of the alerts than those who did not respond. Our findings are vulnerable to response bias, which may also be reflected by the significantly smaller sample of respondents among the NP/PA group (n = 9) compared to the MD/DO group (n = 28). Additionally, this Midwest Health Institution acquired a new EMR system-wide in May 2018. This study did not acknowledge if those who took the survey were among individuals who received enhanced training to become expert end-users of the EMR compared to those who received basic training. Another limitation is our respondents were predominantly non-Hispanic white (83.8%, n = 31) and may not be generalizable to other primary care practices.

Strengths

Strengths of our study include the survey response rate and sample of participants, allowing for a comparison of perceptions between physicians and APPs. It also contributes toward the literature in this area and reveals further potential areas of inquiry, such as how to further improve the usefulness of EMR alerts related to cancer screening.

Conclusion

Healthcare providers in a primary care practice use CDSS alerts in the EMR to facilitate cancer screening ordering. In comparison to the physician group, APPs (NP/PA group) in this cohort were more likely to agree the number of alerts generated by the system was appropriate, use was straightforward, and more alerts would be beneficial. Additional research is needed to evaluate provider barriers that may influence use of CDSS tools, such as provider training (physician versus APPs), age, and years in practice. Future studies could also determine how to prioritize alert recommendations to the provider and integration into the clinical workflow in an outpatient setting.