Evaluation of the entrustable professional activities (EPAs) of the population health promoter domain by North Dakota pharmacists.

BACKGROUND: Entrustable Professional Activities (EPAs) are a list of professional tasks that pharmacy educational organizations support, and accreditation organizations encourage, for assessment by colleges and schools of pharmacy.
OBJECTIVE: This manuscript evaluates the perceived frequency of performing EPAs in the population health promoter (PHP) domain among pharmacists practicing in North Dakota.
METHODS: This survey evaluated the self-reported EPA activities of registered pharmacists living and practicing in North Dakota. For EPAs and supporting tasks in the 6 domains (including the PHP domain), respondents were asked to self-report the number of times during the last 30 days that they perform the task, using a 6 point response scale (0, 1, 2, 3, 4, 5 or more times). There were 990 pharmacists surveyed, and 457 (46.1%) of pharmacists responded.
RESULTS: Within the PHP domain, pharmacists reported performing "Minimize adverse drug events and medication errors" most frequently (mean=3.4, SD=2.0), followed by "Ensure that patients have been immunized against vaccine-preventable diseases" (mean=2.3, SD 2.3), "Maximize the appropriate use of medications in a population" (mean=2.2, SD 2.3), and "Identify patients at risk for prevalent diseases in a population" (mean=1.3, SD=1.9). In these Core EPAs PHP domains, the clinical pharmacists reported the highest level, followed by pharmacy managers and staff pharmacists.
CONCLUSIONS: Pharmacists in North Dakota currently perform some population health promoter activities, but not at a consistent and high level. Most of the health prevention activities were medication-related and oriented towards individual patients (micro-level), rather than at a community (population-based) macro-level. Copyright: © Pharmacy Practice and the Authors.

INTRODUCTION

Ensuring that students successfully transition from didactic education to experiential education, and ultimately into autonomous practice, is a major objective of pharmacy education. To do so, pharmacy educators must link academic outcomes with activities commonly undertaken in practice. They must also ensure that students can undertake these activities with an appropriate (often a minimal) level of supervision when they graduate and pass their board exams.

Medical educators ten Cate and Scheele were the first to investigate the process of integrating practice abilities with academic competencies, which led to the creation of Entrustable Professional Activities (EPAs).1,2 EPAs are a collection of statements, each of which describes an activity that a pharmacist routinely undertakes with a reasonable degree of autonomy in practice.3-6 EPAs are written in a manner that allows clinical preceptors to connect the competency statements currently used in didactic education to professional activities undertaken in clinical settings.5,6 When combined with a rating scale, EPAs with the rating scale represent a competency-based system used to evaluate health care professional training, including pharmacy education.3-5 In doing so, EPAs attempt to provide a “common language” for both academic and experiential educators to assess the knowledge, skills, and abilities of their students.3,7

This manuscript focuses on the development and use of EPAs in pharmacy education. EPAs were introduced into academic pharmacy during the 2015-2016 academic year, when the American Association of Colleges of Pharmacy (AACP) Academic Affairs Committee released its six Core EPA domains for pharmacy: population health promoter, patient care provider, practice manager, information master, interprofessional team member, and self-developer.8 New Doctor of Pharmacy graduates should be able to perform each and every one of these activities with limited supervision (i.e., “Level 3” performance).5 Within pharmacy-based EPAs, this manuscript specifically focuses on the population health promoter (PHP) domain.

EPAs also provide a means to characterize the unique contributions of pharmacists to team-based patient care. This is especially true for the PHP domain, which is inherently interdisciplinary in nature.5,7 Pharmacist preceptors who undertake these activities provide pharmacy students with an opportunity to more fully grasp the impact of their practice on the health of patient populations as a whole, as well as a deeper understanding of how their practice integrates with, and supports, the work of other clinicians to ensure healthy populations.

While pharmacists practicing in any setting will undertake each EPA, it is also important to note that specific EPAs will be undertaken more or less frequently in specific pharmacy practice settings (institutional, community, ambulatory care, etc.).9 The more frequently pharmacists in a given practice setting undertake an EPA, the greater the opportunity for pharmacy students to develop greater levels of entrustability with that activity. Thus, before EPAs can be fully embedded in academic pharmacy, it is vital to assess the frequency with which pharmacists in certain practice settings actually perform specific EPAs in their daily practice. As a corollary, it is useful to determine whether the frequency of use varies by position and preceptor status, in addition to practice setting.

Previous studies have examined the perceived frequency of EPA use (both overall and across practice settings, by preceptor status, and by the type of position/responsibility) in the Patient Care Provider (PCP) and Practice Management (PM) domains.10,11 Substantial differences were found in the perceived frequency of use across various practice settings. However, little is known about the perceived use of EPAs in the other four domains. This is especially problematic, since the continued evolution of the pharmacy profession is critically linked to its ability to develop an expanded role in population health, information management, and collaborations with other clinicians.

The primary objective of this manuscript is to conduct a pilot study evaluating the perceived frequency of use of EPAs in the PHP domain. As a secondary objective, this manuscript explores whether the perceived frequency of EPAs undertaken in the PHP domain varies across practice settings, across the seniority and responsibilities of the position, or by preceptor status.

METHODS

This study is a part of a larger initiative undertaken by the North Dakota Institute of Pharmaceutical Care to assess the frequency with which pharmacists in North Dakota perform EPAs in all six major domains. The methods utilized in this paper are described by other, previously published studies stemming from this initiative.10,11 To implement the initiative, a survey was designed, using established criteria in the survey literature.12-16 The target population consists of all pharmacists who are licensed and currently practicing as a pharmacist (as their primary location of practice) in the state of North Dakota. The research team used its knowledge of survey design, as well as the pharmacy services literature, to create an initial version of the survey.12-16 Five pharmacists agreed to pilot test the survey and provide feedback for improvement, which was incorporated into the survey. Once those revisions were made, the survey was submitted to the NDSU Institutional Review Board, who approved the survey and its methods of administration in the fall of 2017.

The approved version of the survey was implemented using the Qualtrics Software System (www.qualtrics.com). The survey is comprised of four sections. The first section contains a cover letter (inclusive of explaining all rights and responsibilities of participants, in accordance with Institutional Review Board protocols) and an item designed to identify whether respondents meet the study’s inclusion criteria. The second section lists each core EPA, along with supporting tasks.10,11 Based on the literature and the authors’ collective knowledge of pharmacy practice, the authors determined that the vast majority of practicing pharmacists would undertake activities in the practice manager EPA with much greater frequency than activities in other types of domains. This greater frequency of use allows respondents to report their perceptions of these tasks with an extremely high degree of consistency, accuracy and precision. Thus, for the practice manager EPA and all associated supporting tasks, respondents are asked to self-report the number of times during the last 30 days that they perform the task, using an 11 point response scale (0, 1, 2, 3, 4, …, 8, 9, 10 or more times). The authors were less certain that all pharmacists would be able to provide responses with the same degree of consistency, accuracy and precision for activities in other domains, including the PHP domain. Thus, for EPAs and supporting tasks in the other 5 domains (including the PHP domain), respondents are asked to self-report the number of times during the last 30 days that they perform the task, using a 6-point response scale (0, 1, 2, 3, 4, 5 or more times). In the third section, each of the six core EPAs was listed, and respondents were asked to rate (using a 1 “not useful” to 5 “very useful” response scale) the degree to which additional training on the core EPAs would be useful to further the pharmacist’s practice. The final section asks respondents to provide some basic demographics, including gender, age, name of the pharmacy school the respondent attended, year of graduation from a pharmacy program, highest degree earned, current primary practice setting, their primary position/level of responsibility, the location of their practice, and whether or not the respondent served as a preceptor for students attending the state’s lone school of pharmacy during the past year. The final section of the survey concluded with an open-ended question entitled, “Please add anything else you would like to tell us.”

An important consideration in the assessment of EPAs is the location of the practice setting.10-12,14 Due to differences in patient populations, available staffing, financial resources, and local health services infrastructure, pharmacists practicing in rural areas may not undertake the same combination of EPAs (or with the same frequency) as those practicing in urban areas. North Dakota is an extremely sparsely populated state. More than half of its counties have population densities of less than six people per square mile and are designated by the U.S. Health Resources and Services Administration as “frontier counties”.10,11 Concomitantly, four communities in the state (Fargo, Grand Forks, Bismarck, and Minot) have populations of 25,000 or more residents. The latter are typically categorized as “urban” communities, while all other communities are categorized as “rural”.10,11

The North Dakota Board of Pharmacy provided a list of email or physical mailing addresses for all pharmacists currently licensed in the state. Using the physical mailing addresses, the investigators removed the names of pharmacists on the email list who reside well outside of the state’s borders (and cannot commute to North Dakota to practice), the research team was left with a set of 990 possible study participants. As an aside, more than half of North Dakota’s population resides in the Red River Valley along the Minnesota-North Dakota border. The 990 names likely include pharmacists who are dually licensed in North Dakota and Minnesota, but who primarily practice in Minnesota. Thus, this number likely overstates the eligible study population by a considerable margin.

The final, approved version of the survey was emailed to the 990 possible study participants in late September 2017. Consistent with the survey design literature, email reminders were sent 2, 4, and 6 weeks afterwards. Reminders were sent from the pharmacy program’s Senior Associate Dean in week 8, and from the Dean in week 10. A final email reminder was sent by the study’s principal investigator in the 12th week of data collection. Data collection concluded in March 2018.

Data analysis

Responses were stored in a secured database, with access limited to the authors responsible for data analysis (3rd and 4th authors, respectively). To be consistent with Institutional Review Board guidelines, all responses were anonymous.

As noted above, respondents were asked to self-assess the number of times in the previous 30 days that they undertook a core EPA or an associated supporting task using an 11 point response scale (0, 1, 2, 3, 4, …, 8, 9, 10 or more times) for the practice manager domain items, and a 6 point response scale (0, 1, 2, 3, 4, 5 or more times) for all other core EPAs and supporting tasks, including those in the PHP domain. The pattern of responses directly impacts the analysis of data and the reporting of any statistical results. If a given EPA or supporting task is performed only rarely each month, then responses should be relatively evenly distributed across the scale. This allows the data collected from this scale to be treated as an approximation of an ordinal variable, which can be appropriately summarized using traditional measures of central tendency (i.e., means) and dispersion (standard deviations). Concomitantly, EPAs and supporting tasks performed regularly (i.e., on a daily basis) will generate responses that are clustered tightly around the upper end of the scale, producing truncation. In those cases, measures of central tendency and dispersion are biased in a downward fashion, and the data are more appropriately summarized using frequencies or proportions of observations that are, and are not, at the upper end of the scale. A priori, the researchers do not have an expectation about which of these two events will occur. Given these considerations, and consistent with the EPA literature, a decision was made to report results in two forms.10,11 First, for approximately ordinal (i.e., non-truncated) variables, traditional means and standard deviations are reported. For variables that are potentially truncated, the proportion of responses at the truncation point (i.e., the proportion of responses reporting that they perform a task “5 or more times in the last 30 days”) is reported. This allows the reader to more clearly grasp the trends in the data and choose the appropriate descriptive statistic to read and interpret, should truncation be present or absent in the data for a given EPA or supporting task.

A corollary to the study’s research objective was to assess whether significant differences exist in the self-reported frequency of undertaking PHP EPAs varies across practice settings, across the type of the position, or by preceptor status, among other available demographic groups. Consistent with the academic pharmacy EPA literature, the study adopts a very general null hypothesis of no relationship between the frequency of undertaking a specific PHP EPA or supporting task and the pharmacist characteristic being compared.10,11 For variables that are not truncated, the Kruskal-Wallis test is an appropriate means to assess the study’s null hypothesis.17 This test operates under the more specific null hypothesis of no mean differences in the self-reported frequency of undertaking a PHP EPA or supporting task across specific groups of pharmacists (based on pharmacist role, community served, practice setting, etc.). For variables that are truncated, the chi-square test of homogeneity can be applied to assess whether or not a significant association exists between respondents who undertake a given PHP EPA or supporting task 5 or more times per month (versus those who undertake the task less than 5 times per month) and a particular set of pharmacist demographics (again, based on pharmacist role, practice setting, etc.).17 Consistent with the study’s general null hypothesis, the chi-square test of homogeneity also operates under a null hypothesis of no association between the frequency of undertaking a given EPA or supporting task and a specific pharmacist characteristic. Since the researchers have no prior expectations about whether specific variables exhibit, or do not exhibit, truncation, both tests are applied to all variables. This allows the interested reader to view both test results and choose the appropriate test to interpret, depending on whether truncation is, or is not, present in the data for a given EPA or supporting task. All tests utilize 5 percent levels of significance. All tests are conducted using the IBM SPSS Version 24 software package.

RESULTS

The survey was distributed to 990 individuals, of which 457 (or 46%) responded to at least one item in the survey. Of the 457 responses, another 102 were eliminated for failing to meet the study’s inclusion criterion of actively practicing as a pharmacist in North Dakota. Further analysis of these 102 responses indicates that a plurality were not practicing in North Dakota (n=33) or did not work for an employer that provides direct patient care (n=27). The remaining ineligible respondents were retired (n=28), unemployed (n=7) or employed in a non-pharmacy-related career (n=7). This yields 355 individuals who responded to at least one survey item in the demographic or PHP EPA sections of the survey. Unfortunately, a large number of individuals did not address one or more of these survey items, and many (if not the majority) of these respondents failed to answer many of the survey items.16 This result makes it infeasible to compare responders and non-responders. It is also unclear as to whether the additional information added by including responses from individuals who only partly completed the survey outweighs the inconsistency and increased information uncertainty introduced by allowing the sample size to vary from one survey item to the next. To avoid ad hoc or arbitrary decisions about whether and how to interpret missing values, a decision was made to include only those 121 individuals in the data analysis who provided a complete set of responses. Therefore, n=121 becomes the final sample size used in the analysis.

Descriptive statistics for all respondent demographics are presented in Table 1. Within the sample, 70% of respondents were female and 30% were male. The mean age of respondents was 43.2 years of age. Approximately 70% of respondents were less than 50 years of age. Approximately 50% of respondents held a Doctor of Pharmacy as their highest degree, while approximately 30% held a bachelor degree in pharmacy. A plurality of pharmacists (40%; n=49) reported working in an independent community setting, while another 26% (n=32) worked in a health system setting, and only 12% (n=15) reported working in a chain community pharmacy. These percentages are consistent with the pharmacy profession in North Dakota. More specifically, North Dakota has a legal requirement that (except for a few stores that were operating prior to the passage of the law) all community pharmacies in the state be majority owned by pharmacists licensed in the state.18 With regard to position, 38% (n=46) reported holding the position of pharmacy manager, 32% (n=39) reported working as staff pharmacists, 21% (n=26) as clinical pharmacists, and 8% (n=10) in other practice settings. Approximately 60% of pharmacists work in urban communities and 40% in rural communities. Fifty percent of all respondents serve as preceptors for pharmacy students attending the state’s lone school of pharmacy.

Table 1

Mean and proportion of the population health domain with each demographic characteristics (n=121)

Gender; n (%)
Female	80 (0.7)
Male	41 (0.3)
Age; mean (SD)	43.2 (11.5)
Age categories; n (%)
Under 40 years	54 (0.5)
40-49 years	27 (0.2)
50-59 years	28 (0.2)
60 years or older	12 (0.1)
Highest pharmacy-related degree; n (%)
Bachelor Degree	41 ( 0.3)
Doctor of Pharmacy	62 (0.5)
Post-Graduate Residency	15 (0.1)
Other Degree	3 (< 0.1)
Practice Setting; n (%)
Hospital	32 (0.3)
Independent Community	49 (0.4)
Chain Community	15 (0.1)
All Other Practice Settings	25 (0.2)
Respondent’s Role; n (%)
Pharmacy Manager	46 (0.4)
Staff Pharmacist	39 (0.3)
Clinical Pharmacist	26 (0.2)
All Other Roles	10 (0.1)
Population of Community Served; n (%)
Under 5,000 Residents	21 (0.2)
5,000-24,999 Residents	21 (0.2)
25,000 or More Residents	79 (0.6)
Serve as an NDSU Pharmacy Preceptor; n (%)
Yes, Serve as a Preceptor	57 (0.5)

Descriptive statistics for all survey items in the PHP EPA domain (including supporting tasks) are presented in Table 2. The PHP EPA “Minimize adverse drug events and medication errors” was the highest reported EPA, and was performed an average of 3.4 times per month (SD=2.0). Within this EPA, the supporting task identified by respondents as the one most frequently undertaken was “Assist in the identification of underlying system-associated causes of errors” performed an average of 2.0 times per month (SD=1.9). The PHP EPA domain was undertaken the second most frequently was “Ensure that patients have been immunized against vaccine-preventable diseases”, which (at the sample mean) was undertaken 2.3 times per month (SD=2.3). Within this domain, “Determine whether a patient is eligible for and has received CDC-recommended immunizations” was the most frequently performed supporting task (mean=2.2 times per month, SD=2.3), followed by “Administer and document CDC-recommended immunizations to an adult patient” (mean=1.5 times per month, SD=2.2). With the exception of “Minimize adverse drug events and medication errors”), 60% of respondents reported performing a task 4 or fewer times in a 30-day window.

Table 2

Mean and proportion of entrustable professional activities (EPAs) by population health promoter domain (n = 121)

		Number (proportion) of respondents who perform the EPA per week
Population Health Promoter EPA Description	Mean (SD)	< 5 times	5 or more times
1. Identify patients at risk for prevalent diseases in a population.	1.3 (1.9)	102 (0.8)	19 (0.2)
a. Perform a screening assessment to identify patients at risk for prevalent diseases in a population (e.g., hypertension, diabetes).	0.7 (1.5)	110 (0.9)	11 (0.1)
2. Minimize adverse drug events and medication errors.	3.4 (2.0)	55 (0.5)	66 (0.5)
a. Assist in the identification of underlying system-associated causes of errors.	2.0 (1.9)	93 (0.8)	28 (0.2)
b. Report adverse drug events and medication errors to stakeholders.	1.5 (1.8)	102 (0.8)	19 (0.2)
3. Maximize the appropriate use of medications in a population.	2.2 (2.3)	74 (0.6)	47 (0.4)
a. Perform a medication use evaluation.	1.5 (2.0)	99 (0.8)	22 (0.2)
b. Apply cost-benefit, formulary, or epidemiology principles to medication-related decisions.	1.7 (2.1)	91 (0.8)	29 (0.2)
4. Ensure that patients have been immunized against vaccine-preventable diseases.	2.3 (2.3)	76 (0.6)	45 (0.4)
a. Determine whether a patient is eligible for and has received CDC-recommended immunizations.	2.2 (2.3)	76 (0.6)	45 (0.4)
b. Administer and document CDC-recommended immunizations to an adult patient.	1.5 (2.2)	88 (0.7)	33 (0.3)
c. Perform basic life support.	0.1 (0.6)	120 (> 0.9)	1 (< 0.1)

An analysis of responses by practice setting is presented in Table 3. The most commonly reported activity across practice settings was “Minimize adverse drug events and medication errors.” No significant differences exist across practice settings, suggesting that pharmacists in every setting perform this task with similar frequency. Significant differences were noted among practice settings for “Identify patients at risk for prevalent diseases in a population” (Kruskal-Wallis p=0.02; chi-square p=0.02), “Maximize the appropriate use of medications in a population” (Kruskal-Wallis p=0.02; chi-square p=0.02), and “Ensure that patients have been immunized against vaccine-preventable diseases” (Kruskal-Wallis p=0.01; chi-square p=0.01). When examining statistically significant differences across practice settings for these Core PHP EPAs, pharmacists working for chain pharmacies generally reported the highest mean values, while those working for independent community pharmacies reported and lowest means.

Table 3

Entrustable professional activities (EPA) of population health promoter domain by practice setting (n = 121)

Population Health Description; mean (SD)	Hospital [n=32]	Independent Community [n=49]	Chain Community [n=15]	All Other Practices [n=25]	Kruskal-Wallis p value	Chi-Square p value *
1. Identify patients at risk for prevalent diseases in a population.	0.8 (1.5)	1.0 (1.9)	1.3 (1.6)	2.3 (2.3)	0.02	0.02
a. Perform a screening assessment to identify patients at risk for prevalent diseases in a population (e.g., hypertension, diabetes).	0.1 (0.5)	0.7 (1.5)	0.7 (1.4)	1.5 (2.1)	0.01	0.02
2. Minimize adverse drug events and medication errors.	3.6 (1.8)	2.8 (2.2)	3.9 (1.7)	3.8 (2.1)	0.08	0.15
a. Assist in the identification of underlying system-associated causes of errors.	2.4 (2.1)	1.8 (1.8)	2.3 (2.0)	1.8 (1.9)	0.65	0.28
b. Report adverse drug events and medication errors to stakeholders.	2.3 (2.1)	1.1 (1.8)	1.1 (1.3)	1.1 (1.5)	0.04	0.06
3. Maximize the appropriate use of medications in a population.	2.6 (2.5)	1.4 (2.1)	2.7 (23)	3.1 (2.3)	0.02	0.02
a. Perform a medication use evaluation.	2.6 (1.9)	1.0 (1.7)	2.2 (2.2)	2.2 (2.3)	0.05	0.09
b. Apply cost-benefit, formulary, or epidemiology principles to medication-related decisions.	1.4 (2.1)	1.6 (2.1)	2.0 (2.2)	1.8 (2.2)	0.82	0.91
4. Ensure that patients have been immunized against vaccine-preventable diseases.	2.0 (2.3)	2.2 (2.2)	4.3 (1.8)	1.8 (2.2)	0.01	0.01
a. Determine whether a patient is eligible for and has received CDC-recommended immunizations.	2.0 (2.3)	1.9 (2.3)	4.1 (1.8)	2.0 (2.3)	0.01	0.03
b. Administer and document CDC-recommended immunizations to an adult patient.	0.6 (1.7)	1.6 (2.3)	4.2 (1.8)	0.6 (1.7)	<0.01	<0.01
c. Perform basic life support.	0.2 (0.7)	0.0 (0.0)	0.3 (1.3)	0.0 (0.0)	0.03	0.12

Less than 5 versus 5 or more times per week

Table 4 contains an analysis of responses, disaggregated by position. As in Table 3, the item pharmacists reported the highest mean frequency was “Minimize adverse drug events and medication errors”. However, these differences were not statistically significant (Kruskal-Wallis p=0.43; chi-square p=0.29). The next most commonly undertaken item was “Maximize the appropriate use of medications in a population,” which also did not differ significantly by position type (Kruskal-Wallis p=0.19; chi-square p=0.06). Statistically significant differences by position did exist for the core EPA PHP domain “Identify patients at risk for prevalent diseases in a population” (Kruskal-Wallis p=0.02; chi-square p=0.17). In general, clinical pharmacists and pharmacy managers reported undertaking activities and tasks more frequently than staff pharmacists.

Table 4

Entrustable professional activity (EPA) of population health promoter by position (n = 121)

Population health promoter EPA description; mean (SD)	Manager [n=46]	Staff Pharmacist [n=39]	Clinical Pharmacist [n=26]	All Other Positions [n=10]	Kruskal-Wallis p-value	Chi-square p-value*
1. Identify patients at risk for prevalent diseases in a population.	1.6 (2.0)	0.8 (1.6)	1.7 (2.2)	0.3 (1.0)	0.02	0.17
a. Perform a screening assessment to identify patients at risk for prevalent diseases in a population (e.g., hypertension, diabetes).	0.7 (1.5)	0.6 (1.5)	1.1 (2.0)	0.0 (0.0)	0.26	0.29
2. Minimize adverse drug events and medication errors.	3.5 (1.9)	2.9 (2.2)	3.7 (1.8)	3.8 (2.0)	0.43	0.68
a. Assist in the identification of underlying system-associated causes of errors.	2.5 (2.0)	1.5 (1.8)	1.9 (1.9)	1.8 (2.1)	0.13	0.56
b. Report adverse drug events and medication errors to stakeholders.	1.9 (2.0)	0.9 (1.6)	1.6 (1.6)	1.1 (2.1)	0.05	0.20
3. Maximize the appropriate use of medications in a population.	2.3 (2.3)	1.7 (2.2)	2.8 (2.5)	3.0 (2.6)	0.19	0.06
a. Perform a medication use evaluation.	1.5 (1.9)	1.4 (2.0)	1.9 (2.3)	1.0 (2.1)	0.49	0.27
b. Apply cost-benefit, formulary, or epidemiology principles to medication-related decisions.	2.2 (2.1)	1.4 (2.1)	1.4 (2.1)	0.5 (1.6)	0.02	0.55
4. Ensure that patients have been immunized against vaccine-preventable diseases.	2.7 (2.3)	2.2 (2.3)	2.0 (2.2)	1.4 (2.0)	0.36	0.30
a. Determine whether a patient is eligible for and has received CDC-recommended immunizations.	2.4 (2.4)	2.0 (2.4)	2.5 (2.4)	1.0 (1.6)	0.51	0.28
b. Administer and document CDC-recommended immunizations to an adult patient.	2.2 (2.5)	1.8 (2.4)	0.0 (0.0)	0.8 (1.8)	0.01	0.01
c. Perform basic life support.	0.0 (0.0)	0.2 (0.9)	0.2 (0.6)	0.0 (0.0)	0.22	0.62

Less than 5 versus 5 or more times per week

Table 5 reports the PHP domain by preceptor status (NDSU School of Pharmacy preceptor and non-preceptor). Statistically significant differences exist by preceptor status for the core EPA PHP domain “Identify patients at risk for prevalent diseases in a population” (Kruskal-Wallis p=0.01; chi-square p=0.04). For this item, preceptors self-reported undertaking the activity more frequently than non-preceptors. Mean values for “Minimize adverse drug events and medication errors” were very high, although significant differences did not exist across preceptors and non-preceptors (Kruskal-Wallis p=0.36; chi-square p=0.49). Concomitantly, two activity items within this domain, “Assist in the identification of underlying system-associated causes of errors,” (Kruskal-Wallis p=0.03; chi-square p=0.43) and “Report adverse drug events and medication errors to stakeholders,” (Kruskal-Wallis p=0.50; chi-square p=0.04) did elicit significant differences by preceptor status, with preceptors reporting higher mean values. For the remaining items, preceptor responses did not significantly differ from non-preceptors.

Table 5

Entrustable professional activity (EPA) of population health promoter by preceptor status (n = 121)

Population health promoter EPA description; mean (SD)	Not Precept NDSU Students [n=64]	Precept NDSU Students [n=57]	Kruskal-Wallis p-value	Chi-Square p-value*
1. Identify patients at risk for prevalent diseases in a population.	0.9 (1.7)	1.7 (2.1)	0.01	0.04
a. Perform a screening assessment to identify patients at risk for prevalent diseases in a population (e.g., hypertension, diabetes).	0.5 (1.4)	0.9 (1.7)	0.05	0.25
2. Minimize adverse drug events and medication errors.	3.2 (2.1)	3.5 (1.9)	0.36	0.49
a. Assist in the identification of underlying system-associated causes of errors.	1.7 (1.9)	2.4 (1.9)	0.03	0.43
b. Report adverse drug events and medication errors to stakeholders.	1.3 (1.6)	1.7 (2.1)	0.50	0.04
3. Maximize the appropriate use of medications in a population.	2.1 (2.4)	2.5 (2.3)	0.31	0.75
a. Perform a medication use evaluation.	1.4 (2.0)	1.6 (2.0)	0.27	0.86
b. Apply cost-benefit, formulary, or epidemiology principles to medication-related decisions.	1.4 (2.1)	1.9 (2.2)	0.16	0.32
4. Ensure that patients have been immunized against vaccine-preventable diseases.	2.0 (2.2)	2.7 (2.4)	0.09	0.15
a. Determine whether a patient is eligible for and has received CDC-recommended immunizations.	1.8 (2.2)	2.6 (2.4)	0.09	0.07
b. Administer and document CDC-recommended immunizations to an adult patient.	1.5 (2.2)	1.5 (2.3)	0.85	0.85
c. Perform basic life support.	0.1 (0.3)	0.2 (0.8)	0.54	0.47

Less than 5 versus 5 or more times per week

DISCUSSION

One of the most important contributions of Core EPAs is that they create a means to translate academic competencies to professional tasks. In that way, EPAs create a “shared language” inclusive of the evaluation of student performance - between academic and professional pharmacy.7,8 Unfortunately, a dearth of evidence exists demonstrating the frequency with which these EPAs (inclusive of supporting tasks) are undertaken in various areas of pharmacy practice.10,11 The primary contribution of this pilot study was to help address this knowledge gap by assessing the self-reported frequency with which pharmacists practicing in North Dakota undertake Core EPAs in the PHP domain.

Overall, the primary conclusion of the study was that pharmacists perform activities and supporting tasks in the PHP domain, but do so inconsistently and not at a high level. For the four EPAs in the PHP domain, North Dakota pharmacists reported most frequently performing “Minimize adverse drug events and medication errors” (mean=3.4, SD=2.0), followed by “Ensure that patients have been immunized against vaccine-preventable diseases” (mean=2.3, SD 2.3), “Maximize the appropriate use of medications in a population” (mean=2.2, SD=2.3), and “Identify patients at risk for prevalent diseases in a population” (mean=1.3, SD=1.9).

While disappointing, the results of the study were not entirely surprising. The relatively low level of public health involvement is also supported by a 2016 study of North Dakota pharmacists along with those in Iowa and Manitoba, Canada.12,19 Most of the health prevention activities were medication-related and oriented towards individual patients (micro-level) rather than at a community (population-based) macro-level. Perhaps this was because most of the study participants worked in pharmacy settings that were: independents (n=49), hospital (n=32), chains (n=15), and other pharmacy positions (n=25), or they worked in pharmacy positions that were: managers (n=46), staff pharmacists (n=39), clinical pharmacists (n=26), and all other positions (n=10). In both cases, the primary activity is medication-related and most time is spent in dispensing activities, rather than population health promoter activities.20 North Dakota has a unique state law requiring each community pharmacy to have a North Dakota-licensed pharmacist owning 51% or more of the pharmacy, thereby restricting the number of chain stores, and they are typically managed by the pharmacy owner who often serves as the primary pharmacist.18 Clinical pharmacists and pharmacy managers reported performing most of the tasks at a higher frequency than did the independent community pharmacists.

It is also important to note that pharmacists in North Dakota are becoming more involved in certain aspects of care that fall within the PHP domain. For example, one study found that targeted education improved the rate with which community pharmacists provide immunizations.21 Other initiatives are also underway in areas related to the PHP domain, including (but not limited to) opioid misuse.22-24 This suggests that it is also important to assess the self-reported frequency with which pharmacists undertake EPAs regularly over time. As targeted education initiatives are successfully undertaken, the frequency with which pharmacists undertake tasks in the PHP domain will also improve. This change, in turn, provides greater educational opportunities for pharmacy students to improve their proficiency in these areas of practice.

In our study, the PHP EPAs were also reported by preceptor status (NDSU School of Pharmacy preceptor and non-preceptor). In the first core PHP EPA “Minimize adverse drug events and medication errors,” one of two supportive tasks: “Assist in the identification of underlying system-associated causes of errors” was statistically significant, with NDSU preceptors performing those tasks more frequently, than did non-NDSU preceptors. In the second core PHP EPA “Identify patients at risk for prevalent diseases in a population,” the supportive task “Perform a screening assessment to identify patients at risk for prevalent diseases in a population (e.g., diabetes),” showed a statistically significant difference for preceptors more than non-preceptors. This difference may be explained by the availability of human resources. For example, pharmacists may be assigned students on a regular basis precisely because they can have greater time and other resources to reallocate their efforts away from “traditional” activities, and towards other aims, including activities in the PHP domain and precepting students. Precepting students may also create synergies which lead to pharmacists undertaking activities within the PHP domain. As an example, students often take the lead on performing the assessments while being supervised by a pharmacist. This supervisory role is less time consuming, and more efficient than performing the entire screening process by themselves. Consequently, this process often allows for additional overall services to be provided by the pharmacy.

It is also interesting to note that pharmacists who work in hospital and chain community pharmacies performed most of the core EPA PHP domain items and their supportive example tasks more frequently than did independent community pharmacists. This may be due to infrastructure advantages accruing to larger, more integrated, organizations. For example, hospital pharmacists may have access to the patient’s electronic medical record, which allows them greater opportunities to identify and provide services falling within the PHP domain. Chain community pharmacists often receive structured corporate support (directives and resources) to provide services including (but not limited to) immunization delivery and other preventive care programs (point-of-care testing, disease state management) that fall within the PHP domain. Independently owned pharmacies, which are the norm in North Dakota, typically do not have the same level of access to such infrastructure.

Limitations

This study is a part of a larger initiative undertaken by the North Dakota Institute of Pharmaceutical Care to assess the frequency with which pharmacists in North Dakota perform EPAs in all six major domains.10,11 As such, the limitations inherent in those studies (and previously identified in the literature) are applicable here. One major limitation is that several PHP EPAs were not undertaken with the same frequency by all pharmacists. This implies that specific EPAs may have more relevance in certain areas of practice than others. This, in turn, may impact the validity of EPAs as an educational tool. In such cases, future research may be required to refine the construction and assessment of EPAs for specific pharmacist roles or who practice in different settings.

A second limitation is that the number of respondents included in the data analysis (n=121) are a relatively small proportion of the 990 individuals who were offered an opportunity to complete the survey. That is, the survey has a low response rate.16 This may create a study limitation, especially if the n=121 respondents included in the study do not fully reflect the underlying population of pharmacists in North Dakota. Given the low response rate (as well as the other limitations identified in this section), the authors consider the current manuscript as a pilot study. While a low response rate is a study limitation, it is also important to note that the survey’s actual response rate is much higher than 121/990, or 12.2%. To illustrate this, of the 990 people invited to complete the survey, 457 (46.2%) responded. But of these 457 respondents, 102 (22.3%) did not meet the study’s inclusion criteria and should not be included in the determination of response rates. Again, this suggests that the study’s effective response rate is much higher than the stated 12.2%. But even with this adjustment, the response rate for the survey likely remains sub-optimal. Future research that administers a similar survey and obtains a much higher response rate would provide a very valuable re-assessment of the current study.

A third limitation is that the use of survey methods can elicit only self-reported information. Such information is subject to errors of exaggeration or recall bias. This is especially true in instances where pharmacists do not practice on a full time basis, or when they perform a specific task on a very infrequent basis.

The survey’s design creates a fourth limitation. It is difficult to obtain reliable self-reported frequencies from many respondents using a question with a limited response scale. In the case of the PHP EPA items, a 6-point response scale (0,1,2,3,4,5 or more times) was utilized. If the respondent performs a task infrequently, the survey provides valid and reliable inferences. But for tasks performed regularly, the responses become truncated on the right-hand side of the response scale. This type of truncation limits the information that can be gleaned from the survey. Futures surveys that utilize different response scales that are theoretically appropriate and avoid the possibility of truncation would provide deeper and more meaningful inferences than are contained in the current manuscript.

A fifth study limitation concerns the construction of some survey items that were taken from the PHP EPA domain (including supporting tasks).10,11 This study used the 2015-2016 AACP published list of EPA domains, activities and supportive tasks.7 The study findings suggest that the AACP core EPA list may lack validity for some of the survey items.10 Some of the EPA activities and supportive tasks for the PHP domain are cross-cutting in nature (they include more than one concept) and therefore may have confused the respondent. EPAs must be independently executable, observable, and measurable in their process and outcome. There are some EPA’s that do not meet with these characteristics, because they are a mix of activity (EPA) and a competency description. One example is “Minimize adverse drug events and medication errors”. This is a professional competency rather than an EPA description. Searching for these words in PubMed would likely generate many different ways to complete this task. Therefore, two different EPA’s are included in this competency. Another example is “Apply cost-benefit, formulary, or epidemiology principles to medication related decisions.” This includes three different principles into one item. These two examples, and potentially other EPAs, should be revised and then re-tested in future studies.

A sixth study limitation is that the study was conducted in a single, predominately rural state that emphasizes independent community pharmacy practice. Therefore, it is unclear whether, or how, the current study’s findings apply to pharmacy practice in other regions or states with predominately urban areas or those that have limited independent community pharmacy ownership.

Lastly, the study was conducted at a very specific point in time at the end of 2017 and the beginning of 2018. As noted in the Discussion section, the practice of pharmacy evolves over time. In North Dakota, for example, targeted education is being conducted to improve the rate at which pharmacists provide immunizations.21-24 As this evolution occurs, the results contained in this study may lose relevance. Future research that updates our findings would provide additional, meaningful inferences about the use of EPAs in the PHP domain.

CONCLUSIONS

Pharmacists in North Dakota currently perform some population health promoter activities, but not at a consistent and high level. Most of the health prevention activities were medication-related and oriented towards individual patients (micro-level), rather than at a community (population-based) macro-level. With refinement, the EPAs in the PHP domain have potential as a means to assess outcomes in pharmacy education and practice. This study provides useful, but preliminary, basis for future research to further assess the use of EPAs in professional pharmacy practice (both more broadly across EPAs and deeply in the refinement of current EPAs), and how EPAs link to patient care outcomes.