Preceptor Fidelity to the Creation of Precepting-Focused Continuing Professional Development Learning Plans.

OBJECTIVES: To determine preceptor fidelity to the creation of continuing professional development (CPD) learning plans for improvements in clinical teaching. To determine topics of interest for preceptor development programming with a focus on clinical teaching skills.
METHODS: An online CPD program focusing on the reflect and plan stages was launched. The online program introduced preceptors to CPD, encouraged preceptors to review student feedback, and required completion of a CPD precepting plan. Preceptors were provided four guided questions to develop their plans. Preceptor fidelity to the creation of a CPD plan to improve precepting skills was rated as: 1) having a plan for improving a precepting skill, 2) having a plan for improving precepting, but not skill-based, or 3) having a plan lacking a precepting focus. Topics of precepting improvement were identified.
RESULTS: Of the 491 preceptors who completed CPD plans, 46.2% of the plans were skill-based. Of those, 59.5% of the CPD plans included specific learning items. Only 3.9% of preceptors submitted nonprecepting focused CPD plans and the remaining 49.9% of CPD plans were non skill-based, but precepting focused. Giving feedback was the most frequently listed topic with 45.8% of preceptors including it in their CPD plans.
CONCLUSIONS: This online precepting CPD program successfully guided preceptors in developing CPD plans that have the potential to improve student learning experiences. Future precepting CPD programs will need to provide direction on the use of skill-based objectives. Further investigation is needed to determine the effectiveness of the program in changing precepting behavior. © Individual authors.

Introduction

As experiential education comprises at least 30% of the Doctor of Pharmacy curriculum, it is essential that preceptors are competent to offer robust student learning experiences.[1-3] Unfortunately, many preceptors begin clinical teaching without formal training in teaching or precepting.[3] Schools and colleges of pharmacy are responsible for offering preceptor development, but it can be difficult to encourage preceptors to complete training on clinical teaching.[1, 4–6] Using continuing professional development (CPD) to encourage preceptors to develop their own personalized plans may be one avenue to encourage targeted preceptor development.

CPD is an individualized lifelong learning process consisting of four key steps: reflect, plan, learn, and evaluate.[3, 7, 8] These steps are continually revisited while documenting the learner's progress (Figure 1). The American Association of Colleges of Pharmacy (AACP) has promoted the use of CPD for pharmacists[1, 8] with the 2015-2016 AACP Report of the Professional Affairs Standing Committee specifically recommending CPD programs for preceptors.[3] Additionally, the Accreditation Council for Pharmacy Education (ACPE) Accreditation Standards requires schools of pharmacy to “foster the professional development of its preceptors”.[1] While some academic programs have introduced CPD to their preceptors,[7] it has been limited to a general introduction to CPD. Tofade and colleagues introduced preceptors to CPD using AACP educational materials where preceptors saw benefit in the application of CPD to preceptor development.[7] However, the training materials used were for an introduction to CPD for clinical improvements and were not specific to improving clinical teaching. Therefore, a program to promote both CPD and improvement in precepting skills was pursued.

Figure 1.

The CPD Cycle

The University of Wisconsin-Madison (UW-Madison) School of Pharmacy CPD Program for Preceptors was developed with the following objectives: 1) to introduce CPD to preceptors, 2) to encourage preceptors to reflect on student feedback, and 3) to improve preceptors' clinical teaching skills with the ultimate goal of fostering a stronger experiential education program. Innovation in this program is two-fold. The first innovation involved formally combining preceptor CPD with a focus on improved clinical teaching. The second innovation was applying an implementation research technique to evaluate a teaching intervention.

Implementation science techniques can increase adoption, sustainability, and further implementation or dissemination of new interventions into practice.[9-11] After the initial participants completed the precepting CPD program, it was noted that some preceptors described a behavior change that did not involve learning or skill development (e.g., scheduling meetings with student) as the program intended. Therefore, the implementation construct of fidelity, or the preceptors' accuracy in following the program directions, was chosen as part of the program evaluation.[12, 13] By evaluating the preceptors' fidelity to their creation of CPD plans, updates and clarifications in future programing may lead to improved program implementation and improvements in clinical teaching. A secondary objective was to preceptor outcomes. The primary objective of this evaluation was determine topics of interest for preceptor development to determine the fidelity of preceptor creation of CPD plans for programming with a focus on clinical teaching skills.

The CPD Cycle is reprinted with permission from the American Council for Pharmacy Education. https://www.acpe-accredit.org/wp-content/uploads/CPDCycle2014Color.jpg

Methods

Design

This cross-sectional evaluation examined preceptor fidelity to the creation of precepting-focused CPD plans. The online CPD program was designed to guide preceptors through the development of an individualized learning plan to improve their precepting skills (i.e., reflect and plan) with the intent of having a follow up program to complete the remaining steps and continue the CPD cycle the following year. As this project was undertaken for programmatic evaluation, it did not meet the federal definition of research and per the UW-Madison Health Sciences IRB Not Research Determination Decision Tool, IRB review was not required.

Program Overview

A newly developed online Preceptor CPD program introduced preceptors to the concept of CPD and the initial steps of reflect and plan. The learning objectives for the program were to: 1) define CPD, 2) list the components of CPD, 3) describe how CPD differs from the traditional continuing professional education (CPE) model, and 4) discuss the process of using reflection to create a preceptor learning plan. The goal was to develop a CPD plan to improve their precepting.

The program was available to preceptors through the UWMadison School of Pharmacy Preceptor Development Center, which is housed in Canvas (Instructure Salt Lake City, UT), the UW-Madison's learning management system. All UW-Madison School of Pharmacy preceptors have access to the Preceptor Development Center. The Preceptor CPD program consisted of five parts, programed to ensure preceptors completed the program in the following order:

An online presentation titled “Continuing Professional Development (CPD) for Preceptors”

An online personalized CPD plan

A CPD self-assessment quiz

A CPD program evaluation

An opportunity to obtain 0.75 ACPE CPE credits

The 32-minute online presentation was a PowerPoint with imbedded audio delivered to preceptors using Articulate Presenter (Articulate, New York, NY). The presentation began with a discussion of the CPD cycle and a comparison of CPD to traditional CPE. The presentation then transitioned to utilizing CPD specifically to improve precepting skills. During the presentation, preceptors were encouraged to access their students' feedback regarding their precepting and to identify trends for consideration when developing their CPD learning objectives. The presentation reviewed how to access student feedback. Anonymous student feedback is available to preceptors 24/7 via a website continuously throughout the year and is updated in real time as students complete evaluations. Next, preceptor development resources available on the Preceptor Development Center were reviewed.

The online presentation ended with an example of how to complete a precepting-focused CPD plan. The example was a preceptor who wanted to improve giving constructive feedback to their students. The example included a deadline for completing the CPD plan and changing the preceptor's behavior, and incorporated two specific resources to improve their understanding of giving constructive feedback.

Preceptor development resources, assembled by the UWMadison School of Pharmacy, were freely available to preceptors, and were curated to facilitate completion of the CPD plans. This material was shared to facilitate the identification of free high-quality preceptor development materials for the CPD plans. The materials included a program developed by the UWMadison School of Pharmacy for preceptor development, the Journal of the Pharmacy Society of Wisconsin's open-access Preceptor Development Series[14], and the British Columbia Preceptor Development Initiative.[15] The materials were organized by resource, links were provided, and an overview of the kinds of skills each resource covered was briefly included in the online presentation.

The following questions were used by preceptors to guide their plan development and were designed to focus on the reflect and plan stages of the CPD cycle:

Reflecting on your precepting over the last year, what are you doing well?

Where do you want to improve in your precepting? List two areas for improvement.

Write one SMART learning objective for your precepting.

List the planned activities and any resources you intend to use to achieve your SMART learning objective.

The precepting CPD plan consisted of the responses to questions 3 and 4. The responses to all four questions was submitted as a “quiz” in Canvas. Preceptors needed to submit the CPD plan prior to moving on to subsequent steps in the program.

In order to earn CPE credit, seven multiple choice and true/false self-assessment questions were required to be completed by the learner after reviewing the CPD module. A program evaluation was also required to be completed. The CPE was offered to preceptors free of charge as an incentive to commit to improving their precepting, but the CPD program was optional for preceptors to complete.

The CPD program was disseminated to all preceptors (both introductory and advanced pharmacy practice experiences) through a series of announcement emails, inclusion in the UWMadison School of Pharmacy experiential education newsletters, and at least one health system made the CPD program required for their pharmacy staff. While the program was not required by the School of Pharmacy, deadlines were set to encourage preceptor completion. Deadlines were one to two months from the date of the email reminder.

Variables

Two months following the release of the CPD program, the primary instructor (AM) reviewed the submitted precepting CPD plans and noted that many plans described a change in behavior without a learning and development component. For example, a common CPD plan was scheduling weekly follow up meetings with the student to offer feedback. While this would improve the learning experience for the student, it does not require learning, skill-improvement, or professional development and would not satisfy the “learn” step of the CPD cycle. Therefore, this did not meet the expectations for a precepting CPD plan. Following this observation, an evaluation of preceptor fidelity to the creation of CPD plans was conducted. Fidelity was determined by how well the preceptor followed the intent of the CPD program, evaluated through review of the submitted CPD plans (Table 1). Plans that met the intent of the program were those which described improvements in a precepting skill. Plans that partially met the intent of the program described a behavior change related to precepting, but were non-skill based. Lastly, plans that did not meet the intent of the program were unrelated to precepting (i.e., focused on clinical improvements and not clinical teaching). These definitions were determined following review of the submitted CPD plans.

Table 1.

Fidelity Definitions and Examples

	Precepting skill	Non-skill precepting	Non-precepting	Specific Plan	Vague Plan
Definition	Plan described improvement in a precepting skill	Plan described a non-skill based precepting improvement	Plan lacked a precepting focus	Plan included specific preceptor development resources	Plan included generalizations of preceptor development resources to use
Example	“By February 2018, I will use resources on giving feedback and improve the structure of constructive feedback I give students to more consistently share improvements”	“Starting with my next student, I will schedule a time each week to offer feedback to my student”	“I would like to be board certified by the end of 2018”	“I will complete the feedback preceptor development training offered by UW-Madison”	“I will use articles from the Pharmacist's Letter™”
Timebound	Yes	Yes	Yes	No	No

CPD plans that met the intent of the program and described improvements in a precepting skill were also assessed for the specificity of the plan. Specific plans listed explicit resources the preceptor planned to use to learn more about the precepting skill (e.g., the title of an article), whereas vague plans described a body of resources, which included resources outside the scope of the preceptor's SMART learning objective (Table 1).

CPD plans that related to precepting, whether they were skillbased or not, were assessed for topics of preceptor development during the initial review of the CPD plans. There was not a targeted number of topics identified. All plans, whether precepting focused or not, were assessed for being time-bound, that is if the preceptor included a learning plan completion deadline (Table 1). Time-bound could be a calendar date or a teaching time point (i.e., by the next student on rotation).

At the program's release, participants were not given a deadline to complete the learning activities from their CPD plans, but reminders over the summer encouraged them to complete their plans.

Data Analysis

The CPD plans were collected from the learning management system nine months after the program's launch. The CPD plans were evaluated as a group to give general feedback to preceptors; preceptors did not receive individualized feedback. This feedback was shared with preceptors during the annual clinical instructor meeting and incorporated into subsequent preceptor CPD training. The feedback included encouragement to focus on precepting skill-development and a report of the top preceptor development topics of interest. All three investigators completed an initial review of the first 50 CPD plans and through discussion developed the fidelity definitions and identified the precepting topics. Fidelity variables were mutually exclusive; a CPD plan could not include both precepting skill development and non-skill precepting learning objectives. If a CPD plan had more than one objective and at least one was skillbased, then the plan was determined to be a precepting skill plan. All variables were coded in duplicate by two investigators independently and the coded data sets were compared for consistency. Items of inconsistency in fidelity variables were determined through comparison of datasets and resolved through discussion among all three investigators. Topics for precepting improvement used inclusive coding (i.e., if any evaluator felt it was a topic then it was included) and a single CPD plan could include multiple topics. Proportions of CPD plans meeting each fidelity variable and for each program evaluation question were determined. Counts for each precepting topic were calculated.

Results

The program was launched July 2017. As of April 2018, 491 preceptors completed CPD plans and 444 preceptors claimed CPE credit. Five additional preceptors submitted blank CPD plans and were removed from the analysis. As the UW-Madison School of Pharmacy typically has approximately 1,400 preceptors, about one-third of preceptors completed the optional program. Approximately half of the preceptors submitted precepting skillbased CPD plans (227, 46.2%). Of those, 135 CPD plans were rated as specific plans (59.5%) while the remaining 92 CPD plans were rated as vague (40.5%). Only 19 preceptors (3.9%) submitted non-precepting focused CPD plans or did not create a plan (e.g., CPD plan focused on becoming board certified or stated they did not have upcoming students and did not complete a CPD plan). The remaining 245 CPD plans (49.9%) were precepting focused, but not skill-based. Lastly, 337 CPD plans (69%) were time-bound.

There were 12 consistent topics of interest in precepting development that were described in the CPD plans. Descriptions of topics can be found in Table 2. “Other” was used for CPD plans that fell outside of the 12 topics.

Table 2.

Definitions of Topics Reported on CPD Plans

Topic	Preceptor Described
Feedback	Improve quality or increase frequency of both positive or constructive verbal or written feedback. Included descriptions of both formative and summative feedback.
Organize rotation	Prepare resources prior to rotation, e.g., curate resources, select project prior to rotation, student schedules.
Scheduled discussions	Develop or improve scheduled student discussions, e.g., journal club, case discussions, topic discussions.
Setting expectations	Develop or improve communication of student expectations. Included written and verbal, with varying timelines.
Schedule time	Increase frequency of student meetings. Often schedule determined prior to rotation for one-on-one meetings with student.
Non-discussion activity	Involve student in new learning activities, e.g., develop student learning modules, manuscript writing, exposure to new (to the rotation) practice areas.
Self-learning for students	Preceptor self-learning of clinical information, specifically stated for the benefit or teaching of students.
Independence	Foster student independence, e.g., increase student responsibility, allow student to attempt pharmacist roles, coach clinical decision-making.
Spontaneous discussions	Improve ability to have short yet meaningful discussions as opportunities arise in workflow, e.g., one-minute preceptor, teaching with questions.
Orientation	Implement or improve student orientation.
Time management	Improve management of student time, e.g., improve student processes or integrate into the workflow, initiate layered learning, assign student deadlines.
Team teaching	Start or improve team teaching.

Improvement in giving feedback was the most frequently topics are included in Figure 2. Examples of “other” included described topic with 45.8% of preceptors including it in their CPD sensitive situations/difficult conversations, student motivation, plans. Frequencies of the remaining preceptor development leadership, and use of the think-aloud technique.

Figure 2.

Preceptor development topics identified in precepting CPD plans

Discussion

This program successfully guided the development of precepting CPD plans as a majority of preceptors (96.1%) submitted CPD plans describing desired improvements in their clinical teaching. However, there was low fidelity among the submitted CPD plans, as half of the CPD plans were not skill-based and of the skillbased plans 40.5% lacked specificity. Regarding preceptor development skills of interest, feedback was the most frequently identified skill that preceptors wanted to improve (45.8%). While it is possible that some preceptors may have chosen this area for improvement based on the example in the online program, providing effective feedback has been found to be a preceptor challenge in some situations.[16] Additionally, even if feedback was selected by preceptors due to the program example, improved frequency and quality of feedback improves learning.17,18

The low fidelity to incorporating the learning aspect and low specificity of the CPD plans was an unexpected finding. This online program tailored the guided CPD questions to focus on the application of CPD for preceptors on their clinical teaching. It is possible that a live workshop may have allowed for discussion, which could have clarified the need for the learning and skill development component. In general, active learning strategies, which include workshops and practice of new skills, can promote behavior changes among practicing pharmacists.[19-21] While this program was asynchronous and without a workshop component, there is evidence that a guided worksheet can also promote the development and completion of a CPD plan.[20] More successful CPD programs have included guided workshops and/or tools to facilitate CPD,[8, 19, 22, 23] but this approach can create burdensome requirements that do not promote preceptor completion of the program.[4-6] Follow up from this CPD program will help determine the level of engagement needed to improve the fidelity of CPD plan creation and to promote improved clinical teaching behaviors.

One innovative aspect of this project was the use of an implementation outcome in the evaluation of a preceptor development program. While some have described the concept of incorporating implementation science into medical curriculums to varying degrees, many focus on teaching implementation science to health sciences students rather than using those methods to improve a teaching intervention.[13, 24, 25] Baldwin and colleagues assessed the implementation and sustainability of a professional development program by highlighting program fidelity to increase the likelihood of program sustainability.[13] This preceptor CPD program used the Conceptual Model of Implementation Research as the foundation for evaluating implementation.[12, 26] The implementation construct of fidelity was adopted from this model to ensure that the program objective was understood by participants and that preceptors correctly drafted their CPD plans. Based on this evaluation, specific adjustments planned for future iterations of the program include: 1) use of more examples, 2) explanations of how the examples meet the program expectations (specifically the transformation of a poor SMART objective to a stronger SMART objective), and 3) organization of curated preceptor development resources, based on the commonly reported topics in Figure 2.

Several aspects of the program delivery likely contributed to preceptor's productive engagement in the CPD planning process. Pharmacists generally support the concept of CPD and have a personal desire to learn, but require motivation and support to facilitate their engagement in CPD.[27, 28] This program may have helped preceptors to structure their engagement in CPD through guided questions and a menu of resources to facilitate the development of their CPD plan. Additionally, use of CPE for licensure maintenance is a common motivator for professional development activities, which this program also offered.[27] Lastly, this program was delivered as an online module; online learning activities offer flexibility in timing of completion and quick access to information and therefore tends to be preferred by health care professionals.[6, 29]

Finally, additional preceptor development materials can be planned from this evaluation. The topics of interest that preceptors identified can serve as a gap analysis of precepting needs, which is also considered an evidence-based practice for professional development programs.[13] In particular, this evaluation identified giving feedback as an area preceptors would like to improve. Therefore, preceptor development training on giving feedback to students regarding their interprofessional communication and teamwork skills is being developed. Other schools and colleges of pharmacy who adopt a precepting CPD program can also leverage their preceptors' CPD plans to determine preceptor development needs.

Limitations

This evaluation has several limitations. Most notably, this program only evaluated the development of precepting CPD plans and did not determine if those CPD plans were completed or if preceptors changed their clinical teaching behaviors. A second limitation was the emphasis on measuring skills as part of the program evaluation. While the learning step of the CPD cycle was included in the presentation, skill development was not strongly emphasized in the presentation or the guided questions to develop the CPD plan. A final limitation was the response rate of only one-third of preceptors completing the program. Although this yielded nearly 500 plans for evaluation, the results may not represent the rates of plan fidelity may have changed if participation was mandated and additional preceptors were required to complete CPD plans.

Further investigation is warranted regarding the precepting CPD program. While this program assessed the fidelity of each preceptor's plan, it did not compare plans to the student feedback. A precepting CPD plan that is reflective of student feedback could be considered another indicator of program fidelity. This program used a single technique of an online presentation to foster behavior change; however, evidence suggests hands-on practice may be needed for an individual to learn a new skill or change a behavior, and a workshop component should be evaluated for its effects on plan fidelity.[19-21] Other teaching modalities for fostering preceptor CPD should be investigated and compared to this online program to promote effective preceptor improvement while ensuring preceptor time commitment is reasonable.

There are several next steps for this precepting CPD program. We are currently following up with preceptors who completed the program to determine if they implemented the learning activities in the CPD plans they developed. Additionally, as the main goal of the precepting CPD program is to improve clinical teaching, we are following up to determine if preceptors have changed their clinical teaching behaviors and if their behavior changes have improved student learning. We then plan to continue to guide our preceptors through an ongoing CPD program focused on preceptor development to create a continuous focus on improvements in clinical teaching. Lastly, we are planning to investigate why preceptors do or do not complete their CPD plans or change their clinical teaching behaviors. For example, are specific practice sites more likely to engage in the program or implement specific precepting behaviors? This information can aid schools and colleges of pharmacy to determine how to best support their preceptors and ultimately the quality of clinical teaching for their students.

Conclusion

This online precepting CPD program successfully guided preceptors in developing CPD plans that have the potential to improve student learning experiences. However, there was low fidelity among the CPD plans as only half of them focused on skill development. Future precepting CPD programs need further direction on skill-based objectives and further investigation is needed to determine how effective this program is in changing precepting behavior.