Can We Ensure That Workshops Are Effective in Their Goal? Impact of a National Education Workshop on Participants' Subsequent Scholarship.

Medical education conferences offer practical workshops to facilitate physicians' lifelong learning. Little is known about integration of workshop material after conferences. We sought to evaluate the application of workshop content focused on scholarly publication preparation. We developed an email survey to examine participants' progress preparing a publication in 2019, administered 4, 9, and 15 months post-conference. The survey included scaled items and open-ended questions. Thirty-three participants attended the workshop. Participants continued to develop their projects, but noted time, adequate evaluations, and no writing partners as barriers. Following up with workshop participants offers insights into effective application of workshop content.

Background

Completion of residency or fellowship training does not signify the end of learning in medicine. Rather, it indicates entry into the longest stage of training: continuing medical education (CME). Since 1947, CME has served as the primary means by which physicians build new skills and maintain others to continue successful and safe medical practice [1, 2]. In turn, completion of CME is required for clinicians to meet maintenance of certification (MOC) and licensure requirements. For medical educators, CME often provides opportunities to learn new teaching techniques, methods of assessment, or innovative curricular ideas.

Since its inception, the delivery and assessment of CME have evolved substantially to incorporate principles of adult learning theory through active learning strategies [3]. One of the most frequent learning strategies used to deliver CME in medical education settings is the workshop. It is an appealing method for engaging participants. Compared to passive learning techniques, such as didactic lectures, practical workshops have been shown to significantly impact commitment to change and patient care outcomes [4, 5].

While the impact of CME on physician clinical practice has been described [6, 7], the literature about how workshops impact educational and teaching practice is limited to best practice recommendations [8]. Furthermore, medical educators attend meetings, which often require substantial resource commitment in terms of time, money, and travel. Therefore, it is valuable to determine their impact on educational practice [9].

Activity

In 2018, the authors developed an educational workshop designed to impart the skills required to complete a successful educational scholarship submission. After successful acceptance via a peer-reviewed process, it was delivered at a national educator conference. The purpose of this study was to investigate the impact of such a workshop on the educational practice and scholarly output of participants. We hypothesized that participants would report increased academic productivity resulting from participation in the workshop and specific barriers and promoters to completion of their scholarly project.

This mixed-methods study utilized follow-up surveys to assess the impact of a 2-h educational workshop. The workshop, aimed to prepare participants for submission of scholarly work to MedEdPORTAL. It was implemented at the 2019 Annual Meeting of the Council on Medical Student Education in Pediatrics. Workshop attendees consisted of faculty, many of whom held defined educator leadership roles. The 2-h workshop’s objectives were the following: (1) describe the educational activities published in MedEdPORTAL, (2) review the new Educational Summary Report required to publish on MedEdPORTAL, (3) outline steps in successfully publishing an educational activity or product, and (4) create a timeline for submitting educational content on MedEdPORTAL. The workshop included large group discussion and small group hands-on work as instructional strategies to accomplish these objectives.

Follow-up surveys were constructed via expert group consensus. The content focused on the workshop’s impact on participants’ scholarly products, and the factors that may contribute to the participant submitting to MedEdPORTAL or barriers encountered that may have interfered with completing a submission. The survey was administered via email to the cohort longitudinally at 4, 9, and 15 months post-conference and included a combination of Likert scales, yes/no, and open-ended questions.

Results and Discussion

Thirty-three individuals participated in the workshop and agreed to be contacted following the workshop. Response rates were 55% (18/33), 24% (8/33), and 39% (13/33), respectively, for the 4-, 9-, and 15-month surveys. Not all respondents answered each question of the survey, so these have been reported by the number that answered each question.

Prior to the workshop, two respondents had previously submitted and had works accepted to MedEdPORTAL. From the first survey, 15/18 respondents (83%) reported having an idea for a manuscript prior to attending the workshop; 11/17 (65%) reported active current work on the study idea in the initial survey and at the time of the final survey; and 7/12 (58%) respondents continued to work on a project.

In the initial survey, 8/16 (50%) agreed/strongly agreed they had confidence in their abilities to complete a submission as a result of the workshop. In the final survey, 6/11 (55%) agreed/strongly agreed they were confident in their ability to submit to MedEdPORTAL and 5/11 (45%) were confident they would be able to complete the product offered based on the workshop. These results are consistent with studies indicating workshop participants report a commitment to apply the material immediately after the workshop [5, 10].

Of the respondents in the initial survey, 6/10 (60%) had drafted a timeline during the workshop (time was set aside for this activity) and 12/15 (80%) felt developing timelines is a useful activity; however, at the second survey, 5/8 (63%) respondents reported a lack of significant progress with their projects since completion of the workshop. This appeared to worsen over time as at the time of the final survey only 1/11 (9%) respondents had been able to keep their timeline. Chipchase and colleagues also found that workshop participants were motivated to change immediately after the workshop, but ultimately did not [10].

Participants reported their barriers and challenges with their scholarly work. These included lost momentum, discouragement, COVID-19-related distractions, busyness, challenge with completing tasks, and work transitions. A number of key barriers emerged from the surveys, which include (1) time, (2) utilizing appropriate evaluation methods, (3) need for accountability partner/deadline setting, and (4) and need for organization of a team approach (see Fig. 1).

Fig. 1

Reported barriers regarding submission of a scholarly product. The survey was administered via email to the cohort longitudinally at 4 (survey 1), 9 (survey 2), and 15 (survey 3) months post-conference

Participants described the key factors for their success: mentors (40%), local colleague collaboration (27%), research working groups (18%), and others (9%), which included small groups, accountability partners, or other individuals who can help with setting deadlines and following through with them or collaboration with individuals who can critically review work and provide honest feedback (see Fig. 2). Participants differed on whether establishing a regular check-in would be helpful; 25% of respondents would prefer this but a majority (75%) responded “maybe.” Only 17% respondents in the final survey were interested in a Zoom-assisted check-in, while 33% of survey respondents felt that Zoom would be unhelpful, and 50% were unsure. However, Yu and colleagues implemented a longitudinal educational program that was successful in forging a community of practice to improve application of workshop content [11]. Although our participants were ambivalent about regular check-ins, based on this study, it would be worth initiating in a formal manner to ensure successful application of workshop content.

Fig. 2

Reported promoters regarding submitting a scholarly product

Conclusion

Our data suggests that education workshops may enable attendees’ in completing a “promised” scholarly product. Our identification of multi-dimensional barriers and promoters suggests that even prior to implementation, workshops need to be designed with these barriers in mind. It may be useful to ensure that deliberate “priming” of the participants occurs ahead of the workshop. To support the development and completion of scholarly projects, strategies to build and expand on collaborations and communication between participants and others who may help them advance their intended project should be considered in the workshop design process. Such can include continued support from workshop leaders and or from identified local mentors and colleagues.

This study is limited by a small sample, lack of outcomes regarding how many had submitted a project, and the inability to compare with those educators who may have had projects in development but had not attended the workshop. We also had not assessed what stage participants were at in their project and study idea before the workshop. It is possible that participants who were farther along may have benefited more from the workshop than those who were at an earlier stage in their project. Future studies to examine the effectiveness of the medical education workshop should consider and accommodate for these limitations. Still, this study does consider a process for following up with participants as a means of providing support and accountability to apply knowledge and skills attained from a workshop. This process could be applied to the development of other CME activities.

Medical education workshops often focus on outcomes, yet they themselves have not been examined, overall, as an instructional strategy. Future studies should examine longitudinally the actual productivity of such workshops themselves and their overall impact on academic faculty development. Such research can likely inform both workshop leaders and participants to achieve their relevant education scholarship products and enable more effective medical education CME activities.