Reading of the Week: a continuing professional development program for psychiatrists and residents that Osler would have liked.

Introduction: William Osler started the first journal club more than a century ago. As in Osler's time, continuing professional development (CPD) is challenging to deliver in our day. This paper discusses the CPD outcomes of Reading of the Week (ROTW), an innovative online education resource aimed at Canadian psychiatrists and psychiatry residents.
Methods: ROTW consists of a weekly email sent to these physicians through formal partnerships, including 13 residency training programs, and summarizes the latest literature in psychiatric care. An online survey using Moore's continued medical education evaluation framework was conducted to determine the outcomes of ROTW and how to improve it.
Results: One-third of ROTW subscribers (n = 332) responded to the survey. Respondents reported a very high rate of satisfaction (97%). The most significant findings: ROTW improved participants' understanding of psychiatry (93%) and informed their practice (83%). Conclusions: ROTW is a program that addresses challenges related to remaining "up-to-date" amidst the vast amount of resources available. Survey data suggests that ROTW has a high satisfaction rate and achieves practice change, perhaps because it provides a boundless learning option for trainees and providers. Further research is needed better to understand the reasons for the success of this program.

Introduction

Sir William Osler started the first journal club almost a century and a half ago.[1] Though technology has advanced, accessible continuing professional development (CPD) that provides “just-in-time” knowledge translation continues to be challenging to deliver. Further complicating the situation is the amount of new information that physicians must know, which has been exacerbated during COVID-19.[2] Not surprisingly, technology-enabled CPD has been developed. However, a recent review notes the decline of more accessible knowledge exchange experiences and the high use of internet-based continuing medical education activities, accounting for a third of such actions.[3] Providing technology-enabled CPD with knowledge exchange opportunities remains critical given that approximately 80% of physicians in Canada indicated that remaining up to date motivated them to participate in the maintenance of certification programs.[4] E-journal clubs, building on Osler’s idea but unbound by geography, have been developed in urology and other specialties but have not been well studied in psychiatry.[1],[5],[6]

Started more than seven years ago, Reading of the Week (ROTW) is an innovative, free psychiatry CPD resource for Canadian psychiatrists and residents of psychiatry, which summarizes the latest psychiatric literature and offers an accessible synthesis of new evidence to clinicians across the learner lifespan. These summaries–primarily written and edited by David Gratzer, the first author of this paper–are emailed out weekly through formal partnerships with 13 Canadian psychiatry residency programs, the University of Toronto’s Department of Psychiatry, CAMH (Canada’s largest research mental hospital) and the Canadian Psychiatric Association. Readings are promoted through social media, including Twitter. The digital archive (www.davidgratzer.com ) has had about 80,000 visits.

ROTW selections range from public policy to practice, including studies from prominent psychiatry journals. Topic examples: suicide prevention in light of COVID-19 and machine learning to improve psychotherapy; selections also prioritize Canadian studies, giving the program a unique “Canadian-ness.” ROTW also considers more significant debates; a 2019 selection, for instance, discussed social justice in medical education. Selections often include the perspective of those with lived experience. Each ROTW includes commentary, providing context, bridging the gap from evidence to practice, and enlivened by tying to other papers and expert interviews. Like Osler’s journal club, there is an opportunity to exchange ideas, with “letters to the editor” and social media discussions. Unlike Osler’s journal club, this one isn’t restricted to a doctor’s lounge, and the embrace of different platforms (email, website, Twitter, and Facebook) offers a unique CPD approach.

ROTW uses cognitive load theory to inform its design, layout and content synthesis. Cognitive load focuses on the mental burden experienced during the cognitive process of learning new material.[7] Three components of cognitive load are 1) germane–layout of material that enhances learning; 2) extraneous–information that can be altered so that the meaning can be grasped more efficiently; and 3) intrinsic–the inherent complexity of the material. ROTW arranges content from the selection to provide a framework for critical information; uses commentary to synthesize relevant information; and focuses on salient information to improve learning. Feedback on previous iterations has informed the ROTW presentation.

The target audience is practicing psychiatrists and psychiatry residents. ROTW focuses on both groups because new options in CPD (e-learning and other digital options) need to be incorporated in residency and extend to providers in practice to ensure that there is exposure and engagement in this new type of CPD. Data has shown that motivation for lifelong learning can decrease during psychiatry residency. Further evidence has revealed a need for accessible, low-cost CPD for both residents and providers in practice.[8],[9] Therefore, exposure to timely CPD could build skills in lifelong learning (for residents) and offer accessible CPD (for psychiatrists).

Methods

This paper aims to understand and consider the outcomes of ROTW through a questionnaire using an established CPD evaluation framework. We explore outcomes aligned with Moore’s evaluation framework: participation, satisfaction, knowledge change, participant competence, change in practice and change in patient care.

We surveyed all subscribers of ROTW through email. Participation was voluntary. Of the approximately 1,000 ROTW subscribers, 332 completed the survey, a 33% response rate. The survey remained open from February 21 to April 12, 2019; four reminders were sent.

We used Moore’s continuing medical education evaluation framework to evaluate ROTW outcomes, given its use in CPD evaluation.[10] The survey questionnaire contained 24 items, including open and close-ended questions and questions covered on six of the seven domains in Moore’s evaluation framework (see Table 1).

Table 1

Application of Moore’s continuing education framework in the ROTW evaluation

Moore’s Levels of Impact	Adaptation and application in the ROTW
Level 1: Participation & engagement of participants in the education activity	Four closed-ended questions using a 5-point Likert scale assessing the participation and engagement in the ROTW.
Level 2: Satisfaction of the participants from the education activity	Four questions; three closed-ended using a 5-point Likert scale inquiring the satisfaction; and one open-ended question inquiring into suggested improvements.
Level 3: Gain in procedural and declarative knowledge of the participants (knowledge outcomes)	Five questions; three closed-ended using a 6-point Likert scale inquiring into learning/professional development impacts and 2 open-ended questions probing into the impacts.
Level 4 & 5: Competence and Performance (behaviour outcomes)	Two questions: one closed-ended using a 6-point Likert scale assessing if ROTW informed practice and one open-ended probing into the details of how it is used in practice; both levels were combined given the challenge in differentiating them in a cross-sectional, self-reported questionnaire.
Level 6: Patient health (practice outcomes)	Two questions: one closed-ended using a 6-point Likert scale assessing if ROTW has impacted interactions with patients, and one open-ended probing into the details of these impacts.
Level 7: Community health (population outcomes)	Difficult to measure via self-reports, it was excluded.

We analyzed the closed-ended questions using simple frequencies in IBM Statistical Package for Social Sciences (SPSS), version 25. For questions that used a 5- or 6-point Likert scale (strongly disagree/never to strongly agree/always), responses were aggregated (strongly agree and agree responses) to determine agreement with the survey item. The data from the open-ended questions were categorized and themed in Microsoft Excel. The thematic analysis consisted of 635 individual responses to relevant open-ended questions, generating codes and comparing codes with the research team. A final coding book was developed to summarize comments under key themes. The study received an exemption from the University of Toronto’s Research Ethics Board.

Results

Survey respondents were demographically diverse and represented various locations across Canada. Of the respondents, 51% were psychiatry residents and 32%, psychiatrists. In terms of gender, 60% were female (see Table 2).

Table 2

Demographic characteristics of the survey respondents

	N	%
Gender
Male	131	39.6
Female	198	59.8
Primary location to access ROTW
In the Greater Toronto Area (GTA)	159	47.9
In Ontario, outside the GTA	64	19.3
In Canada, outside Ontario	106	31.9
Age Group
25 to 44	227	69
45 to 54	31	9.4
55 or older	70	21
Profession
Psychiatry resident	165	49.7
Psychiatrist (non-resident)	105	31.6
Physician (non-psychiatry specialty)	20	6.0
Allied health care professional	15	4.5

Using Moore’s evaluation framework, we analyzed outcomes across six levels. Participation (level 1) outcomes showed that 59% of the respondents accessed ROTW for more than 12 months. Regarding specific ROTW components, 90% of the respondents reported either “usually” or “always” read the summary of the articles. 97% of respondents were satisfied with the content (level 2). Moreover, 60% shared ROTW at least once, indicating that respondents found ROTW worth sharing with colleagues. In terms of knowledge outcomes (level 3), 93% of respondents agreed that ROTW had improved their understanding of psychiatry research and 87% agreed that ROTW had or would inform their practice, suggesting increased intention among respondents to use ROTW knowledge in their approach.

Responding to an open-ended question, one participant wrote: “[ROTW] gives [me] more confidence to use the information in my clinical practice.” Similarly, a psychiatry resident described using ROTW to improve knowledge: “[ROTW articles] are good preparation for my upcoming residency training program and future practice as a psychiatrist.” For competence and performance outcomes (levels 4 and 5), 46% of respondents agreed that ROTW had or would influence their interactions with patients. One participant wrote: “Several articles about substance use and mental illness have changed how I discuss [these] challenges with clients.” Other respondents observed that ROTW allowed them to identify gaps in knowledge and needs for further learning and professional development. One respondent affirmed: “[ROTW] highlights cutting-edge research that stimulates further learning. It identifies gaps that I also use to inform my lifelong learning.”

Discussion

Based on the evaluation data, psychiatrists and psychiatry residents engaged with ROTW and reported several beneficial outcomes, including high engagement (90% are reading ROTW) and satisfaction. Moreover, 83% of ROTW users said that it has informed or would inform their practice, and 43% felt that it changed their interactions with patients and families. These findings suggest that ROTW is accessible, easy to engage with and supports practice change among clinicians.

Two reasons may explain the high engagement, increase in knowledge, and change in practice that participants reported. First, clinicians have to attend formal CPD events to receive updates on new evidence in mental health; in contrast, ROTW is more self-directed and autonomous. From the perspective of self-determination theory,[11],[12] ROTW offers learners greater autonomy by allowing them to choose summaries and information relevant to their individualized needs from a wide selection available in electronic format and unbound by geography. Second, learners may develop a sense of competence through the commentary and synthesis of the published articles that link reading selections to clinical practice. Curation and assisted navigation to materials related to clinical practice may have increased the feeling of greater clinical competence.

The technology platform needs and implementation barriers for ROTW are minimal. Readings are distributed using Mailchimp without cost and production time, allowing ROTW to draw from the latest in the literature. Additionally, the Readings are spread through social media, which facilitates “real-time” discussion. For some participants, just reviewing the commentary and synthesis on a regular basis (90% “always” or “usually” read the summary) before choosing to read the entire article seemed to be a quick, efficient, less burdensome and hence somewhat a novel way to keep abreast of latest information in the field of psychiatry. Now in its eighth year, ROTW has been feasible and durable with good reach in the context of efficient (minimal) resources.

Although e-journal clubs exist in other disciplines of medicine, this is the first e-journal club for psychiatry reported in the literature.[1],[5],[6] The evaluation outcomes are promising, but there are two limitations: First, survey responses were self-reported. We did not have access to patient charts to verify outcomes related to practice change. Second, our findings may not reflect the experience of all ROTW participants given the response rate of 33% response rate (though comparable to that of other national surveys of healthcare professionals).

Conclusions

Our evaluation provides preliminary data showing that ROTW is a well-received and beneficial “just-in-time” CPD program for psychiatrists and residents. It may be particularly relevant given trends in CPD, especially with COVID-19.[13] Additionally, some data suggest that the information from ROTW is considered a good resource for improving clinical knowledge and its practical implications. ROTW’s use of technology to increase access and reduce barriers signifies that affordable technological solutions may support the relevance and impact of these focused CPD tools on practice with implications for other virtual CPD tools. Future research could explore ROTW outcomes on practice change.