An Innovative Needs Assessment Approach to Develop Relevant Continuing Professional Development for Psychiatrists.

OBJECTIVES: Gaps in psychiatrists' competence can interfere with the delivery of optimal patient care, particularly when these gaps have not been identified. This study aimed to assess the perceived and unperceived continuing professional development needs of psychiatrists practicing in Quebec, Canada.
METHODS: The authors sent an online cross-sectional survey (2018) to members of the 'Association des médecins psychiatres du Québec' and collected data on unperceived needs via the critical incident method (focused on managing challenging clinical cases/situations). Data were analyzed using descriptive statistics, chi-squared tests, thematic analysis, and triangulation of data. Two coders independently analyzed qualitative data.
RESULTS: Of 1150 eligible psychiatrists, 187 (16%) completed the survey. Over half were female (58%), caring for adult patients (60%), and practiced in a university hospital (49%). Top perceived and unperceived learning need areas were neurodevelopmental disorders and psychopharmacology. Three hundred forty-three factors influencing the management of reported challenges were classified as case complexity (53%), patient (22%), environment (19%), and lack of knowledge (4%). Consultation with colleagues (49%) was the most frequently accessed resource for approaching challenging cases. During the previous year and across both self-directed and group learning activities, respondents reported engaging more frequently in in-person than online activities.
CONCLUSIONS: A comprehensive needs assessment integrating perceived and unperceived needs is the cornerstone for planning relevant continuing professional development. The critical incident method is a useful tool to assess psychiatrists' unperceived needs. Critical reflection after solving complex clinical cases might provide an opportunity to optimize psychiatrists' selection of relevant continuing professional development.

Mental disorders continue to be highly prevalent [1] while simultaneously resulting in a lack of treatment relative to physical disorders [2]. Despite the fact that psychiatry represents the largest specialty in Canada (N = 4977) and that psychiatrists are required to fulfill continuing professional development (CPD) obligations to maintain their license, little research has focused on and evaluated their CPD needs. Needs assessments have mostly targeted residents, trainees, or international medical graduates [3] rather than practicing psychiatrists [4]; others focused on assessing educational needs for specific types of practice [5].

Drawing from the Theory of Learning and Change [6], the needs assessment process is the cornerstone of CPD that assists CPD providers in the development and evaluation of targeted and timely CPD activities. This entails systematic information gathering to determine instructional solutions to close the gap between actual and optimal knowledge, skills, and attitudes (i.e., a learning need). A variety of methods are available to help physicians identify their learning needs [4, 7–9]. Perceived needs can be defined as I know what I want and need to know whereas unperceived would be defined as I do not know that I do not know [6]. Traditional needs assessments are often flawed by healthcare professionals’ limited ability to self-assess their professional educational needs [10] and poor response rate. In order to address these limitations, both perceived and unperceived needs [11] of target audiences should be investigated using approaches such as the critical incident method (CIM) [12].

We identified a single study that looked at challenging cases in CPD; this 2018 study elicited reflection on challenging cases or situations encountered in practice among surgeons and medical specialists [13]. It should however be noted that this study did not explore factors which may have influenced the complexity of challenging cases.

The present study aimed to investigate the perceived and unperceived CPD needs of psychiatrists practicing in Quebec (Canada), and the factors that influenced their perception of challenging clinical cases and/or situations. Another aim was to examine resources accessed in response to clinical challenges, as well as CPD characteristics and preferences.

Methods

An online, anonymous, cross-sectional, and voluntary survey exploring psychiatrists’ CPD needs was carried out (2018) in both English and French. All psychiatrists in Quebec (N = 1150) were invited to participate via their professional association (Association des médecins psychiatres du Québec, AMPQ). The survey was available through Survey Monkey for a period of 5 weeks and included 27 multiple-choice questions (MCQs) (with a five-point Likert scale) and 6 open-ended questions covering topics such as perceived and unperceived needs, as well as modalities of delivery and factors influencing selection of CPD. More than half of the MCQs permitted participants to select more than one response. (Variable numbers of responses from participants resulted in varying denominators in the results). To ensure the content validity of the survey, a beta-test was conducted with four psychiatrists from the target audience.

We adapted the CIM for the purpose of obtaining data on psychiatrists’ unperceived needs: this qualitative method was initially developed to collect data on critical incidents from industrial and organizational psychology [12], where an incident was defined as “any observable human activity that is sufficiently complete to permit inferences and predictions to be made about the person performing the act” (p. 327) [14]. CIM has evolved over the last 50 years and has been adopted by a variety of disciplines including medical and nursing education [15]. Drawing from theories on critical reflection and experiential learning, the use of CIM among clinicians seeks to trigger reflection on action by asking participants to retroactively describe challenging clinical cases or situations.

Our survey included six questions which sought to explore psychiatrists’ perceived and unperceived needs. Perceived needs were assessed by asking participants to (1) list two psychiatric topics about which they would like to learn more about in the next 12 months; (2) examine how they identified their learning needs; and (3) select from a list of possibilities pertaining to psychiatric areas and therapeutic approaches, organized by patient population. Unperceived needs were assessed by asking participants to (1) identify two challenging clinical cases/situations encountered in the past year; (2) list the main factors that contributed to the complexity of the case; and (3) list any source(s) of information consulted to resolve said challenging case.

Ethical approval for this study was obtained from the McGill University Faculty of Medicine’s Institutional Review Board. Participants accessed the survey through a direct link sent to them individually by email. This email presented participants with a brief description of the purpose of the survey, as well as of the anonymous nature of the data collection and analysis. Participants were asked to give their informed consent to save and use their data for research purposes via an additional MCQ.

In order to maximize response rate, the study included optional participation in a draw for one of two prizes: free accommodation for their annual professional (AMPQ) CPD conference or a tablet. To maintain anonymity, participants were asked to enter their email addresses for the draw separately from the survey itself, thus assuring that their addresses would not be linked to their responses.

Data Analysis

Data were analyzed for descriptive statistics using SPSS 24.0 software. Chi-squared tests were used to examine bivariate relationships between region of practice, years in practice, and type of practice on factors influencing complexity of challenging cases, where a p value of < 0.05 was significant. Using deductive thematic analysis, all qualitative data were coded independently by two researchers guided by an adaptation of the Theoretical Domains Framework [15]. Discrepancies were resolved by an iterative process of discussion until consensus was reached. Triangulation of data type [16] was conducted as a means of verification. Qualitative and quantitative data describing perceived and unperceived needs derived from the online survey were compared to maximize accuracy.

Results

Demographics

One hundred eighty-seven respondents completed the survey (16% response rate). More than half were females practicing in urban settings, with more than 15 years in practice, and most were caring for adult populations. Forty-nine percent of respondents practiced in university hospitals and 25% in community hospitals (Table 1).

Table 1

Demographics and practice characteristics of respondents

	N	%
Language
English	19	10.2
French	168	89.8
Sexa
Male	77	41.6
Female	108	58.4
Field of practiceb
Adult psychiatry	131	60.0
Child and adolescent psychiatry	39	17.8
Geriatric psychiatry	16	7.3
Consultation liaison	14	6.4
Forensic psychiatry	8	3.6
Other	10	4.5
Region of practicea
Urban	104	55.9
Suburban	38	20.4
Rural	30	16.1
Remote	14	7.5
Type of practicea
Solo	22	11.9
Group	17	9.2
Community hospital	46	24.9
University hospital	91	49.2
Other	9	4.9
Years in practicea
> 25 years	56	30.4
21–25 years	23	12.5
16–20 years	31	16.8
11–15 years	25	13.6
6–10 years	21	11.4
0–5 years	28	15.2

aNot all participants answered this question

bParticipants could select more than one answer

Identification of perceived educational needs generally took place during CPD workshops and conferences, following a clinical encounter, or during discussion(s) with colleagues. More than half of the participants (56%) identified French as the preferred language of CPD activities but felt comfortable in English, while 26% indicated they were bilingual and had no linguistic preference. Ninety percent of participants completed the survey in French (10% English).

Continuing Professional Development

Psychiatrists reported the CPD activities in which they had participated over the previous year in the categories of group learning, self-directed learning, and assessment. Across both self-directed and group learning activities, respondents engaged in more in-person activities (834/1475 or 56.5%, 829/1055 or 78.6% respectively) than online activities (507/1475 or 34.4%, 167/1055 or 15.8% respectively). The preferred format for group activities was in person (rather than online). As was the case for solo activities, the most popular types of activities were in-person conferences (157/1055 or 14.9%), hospital rounds (141/1055 or 13.4%), journal clubs (111/1055 or 10.5%), small group activities (101/1055 or 9.6%), and workshops (94/1055 or 8.9%). The most reported online self-directed learning activities included internet searches such as Medscape, UpToDate, and Dynamed (140/1475 or 9.5%); webinars (118/1475 or 8.0%); and reading peer-reviewed journals (78/1475 or 5.3%). The most common in-person reported self-directed activities were hospital rounds (136/1475 or 9.2%), conferences (133/1475 or 9.0%), and journal clubs (100/1475 or 6.8%). Regarding assessment, participants reported a preference for face-to-face multisource feedback (48/273 or 17.6%), followed by online accredited self-assessment programs (47/273 or 17.2%), and chart audit/feedback (43/273 or 15.7%). A blended format (i.e., online and face to face) across all types of activities was the least popular.

Almost half of participants agreed/strongly agreed that they would like to use more online learning (132/283 or 46.6%), in comparison with simulation-based training (74/283 or 26.1%) and point-of-care technology (77/283 or 27.2%). Participants preferred accessing asynchronous webinars alone (143/176 or 81.2%) whereas in groups they favored synchronous webinars (88/124 or 71.0%). The most popular devices used to complete online CPD activities were desktop computers at one’s clinic/hospital/office (111/475 or 23.4%) or at home (104/475 or 21.9%), followed by laptop computers (108/475 or 22.7%), tablets (73/475 or 15.4%), and smartphones (70/475 or 14.7%).

Email (153/580 or 26.4%) was the preferred method of keeping informed about CPD programs, followed by promotion at live events (135/580 or 23.3%), and word of mouth (108/580 or 18.6%). When selecting CPD activities, participants placed the highest importance on their interest in the subject (120/184 or 65.2%), the credibility of the presenter and the CPD provider (117/184 or 63.6%), and the relevance of the activity to their practice (108/185 or 58.4%), whereas cost was the least important factor (12/185 or 6.5%).

Psychiatrists’ Perceived and Unperceived Needs

Anxiety (31/166 or 18.7%) and neurodevelopmental disorders (31/166 or 18.7%) were the perceived learning needs most frequently identified in the survey (Table 2). With respect to anxiety disorders, respondents reported that trauma, attachment, therapeutic approaches to anxiety, and psychopharmacology were their most relevant needs. Among neurodevelopmental disorders, attention-deficit/hyperactivity disorder and Tourette’s were most frequently cited. Finally, within psychotic disorders (17/166 or 10.2%), participants reported wanting to learn more about treatment-resistant psychosis, psychosis among geriatric populations, and co-morbidity with other medical and/or psychiatric conditions.

Table 2

Frequency of respondents’ perceived and unperceived needs: triangulation of data

	Perceived needs			Unperceived needs
		N	%		N	%
Psychiatric conditions	Anxiety disorders	31/166	18.7	Psychotic disorders	51/242	21.1
	Neurodevelopmental disorders	31/166	18.7	Neurodevelopmental disorders	45/242	18.6
	Psychotic disorders	17/166	10.2	Depression and dysthymia	27/242	11.2
Therapeutic areas	Psychopharmacology	53/155	34.2	Psychopharmacology	97/179	54.2
	Psychotherapy	39/155	25.2	Resistant to treatment	48/179	26.8
	General therapeutic approaches	32/155	20.6	Psychotherapy	17/179	9.5
Other topics	Keeping up to date	35/171	20.5	Co-morbidity with another psychiatric condition	41/164	25.0
	Patient management	18/171	10.5	Co-morbidity with another medical condition	26/164	15.8
	Co-morbidity with another medical condition	17/171	9.9	Diagnosis	22/164	13.4

N = frequency of responses

The most frequently reported perceived needs categorized by patient population were neurodevelopmental disorders in children, bipolar disorders in adolescents, somatoform and neurodevelopmental disorders in adults, and neurocognitive disorders in the elderly (Fig. 1). Pharmacology and cognitive behavioral therapy were the most frequently perceived therapeutic needs across all patient populations (Fig. 2). Additional types of need identified by participants were classified in the following categories: keeping up to date on new approaches in psychopharmacology and psychotherapy (35/171 or 20.5%), patient management (e.g., medicolegal issues) (18/171 or 10.5%), and treating a psychiatric condition co-morbid with a medical condition (e.g., chronic pain) (17/171 or 9.9%).

Fig. 1

Perceived needs–frequency of responses on psychiatric conditions by population

Fig. 2

Perceived needs–frequency of responses on therapeutic areas by population

Psychiatrists felt that they needed more specialized training in neuropsychiatry (77/855 or 9.0%), geriatric psychiatry (64/855 or 7.5%), updates in medicine (57/855 or 6.7%), and suicide (19/855 or 2.2%).

Results obtained after triangulation of data indicated that the same conditions and therapies (i.e., neurodevelopmental and psychotic disorders, psychopharmacology, and psychotherapy) were identified (with differing proportions) as both perceived and unperceived needs (Table 2).

Factors Influencing Complexity of Challenging Cases or Situations

A total of 343 reported factors (based on qualitative responses) having potentially influenced the nature of challenging cases and/or situations were classified into four categories (Fig. 3). The most frequently reported category was the type of clinical case (182/343 or 53.1%), followed by the type of patient (74/343 or 21.6%), environmental context/resources (66/343 or 19.2%), and lack of knowledge (15/343 or 4.4%). Within the type of clinical case category, case complexity represented 164/182 or 90.1% of the type of clinical case and case rarity 18/182 or 9.9%. Case complexity included three subcategories: diagnosis-treatment-management (92/161 or 57.1%), psychopharmacology (44/161 or 27.3%), and comorbidities (25/161or 15.5%). In the type of patient category, patient psychological factors (e.g., anxiety, personality disorders) accounted for 30/70 (42.9%), with social influences from family or friends at 24/70 (34.3%) and patient motivation (e.g., lack of adherence, resistance to treatment) at 16/70 (22.9%). In the category environmental context/resources, lack of or limited access to resources/lack of reliable resources accounted for 36/67 (53.7%) of responses, lack of support (e.g., lack of collaboration) for 19/67 (28.4%), and medicolegal/ethical issues for 12/67 (17.9%).

Fig. 3

Factors influencing the perception of challenging clinical cases or situations

Chi-squared tests of bivariate associations between factors influencing the complexity of challenging cases and (1) region of practice (rural, suburban, urban), (2) years of practice, and (3) type of practice (hospital, group, solo) were not statistically significant.

Resources Used to Solve Challenging Cases/Situations

Resources accessed by participants (N = 186) to solve challenging cases or situations included consultation with colleagues (91/186 or 48.9%), text/reference book(s) (32/186 or 17.2%), individual(s) with knowledge of the problem (22/186 and 11.8%), and formal referrals (13/186 or 7.0%).

Discussion

Prior to the Covid-19 pandemic, we carried out a needs assessment which focused on the perceived and unperceived CPD needs of psychiatrists practicing in Quebec (Canada). The demographic profile of our respondents was largely representative of francophone, experienced psychiatrists working in urban hospitals, and caring for the adult population of the province. Our study used an innovative method (CIM) to trigger reflection about challenging cases/situations and identified a convergence of perceived and unperceived needs among psychiatrists.

Attending conferences in person was the most popular CPD modality among psychiatrists, which is consistent with CPD literature across many specialties [5, 17]. Topic and quality were reported to be the main factors involved in the selection of CPD activities, which mirrors the findings of a recent survey in the USA targeting physicians, nurses, and physician assistants [18]. Despite the ineffectiveness of didactic, stand-alone conferences in affecting practice outcomes [19], previous studies have reported that networking and wellness are important motivators to attend [20].

Our participants also expressed interest in online CPD activities and an online toolbox. The Covid-19 pandemic has been a catalyst for change and for the adoption of digital technologies, enabling the transformation of in-person CPD conferences and activities into virtual formats. It is therefore conceivable that a hybrid CPD model might be adopted post-pandemic. However, successful transition to digital care and to theory-driven digital psychiatric training should include an examination of the inherent complexities and challenges, as well as the monitoring of potential risks [21, 22].

The current study may serve as a benchmark to track changes in psychiatrists’ educational needs and preferences before and after the Covid-19 pandemic.

As the landscape of medical education undergoes profound change, assessment will continue to be a key component of competency-based medical education [23]. Participants in our study preferred the same type of assessment activities as in an earlier study of psychiatrists across Canada [24]. However, psychiatrists and other specialists can be inaccurate in assessing their own performance [10] and, consequently, CPD providers should offer opportunities for informed self-assessment which may be conducive to practice change [25].

Challenging Cases

Our results suggest that psychiatrists’ perceptions of the complexity of challenges in the workplace were influenced by a variety of factors at the individual, organizational, and systemic levels. While a minority of perceived factors in our study related to the systemic level, being unaware of systemic issues could influence patient safety and quality of care [26]. This finding may therefore represent an additional unperceived need of psychiatrists. Our information will be of considerable use for the development of future CPD activities and physician training, which should be grounded in system science [27-29].

A popular approach to challenging cases or situations was discussion with peers, which could trigger the identification of learning needs and be conducive to changes in practice for psychiatrists and other specialists [4, 24]. An earlier study used innovative methods to collect data and to resolve problems on challenging cases and situations at point of care via electronic consultations between primary care physicians and specialists [30]. Collaboration with colleagues was also associated with job satisfaction among licensed psychiatrists in a previous study in Ontario, Canada [31]. Additional studies should obtain data on type and duration of clinical consultation(s) with colleagues: this could inform the development of CPD open-learning environments and be used to debrief challenging clinical cases or situations with guidance from trained coaches [32].

From the perspective of competency-based CPD, our results showed that solving challenging, complex clinical cases aligned predominantly with the medical expert role from the CanMEDS framework. Future studies should explore the intrinsic competencies needed when facing complex situations triggered by environmental factors [13].

Neurodevelopmental and psychotic disorders, psychopharmacology, and psychotherapy were identified as both perceived and unperceived needs in our study. The similarities between both types of needs are consistent with two needs assessments which made use of the CIM and which targeted the needs of family physicians and other specialists practicing in Quebec [33]. The CIM as a type of self-audit [11] may act as a trigger, transforming an unperceived need into a perceived one. Future studies could include semi-structured interviews asking psychiatrists if reflecting on complex clinical cases or situations and consulting with their peers were conducive to selecting relevant CPD activities and/or other resources to prepare them to face similar challenges in the future.

Linking healthcare professionals’ clinical challenges to the identification of learning needs and the development of CPD is a relatively new approach. Our participants’ interest in adopting point-of-care technology for information seeking is congruent with the growing trend in CPD in psychiatry [34] and across specialties [35].

Study limitations include limited generalizability: our sample consisted of a specific subgroup of practicing psychiatrists. The survey response rate was lower than that of similar studies in the literature; in general, studies of physicians tend to have low response rates [36]. Rationale for not completing the survey among nonrespondents was not investigated due to limited resources. The current study received support from the provincial psychiatry association from which participants were recruited. Thus, social desirability could have influenced our findings, reducing generalizability and introducing potential bias. Two strategies were implemented for mitigation: participant anonymity and assurances of confidentiality. The cross-sectional nature of our findings, as well as the use of self-reported measures which should be triangulated with objective measures, represent additional limitations.

This study provides valuable information to develop CPD programming for practicing psychiatrists and could be used as a benchmark to track changes in psychiatrists’ educational needs before and after the Covid-19 pandemic. The CIM is a simple yet effective method which enabled comparison of perceived and unperceived needs and which has been underutilized in the field of CPD. The convergence of perceived and unperceived needs among psychiatrists suggests that critical reflection on how complex clinical cases and situations are resolved in practice might provide a useful opportunity to optimize psychiatrists’ selection of CPD.