Subjective assessment of participants in education programs on clinical practice guidelines in the field of psychiatry.

The Effectiveness of Guidelines for Dissemination and Education in psychiatric treatment (EGUIDE) project, which is a nationwide dissemination and implementation program for clinical practice guidelines (CPGs) in the field of psychiatry, is currently ongoing. In the current study, a subjective assessment of the participants in the EGUIDE programs was assessed using a questionnaire. Then, the relationships between the subjective assessment, the characteristics of the participants, and the clinical knowledge of the CPGs were evaluated. More than 90% of the participants gave a high rating for the components of content, recommendation, knowledge, skill, and adherence, but not for the component of confidence. A positive correlation was found between years of professional experience and the score of confidence. These results suggest that it may be necessary to apply the knowledge and skills of CPGs obtained in the education programs into practice to increase confidence in the proper use of psychiatric therapies based on CPGs.

INTRODUCTION

Clinical practice guidelines (CPGs) that give medical practitioners and patients reasonable treatment recommendations are important in “evidence‐based medicine,” which is the standard manner of today's medical practices. The Effectiveness of Guideline for Dissemination and Education in psychiatric treatment (EGUIDE) project, which is a nationwide dissemination and implementation project for CPGs in the field of psychiatry, is currently ongoing. This project addresses "Treatment Guidelines II. Major Depressive Disorder" by the Japanese Society of Mood Disorders and "Guidelines for the Pharmacological Treatment of Schizophrenia" by the Japanese Society of Neuropsychopharmacology. The project is original in that it evaluates multiple indicators and provides feedback on the lecture and development of CPGs. As of June 2021, 243 medical institutions, including 44 universities, were participating in this project in Japan. The aim was that by promoting the EGUIDE project, psychiatrists will be educated to use the CPGs and that more appropriate treatments, based on CPGs, will be widely provided.

To date, it has been demonstrated that guideline‐based treatment is not always widely used in Japan, , that participation in the EGUIDE programs improves understanding of the CPGs, that revisions of the training materials of EGUIDE programs improve participants' understanding of the CPGs, and that polypharmacy of antipsychotics significantly increases the simultaneous prescription of other psychotropics such as anticholinergics, anxiolytics, hypnotics, and mood‐stabilizing agents in patients with schizophrenia. In the current study, the subjective assessment of the participants in the EGUIDE programs was assessed using a questionnaire. Then, the relationships between the subjective assessment, the characteristics of the participants, and the clinical knowledge of the CPGs were assessed.

METHOD

Participants

Psychiatrists were recruited from October 2016 to March 2018. Four sets of data, the subjective assessments of schizophrenia (SCZ) and major depressive disorder (MDD) programs and the clinical knowledge of the CPGs of SCZ and MDD, were available for a total of 344 participants. The mean age of the participants was 33.8 ± 6.9 years (mean ± SD), and the mean years of professional experience were 5.1 ± 6.1 years.

Subjective assessment of EGUIDE programs

The subjective assessment of each guideline program was evaluated using a questionnaire following the EGUIDE programs. It is a self‐administered questionnaire that consists of the following six components on a scale of 1‐5, with higher scores indicating a higher evaluation: (1) How would you rate the content of this course? The abbreviation for this question is "Content," with 5 = very satisfactory; 4 = moderate satisfaction; 3 = I cannot say I was either satisfied or dissatisfied; 2 = dissatisfied; and 1 = very dissatisfied; (2) How would you rate your recommendation of this course to a colleague or junior colleague? The abbreviation for this question is "Recommendation," with 5: strong recommendation; 4 = moderate recommendation; 3 = I cannot say that I would or would not recommend; 2 = no recommendation; and 1 = no recommendation at all; (3) How would you rate your clinical knowledge of treatment after attending this course? The abbreviation for this question is "Knowledge," with 5 = very increased; 4 = increased; 3 = I cannot say increased or confused; 2 = confused; 1 = very confused; (4) How would you rate your treatment skills after attending this course? The abbreviation for this question is "Skill," with 5 = very increased; 4 = increased; 3 = I cannot say increased or confused; 2 = confused; 1 = very confused; (5) How would you rate your confidence in your ability to perform proper treatment after attending this course? The abbreviation for this question is "Confidence," with 5 = great confidence; 4 = moderate confidence; 3 = I cannot say I am confident or unsure; 2 = no confidence; 1 = no confidence at all; and (6) How would you rate your future choice of treatment according to the guidelines after attending this course? The abbreviation for this is "Adherence," with 5 = will always choose; 4 = will mostly choose; 3 = I cannot say I would choose or not; 4 = no consideration of guidelines; 1 = no consideration of guidelines at all.

Assessment of clinical knowledge of CPGs

Working knowledge of the CPGs of SCZ and MDD was evaluated using a questionnaire both at baseline and after each of the programs. Each self‐administered questionnaire consists of 37 questions, with a total score of 37 points (see Tables S1 and S2).

Statistical analysis

The relationships between the demographic characteristics of the participants (age and professional experience), the total score of clinical knowledge following the programs, and each component score of subjective assessment (the aforementioned content, recommendation, knowledge, skill, confidence, and adherence) were analyzed using Spearman's rank correlation coefficient. All statistical analyses were performed using IBM SPSS 27.

RESULTS

In the subjective assessment, more than 90% of the participants gave a high rating of 4 or 5 for the five components of content (98.8% and 97.7%), recommendation (95.3% and 97.1%), knowledge (98.3% and 98.5%), skill (92.4% and 91.0%), and adherence (95.3% and 96.5%), respectively, in both the SCZ and MDD programs, but not for the component of confidence (42.2% and 44.8%, respectively, Figure 1). When we examined the relationships between age, professional experience, and the individual component scores of the subjective assessment, a positive correlation was found between years of professional experience and the score of confidence both in the SCZ and MDD programs (r = .249, P = 2.97 × 10−6 and r = .151, P =.005, respectively; Table 1). This correlation reached statistical significance when corrected for the multiple testing of the six components in each program (P < .0083). When the relationships between each component score of the subjective assessment and the total score of clinical knowledge of the CPGs were assessed following the programs, no significant association was found in either the SCZ or MDD programs (P > .05; Table S3).

FIGURE 1

Scores of the distribution of subjective assessment. Each cumulative bar chart represents the proportion of the subjective assessment score of the schizophrenia (SC) and major depressive disorder (MDD) guideline programs. The six components were rated on a scale of 1‐5 using a self‐administered questionnaire following the EGUIDE programs, with a higher score indicating a higher evaluation

TABLE 1

Correlations between subjective assessments and age/years of professional experience. Spearman's rank correlation coefficient: The statistical significanec level was set at P <.0083 for the correction of the multiple testing of six components in each program

	Content	Recommendations	Knowledge	Skills	Confidence	Compliance
Schizophrenia
Age (y)
r	.040	.107	−.102	−.115	.126	.058
P value	.455	.048	.060	.033	.019	.287
Professional experience (y)
r	−.008	.088	−.142	−.165	.249	.069
P value	.882	.104	.009	.002	2.97 × 10−6	.202
Major depressive disorder
Age (y)
r	.019	.118	−.073	−.066	.093	−.007
P value	.725	.029	.175	.224	.085	.898
Professional experience (y)
r	−.030	.072	−.181	−.127	.151	.058
P value	.574	.181	.001	.019	.005	.287

Bold value indicates significant positive correlations are observed between "Confidence" and years of professional experience in either the schizophrenia or major depressive disorder program.

DISCUSSION

To our knowledge, this is the first study that assessed the subjective effect of educational program and the related factors in the field of psychiatric CPGs. The subjective assessment of participants in the EGUIDE programs was performed using a questionnaire. High rating scores of content and recommendation suggest that the CPG programs were highly regarded by the participants. High ratings of knowledge and skill suggest that most participants had a better understanding of CPGs. These results are consistent with the previous results of the objective improvement of the participants' clinical knowledge of the CPGs. Although knowledge of the guidelines may not be necessarily linked to treatment outcomes, a high rating score of adherence may lead to future proper treatment choices based on the CPGs.

A high rating for confidence was not achieved in either the SCZ or MDD programs. Most participants might be cautious rather than focusing on building their confidence. While the score of confidence was associated with years of professional experience, it was not associated with clinical knowledge of CPGs following the programs. On the contrary, an education program using case‐based learning and supervision over time increased confidence as well as knowledge. Evidence‐based medicine, which the CPGs are intended to support, relies on the expertise of medical practitioners. , In other words, clinical experience is a prerequisite for the optimal use of the probabilistic recommendations provided by the CPGs in practice. It may be that an application of the clinical knowledge of CPGs acquired in the program in daily practice will lead to the acquisition of confidence in the appropriate use of psychiatric therapies. A longitudinal study of changes in adherence to CPGs and confidence of the proper therapies based on CPGs is currently planned.

The limitations of this study are the number of participants and potential sampling bias, such as years of professional experience. Future researches with increased participants are needed.

CONCLUSION

The educational programs on CPGs contributed to a high subjective assessment of therapy knowledge and skills, as well as guideline‐based treatment choices, but did not lead to an increased confidence in the appropriate use of therapies based on CPGs. To increase confidence, it may be necessary to apply the knowledge and skills obtained in the educational programs to real‐world clinical practice.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS

KOg and SN were involved in data collection and data analysis and wrote the first draft of the manuscript. NHase, JM, KM, and MN were involved in the data analysis and contributed to the interpretation of the data and writing of the manuscript. MM, KOh, MT, TT, NHash, TO, HM, HH, KIc, TI, NY‐F, AH, NS, KF, TNag, YT, RF, KN, TNak, MU, MF, HT, HY, HK, SO, KA, EK, MK, TK, TS, SY, JI, and HI contributed to the interpretation of the data and data collection. KIn and KW were involved in the study design and contributed to the interpretation of the data. RH supervised the entire project, collected the data, and was involved in the design, analysis, and interpretation of the data. All authors contributed to and approved the final article.

APPROVAL OF THE RESEARCH PROTOCOL BY AN INSTITUTION REVIEWER BOARD

This study was approved by the ethics committees of the National Center of Neurology and Psychiatry (A2017‐105) and each participating university, hospital and clinic.

INFORMED CONSENT

All participants provided their written informed consent. Public availability of raw data was not planned in the research protocol approved by an Institution Reviewer Board. We did not obtain informed consent of the public availability.

REGISTRY AND THE REGISTRATION NO. OF THE STUDY/TRIAL

The protocol of this study was registered in the University Hospital Medical Information Network registry (UMIN000022645).

ANIMAL STUDIES STATEMENT

Not applicable.

Tables S1‐S3

Click here for additional data file.