Critical appraisal of journal article by psychiatry PG residents using a new module: Impact analysis.

BACKGROUND: A gap exists from evidence-based medicine (EBM) to clinical practice and there is a felt need to bridge this. Critical appraisal of scientific articles during Journal club by postgraduate (PG) residents will help them to understand and apply the scientific evidence into best clinical practice. Hence, a new module of critical appraisal of journal articles was used for Psychiatry Residents in a Medical College and its impact was assessed.
MATERIALS AND METHODS: Nine psychiatry PG residents from three academic batches participated in the study after informed consent and Institutional ethical committee clearance. They were trained in the Colorado Psychiatry EBM Examination Test module. Subsequently, three consecutive journal article presentations of them were assessed under seven Subtests of the module.
RESULTS: There was gradual increase in the assessment scores of all PG residents with each journal article presentation. They also expressed satisfaction of the assessment method and felt confident of applying the principles of critical appraisal in their clinical practice in future.
CONCLUSION: This study evaluated the impact of a new module of critical appraisal of journal article by psychiatry PG residents. It was found to be acceptable by residents and improved their competency to apply literature-based EBM into their clinical practice. It is recommended for further multicentric evaluation on a larger sample. Copyright:

Evidence-based medicine (EBM) is defined as “The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”[1] EBM is a term used to describe a structure for practicing medicine. This structure incorporates three fundamental components: The best available clinical research, a clinician's judgment, and patient's values and beliefs.[2]

There are five critical steps one needs to follow to practice EBM: (a) Construct a question from the clinical environment, (b) search the medical literature, (c) apprise the evidence, (d) integrate the evidence in clinical practice and (e) evaluate its application.[3] Many studies have assessed the integration of EBM in medical curricula and have noted that practising EBM benefits medical students in improving their attitudes and skills.[4567]

Medical students find the integration of EBM practice in medical education and clinical practice difficult due to challenges in correct and efficient search and retrieve of information to address the clinical question. The PICO method, which stands for P (population/people/patient/problem), I (interpretation), C (comparison), and O (outcome), has been found to be efficient and easy to use. It facilitates students to correctly transfer a clinical situation into an answerable question that fits smugly into literature search engines leading to efficient searches. This method has been used extensively since the days it was enunciated by Richardson et al. in 1995.[8]

Journal club in medical teaching was started by Sir William Osler in 1887 for undergraduate students at MC Master University with an aim to secure training in the difficult science of debate. It is essentially an educational meeting in which a group of individuals discuss published articles, provide a forum for collective effort to keep up with the current literature and an objective to promote discussion and critique of research.[9]

A gap exists from EBM to clinical practice and there is a felt need to bridge this gap. Critical appraisal of scientific articles during Journal club by postgraduate (PG) junior residents (JR) will help them to understand and apply the scientific evidence into best clinical practice. Presently, residents select journal articles randomly and discuss more the methodology rather than about relevance and clinical application of the content. Available studies from Indian Medical Colleges have shown that the current method of journal article appraisal by PG residents is unsatisfactory.[1011] Journal Club is the most common practice to teach the evidence-based practice to resident physician learners. This is a, highly variable experience with visible variable outcomes.[12]

The authors in this study planned to train Psychiatry PG Residents in a Medical College using a newer module in their Journal club critical appraisal and to assess its impact on their competency to apply EBM into their clinical practice.

MATERIALS AND METHODS

Rothberg et al. in 2013 developed a model of teaching EBM to psychiatric residents that addressed the essential skills.[13] Their Colorado psychiatry EBM (CP-EBM) Examination tests knowledge of EBM through 14 open-ended questions. In question 1, residents' ability to assess and ask focussed clinical questions is evaluated. Similarly, questions 2–4 are for the ability to identify search strategy and acquire information, questions 5–9 are for appraisal skills, questions 10–11 are for knowledge of diagnostic and prognostic research designs, question 12 is for the ability to apply outcome measures in clinical practice and questions 13–14 are for the ability to assess a patient with an outcome measure over time. The scale was noted to have high interrater correlation (Cronbach's alpha of 0.95), intraclass correlation of 0.93, Internal Reliability (Cronbach's alpha of 0.84), Item difficulty (from 0.09 to 0.79), and Item discrimination (from 0.24 70 0.78). The CP-EBM Examination module was shared by Rothberg et al., with the authors for this study.

Nine Psychiatry PG Residents (three from each of three academic batches; 1st year-JR I, 2nd year-JR II and 3rd year-JR III) participated in the study after informed consent and Institutional ethical committee clearance. Initially, it was decided to assess the ability to frame the clinical question, search strategy and appraisal skills only, and hence, first seven questions of the module were utilized in the study. They were trained in the application of the CP-EBM Examination Test module for journal club presentations. Subsequently, three consecutive journal article presentations of them were assessed by faculty under seven subtests of the module, namely question construction (1), information source (2), study design (3), search strategy (4), relevance to practice (5), study validity (6) and significance of the study (7). Maximum possible score was 156. This was a qualitative longitudinal study. Progressive changes in their score over three consecutive journal club presentations were assessed. A feedback on the usefulness of the module was also obtained from the participant residents.

RESULTS

In the first journal article presentation after training, the mean score of JR I was 59.66 which increased to 121.00 by the end of the third presentation [Figure 1], while the mean score of JR II increased from 59.00 to 127.00 [Figure 2] and mean score of JR III improved from 68.66 to 137.00 [Figure 3] at the end of the third presentation.

Figure 1

JR I performance in question subjects

Figure 2

JR II performance in question subjects

Figure 3

JR III performance in question subjects

However, there was the difference in their scores in the subtests. In JR I maximum increment was seen in subtest relevance to practice followed by subtest study validity. JR II maximum increment was seen in subtest study validity followed by subtest search strategy whereas in JR III maximum increment was seen in subtest question construction followed by the significance of the study.

DISCUSSION

This study showed the usefulness of a newer assessment module in improving the journal article critical appraisal ability of Psychiatry PG residents. Though there was the improvement in the performance all batches of Residents, there was the difference in their performance in various subtests. Interestingly JR I showed higher improvement in the subtest of Relevance to practice whereas JR III showed maximum increase in the Question construction subtest. These may be due to the changes in appraisal capability over the course of training.

Residents expressed satisfaction of the assessment method and felt confident of applying the principles of critical appraisal in their clinical practice in future. The limitation of this method is the requirement of more time to assess (30 min) each participant as compared to the present method of assessment (10 min) using single page Likert scale pro forma. Furthermore, the participant size was nine PG residents only and hence, suggested that more such studies on the usefulness of this tool, with a cumulative large sample will be required for validation before generalizing for regular use.

The medical doctor is required to engage in patient care, teaching, and research. The learning during PG training enhances the ability to develop the research inclination and deeper understanding of publication. The Department/Unit of Medical Education emphasizes educational research and the implication of training modality as per standards.[14] The added knowledge of the module and training on the defined module improves the analytical skills of PGs.

CONCLUSION

This study evaluated the impact of a new module of critical appraisal of journal articles by Psychiatry PG Residents. It was found to be acceptable by Residents and effective in making them more competent to apply literature-based EBM into their future clinical practice. It is recommended for further multicentric evaluation of this tool on a larger sample.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.