An Integrated Interactive-Spaced Education Radiology Curriculum for Preclinical Students.

INTRODUCTION: The objective of this study is to determine whether a radiology module, together with online spaced education, helps students of an integrated problem-based learning (PBL) curriculum increase their radiology knowledge and long-term retention.
MATERIALS AND METHODS: Second-year students at the American University of Antigua College of Medicine participated in small groups of ten students each into two 2 h of radiology laboratories. The study comprised two cohorts: winter and fall 2013 students (control group) and 2014 students (experimental group). Both groups used face-to-face PBL. The students of the experimental group received additional online-spaced education. The skills were assessed for both groups before the beginning of laboratories and 4 weeks and 7 months after laboratories.
RESULTS: There was no significant difference on pretest between the control and experimental groups. On completion of the radiology laboratories, comparison of test results before and after training showed net improvement for both groups. The corresponding difference for the experimental group was higher compared to the one for the control group (7.83 vs. 6.21, P < 0.001). The difference between the scores on delayed test and pretest showed that the students of both groups demonstrated average knowledge improvement even though their level of performance was slightly below the posttest. The corresponding difference for the experimental group did not differ much from the posttest (P > 0.05), and no significant difference of scores was observed 7 months later for either group. Further, a higher percentage of the students in the experimental group strongly agreed that their learning objectives were met (92% vs. 71%, P > 0.001), and this trend persisted throughout the study.
CONCLUSION: Online spaced education combined to a face-to-face PBL enhances not only the student's knowledge of basic radiology along with his/her self-assessment skills but also the long-term retention of radiology material and satisfaction with the integrated interactive system-based module. Future research is needed to see if medical students in need of additional education support may benefit from spaced education in the field of remediation.

RCT Entities:   Population Interventions Outcomes

Introduction

In the fall 2012, the American University of Antigua (AUA) College of Medicine integrated the basics of radiology into its third semester of the introduction to clinical medicine (ICM) 3 course. ICM is a problem-based learning (PBL) course that introduces students to the integration of basic sciences into the clinical setting.

Recent studies have examined strategies to incorporate radiology into the undergraduate medical program.[12345678] Online-spaced education, referred to as “Qstream,” is an e-learning platform in which “information is presented and repeated over spaced intervals and is learned and retained more effectively.”[9] Qstream is shown to facilitate knowledge recall and behavior.[101112]

The purpose of this study, conducted between the fall 2013 and the winter 2014 at the College of Medicine, is to assess the impact of Qstream on the student's acquisition and retention of interpretative radiology skills. In our study, we compared the long-term retention of the basics of radiology and interpretive skills in the students undergoing training using PBL format/Qstream.

The Alliance of Medical Student Educators in Radiology curriculum[13] provided the foundation for the outline of the core concepts used in the ICM laboratories.

Materials and Methods

Study design

The AUA College of Medicine Ethics Research Council approved this case–control study.

Study setting and population

The control group

The control group comprised the class of ICM 3 throughout the 1st year of the introduction of the radiology module. The group was exposed to the module in a standard PBL format and had 187 and 210 students for winter and fall 2013, respectively. Their age ranged from 20 to 25 years (average 23 years). The students were split up into small groups of 10–12 each, and the students attended the radiology course (i.e., 100% of attendance to laboratories).

All the small groups participated into two 2 h of the basics of radiology laboratories (August, September, October for the fall semester; February, March, April for the winter semester) dedicated to exploring appropriate imaging and using imaging to make a diagnosis. Furthermore, two review lectures were provided, the first for ICM 3 students at the end of the training during a single large group event, and the second, as need arose for ICM 4 students, toward the end of the 4th semester.

A fellowship-trained neuroradiology, chest and musculoskeletal (MSK) imaging radiologist (Eli Tshibwabwa, with clinical and research interest in medical education) served as instructor and facilitator of the module for the two groups. Indeed, previous works on the incorporation of radiology in the preclinical program have demonstrated that one radiologist educator with PBL involvement could facilitate these types of small group teachings.[125]

In the context of PBL, application of the new knowledge is intended to enhance understanding and retention of the student's skills knowledge. Indeed, the student is expected to make links with prior relevant experience or reading, ask questions to stimulate the group's discussions, thus enhancing his/her own and the group's learning experience.[141516] The new module integrates the basics of diagnostic imaging into critical concepts learned in ICM and designates time for applying radiology concepts in tutorials. Indeed, PBL format requires learners to research issues of interest surrounding the clinical vignette under discussion.[14151617] Therefore, the students in the current study were advised to review the foundational radiology materials ahead of their laboratories. For instance, specific terminologies are used; thus, students need to come well prepared and know how opacities or a lucent structure refer to chest, abdomen, and MSK plain radiographs and hyperdense or hypodense structures to computed tomography (CT) scan. They also need to know how “anechoic/posterior enhancement” and hyperintense/hypointense terminologies pertain to ultrasound and magnetic resonance imaging (MRI), respectively. Further, they are required to describe axial/coronal/sagittal CT/MRI scans of the brain or abdomen.

To assist students in determining where to start for their upcoming tutorial, all the relevant learning materials were disseminated through the school's learning management system (Blackboard: Elearning.auamed.net). The available content was structured as follows: an accompanying PowerPoint presentation (PPT) on basics of radiology for the chest, abdomen, genitourinary, MSK, and nervous systems (with integrated interactive elements), which was prepared by the first author. The goals and overall learning objectives of the current module [Table 1] were included in the PPT. The platform of blackboard was only used as an electronic delivery system for learning material. Since blackboard was already in existence at the college of medicine during the study period, the use of this system by the students would not be a confounding factor in the assessment of PBL.

Table 1

Goals and learning objectives of the basics of radiology labs for the chest, gastrointestinal tract, genitourinary system and the nervous and musculoskeletal systems

The sessions took place in the ICM laboratory. As in a previous research,[2] the room was adequately lit, and the PPT projected on a large high-definition television display to allow for all the students in the laboratory to view them and to effectively participate in the discussion. The learning experience focused on describing the modalities for imaging body systems, deciding which modality to use and when to order it, and becoming familiar with the overall learning objectives and images of some important clinical scenarios frequently investigated with imaging modalities [Table 1].

In tutorials, under guidance of the facilitator, the group may ask questions to challenge peers in a way to support the student's learning. Furthermore, the face-to-face setting allowed for interaction among the students and between the group and the facilitator. The feedback from peers and faculty leads to the student's skills of self-assessment. In this context, the teaching promotes collaboration among the students, and this professional competency is necessary for the practice of medicine.

Experimental group

The experimental group was made of the class of ICM 3, during the second half of the 1st year and first half of the 2nd year of the introduction of the module. The students’ age and the small group size were of the same range as the control group. All the students attended the training (100% of laboratories attendance).

Intervention

The students of the experimental group received the face-to-face PBL with an online Qstream supplement integrated into their learning and were required to complete this Qstream program. However, the duration to impart the theoretical radiological interpretative skills to the group was identical to the control group, i.e., 2 × 2 h of laboratories. Further, the experimental group received the same preparatory resources as the control group.

The second author (RM) was granted access and editing privileges and created the Qstream link.[9] The course was made live, and a link was sent to the group through the College of Medicine's learning management system for each student to begin online spaced education. To this end, the first author created a 10-item Qstream course of the basics of radiology for students to practice, and all the authors could follow the student's progress with the course. Although different from the vignettes discussed during ICM laboratories, the health problems for the 10 items also addressed the core radiological concepts outlined in Table 1. Allowing the students in the “Qstream” to work along the same overall learning objectives of the module allowed for consistency and comparability of the control and the intervention groups. An example of Qstream item is provided along with its evaluative component and educational component. On completion of their training, students were given 4 weeks to complete the “Qstream course.” Here is how the Qstream is structured.[91011] each learner receives two questions every day through E-mail. The Qstream program is highly interactive as “The participant submits an answer, the response is recorded in a central server, and the student receives immediate feedback.”[10] If the question is not answered correctly, it is repeated 1 week later. If a question is answered correctly two times in a row, the item is removed from the course.[10] When students complete the course and all the questions are retired, they receive a certificate.

Evaluation

Pre- and post-tests and delayed test including identical multiple-choice questions (MCQs) in various arrangement order of topics at each testing were used to assess the students’ skills of interpreting diagnostic images.

Before the beginning of the laboratories, a pretest was performed to assess student interpretative skills. The required learning objectives [Table 1] provided the framework for 10 MCQs. The authors ensured that these questions had appropriate level of difficulty, knowledge validity, and testing of readily identifiable concepts referenced to the goals and learning objectives. To allow for comparability of the groups, there was one set of 10-item MCQs constant throughout the study, covering the same radiology concepts though presented with different scenarios and varied order of arrangement. The questions were administered without prior notice and not aimed at a block examination.

Posttest was carried out 4 weeks after completion of the laboratories, i.e., toward the end of the semester before a review lecture. It was anticipated that, by the time of this review, students in the intervention group would have completed the Qstream course. On completion of the posttest, no further radiology laboratories were given although the students had to continue with other ICM3 and ICM4 curriculum courses. Immediately after the posttest, the students were asked to assess their overall experience and satisfaction with the radiology module. Since there was only one instructor/facilitator for the two groups, there were no differences in skill sets of instructors to contend with.

A delayed test was then performed toward the close of the ICM4 courses. In advance of their Step 1 and Step 2 of the United States Medical Licensing Examination, the ICM4 students usually requested a review session aimed at the ICM3 basics of radiology, i.e., chest, gastrointestinal tract, genitourinary system, MSK system, and nervous systems. These requests were done each year for May and November of the following semester for the fall and winter groups, respectively. This length of time was the justification for using 7 months to define long-term retention. As in the previous posttest, again, the students were polled to determine whether the module helped them meeting their learning expectations.

The assessment of the students’ skills in interpreting basic radiology findings was done using a 10-point Likert scale. Evaluation of differences between pre- and post-test and delayed test students’ scores was made with a paired t-test. Results of the student's impressions on the module were expressed as percentage of very strongly agree, strongly agree, disagree, strongly disagree, and very strongly disagree. Further, differences in student-reported satisfaction between groups were analyzed to see if they are significant or not. Chi-square test was used to compare control versus experimental for very strongly agree and differences in student-reported satisfaction between groups.

Data collection

To record the outcomes measures, a rapid audience response device (“clicker”) in conjunction with Turning Point and PowerPoint software was used. As with a previous research elsewhere, again, in the current study, the students were issued a ResponseCard RF Audience Response device, henceforth referred to as a “clicker” available in all the classrooms.[2] The clicker interfaces with the software Turning Point (Turning Technologies Headquarters, 255 West Federal Street, Youngstown, Ohio, 44503, USA) and Microsoft PowerPoint to record the students’ selection of answers to various MCQs and to provide immediate feedback on student performance on the questions or content area, i.e., correct answer and additional explanation, the students’ overall satisfaction with the module. The clicker [Figure 1] was chosen for audience voting and data collection because it has been deemed as an efficient method of collecting data within the classroom while allowing students’ participation. At each test, the facilitator included the MCQs into the PPT; these were displayed on large screens for all the students to answer using their “clicker,” but answers were anonymous. The clicker allows students to answer polling questions with a click of a button.

Figure 1

An original sample of ResponseCard RF Audience Response device henceforth referred to as a “clicker.”

Results

The focus of the learning format for both groups was the PBL. Attendance at laboratories was mandatory and considered as part of the professional behavior expected of the medical students.

Of students in the control group, 173 (92% out of 187) of the winter and 199 (94.5% out of 210) of the fall attended the review session, completed the posttraining test, and were included in this study. 128 of these 173 students (73.9%) and 163 of 210 students (77.6%) completed the delayed test and were included in this study.

All the students in the experimental group (100%) attended the training and were included in the pretest. Qstream was well accepted by the students: Indeed, 186 out of 190 students for winter 2014 (97.8%) and all the 287 for fall 2014 (100%) completed Qstream, and all attended the large group review and thus were included in the posttest. However, 143 of these 186 students (77%) and 238 of these 287 (82.5%) completed the delayed test at the beginning of their 1st day of ICM4 radiology lectures and thus were included in this study.

Tables 2–4, respectively, summarize the test scores of the control and experimental groups before and after training and 7 months later.

Table 2

Results of the students’ answers to multiple-choice questions before radiology training

Table 4

Results of the students’ answers to multiple-choice questions 7 months after the end of the basics of radiology labs

As shown in Table 2, there was no significant difference in the pretest, i.e., mean score and standard deviation between the control and experimental groups. Out of a maximum possible score of 10, both groups had almost similar scores (4.38 vs. 4.59, P > 0.05).

At the review session, on completion of the laboratories, comparison of test results before and after training showed net improvement for both groups since the control and the experimental groups have each shown higher mean scores than before the training [Table 3]. However, the corresponding difference for the experimental group was higher as compared to the one for the control group (7.83 vs. 6.21, P < 0.001).

Table 3

Results of the students’ answers to multiple-choice questions four weeks after ICM radiology labs

Even though the level of the performance was slightly below the posttest scores, the difference between the scores on delayed test and pretest [Table 4] had shown that the experimental group had maintained higher scores compared to the control group (7.26 vs. 6.13, P < 0. 001). Further, the corresponding difference for the experimental group did not differ much from the posttest (P > 0.05). However, for the control group, there was no significant difference of scores 7 months later [Table 4].

Results of the students’ overall experience and satisfaction with the radiology module are shown in Table 5. Differences in student-reported satisfaction between groups were significant. A higher percentage of the students in the experimental group strongly agreed that their learning objectives were met with the radiology module (92% in experimental group vs. 71% in control (P < 0.001). Seven months after the training, nearly these differences were shown to be constant throughout the study.

Table 5

Students’ responses regarding the question on “Were your learning objectives met with the radiology module?”

In the setting of Qstream, 71% of the participants saw at least one question twice before completing the posttest. For instance, regarding the question on “which radiologic term best describes the ultrasound appearance of gallstones,” there were at least 983 attempts by 477 participants.

Discussion

In a face-to-face PBL, “recall occurs best for the student in this system when he (she) is faced with a problem, not when he (she) is faced with subject-oriented questions.”[14] Further, several studies have indicated this positive impact of PBL on the retention of the subject course.[151617] However “the impact of a program may be limited if there is no follow-up to reinforce the message.”[18] In contrast to PBL, “Qstream” uses the “spaced education” methodology that is proven to increase knowledge acquisition, boost learning retention, improve clinical behavior,[10192021] and increase knowledge retention by 170%.[922] The authors of the current study had hypothesized that Qstream integrated into the small group learning would enhance students’ learning because “testing is not merely a means to measure a learner's level of knowledge, but rather causes knowledge to be stored more effectively in long-term memory.”[9] In posttest and 7 months later, the scores were 6.21 and 6.13 for the control group and 7.83 and 7.26 for the experimental group. Although both groups on their own show no significant changes with respect to retention, this study demonstrates that the scores are consistently higher for the experimental group. Further, the results also indicate better long-term retention of the basics of radiology and interpretive skills in the students undergoing training using PBL format/Qstream.

Previous studies with the inclusion of spaced education into the medical student courses have reported varied completion rates ranging from 55% to 74%.[92123] However, the strength of the current study lies (i) in the larger size of the population from one single institution; (ii) 100% of participation into “Qstream” along with completion rates, in addition to the relatively high rates of students on the delayed test (27%–30% of attrition rates [Table 4]; (iii) along with constant and significant obvious students’ enthusiasm for Qstream [Table 5]. This observation leads to a sense of the student ownership of the Qstream activity. Furthermore, similar level of spaced education acceptance and enthusiasm was found in other studies.[91013212223] This type of teaching can be considered as a way of continuous communication and interaction with students outside of the classroom. Karpicke and Blunt emphasize that “a retrieval event may actually represent a more powerful learning activity than an encoding event.”[24] Indeed, several studies have shown that interactive-spaced education enables the retrieval of useful information.[1325] Furthermore, spaced education, consisting of clinical scenarios with questions and explanations distributed on a weekly basis, can significantly improve students’ retention of medical knowledge.[2126] There is also evidence that, even without feedback, free recall testing has positive effects on retention.[27] However, the immediate feedback to students in spaced education is shown to reinforce his/her self-assessment skills.[928] This would be in addition to the learner's self-assessment skills provided by PBL.

The reader may argue that there was no justification for using 7 months to define long-term retention. However, studies in other disciplines or across years have much different time periods, ranging from 4 weeks[25] to approximately 3 and 5 months,[1029] 8 months,[30] and 2 years after completion of the training.[931] Furthermore, data from large randomized controlled trials demonstrate that all the components of spaced education delivery can be personalized to meet the specific needs of the learners.[9] Thus, the 7-month period is consistent with an acceptable time period.

With regard to limitations, ANOVA could have been used to directly compare pretest and posttest results between the control and intervention groups. However, the t-test can be used in “before-after” studies or when the samples are the matched pairs. The paired test is shown to be appropriate to compare significance between groups, for example, in the comparison between face-to-face PBL versus PBL with additive spaced education. However, unavoidable limitation lies in the impossibility to identify individual student and follow his/her personal progress due to the anonymity of the audience response.

Conclusion

This study demonstrates that online-spaced education radiology course combined to a face-to-face PBL enhances not only the student's knowledge of basic radiology skills but also the long-term retention of radiology material. This is accomplished by placing the students’ interest into an integrated interactive system-based curriculum. Self-assessment skills are also reinforced by immediate feedback provided by spaced education. Therefore, students in this study may apply their acquired knowledge to other preclinical courses and toward their medical career goals. Future research is needed to see if medical students in need of additional education support may benefit from spaced education in the field of remediation.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.