An online curriculum in movement disorders for neurology housestaff.

INTRODUCTION: In many neurology residency programs, outpatient neurology subspecialties are underrepresented. Trainee exposure to these subspecialties, including movement disorders, is limited by paucity and variability of clinical experiences. We designed a structured educational tool to address this variability and allow for standardization of elements of movement disorders teaching.
METHODS: We designed and implemented a web-based curriculum in movement disorders for neurology housestaff, in order to improve participant knowledge. The curriculum includes an introduction with a structured framework for the description of abnormal movements and 10 interactive modules focusing on common movement disorders. The curriculum was piloted with nine neurology housestaff at Yale-New Haven Hospital. Evaluation of the curriculum was performed using pre- and post-tests, a survey, and semi-structured interviews.
RESULTS: The mean pre-test score was 0.7 (±0.19), and the mean post-test score was 0.95 (±0.05) (t = 3.27). Surveys demonstrated mean Likert values >4/5 for all questions in all categories (knowledge acquisition, quantity, enthusiasm and technical). Semi-structured interviews revealed the following themes: 1) the modules increased participant comfort with the topic, 2) the format was engaging, and 3) the curriculum accommodated different learning styles. All participants remarked that the structured framework was a particular strength.
CONCLUSION: We have created, implemented, and evaluated a foundational curriculum in movement disorders for neurology trainees, using readily-available technology. Housestaff responded positively to the curriculum, both in terms of content and format. This curriculum can be implemented in a variety of educational settings, as a central component of a standardized approach to movement disorders teaching.

Introduction

Several lines of evidence support the fact that there is a need for greater movement disorders teaching in residency. There are more movement disorders fellowship positions than applicants; in 2018, 60 applicants entered SFMatch for 77 movement disorders fellowship positions. At the same time the population of patients with Parkinson's disease, the foundation of movement disorders neurology, doubled from 1990 to 2015, with an expected further doubling by 2040 [1]. Yet in neurology residency, clinical exposure to movement disorders may be limited and/or relegated to later in training due to inpatient coverage and training demands.

Residents are often given lecture-based sessions to supplement clinical exposure. Lectures have many disadvantages including 1) poor attendance, 2) temporal incongruency with clinical experience, and 3) not stylistically promoting learning. Learners have a variety of learning style preferences, and accommodating these can affect the productivity of learning [2]; a multimodal approach may be particularly effective [[3], [4], [5]]. Traditional lecture formats often lack multimodal or interactive elements, and are associated with low resident engagement [6]. Duration of the learning experience and the ability to control the pace of learning are also valuable to learners [7]. In one study, the majority of viewers of instructional videos were not retained beyond 6 min [8]. Lectures of one plus hours clearly exceed these limits.

Didactic alternatives such as flipped curriculum, problem- or team-based learning have been employed [9], but still require residents to attend didactic time, failing to eliminate the issues of attendance and temporal incongruency with clinical experiences. The proliferation of new technologies gives the medical educator a cornucopia of options for updating trainee education. Virtual learning modules can serve to standardize didactic experiences, and are associated with high resident satisfaction, as demonstrated with a video-based EEG curriculum for residents [8]. Our module-based curriculum in movement disorders addresses many of these concerns: the modules are brief, interactive, patient video-based (authentic), and may be completed at the convenience of the learner. To our knowledge, no such curriculum currently exists in movement disorders.

Methods

We employed a mixed method design with the aim to study whether the implementation of an online curriculum in movement disorders for neurology housestaff, a) improves participant knowledge of movement disorders, and b) is acceptable and engaging to housestaff learners. This research protocol was approved to the Yale Human Investigation Committee.

Curriculum design

We created a video-based lecture that outlined a simple approach to describing and classifying common movement disorders. The framework was formulated by Sara Schaefer, with input from three movement disorders faculty (EL, ST, AP), one non-movement disorders neurology attending with formal training in medical education and the development of video-based lectures (JM), and one movement disorders fellow (AVR) at Yale School of Medicine. The framework provides descriptive terms in categories (voluntariness, rhythmicity, pattern, quality of the movement), and combines terms to guide the learner through the core features of different movement disorders phenomenologies (tremor, myoclonus, etc.).

We made an introductory video using PowerPoint and Camtasia video-editing software, introducing fundamental elements of each movement disorders phenomenology, and explaining some of the descriptive terms in detail using analogy (e.g. “oscillatory is like a see-saw”).

We created ten modules with the following topics: hemifacial spasm, essential tremor, myoclonus, dystonia, Parkinson's disease, essential tremor and Parkinson's disease, progressive supranuclear palsy, chorea/Huntington's disease, functional movement disorders, and tics, and placed all modules on a centralized website. Categories were selected based on commonality in clinical practice, dominance of physical exam findings, and availability of patient videos.

Patient videos were collected in the Yale Neurology Movement Disorders Clinic. Additional patients were recruited through established clinical trials that involve patient video collection at Yale, after IRB modifications were approved. Thirty-five patients consented to have their videos used for the project. Additional non-identifiable imaging studies were used.

One hundred twenty videos were edited using Camtasia, at times using PowerPoint slide decks with animation. Additional video modifications using Camtasia, including voiceover, illustrative arrows, split screen, freeze framing, etc. were utilized as needed (Fig. 1). Graphic art including illustrations, photos, and animations were made as needed by the project development team.

Fig. 1

Sample video screenshots from module videos.

Modules were created using the Qualtrics survey platform (Fig. 2). Each module included multiple embedded videos with multiple-choice and open-ended questions and answers, explanations, and descriptive text. Display logic within the platform specified conditional answers to enhance interactivity (i.e. learners were given “correct” or “incorrect” prompts depending upon their answers). Repetition of the framework from the introductory video guided the learner through the classification of a range of abnormal movements according to description (e.g. oscillatory), then phenomenology (e.g. tremor), then diagnosis (e.g. essential tremor). Some diagnostic tests and treatment were also covered. The modules were designed to be completed by an average learner in under 15 min. Individual videos did not exceed 2.5 min.

Fig. 2

Screenshot of development of a sample module within the Qualtrics Survey Platform, demonstrating sample question and display logic.

Participants

Yale neurology residents and non-movement disorders fellows from PGY-2 through PGY-5 were recruited for the study. At that time at Yale, our movement disorders curriculum consisted of a two-week didactic block for all residents, covering a variety of topics in one-hour lectures by various available faculty. This curriculum occurred as scheduled during the 2017–2018 academic year, preceding the study period. An email explaining the study with attached consent form was provided to all eligible neurology housestaff. Participation was voluntary. Six residents and three fellows agreed to participate (4 PGY-2s, 2 PGY-3, 3 PGY-5s). Only one participant (PGY-2) was planning to pursue a fellowship in movement disorders.

Curriculum evaluation

A pre-test and post-test were designed using the rules set forth by the National Board of Medical Examiners guide for item writing [10]. Twenty total questions were prepared, categorized as most related to 1) phenomenology, 2) workup, 3) diagnosis, or 4) treatment, independently reviewed by two movement disorders specialists and one general neurologist, and revised. The questions were divided evenly by category between the two tests.

Participants completed anonymous survey evaluations of the modules (Supplemental materials). The survey was adapted from a previous survey designed for learners to evaluate an EEG curriculum [8]. The survey used Likert scales for participants to rate technical aspects of the modules (video image and sound quality, ease of access, questions linked to answers, table of contents, text, embedded images), as well as the content and educational potential of the modules (how well educational concepts were conveyed, length and number of modules, learning experience, whether they would recommend the modules to other learners). Additional open-ended questions asked about what participants liked and didn't like about the modules, and recommendations for future content.

Participants were given two months to complete pre- and post-tests, all modules, and the survey. Participation across all aspects of the study was tracked with a personal identifier for each participant designed to ensure anonymity.

Upon completion of the modules, all participants were asked to participate in brief semi-structured interviews conducted by LV and PJ. Interview questions focused on the format, content, and educational value of the modules, as well as suggestions for future use (Supplemental materials).

Qualitative analysis

Eight of nine participants were interviewed, and interviews transcribed, deidentified by LV and PJ to ensure anonymity prior to review by study members involved in program leadership, and inputted into Dedoose. Two interviews were coded, and codes compiled to determine a final code book. Single coding using the final code book (Supplemental materials) was performed iteratively on all deidentified interviews. After eight participants were interviewed saturation was reached, such that no further themes were identified. Therefore the ninth participant was not interviewed.

Quantitative analysis

Survey questions were clustered into four categories: knowledge acquisition, enthusiasm, quantity, and technical. Technical questions involved Likert scale answers of poor, fair, good, very good, and excellent, while the other three categories involved Likert scale answers of strongly disagree, disagree, neutral, agree, and strongly agree. Likert scale answers were assigned values from 1 to 5 (poor or strongly disagree = 1, excellent or strongly agree = 5).

Mean scores and standard deviations were calculated within each question. Categorical means were calculated using the individual question means within each category. A one-way analysis of variance (ANOVA) was calculated on both individual and category means.

A paired t-test was performed to determine if there was significant increase in test scores between the pre-test and post-test for the six participants who completed both. A t-test was performed to determine if the three subjects who completed the pre-test but not the post-test differed significantly from the six participants who completed both with regards to the pre-test score.

Results

Qualitative results: semi-structured interviews

In the interviews, residents' previous experiences with movement disorders exposure and didactic education echoed the issues raised above regarding limitations of movement disorders education in neurology residency. Participants mentioned issues of variability of didactic lectures depending on the educational expertise of the lecturer. Residents commented that they often weren't able to attend the relevant lectures. Even if they could attend they often could not focus due to other resident and patient care obligations.

They reported that they “learned movement disorders piecemeal,” and that they never learned the basics, “we get a lecture on Parkinson's disease, but it's like: ‘here's the symptoms of Parkinson's and here is what you do for it.’… but they don't really talk about recognizing like, you know, what is a tremor?” As a result, residents felt that they weren't previously approaching movement disorders in an organized fashion, “I would just kind of look at something and try to just be like ‘oh it's this diagnosis’ rather than… describing what it is and then trying to figure out which diagnoses fit that description.”

Several overarching themes emerged from the interviews regarding the impact of the modules, including: 1) the modules increased participants' comfort with movement disorders, 2) the format was engaging, and 3) the modules appealed to learner preferences (Table 1). Without being specifically asked about the framework, every participant commented that it was a particular strength. The framework increased their confidence, accuracy, and ability to develop a starting point that could be used to explore further with additional resources.

Table 1

Qualitative themes from semi-structured interviews with illustrative quotes.

Theme	Sub-theme	Example
Increased comfort with the topic	Better able to describe	“it helps to remind yourself of, of different morphologies and just like, the lingo.”
	Using a bottom-up rather than a top-down approach	“I would just kind of look at something and try to just be like ‘oh it's this diagnosis’ rather than like more describing what it is and then trying to figure out which diagnoses fit that description.”
	Confidence in identifying the movement properly	“if I saw a movement I had to think about was it hyperkinetic, hypokinetic, these are terms that all made sense, I had not thought about it in an organized… process like that before. …And I feel more confident.”
	Establishing a starting point from which to gather more information	“when you look at a movement how do you first think about it, and I think that that is the best thing as a junior learner, is that you can at least categorize it, you can use a resource to get to a diagnosis.”
Format was engaging	Fun	“it was like actually kind of fun to learn.”
	Use of videos and visual aids	“I think that was like very cool, like when you had videos, they were kinda like phasing the image and like pointing out symptoms or signs. So, I think… that was very, very helpful.”
	“Bite-sized”	“so this was something that was bite-sized, enough that you could work on it in like little periods of time and get something out of it.”
	Interactive	“And then I liked that there was an interactive component to sort of test your knowledge and make sure you're getting it along the way.”
	User-friendly	“I thought it was user friendly, easy to understand, easy to navigate.”
	Flexible	“it was helpful to learn on your own time, either at work or outside of work, so it was nice to have the flexibility to do that.”
Accommodated different learning styles	Visual	“So you, it is like, you saw it, it sticks in your mind. Think it has much more value than, than just reading about it.”
	Multimodal	“having it be like multiple modalities of providing information to you so you know you're reading this screen/slide whatever, you're hearing it read to you, there's videos to show you to sort of reinforce the written description, and then there's sort of the interactive component where you're answering questions and getting an explanation as to why it was right or wrong, so I think that it's really good for a busy resident but also people across different learning styles I think can do really well with that.”

The format of the modules was well received. Several participants described the format as “fun”. The visual aspects of the modules contributed to the learners' engagement, as did their short length (“bite-sized”) and interactivity. Learners found that the modules incorporated different learning styles (visual, multimodal) that they did not find to be accommodated by other educational formats such as lectures. The ability of residents to access the modules online, anytime, anywhere, was repeatedly mentioned as a significant advantage.

Finally, participants commented about their desire for more learning materials like this, both within movement disorders and in other neurological subspecialties.

Quantitative results: surveys

Six participants completed the survey (3 PGY2s, 1 PGY3, 2 PGY5s). Individual question and categorical data are summarized in Table 2. Means for all questions were ≥4.0 and 17/18 question means were ≥4.5. There was no statistical difference between categories (p = 0.32). All participants strongly agreed with the following enthusiasm statements: “completing the modules was a good use of my time” and “I would recommend the modules to a resident in another neurology training program, or to next year's residents.” All participants agreed or strongly agreed that they enjoyed completing the modules. All participants agreed or strongly agreed with all knowledge statements, with the exception of one participant who was neutral that “the modules helped me to finally ‘get’ concepts that I had heard before, but didn't understand.” All participants agreed or strongly agreed with both statements on quantity: that there were the right number of modules and that they were a good duration. Every participant rated every technical aspect of the modules as “very good” or “excellent”.

Table 2

Individual question and categorical means of survey questions (SD = standard deviation).

Category	Question (abbreviated)	Mean (SD)	Categorical mean (SD)
Technical	Video image	4.67 (0.52)	4.74 (0.20)
	Video sound	5 (0)
	Ease of access	5 (0)
	Questions linked to answers	4.5 (0.55)
	Table of contents	4.5 (0.84)
	Text	4.83 (0.41)
	Embedded images	4.67 (0.52)
Quantity	Right duration	4.83 (0.52)	4.75 (0.83)
	Right number	4.66 (0.52)
Enthusiasm	Good use of time	5 (0)	4.88 (0.14)
	Enjoyment	4.83 (0.41)
	Recommend for other subspecialties	4.67 (0.82)
	Recommend for other residents	5 (0)
	Appropriate for learning level	4.5 (0.55)
Knowledge	Understand basic concepts	4.67 (0.52)	4.53 (0.22)
	Understand difficult concepts	4.5 (0.55)
	“Get” concepts didn't understand	4.17 (0.75)
	Confident in diagnosis	4.83 (0.41)

Quantitative results: pre- and post-tests

The mean pre-test score was 0.7 (±0.19) (i.e. mean 70% or 7/10 correct), and the mean post-test score was 0.95 (±0.05). The test for location showed the mean was significantly >0, demonstrating a statistically significant change in score (t = 3.27, p = 0.022). A comparison of those who completed versus those who did not complete the post-test revealed that those who only performed the pre-test had a slightly lower mean score (0.63 vs. 0.7), but this result was not statistically significant (p = 0.58).

Discussion

The neurology trainees in this study responded positively to multiple aspects of the modules-based curriculum. Most universally mentioned was the structured framework provided in the introductory video and reinforced throughout the curriculum. Giving learners a structured framework from which to work in advance of the application phase of learning has been shown to promote deeper understanding of concepts [11]. One theoretical model of learning, the “thinking fast and slow” paradigm, differentiates the cognitive processing in beginners versus experts [12]. The expert uses pattern recognition in a faster method of deduction, while the beginner uses an analytical, systematic approach in a slower process that is less efficient but less error-prone. A new learner using the “fast” approach can lead to error: if a resident doesn't understand that tremors are oscillatory and rhythmic, he/she may have difficulty differentiating tremor from multifocal myoclonus. This has borne out in studies; presenting residents with an organized, terminally-branching algorithm was shown to reduce errors, improve diagnostic accuracy, and lower cognitive load when compared to “hybrid schemes” that forced residents to weigh multiple variables simultaneously [13].

Using qualitative and quantitative analysis, we aimed to determine if the curriculum, a) contributed to an increase in knowledge about how to diagnose and manage movement disorders patients, and b) was engaging and acceptable to housestaff learners. Participants' performance on pre- and post-tests supported that knowledge acquisition took place. Even with a small number of participants completing pre- and post-tests (n = 6), there was a statistically significant increase in score, with every participant scoring higher on the post-test with the exception of one participant who scored equally well on both tests (score = 90%).

On the survey, the mean for all categories was >4.5/5.

Both the survey data and the semi-structured interviews highlighted engagement by learners. In the survey, the enthusiasm category had the highest mean Likert score with the lowest standard deviation (4.88, SD 0.14), including that all participants “strongly agreed” that “the modules were a good use of my time.”

Finally, the flexibility of the modules was appreciated by housestaff, who not only commented that they could complete them on their own time, but also suggested coupling the curriculum as independent study immediately prior to a clinical experience to maximize learning:

“if I could have watched those videos and then gone and seen patients with movement disorders, then I would have been like is this hyperkinetic or hypokinetic, you know I could have directly applied them, that day, it would have been awesome, and it would have really cemented it.”

This resident was unwittingly describing a form of “flipped classroom,” which couples pre-didactic independent study with a didactic or clinical experience that applies that knowledge. This strategy was achieved in the Yale neurology residency program with a foundational training in electroencephalography introduced residents to basic concepts through brief video-based lectures, and then residents advanced their understanding of these concepts during reading sessions with faculty and clinical fellows on their rotations [8].

Limitations to our results include the sample size, and confinement to a single institution which hinders any conclusions regarding generalizability of this curriculum to programs with meaningful differences in size, outpatient vs. inpatient responsibilities, and exposure to movement disorders. Participation was voluntary within the program, which raises concern for participation bias. Although eight of nine participants completed all modules (the remaining participant completed 7/10 modules), not all participants completed the survey and pre- and post-tests, again introducing participation bias. In addition, pre- and post-tests were not randomized, and there was no control group. Therefore, conclusions drawn about knowledge acquisition when comparing pre- and post-test scores are potentially confounded by the possibilities that the tests were of different difficulty levels, that test-enhanced learning was a factor, or that residents learned enough movement disorders in their regular curriculum and clinical activities during the two-month study period to demonstrate a statistically significant increase in score. To address these limitations, we are conducting a multi-center randomized controlled trial to further assess the acceptability, effectiveness, and generalizability of this curriculum.

This pilot curriculum lays the groundwork for understanding how this next generation of neurology trainees view their education, and how the supplementation of their clinical experience could be optimized. Such creative strategies for didactic education may be particularly valuable for outpatient subspecialties that are relatively underrepresented in a typical resident schedule.

Concluding statements

Herein we have described the design and use of a basic curriculum in movement disorders for neurology residents and fellows. The curriculum was created with the millennial learner in mind, focusing on learner preferences and incorporating pedagogical theory. The creation of this curriculum was feasible with the use of a limited supply of available platforms that included Camtasia, PowerPoint, Qualtrics, and WordPress. Trainees responded positively to the curriculum, both in terms of the content and in terms of format. In the future, this and other similar online curricula should be studied on a larger scale at multiple institutions to determine generalizability of this data.

Declaration of competing interests

Nothing to declare.