An observational study of an approach to accommodate a fourth-year to third-year neurology clerkship curricular transition.

Background: Curricular transformation can result in bulges in students' clinical placements.Objective: To report on learner outcomes associated with a competency-based opt-out approach for a required 4th-year neurology clerkship.Methods and Study Design: During Oregon Health & Science University's recent undergraduate medical education curricular transformation, a 4-week required neurology clerkship transitioned from the fourth-year to the third-year in academic year 2016-17. Because this would have resulted in the neurology clerkship accommodating double enrollment for an entire academic year, 4th year medical students from the prior curriculum (graduating class of 2017) were offered the option of opting-out of the required neurology clerkship if they demonstrated competency by passing the USA National Board of Medical Examiners (US-NBME) clinical neurology subject examination and completing a neurology faculty-observed history and complete neurological examination.
Results: Fifty-seven of 133 fourth-year students (42.9%) chose to complete the required neurology clerkship with 77 (57.9%) choosing to opt-out. All opt-out students passed the neurological exam assessment and scored similarly to the students who took the clerkship (US-NBME Neurology Subject Exam mean raw score in the opt-out group 76.9 vs, 77.6; p = 0.61). Students grades did not differ. Students who opted-out tended to pursue surgical careers (e.g., general surgery -10.8% opted-out vs 0% clerkship, OB/GYN - 6.8% opted-out vs 0% clerkship, orthopedic surgery 5.4% opted-out vs 0% clerkship) where those who took the clerkship tended to choose medical residency training disciplines (family medicine -16.1% clerkship vs 10.8% opting-out; internal medicine - 32.1% clerkship vs 14.9% opting-out; psychiatry 10.7% clerkship vs 2.7% opting-out (p = 0.042)
Conclusion: While undertaking the neurology clerkship would have been the desired approach, students appear not to have been harmed by the opt-out approach regarding performance on the US-NBME clinical neurology subject exam. Choices regarding opting-out versus taking the neurology clerkship appear to be associated with career choice.

Introduction

Innovations in medical education are occurring across the physician training continuum [1-3]. Altering the structures of both undergraduate and graduate medical education makes sense given rapid changes in clinical medicine occurring over the past several decades [2,3]. Changes to the amount or integration of basic science has resulted in several educational innovations [4-6], many of which include compressing basic science education into a shorter time period early in training, with some programs offering integrated material during the clinical years [7,8].

Educational innovations can greatly affect learners’ schedules. By shortening the duration of basic science education between a new and a prior curriculum, a ‘bulge’ is created in students’ clinical placements that can be challenging to overcome, given the reality of competing demands for clinical patient care and learner supervision [9]. Ensuring suitable patients exist in a given setting, the need for more diversity in clinical settings, and lack of preceptor familiarity with newer curricular models/teaching approaches have been studied [10-12]. What is lacking from the literature are studies that progress beyond the management of student placements toward researching learner outcomes when innovative options to reduce the bulge are undertaken.

The objectives of this paper are to: 1) describe a novel approach we undertook for handling a bulge in clinical placements as part of a 4th year required neurology clerkship; 2) describe the competency-based methods and analyses we undertook to compare students who took the neurology clerkship to students who opted out; and 3) report our findings from this observational study, including comparisons of career choice among students’ who took the neurology clerkship to students who opted out, which further informs their choices regarding the neurology clerkship.

Methods

Oregon Health & Science University (OHSU) transformed to a new time-varying competency-based curriculum, called ‘YourMD’ in 2014. The redesign included more flexibility for elective time, enrichment opportunities that allowed students to explore different disciplines in medicine, and tailored coaching toward fostering the development of master adaptive learners [13-15]. The schedule for this transformation was dictated, in part, by receipt of an American Medical Association Grant, Accelerating Change in Medical Education [16] where OHSU’s School of Medicine committed to rapidly transform to an entirely new curriculum in the first year of the grant, and graduating its first class in 2018. As part of the transformation, the first two years in the prior curriculum were reduced to 18 months, which resulted in doubling of the number of medical students per clinical rotation for all established 3rd year required clerkships between February and May of 2017, when the overlap of the new and old curriculum were occurring. Thus, the initial bulge for OHSU occurred over a 12-week period. Prior to the transition, the required curriculum occurred during the 4th year of training. The curricular format for this clerkship consisted of a four week rotation with both inpatient and outpatient experiences. Students were typically assigned to two weeks on a general neurology resident team, one week on a neurocritical care inpatient service, and one week in a neurology clinic. Students’ required assignments included an observed neurological history and physical examination, a written mechanism of disease case report based on a patient seen on the rotation, and the U.S. National Board of Medical Examiners (US-NBME) shelf examination in clinical neurology. Summative evaluation for a 5-tiered grade was assessed based on clinical evaluations of the student by faculty and residents, rubric scoring on the case report, and a national percentile score on the NBME shelf examination. The observed neurological history and physical examination was completed by a faculty member and successful accomplishment by the student was needed for them to receive a passing grade on the rotation.

As part of the curricular transition this four-week neurology clerkship was moved from the fourth year to the third year of the program, which further intensified the need for clinical placements. A solution to the ‘bulge’ in student placements proposed by the neurology clerkship director (author JAK) was to offer 4th year medical students, those in the prior curriculum, the option of opting-out of the required neurology clerkship as long as they passed the (US-NBME) clinical neurology subject examination and completed a neurology faculty-observed history and complete neurological examination. The goal was to reduce learner load by half of the graduating class while allowing for assessment of competency in clinical neurology. This proposal was approved by the OHSU Undergraduate Medical Education curriculum committee, the Faculty Senate, and by OHSU’s Institutional Review Board (IRB # 10873).

OHSU’s opt-out students were given access to all clerkship online resources including the American Academy of Neurology medical student self-assessment exam to prepare for assessments. Access to these materials were evaluated by counting the number of downloads that occurred. Several aspects of the innovation were assessed by comparing those who opted-out versus those who decided to undertake the neurology clerkship. Variables included student demographics, including age and gender; academic performance, including USA Medical Licensing Examination (USMLE) Step 1 raw scores and pass/fail rates, the US-NBME internal medicine and clinical neurology subject exam scores, student grades, frequency of course materials and lecture resources accessed, and chosen residency training disciplines.

USMLE Step 1 assesses learners’ understanding and ability to apply important concepts of sciences basic to the practice of medicine, with special emphasis on principles and mechanisms underlying health, disease, and modes of therapy [17]. While the USMLE program does not disclose how the three-digit score is calculated, most examinees score between 140–260, the passing score is 194 and the national mean and standard deviation are approximately 229 and 20, respectively [17].

US-NBME subject examinations are achievement tests that require medical students to solve scientific and clinical problems and are constructed to assess a broad range of curricular approaches [18]. They also provide comparisons to students’ scores across the USA.

The USMLE Step 1 and US-NBME internal medicine and clinical neurology subject examinations were blinded according to whether students opted in or out of the neurology clerkship. The observed neurological examination was assessed using a checklist of major elements of the neurological examination (Mental Status Exam, Cranial Nerves, Motor Exam, Sensory Exam, Coordination Exam, Station and Gait Exam). The examiner was asked to initial the form if the student performance demonstrated competence on each part of the examination and written comments were provided to students as feedback.

Examiners were not blinded to which group the students they were assessing were in. The reason for this is that students who took the established rotation had traditionally been evaluated by the attendings with whom they were working on the inpatient ward. To evaluate students who opted out, we created a scheduling system to conduct the observed neurological examination. Thus, faculty doing these assessments were aware that these students had opted out.

Data analyses

Descriptive statistics, including means, standard deviations and frequencies were used to characterize the study population and their assessments. Continuous variables (e.g., mean age and test scores) were assessed using independent samples t-tests, and categorical variables (e.g., gender and course materials accessed) were assessed using Chi Square. All tests were two-tailed with the alpha level for determining statistical significance set at 0.05. All analyses were conducted using SPSS version 25.

Results

Fifty-seven of 133 fourth-year students (42.9%) in academic year 2016–2017 chose to complete the required neurology clerkship with 77 (57.9%) choosing to opt-out (Table 1). Students who chose to opt-out were enrolled in the clerkship for the entire 11–12 week time period and were required to complete all assessments within that term. Opt-out students were allowed to either concurrently pursue residency interview visits or took electives during this time period. Characteristics of students in terms of age, gender, USMLE pass rates, and the U.S. National Board of Medical Examiners (NBME) Internal Medicine subject exam scores did not differ among the students who opted in compared to those who opted out, though raw USMLE Step I scores were statistically higher among students who opted-out versus those who took the clerkship (Mean score of 231.1 vs 222.9; p = 0.01) (Table 1). Students in the opt-out group accessed fewer course locally-derived resources compared to those taking the clerkship with one exception. The percent of students in the opt-out group who never accessed main course materials was 88.2% compared to 66.7% of those who underwent the clerkship (p = 0.04), and the mean number of times students in the opt-out group accessed the materials, among those who did access them, was <1, where the mean number of times those taking the clerkship accessed these materials was 10.8; p = 0.008). Similarly, the percent of opt-out students who never accessed the OHSU Lecture Resources was 75% compared to 45.6% of those who took the clerkship (p = 0.01), and the mean number of times those resources were accessed among the students who opted out was <1 compared to a mean of 2.9 among the students who took the clerkship (p = 0.002). Opt-out students accessed the online American Academy of Neurology (AAN) self-assessment medical student examination, a 100-item formative knowledge-based assessment tool, similarly to those taking the clerkship (1.9 compared to 2.9 respectively; p = 0.11)

Table 1.

OHSU medical student characteristics and course material access behaviors according to whether they took the neurology required clerkship or chose to opt-out

Assessment Variables	Neurology Clerkship Students (n = 57)	Option B Test Out Students (n = 76)	p value	Test Statistic(df~)
Medical Student Characteristics
Age	Mean (SD††)30.3 (5.2)	Mean (SD)30.1 (3.3)	0.89*	t = 0.137 (df = 131)
GenderMaleFemale	n (%)27 (47.4%)30 (52.6%)	n (%)45 (59.2%)31 (40.8%)	0.18†	Chi-Square = 1.840(df = 1)
USMLE Step 1 Raw Score	Mean (SD)222.9 (18.6)	Mean (SD)231.1 (18.2)	0.01*	t = −2.618 (df = 120)
USMLE Step 1 Pass/FailPassFail	n (%)54 (94.7%)3 (5.3%)	n (%)74 (97.4%)2 (2.6%)	0.43†	Chi-Square = 0.623(df = 1)
NBME IM Subject Exam Score	Mean (SD)74.1 (8.4)	Mean (SD)74.2 (9.4)	0.93*	t = −0.089 (df = 131)
Course Materials Accessed
# Times Main Course Materials AccessedNoneMean #, if not 0Range, not including 0	n (%)38 (66.7%)Mean (SD)10.8 (28.3)Range2-185	n (%)67 (88.2%)Mean (SD)<1 (2.1)Range1-14	0.04†0.008*	Chi-Square = 9.051(df = 1)t = 3.162 (df = 131)
# Times OHSU Lecture Resources AccessedNoneMean #, if not 0Range, not including 0	n (%)26 (45.6%)Mean (SD)2.5 (3.5)Range1-15	n (%)57 (75%)Mean (SD)<1 (2.0)Range1-11	0.01†0.002*	Chi-Square = 11.989(df = 1)t = 3.435 (df = 131)
# Times Online AAN Neurology Resources AccessedNoneMean # (SD†), if anyRange, not including 0	n (%)19 (33.3%)2.9 (4.4)Range1-27	n (%)37 (48.7%)1.9 (2.7)Range1-10	0.45†0.11*	Chi-Square = 3.149(df = 1)t = 1.607 (df = 131)

*Independent samples t-test

†Chi Square

~ df-degrees of freedom

All opt-out students passed the neurological exam assessment (Table 2) and scored similarly to the students who took the clerkship (US-NBME Neurology Subject Exam mean raw score in the opt-out group 76.9 vs, 77.6; p = 0.61). Students grades did not differ either. Students who opted-out tended to choose surgical residency training disciplines (e.g., general surgery −10.8% opted-out vs 0% clerkship, OB/GYN – 6.8% opted-out vs 0% clerkship, orthopedic surgery 5.4% opted-out vs 0% clerkship) where those who took the clerkship tended to choose medical residency training disciplines (family medicine −16.1% clerkship vs 10.8% opting-out; internal medicine – 32.1% clerkship vs 14.9% opting-out; psychiatry 10.7% clerkship vs 2.7% opting-out (p = 0.042) (Table 2).

Table 2.

OHSU medical student performance outcomes according to whether they took the neurology required clerkship or chose to opt-out

Medical Student Outcomes	Neurology Clerkship Students (n = 57)	Option B Test Out Students (n = 76)	p value	Test Statistic (df~)
US-NBME Neurology Subject Exam ScoreRaw ScorePercentile	Mean (SD)77.6 (8.7)49.6 (32.4)	Mean (SD)76.9 (7.0)45.3 (26.3)	0.61*0.41*	Raw t = 0.43 (df = 131)% t = 0.745(df = 131)
Student GradesHonors (%)Near Honors (%)Satisfactory (%)Marginal (%)Failure (%)	n (%)16 (28.1%)12 (21.1%)28 (49.1%)1 (1.8%)0	n (%)11 (14.3%)22 (28.6%)38 (49.4%)5 (6.5%)1 (1.3%)	0.19†	Chi-Square = 5.582(df = 4)
Residency Training Disciplines	n (%)	n (%)	0.042†	Chi-Square = 25.612(df = 15)
Anesthesia	2 (3.6%)	5 (6.8%)
Emergency Medicine	5 (8.9%)	11 (14.9%)
Family Medicine	9 (16.1%)	8 (10.8%)
General Surgery	0	8 (10.8%)
Internal Medicine	10 (32.1%)	11 (14.9%)
Medicine Preliminary	5 (8.9%)	6 (8.1%)
OB/GYN	2 (3.6%)	5 (6.8%)
Orthopedic Surgery	0	4 (5.4%)
Otolaryngology	0	2 (2.7%)
Pediatrics	4 (7.1%)	2 (2.7%)
Psychiatry	6 (10.7%)	2 (2.7%)
Neurology	1 (1.8%)	0
Surgery Preliminary	2 (3.6%)	5 (6.8%)
Transitional	1 (1.8%)	3 (4.1%)
Pathology	1 (1.8%)	1 (1.4%)

*Independent samples t-test

†Chi Square

~ df-degrees of freedom

Discussion

This study is among the first, to our knowledge, to rigorously study a solution to manage a planned, temporary one-year expansion in medical student clinical placements to accommodate a required clerkship’s curricular move. We learned it was feasible to adequately reduce 4th year medical student clerkship monthly enrollment by offering the four-week neurology clerkship or opt-out by completing a series of required assessments. We considered the assessments we chose as adequate for assessing minimal competency levels for knowledge and clinical examinations needed to care for patients with neurological disease. We worked hard to collect detailed data that would help us understand the potential impact of this solution for dealing with a significant bulge in student placements.

We learned that over half of 4th year medical students did choose to opt-out, which did indeed reduce enrollment numbers so we were able to successfully place all 4th year students who opted in and all third-year students in clinical sites throughout the 2016–2017 academic year. Those students who chose to opt-out did not differ from those who chose to take the neurology clerkship in terms of age, gender, USMLE pass rate or National Board of Medical Examiners (US-NBME) Internal Medicine Shelf Exam scores. We did find that raw USMLE Step 1 scores were statistically higher among the students who chose to opt-out. It may be that those with higher raw Step 1 scores chose to opt-out because they felt less pressure to take the clerkship given their higher scores on this important licensing exam. This possibility is reinforced by the lower level of accessing course materials related to the Clerkship that occurred among the opt-out students as well as the lack of statistical difference in NBME Neurology Self-Exam Scores. It is possible that opt-out students accessed other materials to prepare for this shelf exam, the assessment of which was beyond the scope of this study. This finding may also have been influenced by the fact that students who opted-out were more likely to be pursuing surgical sub-specialty residency programs and those students tend to have higher USMLE scores due to their competitive nature.

We found it interesting that students who decided to take the neurology clerkship were more likely to choose career paths in the medical specialties, such as internal medicine, family medicine and psychiatry, whereas those interested in surgical professions, such as OB/GYN, orthopedic surgery and general surgery tended to opt-out. We were surprised that those who choose emergency medicine tended to opt-out of the neurology clerkship, given that neurological disorders are a fairly common reason for admission to emergency rooms and are associated with significant morbidity and mortality [19]. Perhaps it is the referral nature of many aspects of emergency medicine that led those who chose this career path to opt-out, given that between 8% and 15% of all patients admitted to the ER in general hospitals require assessment by a neurologist [20]. Another issue that may be affecting our findings is ‘neurophobia,’ where medical students perceive that neurology is the most difficult medical specialty [21] and they often struggle with the integration of basic and clinical sciences in this discipline. It may be that neurophobia is negatively influencing medical students’ selection of neurology as a future career [22]. Strategies to overcome neurophobia have been highlighted in a recent publication on evidence-based strategies [22]. Unfortunately, we did not include questions about neurophobia in our study and so cannot comment directly on its potential impact.

Our internal institutional approvals by the Curriculum Committee and Faculty Senate for this approach to handle students’ clinical placements were likely related to the fact that the Liaison Committee on Medical Education (U.S. medical school accrediting body) does not require medical schools to offer required neurology clerkships, though a recent report from the Association of American Medical Colleges revealed that in academic year 2016/2017 that 86% of medical schools had a neurology clerkship requirement [23]. This has been an increasing trend over the past decade, suggesting that core neurology skills are perceived as needed by the vast majority of medical schools in the USA. Though opting out of a four-week neurology clerkship was not optimal, sometimes difficult choices need to be made during curricular transformations, and we in no way would advocate for dropping such an important clerkship from any curriculum. Strengths of this study include the very recent detailed data collected on an entire cohort of medical students, including their career choices. While other studies have reported on managing clinical placements [10-12], these prior studies have focused on theoretical aspects [10], staff perspectives [11] or international experiences [13] without reporting detailed educational outcomes, as we have done here.

Limitations include that a single medical school was involved, which limits generalizability. Also, we did not include students’ ethnic or socioeconomic status as variables in analyses, which may have influenced career choice and other variables. Lastly, there is no long term follow up available at this time regarding how the decision to opt-out affected students’ future care of patients with neurologic illness. As mentioned, we are not advocating for routinely replacing a neurology clerkship with an opt-out approach; however, this model or one like it could be replicated elsewhere, for medical schools undertaking curricular transitions that result in a surplus of students beyond available clinical placements.

In conclusion, it is feasible to implement an opt-out versus undertaking a neurology clerkship to manage planned expansions in students’ clinical placements and rigorously studying student outcomes associated with it. Students do not appear to have been harmed by the opt-out approach in terms of their performance on the NBME Neurology Shelf Exam, their history taking or physical examination skills, or their grades. Their choice regarding opting-out or taking the neurology clerkship appear to be associated with their career choice.

Table A1.

Author contributions

Name	Location	Role	Contribution
Jeff Kraakevik, MD	Oregon Health & Science University, Portland, OR	Author	Study concept and design, acquisition of data, analysis and interpretation of data, drafting of initial manuscript, and approval of final version
Meredith Frederick, MD	North Colorado Medical Center, Banner Health, Greeley, CO	Author	Study concept and design, acquisition of data, critical revision of manuscript for intellectual content, and approval of final version
Nicole Ryan, BS	Oregon Health & Science University, Portland, OR	Author	Acquisition of data and critical revision of manuscript for intellectual content, and approval of final version
Leslie Haedinger, MS	Oregon Health & Science University, Portland, OR	Author	Acquisition of data and critical revision of manuscript for intellectual content, and approval of final version
Patricia Carney, PhD, MS	Oregon Health & Science University, Portland, OR	Author	Study concept and design, analysis and interpretation of data, drafting of initial manuscript, and approval of final version