The Performance and Trajectory of Medical Students With Disabilities: Results From a Multisite, Multicohort Study.

PURPOSE: To conduct a post-Americans with Disabilities Act Amendments Act of 2008 multisite, multicohort study called the Pathways Project to assess the performance and trajectory of medical students with disabilities (SWDs).
METHOD: From June to December 2020, the authors conducted a matched cohort study of SWDs and nondisabled controls from 2 graduating cohorts (2018 and 2019) across 11 U.S. MD-granting medical schools. Each SWD was matched with 2 controls, one from their institution and, whenever possible, one from their cohort for Medical College Admission Test score and self-reported gender. Outcome measures included final attempt Step 1 and Step 2 Clinical Knowledge scores, time to graduation, leave of absence, matching on first attempt, and matching to primary care.
RESULTS: A total of 171 SWDs and 341 controls were included; the majority of SWDs had cognitive/learning disabilities (118/171, 69.0%). Compared with controls, SWDs with physical/sensory disabilities had similar times to graduation (88.6%, 95% confidence interval [CI]: 77.0, 100.0 vs 95.1%, 95% CI: 90.3, 99.8; P = .20), Step 1 scores (229.6 vs 233.4; P = .118), and match on first attempt (93.9%, 95% CI: 86.9, 100.0 vs 94.6%, 95% CI: 91.8, 97.4; P = .842), while SWDs with cognitive/learning disabilities had lower Step 1 scores (219.4; P < .001) and were less likely to graduate on time (81.2%, 95% CI: 69.2, 93.2; P = .003) and match on first attempt (85.3%, 95% CI: 78.0, 92.7; P = .009). Accommodated SWDs had Step 1 scores that were 5.9 points higher than nonaccommodated SWDs (95% CI: -0.7, 12.5; P = .08).
CONCLUSIONS: Structural barriers remain for SWDs with cognitive/learning disabilities, which could be partially mitigated by accommodations on high-stakes exams.

Medical students with disabilities (SWDs) are an important part of a diverse health care workforce. [1-8] A renewed interest in this population is driven—in part—by the 69% growth in medical student disability disclosure since 2016 and the limited data on this population’s long-term success in medical school. [9] The performance and trajectory of this population, including board exam scores, time to graduation, leaves of absence (LOAs), matching to residency, and specialty choice, are important data for faculty, administrators, and curricular designers as they work to create a more inclusive medical education environment. [10-15]

Previous studies have addressed the performance of SWDs but have had limitations, [13,14] including lack of data on United States Medical Licensing Examination (USMLE) Step 1 exam accommodations, the use of data from a single institution, and the use of data from cohorts that entered medical school before the Americans with Disabilities Act Amendments Act of 2008 (ADAAA), [15,16] which expanded the definition of disability and preceded the increase in disability disclosures noted above. [9] These studies [13,14] included a call for future research to investigate the interplay of medical student performance and trajectory by category of disability and accommodation status on the Step 1 exam. In addition, qualitative data suggest a connection between disability status and an avowed interest in entering primary care and focusing on underserved populations. [15] To date, the association between SWDs and entering primary care has not been tested.

Therefore, in this Pathways Project study, we expand and build on prior work, [13,14] conducting a post–ADAAA multisite, multicohort study to assess the performance and trajectory of SWDs in the context of disability category, gender, and accommodation use on Step 1. We also review time to graduation, controlling for dual degree status; whether SWDs were more likely to take an LOA; whether SWDs were less likely to match on first attempt (post–Supplemental Offer and Acceptance Program); and whether SWDs were more likely to match to primary care.

Findings from this study may have implications for the admission of SWDs into medical school and for student support services. Moreover, identification of category-specific performance differences can inform the focus of future research looking at curriculum- and systems-level barriers faced by SWDs.

Method

Using school-level data from previous studies, [9,17] we identified U.S. MD-granting medical schools with the largest percentages of SWDs across all 4 Association of American Medical Colleges geographic regions. We invited the 3 schools with the largest populations of SWDs from each of the 4 regions to join the study; all schools accepted the invitation. Eleven schools provided final deidentified data. One school withdrew, citing COVID-19–related administrative duties.

Between June and December 2020, we conducted a matched cohort study examining the performance and trajectory of SWDs from 2 graduating cohorts (2018 and 2019). These data allowed us to evaluate students’ performance and trajectory in cohorts that matriculated following the ADAAA, which broadened the definition of disability. [16] Inclusion criteria for SWDs included having a disability as determined by the institution’s disability office and receiving accommodations. All authors were masked to the identity of students and each institution followed a strict protocol for data collection to protect student identity, with disability professionals populating a standardized spreadsheet with deidentified data on SWDs. Matching for controls was conducted by each school’s administrators who received a deidentified list of gender and Medical College Admission Test (MCAT) scores of SWDs.

Comparison group matching

Each SWD was matched with 2 nondisabled controls from their institution, which served as comparison groups. For one of the controls, students were matched based on final MCAT scores, aiming for a confidence band of ±2 score points for total scores, which provides a narrower range of scores and a higher precision of matching. Given the research showing gender-based differences in Step 1 performance, [18,19] we also matched on self-reported gender at admission for the other control. To minimize potential cohort (or graduation year) effects, matching was performed within each graduating class whenever possible. Matching on gender was successful for all students. Over 80% of SWDs were matched with peers who had MCAT scores within 2 points of theirs; the remainder were matched within 3 points (12%), 4 points (4%), and 5 points (1%). Similarly, 80% of SWDs were matched within their institution and cohort, while the remainder were matched with students from their institutions but from the alternate cohort.

Data

The study sample consisted of 171 SWDs and 341 nondisabled controls; in one instance, an SWD was only able to be matched with a single nondisabled peer using the defined parameters. Outcome measures included final attempt scores on the USMLE Step 1 and Step 2 Clinical Knowledge (CK) exams; time to graduation, which was considered on time if it was within 5 years of matriculation for MD students, within 6 years for all masters-level dual degree students (e.g., MD–MPH, MD–MBA), and within 8 years for MD–PhD students; whether students took an LOA (yes/no); whether students matched on first attempt (post–Supplemental Offer and Acceptance Program; yes/no); and whether students matched into a primary care specialty (using the broadest definition of primary care to include family medicine, internal medicine, pediatrics, and obstetrics and gynecology [20]).

Time to graduation was set (as noted above) and disability category was dichotomized to allow for comparison with previous studies. [13,14] SWDs with sensory (e.g., deaf or hard of hearing, visual disability), chronic health, and mobility disabilities were categorized as physical/sensory. SWDs with attention-deficit/hyperactivity disorder (ADHD), psychological disabilities (e.g., depression, bipolar, anxiety), and learning disabilities (e.g., dyslexia, other reading disabilities, processing speed disorder) were categorized as cognitive/learning. The 5 SWDs who were unable to be categorized in one of these groups were dropped from the disability category analysis. Further analysis was conducted by separating students from the cognitive/learning group into psychological disabilities and ADHD/learning disabilities; however, this sensitivity analysis did not result in any significant differences in model conclusions; therefore, the 2-group classification of SWDs was retained.

Analysis

Demographic characteristics were summarized for the overall study sample and stratified by SWDs and nondisabled control groups. Prevalence of taking an LOA, graduating within 5 years for MD students (or 6–8 years for dual degree students), and primary care match were compared between SWDs and controls using marginal predicted probabilities (i.e., percentages) and associated 95% confidence intervals (CIs) from mixed-effects logistic regression models, including random intercepts for school and matched groups.

Final attempt Step 1 and Step 2 CK scores were assessed using linear mixed models, which included random intercepts for school and matched groups to account for clustering effects. Unadjusted models included fixed effects by disability group, as both 2-group (SWDs vs nondisabled controls) and 3-group (SWDs with physical/sensory disabilities vs SWDs with cognitive/learning disabilities vs controls) variables. Marginal means from the resulting models were used to compare average values between disability groups. Match to residency on first attempt was analyzed using a mixed-effects logistic regression model, including a random intercept for school only. Adjusted models for Step 1 scores, Step 2 CK scores, and match on first attempt focused on the 3-group variable and the inclusion of effects for gender, MCAT score, and graduating cohort were considered.

A secondary analysis using school-reported use of accommodation on Step 1 was performed. A 3-group variable (SWDs with accommodation on Step 1, SWDs without accommodation on Step 1, and nondisabled controls) evaluated whether accommodation use resulted in SWD scores that were closer to those of the nondisabled controls. A linear mixed model (as described above) with this 3-group variable and gender, MCAT score, and graduating cohort as fixed effects was assessed. Inclusion of fixed effects, random effects, and interactions between disability groups and all other variables in each model was investigated using likelihood ratio tests with an alpha value of 0.10. A significance level of P < .05 was used in determining significant associations. All analyses were performed using Stata 15.1. [21]

The University of Michigan Institutional Review Board deemed the study exempt.

Results

A total of 171 SWDs and 341 nondisabled controls were examined; a small percentage of all students were pursuing dual degrees (44/512, 8.6%) and the majority of SWDs had cognitive/learning disabilities (118/171, 69.0%), which is consistent with other studies (Table 1). [9,17,22,23]

Table 1

Demographics of Students in a Post–ADAAA Multisite, Multicohort Study Assessing the Performance and Trajectory of SWDs, 11 U.S. MD-Granting Medical Schools, June–December 2020

LOA and time to graduation

Among SWDs, 31.8% (95% CI: 20.4, 43.3) took an LOA compared with 10.7% (95% CI: 5.0, 16.4; P < .001) of nondisabled controls based on the model adjusting for clustering by matched pairs and school. SWDs in both disability categories had higher probabilities of taking an LOA than controls, including 34.5% (95% CI: 21.5, 47.4; P < .001) of SWDs with cognitive/learning disabilities and 29.2% (95% CI: 12.7, 45.6; P = .014) of SWDs with physical/sensory disabilities. There was not a significant difference between SWDs with cognitive/learning and physical/sensory disabilities (P = .52).

Modeled probabilities found that SWDs were less likely to graduate on time (i.e., within 5 years for MD students and within 6–8 years for dual degree students) than controls (84.2%, 95% CI: 73.9, 94.4 vs 95.1%, 95% CI: 90.3, 99.8; P < .001). This difference was driven mainly by those with cognitive/learning disabilities, who had an 81.2% (95% CI: 69.2, 93.2) probability of graduating on time, which was significantly lower than controls (P = .003). However, those with physical/sensory disabilities had an 88.6% (95% CI: 77.0, 100.0) probability of graduating on time, which was not significantly different from controls (P = .20) or those with cognitive/learning disabilities (P = .21).

Step 1 scores

Unadjusted analysis found significant differences in final attempt Step 1 scores between groups, with all SWDs having lower mean scores than controls (222.0 vs 233.4; P < .001; Table 2). Performance of SWDs with physical/sensory disabilities did not significantly differ from controls (229.6; P = .118), while students with cognitive/learning disabilities had significantly lower scores than controls (219.4; P < .001).

Table 2

Meana Final Attempt Step 1 and Step 2 CK Scores by Disability Groupb in a Post–ADAAA Multisite, Multicohort Study Assessing the Performance and Trajectory of SWDs, 11 U.S. MD-Granting Medical Schools, June–December 2020

Step 2 CK scores

Similar to Step 1 outcomes, all SWDs had lower mean final attempt Step 2 CK scores than controls (236.3 vs 245.0; P < .001; Table 2). However, when examined by disability category, those with physical/sensory disabilities did not significantly differ from controls (243.0; P = .361), while those with cognitive/learning disabilities had significantly lower scores than controls (233.6; P < .001).

Step 1 and Step 2 scores by gender and disability group

Adjusted analysis for final attempt mean Step 1 scores found a significant interaction between disability group and gender (likelihood ratio test P = .03; Table 3). Male students with cognitive/learning disabilities, on average, had mean scores that were nearly 18.0 points lower than nondisabled male controls (95% CI: –22.3, –12.9; P < .001; Table 3) and about 13.0 points lower than male students with physical/sensory disabilities (95% CI: –21.4, –5.3; P = .001; Figure 1). Similarly, female students with cognitive/learning disabilities had significantly lower scores than controls, by about 9.0 points on average (95% CI: –13.7, –5.0; P < .001; Table 3) but did not significantly differ from female students with physical/sensory disabilities (B = –3.1, 95% CI: –10.1, 3.9; P = .380; Figure 1). For males, there was not a significant difference between controls and students with physical/sensory disabilities (B = –4.2, 95% CI: –11.4, 2.9; P = .245; Table 3). However, females with physical/sensory disabilities had scores that were only slightly lower than, but still significantly different from, controls (B = –6.3, 95% CI: –12.4, –0.1; P = .045; Table 3).

Table 3

Linear Mixed Modela Results for Final Attempt Step 1 Scores in a Post–ADAAA Multisite, Multicohort Study Assessing the Performance and Trajectory of SWDs, 11 U.S. MD-Granting Medical Schools, June–December 2020

Figure 1

(Panel A) Final attempt marginal mean Step 1 scores and mean differences by (Panel B) gender and (Panel C) disability group (P/S or C/L) based on linear mixed model results in a post–ADAAA multisite, multicohort study assessing the performance and trajectory of SWDs, 11 U.S. MD-granting medical schools, June–December 2020. Panel A shows the estimated marginal means and corresponding 95% CIs from the linear mixed model for Step 1 scores for each disability and gender group. Panels B and C show the differences in means by gender and disability group, respectively, with corresponding 95% CIs; the 95% CIs that cross the dotted line indicate a nonsignificant difference in means at alpha = 0.05. SWDs with sensory (e.g., deaf or hard of hearing, visual disability), chronic health, and mobility disabilities were categorized as P/S. SWDs with attention-deficit/hyperactivity disorder, psychological disabilities (e.g., depression, bipolar, anxiety), and learning disabilities (e.g., dyslexia, other reading disabilities, processing speed disorder) were categorized as C/L. Abbreviations: P/S, physical/sensory disabilities; C/L, cognitive/learning disabilities; ADAAA, Americans with Disabilities Act Amendments Act of 2008; SWD, medical student with disabilities; CI, confidence interval; NDC, nondisabled control.

On average, males in the control group outperformed females in the control group in terms of their final attempt mean Step 1 score (P = .008; Figure 1). Gender did not significantly influence Step 1 scores in either the cognitive/learning (P = .238) or physical/sensory group (P = .136). As detailed in another publication, [24] we also found that MCAT scores were positively associated with Step 1 scores (P < .001; Table 3).

Group differences in the adjusted results for final attempt mean Step 2 CK scores were consistent with unadjusted findings (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/B205). Males tended to have lower scores than females on Step 2 CK, the opposite of the gender differences in the Step 1 findings. A similar association with MCAT score was found, with higher MCAT scores positively associated with higher Step 2 CK scores.

Step 1 scores by accommodation status

Accommodation information for the Step 1 exam was available for 113/171 (66.1%) SWDs. Of those, 28 (24.8%) received accommodations. When compared with nondisabled controls, nonaccommodated SWDs had average Step 1 scores that were 12.2 points lower (95% CI: –15.9, –8.4; P < .001). Scores for accommodated SWDs remained significantly lower than those for controls, but only by 6.3 points (95% CI: –12.3, –0.3; P = .04). Accommodated SWDs had higher mean scores than nonaccommodated SWDs by 5.9 points on average (95% CI: –0.7, 12.5; P = .08).

Match to residency on first attempt

In unadjusted analysis, SWDs with cognitive/learning disabilities were less likely to match to residency on first attempt than nondisabled controls (85.3%, 95% CI: 78.0, 92.7 vs 94.6%, 95% CI: 91.8, 97.4; P = .009). There were no significant differences between the physical/sensory and control groups (93.9%, 95% CI: 86.9, 100.0; P = .842) or between the cognitive/learning and physical/sensory groups (P = .077). After adjustment for Step 1 score, there were no longer any group differences between SWDs with cognitive/learning disabilities and controls (odds ratio [OR] = 0.56, 95% CI: 0.27, 1.17; P = .124), and Step 1 score was the only significant association, with each 1-point increase in Step 1 score increasing the odds of matching on first attempt by 4% (OR = 1.04, 95% CI: 1.02, 1.07; P < .001; Table 4).

Table 4

Mixed-Effects Logistic Regression Model for Residency Match on First Attempta and Residency Match to Primary Careb in a Post–ADAAA Multisite, Multicohort Study Assessing the Performance and Trajectory of SWDs, 11 U.S. MD-Granting Medical Schools, June–December 2020

Matching to primary care

The physical/sensory group matched into primary care residencies at a rate of 67.0% compared with 54.4% for the cognitive/learning group and 48.6% for the control group. In a model adjusting for Step 1 score, those with physical/sensory disabilities had higher odds of matching into primary care than nondisabled controls (OR = 2.11, 95% CI: 1.05, 4.26; P = .037; Table 4) and those with cognitive/learning disabilities (OR = 2.32, 95% CI: 1.05, 5.15; P = .038; data not shown). There was no significant difference between the cognitive/learning and control groups in their likelihood of going into primary care (OR = 0.91, 95% CI: 0.56, 1.48; P = .700).

Discussion

This study expands understanding of the performance and trajectory of SWDs in U.S. MD-granting medical schools, showing that 2 graduating cohorts of SWDs across 11 institutions generally graduated on time and matched to residency. Mean scores on Step 1 and 2 CK were lower for SWDs compared with nondisabled controls, a finding that supports previous studies. [13,14] However, this difference dissipates for some SWDs when analyzed by disability category. SWDs with physical/sensory disabilities perform similarly to controls, while SWDs with cognitive/learning disabilities score lower than controls but at smaller margins than those reported in a prior study. [14] When SWDs were accommodated on Step 1, scores increased by 5.9 points on average. Though not statistically significant, this increase suggests that accommodations have a measurable impact on Step 1 scores for SWDs. Moreover, this increase eliminates approximately half of the score difference between SWDs and controls, suggesting that lack of accommodation on Step 1 may be a key driver of score differences. Importantly, only about 25% of students in our sample used accommodations on Step 1. Thus, it may be that a lack of access to accommodations on Step 1 contributes to the differences in performance seen for the cognitive/learning disabilities group, a group that represented 69.0% of our SWD sample.

In contrast to leading studies, [18,19] a gender effect for Step 1 scores was found, with males scoring higher than females in the nondisabled control group on Step 1, while females scored higher on Step 2. There were also differences in disability group by gender, with females from both disability groups performing lower than controls on Step 1 but only males with cognitive/learning disabilities differing from controls, though this group’s score difference was much larger. Reasons for these converse findings are not clear and require additional exploration.

SWDs were more likely to take an LOA than nondisabled controls, which may account for the delay in time to graduation noted in a previous study. [13] Our study controlled for students in dual degree programs, which abates the differences in time to graduation for students with physical/sensory disabilities noted in the previous study [13]; however, a significant difference remains for those with cognitive/learning disabilities. Further evaluation was conducted within the cognitive/learning category between psychological disabilities and ADHD/learning disabilities. No significant differences were found, suggesting that LOA and time to graduation are not driven by a discrete type of cognitive/learning disability. It may be that students with cognitive/learning disabilities are negatively impacted by structural barriers within the curriculum, namely time-dependent activities and assessments. Their need for additional time within the program may necessitate a decompression of coursework or additional study time between courses and clinical rotations. For those denied accommodations on Step 1, an LOA may be necessary to address an appeal of the denial or for extra Step 1 preparation time, causing a delay in time to graduation. It may also be that students are not diagnosed with cognitive/learning disabilities until they enter the fast-paced environment of medical school, limiting their experience with requesting and using accommodations and requiring a period of adjustment that necessitates an LOA.

There were no differences between SWDs and nondisabled controls in the proportion of students who matched into residency after adjusting for Step 1 score, confirming previous findings. [14] Given that Step 1 scores influence the likelihood of matching into residency and SWDs have lower Step 1 scores than controls, differences in match results may be indirectly driven by disability status via Step 1 score rather than by disability status alone. When taken together with our finding on accommodated SWD Step 1 scores, it appears that nonaccommodation may have significant implications for residency matching and requires further study.

When compared with nondisabled controls, students with physical/sensory disabilities only significantly differed in the likelihood of taking an LOA and not in time to graduation or Step 1 or Step 2 CK exam scores. We postulate that for students in this category, barriers may be more apparent to administrators and, thus, more easily removed. In addition, these students may have more experience addressing their disability needs (e.g., via requesting accommodation) and with communicating their disability-related needs.

The personal accounts of physicians with physical/sensory disabilities, [25-32] prior studies showing positive performance, [14,15] and findings from this study should quell concerns about the ability of students with physical/sensory disabilities to matriculate, graduate, and match to residency. Moreover, our study finds that students with physical/sensory disabilities were significantly more likely to match to primary care residencies than controls, mirroring the trajectory of other underrepresented groups. [33-36] While interest in primary care may be influenced by demographic characteristics, [36,37] a student’s lived experience, [4] and educational pathway (e.g., rural upbringing, community college), [38-44] it may also be that increasing the representation of physicians with physical/sensory disabilities could have a direct and positive impact on the number of physicians practicing primary care.

Our study has limitations. First, while we attempted to control for variance by matching for MCAT score and gender, we were unable to control for race and ethnicity because these data were not available from all schools. Sampling from schools with higher percentages of SWDs may introduce bias based on the culture, climate, and robust nature of disability support at these institutions. Only final attempt Step 1 scores were analyzed; however, this was done to increase the odds of capturing the impact of accommodation use, which is sometimes only granted after Step 1 failure. Finally, there were incomplete data on Step 1 accommodation provided by the institutions, as the National Board of Medical Examiners does not report accommodation decisions to schools, which may dilute the significant impact of this variable on performance and trajectory. Importantly, experiences within the categorically defined disability groups we used may vary in ways that were not captured by our dichotomization, although subgroup analysis of the cognitive/learning group produced results that were reassuringly similar to those of the larger group. This dichotomization was necessary to ensure confidentiality for students with low prevalence disability types, to improve the power of the study, and to allow for comparisons with previous studies. Further, our study was not designed or powered specifically on the split of SWDs into the disability groups, and the small sample size of the physical/sensory group could contribute to some of the nonsignificant findings when compared with controls. Finally, our study only analyzed data from graduated cohorts and did not include data on attrition; however, the overall medical school attrition rate is low (3.3%) and, thus, is unlikely to significantly impact our findings. [45]

Conclusions

Inclusion of SWDs is an important step to further diversifying the physician workforce, with implications for informed care for patients with disabilities. [1] This study expands on prior research, [13,14] demonstrating gaps in performance on the Step 1 and Step 2 CK exams for SWDs and greater time to graduation for students with cognitive/learning disabilities. These performance differences may be partially due to nonaccommodation on the Step 1 exam, disability category–specific stigma that discourages disclosure of disability and request for accommodation on board or licensing exams, or fear of the impact of disclosure on residency application. [46,47] Our findings heighten concerns about the structural barriers in medical education for students with cognitive/learning disabilities and the supports and structures needed to ensure equitable access for this population. [48] As the prevalence of SWDs increases, [9] schools must investigate and address structural inequities in medical education and access to accommodations on high-stakes exams to ensure that assessments are fully accessible, thereby better representing the actual knowledge of SWDs.

Acknowledgments:

The authors would like to thank the participating institutions and the following individuals for their partnership in collecting data and contributing to critical reviews of this article. These individuals did not receive any financial compensation for this work: Joanna Arnold, Barbara Blacklock, Jaime Bograd, Patrick Bridge, Sarah Scott Chang, Tonya Fancher, Mark Grichanik, Erin Griffin, Beth Holman, Allison Kommer, Carleigh Kude, Marie Lusk, Emily Magee, Mikiba W. Morehead, Charlotte O’Connor, and Richard D. Peppler.