The Cost of Board Examination and Preparation: An Overlooked Factor in Medical Student Debt.

Objective Board examinations in undergraduate medical education are imperative for competency assessment and a standard licensure process. While the cost of attendance and mean indebtedness of medical students have been quantified, the financial burden experienced by medical students from board preparation and examination has never been quantified. Materials and methods A total of 290 fourth-year osteopathic medical students from 38 osteopathic medical schools completed an anonymous survey that asked them to select the resources they had purchased for board preparation. Along with demographic information, respondents were asked which board examinations they had taken during their medical school education. The price for each resource was located by going to the resource website and finding the "list price" of a brand-new copy/version of that resource. If a price was not found, a current Amazon.com "list price" was utilized. These prices best approximate the maximum a student would spend per resource. Response and statistical analysis such as analysis of variance, post hoc comparison (Scheffé and Bonferronis test), and chi-square tests were conducted using the Statistical Package for Social Sciences (SPSS) Statistics, version 25.0 (IBM SPSS Statistics, Armonk, NY). Results This study found that osteopathic medical students spent, on average, $7,499 (s.d.=$2,506) for board preparation and examination. This cost when isolated is $3,370 for the cost of taking board examinations and $4,129 for the cost of board preparation. Respondents from the West were found to spend most at $9,432, while students from the Northeast spent the least, $7.090. Additionally, non-traditional medical students, those who matriculated after the age of 30 were found to spend more than individuals who began when they were under the age of 25 or between the ages of 25-30. The two most commonly used resources for both Level 1/Step 1 and Level 2/Step 2 examinations were COMBANK and UWorld.  Conclusions/relevance This study is the first of its kind to quantify the mean cost of board preparation and examination in undergraduate medical education at $7,499. When considering the mean indebtedness of the osteopathic graduating class of 2017-2018, 2.94% of medical education debt can be attributed to the cost of board preparation and assessments. As competitiveness for graduate medical education increases, individuals will spend more money to ensure a competitive board exam performance, a key selection factor. Stakeholders in undergraduate medical education are encouraged to further understand the interplay between medical student debt and the cost of board examinations and preparation.

Introduction

With every passing year, medical student debt increases [1]. A majority of medical students take out loans to pay for medical school tuition and fees, learning materials, and housing and boarding, among other expenses. For the 2017-2018 school year, the median cost of attending an osteopathic medical school was $222,972 for public schools and $261,133 for private schools [2]. Debt levels are likely to increase given that tuition costs are anticipated to rapidly increase at public medical schools [3].

Although medical student debt has been quantified, there has been scant research isolating the financial cost of board exam preparation that medical students incur during medical school. To become a licensed physician, osteopathic medical students, who comprise about a fifth of all medical students in the United States [4], must complete the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 1, Level 2 Cognitive Evaluation (CE), Level 2 Performance Evaluation (PE), and Level 3 exams. To increase their competitiveness for residency programs, many osteopathic medical students concurrently take USMLE board examinations, the allopathic equivalent of COMLEX-USA [5]. With an additional board examination to prepare for, osteopathic medical students may opt to purchase more preparation materials. With a single accreditation system for graduate medical education almost implemented, more osteopathic medical students may take the USMLE.

The original intent of the board examination was to ensure competency assessment and a standard licensing process for future physicians. However, the results of board examinations, particularly the United States Medical Licensing Examination (USMLE) Step 1 and COMLEX Level 1, are used by residency programs to identify competitive candidates [5-6]. This has consequently heightened the focus on board examination performance. Companies including, but not limited to, Kaplan Test Preparation and Becker, have capitalized on medical students’ concerns regarding competitiveness on board examinations by offering preparation materials that they claim, if properly utilized, can lead to higher board scores [7].

Medical students also utilize external resources to augment their medical school curriculum [8]. Administrators at medical schools have noticed this trend and sometimes purchase commonly utilized resources such as first aid for the USMLE Step 1 in bulk for their students. Prior research has found that the earlier a student begins board preparation, the higher the USMLE Step 1 score [9]. Moreover, individuals who began board preparation earlier tend to have greater usage rates of first aid for the USMLE Step 1 and complete a higher number of practice questions [9].

With an increased emphasis to perform competitively on board examinations, medical students often purchase a wide array of resources. Resources come in many forms such as bundled preparation programs, review books, question banks, flashcards, videos, and other formats. The objective of this study was to quantify costs associated with board examination and preparation incurred by osteopathic medical students in the United States.

Materials and methods

The sample size of this study comprised (n = 290) fourth-year osteopathic medical students from the continental United States. The Ohio University Office of Research and Sponsored Programs approved this study design and provided Institutional Review Board (IRB) approval. To reach these students, the Student Government Association (SGA) Presidents were contacted via email by the SGA President at Ohio University Heritage College of Osteopathic Medicine. Each SGA President then distributed the survey to their respective student bodies. The survey was administered over a two-week period from February 8, 2018 to February 21, 2018.

In total, 38 osteopathic medical schools and campuses chose to distribute the survey. As a result, the survey was distributed to a total of 6,162 fourth-year osteopathic medical students, out of which 290 osteopathic medical students responded, yielding a 4.71% response rate. Participating schools were located in the four main regions: Northeast (n = 7), West (n = 9), South (n = 12) and Midwest (n = 10; Figure 1).

Figure 1

Geographic locations of the participating osteopathic medical schools

The survey was utilized to obtain representation from the many backgrounds of osteopathic medical students and capture their different perspectives on board examination and preparation. The survey can be found in Appendix A. Qualtrics software (Provo, UT: Qualtrics Inc) was utilized to administer the survey, and the settings ensured no IP addresses were collected to guard anonymity. Furthermore, the survey did not ask for respondents’ names or medical schools. Respondents were given two weeks to participate in the study.

Participants were asked to indicate the region of the United States in which their medical school was located, their gender and race/ethnicity with which they primarily identified, and the age at which they matriculated medical school. These variables were the focus of this study as previous research has found differences in board examinations scores between races, genders, and traditional versus nontraditional students [10-12]. Participants were also asked to indicate their current year in medical school. The options for race/ethnicity were in concordance with the United States Census Bureau.

Respondents were asked about the following board examinations: 1) COMLEX - USA Level 1; 2) USMLE Step 1; 3) COMLEX-USA Level 2 CE; 4) USMLE Step 2 CK; 5) COMLEX-USA Level 2 PE, and; 6) USMLE Step 2 CS. For each exam, the respondents were given four options: 1) never taken but plan to take, 2) never taken and do not plan to take, 3) taken once, and 4) taken more than once.

Respondents were then asked to select all the listed options of resources that they had purchased in preparation for COMLEX-USA Level 1 and/or USMLE Step 1. 162 resources were provided in this comprehensive list. However, due to an oversight error, the resource First Aid for the USMLE Step 1 was not included in the list of options. An option to list “other” resources was also provided on the survey. In addition to review materials, respondents were asked how many practice exams they had purchased in preparation for USMLE Step 1 and/or COMLEX Level 1. The options included a scale from 0 to 6 for NBME Step 1 practice exams and 0 to 3 for NBOME COMSAE Phase 1 practice exams.

Participants were asked to select all the listed options of resources that they had purchased in preparation for COMLEX-USA Level 2 CE, USMLE Step 2 CK, COMLEX-USA Level 2 PE, and USMLE Step 2 CS. 135 resources were listed, and again, respondents were provided with the option of “other” to list other resources they had purchased not listed on the survey. Additionally, respondents were again asked how many practice exams they had purchased in preparation for USMLE Step 2 CK, USMLE Step 2 CS and COMLEX-USA Level 2 CE. The options included a scale from 0 to 3 for NBME Step 2 CK practice exams, 0 to 2 for NBME Step 2 CS practice exams, and 0 to 1 for NBOME Level 2 CE practice exam.

The price for each resource was located by going to the resource website and finding the “list price” of a brand-new copy/version of that resource. If a price was not found, a current Amazon.com “list price” was utilized. These prices best approximate the maximum a student would spend per resource. While an overall cost of board preparation and examination was calculated, this was separated to appreciate the individual cost of (1) board preparation and (2) examination. Responses were analyzed using the Statistical Package for Social Sciences (SPSS) Statistics, version 25.0 (IBM SPSS Statistics, Armonk, NY).

Analysis of variance (ANOVA) was utilized to compare the means of various categories. With the nature of categorical independent variables (e.g., age of matriculation), the Scheffé test was employed to explore post hoc multiple comparisons. Otherwise, Bonferroni post hoc comparisons were utilized. Chi-square test was used to interpret categorical data. The conventional p = 0.05 cutoff was applied to all inferential statistical tests.

By the time osteopathic medical students graduate they spend, on average, $7,499 (s.d.=$2,506) for board preparation and examination.

Results

Respondent demographics can be seen in Figure 2.

Figure 2

Demographic data

This figure illustrates the demographic data of the survey respondents by gender, race/ethnicity, age of matriculation, and regional location of medical school. With regards to Race/Ethnicity, individuals were given multiple options to select all that applied. Due to the low sample count for American Indian or Alaska Native, and Native Hawaiian or Pacific Islander, they were aggregated as "Other".

When the total cost of both taking and preparing for board examinations is separated to 1) cost of taking board exams and 2) cost of board preparation, the impact of each cost is appreciated. The isolated cost of taking board examinations was $3,370 for fourth years, while the cost of board preparation was $4,129 for fourth years (Figure 3).

Figure 3

Timeline of board examinations and the various average costs incurred for taking and preparing for board examinations

This illustration provides a simplistic understanding of when osteopathic medical students take board examinations and the average cost of taking examinations, average cost of board preparation, and the average total cost of both taking and preparing for board examinations.

The cost of taking and preparing for board examinations did not result in much a variation when categories of race were compared. Costs accrued by Whites were $7,464 (n = 224), Black or African Americans spent $7,974 (n = 8), Asians incurred $7,674 (n = 32), and others accrued $7,163 (n = 8). ANOVA and Scheffé post hoc test did not find statistically significant differences (F = 0.34; df = 5; p = 0.892).

Albeit not statistically significant (F = 0.00; df = 2; p = 0.998), when the cost of taking and preparing for board examinations is separated by gender, males and females were found to spend nearly identical amounts with females (n = 158) found to have spent $7,499, while males (n = 132) incurred $7,498.

When respondents were categorized by the region of their medical school, respondents from the West (n = 35) spent the most amount equaling $9,432, while respondents from the Northeast (n = 65) spent the least at $7,090. Individuals from the South (n = 88) spent $7,379 and Midwest respondents (n = 101) accrued $7,199. ANOVA detected statistically significant results (F = 1.98; df = 3; p = 0.015). Bonferroni post hoc comparisons were detected to be statistically significant when comparing West to the other three regions (p < 0.001).

Categorizing respondents by age at the time of matriculation showcased an increasing trend of average cost of board examination and preparation with age. Individuals under the age 25 (n = 188) accrued $7,199, while respondents between ages 25-30 (n = 84) spent $7,721 and participants over the age 30 (n = 18) incurred $9,590. ANOVA detected statistically significant results (F = 8.68; df = 2; p < 0.001).

While the focus was to appreciate the cost of board examination and preparation, a few resources were found to be more frequently chosen by the respondents in their preparation. The five most commonly bought resources for each set of exams are depicted in Figure 4. Out of these 10 popular resources, question banks offered by COMBANK and UWorld for Level 1/Step 1 and Level 2/Step 2 were the most frequently purchased.

Figure 4

The top five resources utilized in preparation for each set of board examinations

Blue highlights the resources used in COMLEX-USA Level 1 and/or USMLE Step 1, while red indicates resources utilized in COMLEX-USA Level 2 CE/PE and/or USMLE Step 2 CK/CS preparation. Percentage of resource frequency is listed in white within each column.

Discussion

Medical student debt, commonly calculated as the total amount of debt accrued by graduating medical students, continues to garner warranted attention in the literature. While the debt continues to rise faster than the rate of inflation, many speculate if this rate is sustainable [9,13]. This issue justifies questioning when increasing debt levels will deter potential applicants [14-15].

Although undergraduate medical education curricula are designed to fully prepare medical students for graduate medical education, students must simultaneously engage in self-directed study for board examinations. To deem a medical student a competitive applicant for a particular residency position, board performance is considered, if not the most, one of the most important qualifying factors [16]. For medical students, the major focus becomes competitive performance on board examinations, leading to substantial monies invested in board preparation material. The cost of board preparation directly affect the expenditures required to complete medical school.

Thus far, there has been limited research on the cost of preparation for board examinations. This study quantitatively assesses the costs of taking and preparing for board examinations incurred by osteopathic medical students, which was found to be, on average, $7,499 (s.d.= $2,506) per student. The cost of board preparation and testing affects the cost of attending medical school and in this study is equivalent to 2.94% of the average student debt in the osteopathic graduating Class of 2017-2018 [2].

While the results of this study indicate that there were no statistically significant differences in the amount of money spent for board preparation and assessments between categories of race, African American or Black respondents spent approximately $500 more than White respondents. This result is in concordance with the current literature. Prior research has found that underrepresented minorities accumulate almost $30,000 more in student debt compared to White students [17]. However, it should be noted that medical student debt reflects many costs incurred by the medical student. Even so, the cost of board preparation and examination are required costs of attending medical school. These required costs aggregate and may disproportionately impact students from lower to middle financial classes, many of whom are first generation or underrepresented minorities in health care. Consequently, the high cost of medical education has been ranked highly by underrepresented minorities as the primary obstacle to pursuing a medical degree [14,18-19].

Areas for further research

Admittedly, one can speculate that osteopathic medical students spend more than allopathic medical students on board preparation since the latter are eligible for only one type of board examination (USMLE). However, a similar research study design could approximate the costs of board preparation incurred by allopathic medical students. In a similar vein, costs should also be measured for international medical graduates seeking to obtain a residency position in the United States. Together, this triumvirate of research foci can more comprehensively examine the impact of board preparation on medical student debt.

Limitations

One limitation to note involves an oversight in the exclusion of First Aid for the USMLE Step 1 in the survey as an option. While 64 respondents identified this resource using the “Other” category, it is anticipated that other respondents who did not specify this resource had purchased it. Given a $55 list price for this resource, the figures calculated may underrepresent the total cost of board preparation by approximately 0.73%, which we believe does not significantly change the results from this study.

While 290 respondents participated in the study, the low survey response rate (4.71%) can possibly be attributed to survey fatigue by medical students, who are regularly asked to participate in surveys. Another reason that might help explain this response rate is that not only are medical students invited to participate in numerous surveys, those surveys offering monetary incentives are more likely to be completed. This survey did not financially reward students for their participation. Furthermore, some of the demographic categories that respondents were asked to identify yielded sample sizes too small to include in the study, potentially compromising the anonymity of participants.

However, this study’s respondent demographics are quite similar to the published survey data of graduating medical students from the American Association of Colleges of Osteopathic Medicine (AACOM), with their respondent demographics comprising of 44.7% female, 84.2% White, 9.0% Asian, and 3.2% Black [2,20-21]. In comparison, our respondent demographics were 54.5% female, 84.1% White, 11.0% Asian, and 2.8% Black. When comparing the age of matriculation, AACOM found 74.5% of students matriculated under the age of 25, with 19.2% between the ages of 25 and 30, and 5.4% commencing medical school after the age of 30, while our respondents were 64.8%, 29.0%, and 6.2% in the aforementioned categories [21]. Due to these similarities, we believe that the low response rate still created a sample size representative of the overall graduating osteopathic medical students, and the results found in this study can be appreciated broadly.

Finally, other variables that may also attribute to the true cost of board preparation and examination include the total costs calculated may not be directly assumed by the respondent in all cases. At some institutions, administrators provide these resources to students at no cost. Regardless, whether directly or indirectly, medical students ultimately bear the burden of these costs.

Conclusions

Quantifying the cost of board examination and preparation sheds light on another direct factor impacting the cost of attending medical school. Osteopathic medical students spend on average $7,499 to ensure preparedness for board examinations. With a single accreditation system almost implemented, more osteopathic medical students anticipate taking the USMLE Step 1 exam. As competitiveness for graduate medical education increases, individuals will spend more money to ensure a competitive board exam performance, a key selection factor. Stakeholders in undergraduate medical education are encouraged to further understand the interplay between medical student debt and cost of board examinations and preparation.