Factors associated with successful passage of the American College of Veterinary Internal Medicine general examination.

BACKGROUND: Board certification relies on passing the American College of Veterinary Internal Medicine (ACVIM) general examination. Pass rates might depend on properties of residency training programs (RTP). HYPOTHESIS: We hypothesized that <4 weeks of dedicated study time, lack of board preparation lectures, status as a re-taker, and private practice RTP would result in lower pass rates of the ACVIM general examination.
SUBJECTS: Two hundred forty-eight ACVIM general examinees.
METHODS: Cross-sectional study. Examinees were surveyed using a Qualtrics survey over a 3-year period. Factors included: study weeks, on-call duty, board preparation lectures, academic or private practice program, and status as a re-taker.
RESULTS: First-attempt examinees were more likely to pass (P < .0001, OR 5.12, 95% CI [2.53, 10.52]). For first-attempt examinees, on-call duty during study weeks resulted in a lower pass rate (P = .002, OR 0.31, 95% CI [0.16, 0.67]). General didactic and specific board-preparation lectures resulted in higher pass rates (P = .003, OR 3.08, 95% CI [1.44, 6.61]; P = .02, OR 3.04, 95% CI [1.20, 7.68]). Diplomate-led board-preparation lectures resulted in higher pass rates than resident-led (P = .007, OR 10.67, 95% CI [1.75, 64.91]). Using a mixed effect logistic model, predicted pass rates were highest with both lack of on-call duty and presence of didactic lectures (predicted pass rate 95%, 95% CI [0.87, 0.98]). CONCLUSIONS AND CLINICAL IMPORTANCE: To optimize pass rates, RTP should provide study time without on-call duty. Provision of didactic lectures and specific board-preparation lectures by diplomates assist in candidate preparation.

American College of Veterinary Internal Medicine

INTRODUCTION

For all candidates in an American College of Veterinary Internal Medicine (ACVIM) specialty residency training program, the General Examination is the first point of assessment of their knowledge base and clinical acumen; it must be passed for a candidate to achieve certification as a diplomate. The ACVIM certification manual provides some guidelines for residency training programs (RTPs), including a mandated 80‐hours of attendance at journal club and a requirement to have completed 18 months of an approved RTP before sitting the General Examination. Certain specialty colleges have additional requirements for the RTP but not general requirements for General Examination preparation. As such, there could be wide variation in RTP candidate‐preparedness including number of weeks allotted for study, emergency or on‐call time during the study period, didactic lectures for examination preparation, among others.

Predictors of success for passing the American Board of Internal Medicine or Pediatrics certifying examinations include: scores on in‐training examinations, number of months of overnight call before the exam, program size (<12 examinees per year), faculty to trainee ratio, and average hours per week of didactic lectures, and the presence of a formal mentoring program. , , Because these certifying examinations occur at a standard date after completion of residency training, there are no “study weeks” allotted for certifying examinations. In the veterinary field, factors associated with passing specialty examinations are largely unreported. Mock board exams are positively correlated with veterinary gross pathology examination pass rates. In veterinary ophthalmology, academically trained residents are more likely to pass the written portion of the national specialty board exam; these residents also have more time off clinical duty overall and more direct supervision on clinical duty.

There were several objectives to this study. The first was to determine the variation in RTP board preparation by soliciting information from the candidates about their RTP relevant to board preparation: number of weeks of study, presence or absence of on‐call or emergency duty during the study period, presence or absence of didactic lectures and specific board‐preparation lectures as well as whether lectures were taught by diplomates/faculty or residents and type of residency training (academic or private practice). The second objective was to seek correlation between the pass rate and these variables, as well as whether a candidate had made previous attempt(s) and the specialty area of the resident. Our hypotheses were that less than 4‐weeks of dedicated study time, the lack of specific board preparation lectures, status as a re‐taker of the examination, and private practice residency training would be associated with a lower pass rate.

MATERIALS AND METHODS

The survey instrument was composed using standardized survey software (Qualtrics). The ACVIM Board of Regents approved distribution of the survey for ACVIM candidates and the study received IRB exemption. The survey was emailed by anonymous link to all candidates sitting the ACVIM general examination in the years 2017, 2018, and 2019. The link was distributed to each candidate class within 3‐6 months of completing the examination. Candidates were given a 2‐week window to complete the survey. At the time that the survey link was distributed, candidates had already received their examination results.

Informed consent was gathered before the start of the survey (Data S1). Consent to complete the survey was indicated by progressing through the survey. Candidates who did not give consent to fill out the survey or elected not to use the anonymous link were not included in data analysis. Information collected from each candidate that consented included: ACVIM ID number, location of residency (private practice or academia), specialty [small animal internal medicine (SAIM), large animal internal medicine (LAIM), neurology, cardiology, oncology, present employment (residency, academia, private practice), whether this was the first attempt to pass the exam, and how many attempts had been previously made. Additionally, for the most recent exam attempt, all candidates were asked about the number of weeks they were given off clinical duty for study purposes, whether they felt that was adequate, whether their RTP provided formal didactic lectures, whether the RTP provided specific board preparation lectures, whether these were provided by diplomates/faculty, residents, or a combination, and whether they were assigned to on‐call or emergency duty during the study period. Finally, for candidates who were retaking the examination, they were asked to rank variables they believed to be associated with previous unsuccessful attempts, including, not enough study time, lack of guidance on what to study, studying the wrong area or using the wrong resources, test anxiety, illness during the examination period, or other (with free choice text).

Candidates were matched by their ACVIM ID number to their status of having passed or failed the examination. Candidates were classified as passing if they passed all portions of the exam taken [up to 2 sections, general‐general and general‐small animal (SA) or general‐large animal (LA)]. They were classified as failing if they did not pass either portion of the exam that was taken in that year.

Candidates were stratified into first‐attempt examinees and repeat examinees. For first‐attempt examinees, a chi‐square test of independence was used to compare variables associated with exam performance (pass or fail). A general mixed effect logistic model was used to predict performance based on all significant predictors.

For candidates that were retaking the exam, a Generalized Estimating Equations approach was used to evaluate the associations since the observations were not independent. Statistical significance was set at P < .05. Statistical analyses were performed using R software version 3.6.1 with additional packages: lme4, and geepack.

RESULTS

There were 262 surveys completed by 248 candidates, 112 in 2017, 65 in 2018, and 85 in 2019. In those years, a total of 681 candidates took the general examination for a response rate of 38%. Sixteen candidates filled out the survey more than once as they did not pass on the first attempt so completed the survey after the first attempt and any retakes.

There were 219 surveys by first‐attempt examinees out of a total of 503 first‐attempt examinees over that period and 41 surveys from retakers out of a total 178 retake examinees. The overall pass rate by first‐attempt examinees that responded to the survey was 84%, 95% CI [78, 89] and 51%, 95% CI [35, 67] for retakers. First attempt examinees were more likely to pass the exam (P < .0001, OR 5.15, 95% CI [2.53, 10.52]). The overall average pass rate for all examinees for the exam over these 3 years was 77%; 26% of these examinees overall were retaking an exam. Surveys from retakers composed 16% of the responses; retakers were slightly underrepresented compared to the overall pool of examinees.

First‐time examinees

Examinee respondents were largely (87%, 191/219) in an academic RTP, with the remainder in a private practice RTP. The proportion of first‐time examinee respondents in academia was slightly higher than of overall examinees, of which 80% were in academic residencies (548/681). The pass rate was not different based on site of training: pass rate 85%, 95% CI [80, 90] in academia vs pass rate 75%, 95% CI [55, 89] in private practice (P = .16).

Respondents' areas of specialization within ACVIM were: SAIM 33% (73/219), pass rate 85%, 95% CI [75, 92]; oncology 22% (49/219), pass rate 92%, 95% CI [80, 98]; LAIM 19% (42/219), pass rate 71%, 95% CI [55, 84]; cardiology 15% (32/219), pass rate 88%, 95% CI [71, 96]; and neurology 10% (23/219), pass rate 83%, 95% CI [61, 95]. There was no effect of specialty on pass rate (P = .11) among respondents. The proportion of respondents in each specialty was similar to the proportion of all examinees in each specialty: SAIM 35%, oncology 17%, LAIM 20%, cardiology 12%, and neurology 17%.

In terms of time off clinical duty for study, 67% (147/219) of the candidates received 4 to 5 weeks, 22% (49/219) received 6 to 7 weeks, 8% (18/219) received 2 to 3 weeks, and 1% (5/219) had either 0 weeks or 8 or more weeks. There was not a significant linear trend between study weeks and pass rates (P = .4), though there were few candidates with fewer than 4 or greater than 7 study weeks (23 total candidates). When the candidates were stratified into <4 or ≥4 study weeks, the pass rate was 75%, 95% CI [64, 99] vs 85%, 95% CI [80, 90], P = .10. Three candidates were not provided time off clinical duty for study; the pass rate of these individuals was 67%, 95% CI [9, 99] (P = .4).

For candidates that received 2 to 3 weeks off clinical duty, 78% (14/18) felt it was far less or less than enough time; 22% (4/18) felt it was sufficient. For candidates that received 4 to 5 weeks off clinical duty, 80% (118/147) felt it was sufficient or more than sufficient. For candidates receiving 6 or more weeks, 86% (44/51) felt it was sufficient or more than sufficient. Candidates that reported feeling that they had less than enough or far less than enough study time had a pass rate of 62%, 95% CI [47, 75] vs 91%, 95% CI [86, 95] for candidates that felt they received sufficient or more than sufficient study time (P < .0001, OR 6.17, 95% CI [2.83, 13.45]).

In terms of on‐call or emergency duty, 77% (169/219) of those had no on‐call or emergency duty during the study period, with the remainder having some on‐call or emergency duty during study. A requirement to perform on‐call or emergency duty was associated with a lower pass rate, 70%, 95% CI [55, 82] vs 88%, 95% CI [82, 93] (P = .002, OR 0.31, 95% CI [0.16, 0.67]).

Most respondents (77%, 168/219) reported that their training programs provided didactic lectures such as pathophysiology lectures, lectures as part of a Master's course, or similar whereas a minority (35%, 77/219) provided specific board preparation lectures. Examinees whose programs provided didactic lectures had a higher pass rate (88%, 95% CI [82, 93]) than those at programs without didactic lectures (71%, 95% CI [56, 83]; P = .003, OR 3.08, 95% CI [1.44, 6.61]).

Candidates were also asked whether their training programs provided specific board preparation lectures. For programs that did provide these, 63% (48/76) of those provided 11 or more hours of lectures, 22% (17/76) provided 6 to 10 hours of lectures, and 14% (11/76) provided 1 to 5 hours of board preparation lectures. Faculty/diplomates provided the board preparation lectures for 62% (47/76) of respondents. Another 26% (20/76) reported that lectures were provided by a mix of residents and diplomates, and the remainder (9/76) reported that lectures were provided by residents only. Examinees provided with board preparation lectures were more likely to pass (92%, 95% CI [85, 97]) vs examinees without board preparation lectures (80%, 95% CI [72, 86], P = .02, OR 3.04, 95% CI [1.20, 7.68]). However, when lectures were provided by residents, the pass rate was lower (67%, 95% CI [30, 93]) as compared to board preparation lectures provided by faculty/diplomates (94%, 95% CI [82, 99]) or a mix of both residents and faculty/diplomates (100%, 95% CI [83, 100]; P = .007, OR 10.67, 95% CI [1.75, 64.91]).

Most respondents, 44% (97/219), received didactic lectures without specific board preparation lectures. The pass rate among these candidates was 85% (95% CI [76, 91]). The pass rate with both didactic and specific board preparation lectures (32%, 71/219 of first‐time examinees) was the highest at 93% (95% CI [84, 98]). Few respondents had received board preparation lectures without didactic lectures (3%, 6/219) and had a pass rate of 83% (95% CI [36, 100]). The absence of any lecture was associated with a pass rate of 69% (95% CI [53, 81]). The presence of didactic or specific board preparation lectures increased pass rates (P < .01, OR 3.29, 95% CI [1.51, 7.17]).

For development of the mixed effect logistic model to predict exam performance in first‐attempt examinees, the following predictors were evaluated: presence of emergency or on‐call work during the study period, presence of didactic lectures, presence of board preparation lectures, and the source/provider of board preparation lectures. The final model included presence of emergency/on‐call work and the presence of didactic lectures (on‐call/emergency duty P = .008, didactic lectures P = .03). Examinees with didactic lectures during their residency training program and no on‐call or emergency work during the study period had the highest predicted pass rates of 95%, 95% CI [87, 98]. Examinees with no didactic lectures that also had on‐call duty during the study period had the lowest predicted pass rates of 53%, 95% CI [32, 72].

Repeat examinees

There were 41 surveys by candidates who had made 1 or more previous attempts to pass the examination. The overall pass rate by examinees retaking an exam was 51%, 95% CI [35, 67]. When asked to rank the factors associated with a previous failed attempt(s), the most cited reason was lack of guidance on what to study, listed as the most important factor by 44% (18/41) respondents, followed by not enough time off clinical duty to study and prepare, listed as the most important factor by 27% (11/41) respondents (Table 1). Other reasons cited as free text included: disproportion of certain specialties on the exam, poor mentorship, need to study for >1 exam, and personal issues.

TABLE 1

Self‐described causative factors for previous examination failures

Reason	Number of candidates ranking as number 1	Average rank among candidates
Lack of guidance on what to study	18	2.0
Not enough off clinic study/preparation time	11	2.8
Studied the wrong topic areas or used the wrong resources	4	2.9
Test anxiety	6	3.2
Other	4	4.9
Illness during examination	0	5.3

Note: Candidates that had previously taken the examination were asked to rank factors they believed to be reasons their previous attempt(s) had been unsuccessful, with 1 being the most important factor and 6 being the least important. Factors in “other” that were ranked as the most important or second most important factor included: specialty not being equally represented on the exam (2 respondents), and with 1 respondent each, poor mentorship, personal issues, requirement for taking 2 examinations, and lack of simulated test questions by mentors.

Most of the repeat examinees had been trained in an academic RTP (83%). There was no difference in the pass rate between those trained in academic institutions and those trained in private practice. Candidates retaking the exam were in the specialty of LAIM (49%, 20/41), oncology (20%, 8/41), SAIM (17%, 7/41), neurology (12%, 5/41), and cardiology (2%, 1/41). Candidates retaking the exam were largely employed in private practice (71%, 29/41, pass rate 45%, 95% CI [27, 63]), with 8 candidates in academia (pass rate 63%, 95% CI [29, 96]) and 3 candidates not currently employed (pass rate 100%, 95% CI [0, 100]). One respondent was currently in a residency training program at the time of the examination. There was an effect of current employment on pass rates, with respondents in private practice having the lowest pass rate (P < .0001).

Among repeat examinees, 46% of respondents were required to do on‐call or emergency duty during the study period (pass rate 42%, 95% CI [20, 67]). The pass rate of respondents without emergency duty was 59%, 95% CI [36, 79] (P = .21).

Seven respondents were not given time off clinical duty for study with a pass rate of 29%, 95% CI [4, 71]. All but 1 of these respondents was working in private practice at the time of the examination. The 7th was currently unemployed, and passed the examination. The pass rate of the remaining 34 candidates given time off clinical duty was 56%, 95% CI [38, 73]. Time off clinical duty did not significantly affect pass rates among repeat examinees (P = .22). The number of weeks of study did not significantly affect pass rates, though there were few candidates in each category: pass rate 71%, 95% CI [42, 92] for 14 repeat examinees with 4 to 5 weeks off clinics; pass rate 67%, 95% CI [9, 99] for 3 examinees with 6 to 7 weeks of study; pass rate 43%, 95% CI [18, 71] for 14 examinees with 2 to 3 weeks off clinics; pass rate 33%, 95% CI [1, 90] for 3 examinees given 8 to 9 weeks off clinics; and 29%, 95% CI [4, 71] for 7 examinees with 0 weeks off clinics (P = .13).

For repeat examinees, most (66%, 27/41) received didactic lectures during their training programs. The pass rate for examinees with didactic lectures during the RTP was 44%, 95% CI [25, 65] and 64%, 95% CI [35, 87] for examinees without didactic lectures during the RTP. Using the Generalized Estimating Equations approach, the presence of didactic lectures was not predictive of pass rate (alpha −.18, SD 0.197). Only 3 received specific board preparation lectures at their current site of employment. The pass rate of these 3 individuals was higher than the remaining 38 repeat examinees: 67%, 95% CI [9, 99] vs 50%, 95% CI [33, 67]. The pass rate of these 3 was not significantly higher than the remaining repeat examinees (alpha −.08, SD 0.385).

DISCUSSION

For first‐time examinees, the majority (67%) received 4 to 5 weeks of study that was felt to be sufficient or more than enough by 80% of those respondents. The majority of examinees receiving fewer than 4 weeks of study did not feel that they had adequate study time (78% responded it was not sufficient). Candidates that reported feeling that they had sufficient time to study were more likely to pass (P < .001). Pass rates were very similar between candidates given 4 to 5 weeks and those given 6 to 7 weeks, suggesting that 4 to 5 weeks of study is adequate for first‐time examinees.

A potential confounding factor to this conclusion is that examinees were notified of their exam outcome before completing the survey. Candidates that passed might have been more likely to assess their study period as sufficient or more than enough vs candidates that did not pass. In future studies, this could be evaluated by soliciting responses before examinees are notified of the results. This was not possible in the current study because examination staff who assisted in administering the survey were not available until after examination results were delivered to the examinees.

Having on‐call or emergency duty during the study period adversely affected first‐time examinees' pass rates, resulting in an 18% decrease in the pass rate compared to those without these duties, suggesting that these duties should be halted during the designated study period to promote success on the general examination.

When examinees were provided didactic rounds and board preparation lectures, they were more likely to pass. It is possible that this reflects the value of these didactic learning opportunities. Alternatively, training programs that provide didactic and board preparation lectures might reflect the quality of the training program as a whole and the dedication of the supervising faculty/diplomates to ensure candidate success in board examinations rather than a direct benefit of the lectures themselves. Regardless, to optimize a resident's likelihood of passing the general examination, didactic lectures and specific board preparation lectures should be provided, ideally by faculty/diplomates or a mix of diplomates and residents. Resident‐led board preparation lectures did not improve the examinees likelihood of passing the general examination. For the 9 respondents who received resident‐led board preparation lectures, the pass rate (67%) was similar to respondents who had neither didactic lectures nor specific board preparation lectures, emphasizing that there is limited value to board preparation lectures unless provided, at least in part, by faculty/diplomates. This could be due to diplomate/faculty familiarity with the material that is typical for the general examination, improved teaching efficacy by faculty/diplomates, or some other factor. Training programs with fewer specialists could consider collaborating with other smaller training programs to provide board preparation rounds led by diplomates across several training programs. Though the current study did not investigate the effect of virtual vs in person board preparation, the provision of virtual board preparation lectures by diplomates, if necessary, might be reasonable to provide candidates with this valuable learning opportunity.

There were fewer surveys returned from repeat examinees, and consequently, this study was underpowered to detect factors associated with a higher likelihood of passing the examination. In terms of study time, 4 to 5 weeks of study time and time of emergency and on‐call duty might provide the highest pass rates as for first‐time examinees, though there was not enough data to reach this conclusion. With only 3 individuals receiving board preparation lectures during their employment, it is difficult to determine the value of these lectures for examinees retaking the test. For candidates that are entering the job market before passing the examination, it might be of benefit for candidates to negotiate 4 to 5 weeks off clinical duty for study (including protected time from emergency and on‐call duty) and to request specific board preparation lectures from other diplomates at the practice. If that is not possible, virtual board preparation lectures with diplomates in associated practices might have some benefit.

Respondents were not asked about the size of their training program. In human medicine, residency programs that have greater than 12 residents per year within that specialty have higher pass rates on their specialty exams. In veterinary medicine, there are generally fewer residents training year per specialty within a hospital or referral practice than in human medicine, so this might not be relevant. There could, however, be benefit in ACVIM training programs within a referral practice or across several referral practices coordinating resources in provision of didactic rounds and board preparation lectures.

There are additional factors both on a programmatic level or an examinee level that could have affected the likelihood of passing the examination that were not evaluated by our survey. For instance, we did not assess whether examinees have families or other caring commitments that would affect their study period, the age of the examinee, number of years elapsed between the residency and the examination for repeat examinees, and examinee performance during veterinary school. We aimed to keep the survey brief to improve the total number of respondents though in doing so might have missed these or other relevant factors. Additionally, there were fewer number of respondents that were repeat examinees so making conclusions about this subset of respondents was difficult.

Overall, there is support that residency training programs should provide at least 4‐5 weeks off clinical duty for study time and that this time should be free from emergency and on‐call duty. Additionally, programs should provide regular didactic rounds, and ideally specific board preparation rounds by faculty/diplomates. Given the difference in pass rates in first‐time examinees vs repeat examinees, it becomes even more crucial to enable success of trainees on the first attempt.

CONFLICT OF INTEREST DECLARATION

Authors declare no conflict of interest.

OFF‐LABEL ANTIMICROBIAL DECLARATION

Authors declare no off‐label use of antimicrobials.

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEE (IACUC) OR OTHER APPROVAL DECLARATION

Authors declare no IACUC or other approval was needed.

HUMAN ETHICS APPROVAL DECLARATION

Authors declare human ethics approval was not needed for this study.

Data S1. ACVIM General Exam survey.

Click here for additional data file.