Student Evaluations, Outcomes, and National Licensure Examinations in Radiology Education: A Narrative Review of the Literature.

OBJECTIVE: The purpose of this study was to examine literature on radiological student evaluation and outcome assessments including national board examinations.
METHODS: A review of the literature was performed using relevant key words. Articles were retrieved through December 2012 using PubMed, ScienceDirect, ERIC, Proquest, and ICL databases along with a manual review of references.
RESULTS: Of the 4716 unique abstracts reviewed by the author, 54 were found to be relevant to the purpose of this study. Student grade point average correlates with board scores in the nursing, chiropractic, and medical professions. Scores on the chiropractic college admission test and undergraduate grade point average correlate with success in professional college. There is a correlation between board scores and college attended. Board preparation programs do not appear to affect board examination scores.
CONCLUSION: Although evaluations can be effective teaching tools, they are not used by many radiology programs. Some programs have inadequate evaluations and do not allow students to review their evaluations. There are no definitive links between mastery of radiology and specific evaluations, outcomes, or pre-professional/clinical grades. Studies suggest that board examination scores reflect long-term mastery of knowledge rather than short-term memorization of facts.

Introduction

Radiology educators have only recently asked questions and sought meaningful answers about how students learn.1, 2 As a result, little literature examines how radiology is taught, learned, or evaluated at the resident level, and even fewer articles examine undergraduate radiology education specifically.

There is a substantial amount of similarity found in resident and undergraduate radiology training, and these share similar needs and often use the same resources. However, there is currently no review of the literature that summarizes evaluation and outcome assessments for these training programs. Therefore, the purpose of this study was to examine literature on resident and undergraduate radiological student evaluation and outcome assessments including national board examinations.

Methods

This article reviews literature though 2012 that was retrieved from the MEDLINE, ScienceDirect, ERIC, Proquest, and ICL databases along with manual review of references. The comprehensive sampling strategy used the terms radiology OR diagnostic imaging AND education OR teaching OR resident OR medical student OR chiropractic student OR curriculum OR medical education OR medical school OR medical students OR medical curriculum OR chiropractic education, OR chiropractic school OR chiropractic students OR chiropractic curriculum. Articles were limited to those in the English language and to humans.

Results

The resultant 4716 unique article abstracts and/or titles were reviewed by the author. All articles that appeared germane to student evaluations, outcomes, and national licensure examinations in radiology education were obtained and reviewed by the author, which led to the inclusion of 54 articles in this paper.

Discussion

Evaluations and Outcomes

Evaluation tools such as examinations, case presentations, and periodic performance evaluations can be used as both evaluations of performance4, 5, 6, 7, 8, 9 and effective teaching tools.7, 10, 11, 12, 13, 14 Some programs require that residents complete “credentialing examinations” at set times during their residency; these examinations serve to document residents’ proficiency before they are allowed to interpret films within a given department.

Although the Accreditation Council for Graduate Medical Education requires that radiology residents receive formal evaluations at least 4 times a year, 22% of programs appear not to enforce this policy, and only approximately 25% provide the residents with copies of these evaluations. However, 92% allowed the resident to view the evaluations. A 2012 study found that students prefer that radiology examinations be both summative and formative, thus allowing them to learn from their mistakes and address identified areas of weakness. The importance of timely, useful feedback is well understood in the educational literature.18, 19 Similarly, a 2007 survey of clerkship directors found that only 33% of clerkships assessed chest radiology interpretation skills even when these were covered within the clerkship. Interestingly, electronic evaluation systems appear to improve both the response rate and timeliness with no impact on the quality of responses when compared with paper-based evaluation systems.

A few researchers examined the importance of evaluation and the correlation of various evaluations with student outcomes. For example, resident rotation evaluation scores appear to correlate with the American Board of Radiology standardized written examination,22, 23 and student scores on a radiology station in a Comprehensive Clinical Skills Assessment correlated with National Board Examinations and grade point average (GPA), whereas applicant rank order (how a resident compared against his or her peers during the residency matching process) does not correlate with these examinations. Preclinical medical school grades of honors/A grades in anatomy and pathology show some degree of correlation with radiology board scores, as do honors grades in some clinical courses and student scores on the United States Medical Licensure Examination.26, 27

Other discussions of possible correlation of outcomes focus on the impact of pre-radiology clinical year(s) on radiology resident performance.5, 28 However, this discussion is mainly editorial in nature, with a handful of equivocal studies looking at outcomes. A few researchers examine possible preexisting factors relating to radiology residents’ success, in particular, their perceptual abilities. These articles revealed the possibility of improving resident selection by discriminating, in a very preliminary manner, between applicants based on their perceptual aptitude.30, 31, 32

Method of examination delivery is also discussed mainly in relation to the conversion of the profession to digital image delivery for interpretation, education, and evaluation. These articles focus on the ease of digital examination style in relation to creating and proctoring the examinations, as well as the cost effectiveness of the examination delivery.

Researchers explore the relationships of examination style in radiology to measuring student mastery levels and other factors involved in undergraduate evaluations. One author in particular examines the differences in students’ perception of examination style on their level of examination preparedness, anxiety levels, and study habits in relation to a practical radiology examination. Peterson found a minimal difference in students’ anxiety levels with the open-book format vs the closed-book format, whereas the remainder of the variables remained unaffected. The examination style consisted of short-answer questions, problem solving, and case-based questions. A study examining student perceptions in regard to learning radiology found that students believe that their mastery of the material is directly linked to their perception of the difficulty level of the examination and thus the effort they put into preparing for the examination, a phenomenon known as strategic studying in educational literature. An interesting pair of articles describes the use of a clinical competency examination in radiology in the undergraduate setting.35, 36 These researchers presented the creation and implementation of such an examination and discussed the results of the pilot examination. The authors were able to arrive at 2 conclusions: first, that the use of content-based curricula may result in poor clinical competency and, secondly, that the implementation of ongoing clinical competency measures must be administered regularly within a given curriculum. A recent article described the reliability and validity of an x-ray difficulty scale for selecting chest radiographs for examinations, a useful tool for creating examinations of appropriate difficulty for a given set of students. The University of Michigan Medical School has shown that adding a 15-minute case of the week, taught by radiology faculty, significantly improved the students’ comprehensive clinical assessment examination scores.

National Board Examinations

Several authors have found that student GPA in chiropractic education correlates well with board scores.39, 40, 41, 42 This has also been demonstrated by a variety of authors in both the nursing43, 44, 45, 46 and medical professions. In addition, just as scores on the chiropractic college admission test correlate with success in professional college and on board examinations,47, 48 so, too, does undergraduate GPA.40, 49, 50, 51, 52, 53 A study published in 1979 found a correlation between board scores and the college the students attended. Board preparation programs do not appear to affect board examination scores.49, 51 These studies suggest that national board examination scores are reflective of long-term mastery of knowledge rather than short-term memorization of facts.

Limitations

This study was limited to articles available in the English language and therefore is not comprehensive of all literature worldwide. The search did not include the gray literature or other potentially relevant sources. It is possible that the search terms did not identify all relevant articles.

Conclusions

Evaluations can be used as effective teaching tools; however, many radiology programs are not using them in this manner. Some undergraduate and residency programs have inadequate evaluations and often do not allow students to review their evaluations. In addition, no definitive conclusions have been found linking mastery of radiology with any specific evaluation, outcome, or pre-professional or preclinical grades. This area of research is markedly understudied, and the little research completed only offers preliminary clues for areas to examine in future research.

Funding Sources and Conflicts of Interest

No funding sources or conflicts of interest were reported for this study.