Charles M Maxfield1, J Felipe Montano-Campos2, Teresa Chapman3, Terry S Desser4, Christopher P Ho5, Nathan C Hull6, Hillary R Kelly7, Tabassum A Kennedy8, Nicholas A Koontz9, Emily E Knippa10, Theresa C McLoud11, James Milburn12, Megan K Mills13, Desiree E Morgan14, Rustain Morgan15, Ryan B Peterson16, Ninad Salastekar17, Matthew P Thorpe6, Jessica G Zarzour18, Shelby D Reed2, Lars J Grimm19. 1. Vice-Chair of Education, Department of Radiology, Duke University Medical Center, Durham, North Carolina. Electronic address: charles.maxfield@duke.edu. 2. Department of Population Health Sciences, Duke University School of Medicine, Durham, North Carolina. 3. Residency Program Director, Department of Radiology, Seattle Children's Hospital, University of Washington School of Medicine, Seattle, Washington. 4. Department of Radiology, Stanford University Medical Center, Stanford, California. 5. Residency Program Director, Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, Georgia. 6. Department of Radiology, Mayo Clinic, Rochester, Minnesota. 7. Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts; Massachusetts Eye and Ear, Harvard Medical School, Boston, Massachusetts. 8. Department of Radiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin. 9. Department of Radiology and Imaging Sciences, Indiana University School of Medicine, Indianapolis, Indiana. 10. Department of Radiology, University of Texas Southwestern Medical Center, Dallas, Texas. 11. Vice-Chair of Education, Residency Program Director, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts. 12. Residency Program Director, Department of Radiology, Ochsner Health System, New Orleans, Louisiana. 13. Department of Radiology and Imaging Sciences, University of Utah, Salt Lake City, Utah. 14. Vice-Chair of Education, Department of Radiology, University of Alabama at Birmingham, Birmingham, Alabama. 15. Residency Program Director, Department of Radiology, University of Colorado Anschutz Medical Campus, Aurora, Colorado. 16. Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, Georgia. 17. Department of Radiology, SUNY Upstate Medical University, Syracuse, New York. 18. Radiology Residency Program Director, Department of Radiology, University of Alabama at Birmingham, Birmingham, Alabama. 19. Department of Radiology, Duke University Medical Center, Durham, North Carolina.
Abstract
OBJECTIVES: Reporting of United States Medical Licensing Examination Step 1 results will transition from a numerical score to a pass or fail result. We sought an objective analysis to determine changes in the relative importance of resident application attributes when numerical Step 1 results are replaced. METHODS: A discrete choice experiment was designed to model radiology resident selection and determine the relative weights of various application factors when paired with a numerical or pass or fail Step 1 result. Faculty involved in resident selection at 14 US radiology programs chose between hypothetical pairs of applicant profiles between August and November 2020. A conditional logistic regression model assessed the relative weights of the attributes, and odds ratios (ORs) were calculated. RESULTS: There were 212 participants. When a numerical Step 1 score was provided, the most influential attributes were medical school (OR: 2.35, 95% confidence interval [CI]: 2.07-2.67), Black or Hispanic race or ethnicity (OR: 2.04, 95% CI: 1.79-2.38), and Step 1 score (OR: 1.8, 95% CI: 1.69-1.95). When Step 1 was reported as pass, the applicant's medical school grew in influence (OR: 2.78, 95% CI: 2.42-3.18), and there was a significant increase in influence of Step 2 scores (OR: 1.31, 95% CI: 1.23-1.40 versus OR 1.57, 95% CI: 1.46-1.69). There was little change in the relative influence of race or ethnicity, gender, class rank, or clerkship honors. DISCUSSION: When Step 1 reporting transitions to pass or fail, medical school prestige gains outsized influence and Step 2 scores partly fill the gap left by Step 1 examination as a single metric of decisive importance in application decisions.
OBJECTIVES: Reporting of United States Medical Licensing Examination Step 1 results will transition from a numerical score to a pass or fail result. We sought an objective analysis to determine changes in the relative importance of resident application attributes when numerical Step 1 results are replaced. METHODS: A discrete choice experiment was designed to model radiology resident selection and determine the relative weights of various application factors when paired with a numerical or pass or fail Step 1 result. Faculty involved in resident selection at 14 US radiology programs chose between hypothetical pairs of applicant profiles between August and November 2020. A conditional logistic regression model assessed the relative weights of the attributes, and odds ratios (ORs) were calculated. RESULTS: There were 212 participants. When a numerical Step 1 score was provided, the most influential attributes were medical school (OR: 2.35, 95% confidence interval [CI]: 2.07-2.67), Black or Hispanic race or ethnicity (OR: 2.04, 95% CI: 1.79-2.38), and Step 1 score (OR: 1.8, 95% CI: 1.69-1.95). When Step 1 was reported as pass, the applicant's medical school grew in influence (OR: 2.78, 95% CI: 2.42-3.18), and there was a significant increase in influence of Step 2 scores (OR: 1.31, 95% CI: 1.23-1.40 versus OR 1.57, 95% CI: 1.46-1.69). There was little change in the relative influence of race or ethnicity, gender, class rank, or clerkship honors. DISCUSSION: When Step 1 reporting transitions to pass or fail, medical school prestige gains outsized influence and Step 2 scores partly fill the gap left by Step 1 examination as a single metric of decisive importance in application decisions.