Rushad Patell1, Paola Raska2, Natalie Lee3, Gina Luciano4, Deborah J DiNardo5, Amiran Baduashvili6, Mel L Anderson7,8, Frank Merritt7, Michael B Rothberg9. 1. Department of Hematology-Oncology, Beth Israel Deaconess Medical Center, Boston, MA, USA. 2. Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH, USA. 3. National Clinician Scholar, Medicine, Perelman School of Medicine, Pennsylvania, PA, USA. 4. Division of General Medicine and Geriatrics, Baystate Health, Springfield, MA, USA. 5. Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA. 6. Section of Hospital Medicine, Department of General Internal Medicine, Weill Cornell Medical College, New York, NY, USA. 7. Department of Medicine, University of Colorado Denver School of Medicine, Aurora, CO, USA. 8. Medicine Service, Rocky Mountain VA Regional Medical Center, Aurora, CO, USA. 9. Department of Internal Medicine , Cleveland Clinic, 9500 Euclid Avenue, Mail Code G10, Cleveland, OH, 44195, USA. rothbem@ccf.org.
Abstract
BACKGROUND: Medical educators need valid, reliable, and efficient tools to assess evidence-based medicine (EBM) knowledge and skills. Available EBM assessment tools either do not assess skills or are laborious to grade. OBJECTIVE: To validate a multiple-choice-based EBM test-the Resident EBM Skills Evaluation Tool (RESET). DESIGN: Cross-sectional study. PARTICIPANTS: A total of 304 medicine residents from five training programs and 33 EBM experts comprised the validation cohort. MAIN MEASURES: Internal reliability, item difficulty, and item discrimination were assessed. Construct validity was assessed by comparing mean total scores of trainees to experts. Experts were also asked to rate importance of each test item to assess content validity. KEY RESULTS: Experts had higher total scores than trainees (35.6 vs. 29.4, P < 0.001) and also scored significantly higher than residents on 11/18 items. Cronbach's alpha was 0.6 (acceptable), and no items had a low item-total correlation. Item difficulty ranged from 7 to 86%. All items were deemed "important" by > 50% of experts. CONCLUSIONS: The proposed EBM assessment tool is a reliable and valid instrument to assess competence in EBM. It is easy to administer and grade and could be used to guide and assess interventions in EBM education.
BACKGROUND: Medical educators need valid, reliable, and efficient tools to assess evidence-based medicine (EBM) knowledge and skills. Available EBM assessment tools either do not assess skills or are laborious to grade. OBJECTIVE: To validate a multiple-choice-based EBM test-the Resident EBM Skills Evaluation Tool (RESET). DESIGN: Cross-sectional study. PARTICIPANTS: A total of 304 medicine residents from five training programs and 33 EBM experts comprised the validation cohort. MAIN MEASURES: Internal reliability, item difficulty, and item discrimination were assessed. Construct validity was assessed by comparing mean total scores of trainees to experts. Experts were also asked to rate importance of each test item to assess content validity. KEY RESULTS: Experts had higher total scores than trainees (35.6 vs. 29.4, P < 0.001) and also scored significantly higher than residents on 11/18 items. Cronbach's alpha was 0.6 (acceptable), and no items had a low item-total correlation. Item difficulty ranged from 7 to 86%. All items were deemed "important" by > 50% of experts. CONCLUSIONS: The proposed EBM assessment tool is a reliable and valid instrument to assess competence in EBM. It is easy to administer and grade and could be used to guide and assess interventions in EBM education.
Entities:
Keywords:
evidence-based medicine; medical education; test