Kathleen D Ramos1, Sean Schafer, Susan M Tracz. 1. University of California San Francisco, Fresno Medical Education Program, Department of Family and Community Medicine, 445 South Cedar Avenue, Fresno, CA 93702, USA. katie.ramos@ucsfresno.edu
Abstract
OBJECTIVE: To describe the development and validation of a test of knowledge and skills in evidence based medicine. DESIGN: Cross sectional study. SETTING: Family practice residency programme in California; a list server for those who teach evidence based medicine; and an evidence based medicine seminar series. PARTICIPANTS: Family practice residents and faculty members (n=43); volunteers self identified as experts in evidence based medicine (n=53); family practice teachers (19) beginning a seminar series on evidence based medicine. INTERVENTION: The Fresno test is a performance based measure for use in medical education that assesses a wide range of evidence based medicine skills. Open ended questions are scored with standardised grading rubrics. Calculation skills are assessed by fill in the blank questions. MAIN OUTCOME MEASURES: Inter-rater reliability, internal reliability, item analyses, and construct validity. RESULTS: Inter-rater correlations ranged from 0.76 to 0.98 for individual items. Cronbach's alpha was 0.88. Item difficulties ranged from moderate to difficult, all with positive and strong ability to discriminate between candidates. Experts scored consistently higher than novices. On the 212 point test, the novice mean was 95.6 and the expert mean was 147.5 (P<0.001). On individual items, a higher proportion of experts than novices earned passing scores on 15 of the 17 items. CONCLUSION: The Fresno test is a reliable and valid test for detecting the effect of instruction in evidence based medicine. Its use in other settings requires further exploration.
OBJECTIVE: To describe the development and validation of a test of knowledge and skills in evidence based medicine. DESIGN: Cross sectional study. SETTING: Family practice residency programme in California; a list server for those who teach evidence based medicine; and an evidence based medicine seminar series. PARTICIPANTS: Family practice residents and faculty members (n=43); volunteers self identified as experts in evidence based medicine (n=53); family practice teachers (19) beginning a seminar series on evidence based medicine. INTERVENTION: The Fresno test is a performance based measure for use in medical education that assesses a wide range of evidence based medicine skills. Open ended questions are scored with standardised grading rubrics. Calculation skills are assessed by fill in the blank questions. MAIN OUTCOME MEASURES: Inter-rater reliability, internal reliability, item analyses, and construct validity. RESULTS: Inter-rater correlations ranged from 0.76 to 0.98 for individual items. Cronbach's alpha was 0.88. Item difficulties ranged from moderate to difficult, all with positive and strong ability to discriminate between candidates. Experts scored consistently higher than novices. On the 212 point test, the novice mean was 95.6 and the expert mean was 147.5 (P<0.001). On individual items, a higher proportion of experts than novices earned passing scores on 15 of the 17 items. CONCLUSION: The Fresno test is a reliable and valid test for detecting the effect of instruction in evidence based medicine. Its use in other settings requires further exploration.
Authors: D Michael Elnicki; Meenakshy K Aiyer; Maria L Cannarozzi; Alexander Carbo; Paul R Chelminski; Shobhina G Chheda; Saumil M Chudgar; Heather E Harrell; L Chad Hood; Michelle Horn; Karnjit Johl; Gregory C Kane; Diana B McNeill; Marty D Muntz; Anne G Pereira; Emily Stewart; Heather Tarantino; T Robert Vu Journal: J Gen Intern Med Date: 2017-06-20 Impact factor: 5.128
Authors: Sharon E Straus; Michael L Green; Douglas S Bell; Robert Badgett; Dave Davis; Martha Gerrity; Eduardo Ortiz; Terrence M Shaneyfelt; Chad Whelan; Rajesh Mangrulkar Journal: BMJ Date: 2004-10-30
Authors: Eric S Holmboe; Judith L Bowen; Michael Green; Jessica Gregg; Lorenzo DiFrancesco; Eileen Reynolds; Patrick Alguire; David Battinelli; Catherine Lucey; Daniel Duffy Journal: J Gen Intern Med Date: 2005-12 Impact factor: 5.128
Authors: Sarang Kim; Laura R Willett; David J Murphy; Kerry O'Rourke; Ranita Sharma; Judy A Shea Journal: J Gen Intern Med Date: 2008-09-04 Impact factor: 5.128
Authors: Regina Kulier; Khalid S Khan; A Metin Gulmezoglu; Guillermo Carroli; Jose G Cecatti; Maria J Germar; Pisake Lumbiganon; Suneeta Mittal; Robert Pattinson; Jean-Jose Wolomby-Molondo; Anne-Marie Bergh; Win May Journal: Reprod Health Date: 2010-05-14 Impact factor: 3.223