Sreeja Natesan1, Brett Todd2, Robert S Hsu3, Ronnie Kuo Ren4, Ryan Clark5, Geoff Jara-Almonta6, Joao Ricardo Nickenig Vissoci1, Khuansiri Narajeenron7. 1. Division of Emergency Medicine Duke University Durham North Carolina USA. 2. Department of Emergency Medicine Oakland University William Beaumont School of Medicine Beaumont Health Royal Oak Michigan USA. 3. Christiana Care Emergency Medicine Residency Program Sidney Kimmel Medical College Thomas Jefferson University Cherry Hill New Jersey USA. 4. University of Massachusetts Worcester Massachusetts USA. 5. Department of Emergency Medicine UMMS-Baystate Williamsburg Massachusetts USA. 6. Icahn School of Medicine at Mt Sinai Dept of Emergency Medicine New York City Health and Hospitals Elmhurst Hospital Center Department of Emergency Medicine New York New York USA. 7. Department of Emergency Medicine, Faculty of Medicine Chulalongkorn University King Chulalongkorn Memorial Hospital, The Thai Red Cross Society Bangkok Thailand.
Abstract
BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) emphasizes constructive feedback as a critical component of residency training. Despite over a decade of using competency-based milestone evaluations, emergency medicine (EM) residency programs lack a standardized method for assessing the quality of feedback. We developed two novel EM-specific feedback surveys to assess the quality of feedback in the ER (FEED-ER) from both the resident and the faculty perspectives. This study aimed to evaluate the surveys' psychometric properties. METHODS: We developed FEED-ER using a Likert scale with faculty and resident versions based on the ACGME framework and a literature review. The preliminary survey consisted of 25 questions involving the feedback domains of timeliness, respect/communication, specificity, action plan, and feedback culture. We conducted two modified Delphi rounds involving 17 content experts to ensure respondent understanding of the items, item coherence to corresponding feedback domains, thematic saturation of domain content, and time duration. A multicenter study was conducted at five university-based EDs in the United States and one in Thailand in 2019. We evaluated the descriptive statistics of the frequency of responses, validity evidence, and reliability of FEED-ER. RESULTS: A total of 147 EM faculty and 126 EM residents completed the survey. Internal consistency was adequate (Cronbach's alpha > 0.70) and test-retest reliability showed adequate temporal stability (ICC > 0.80) for all dimensions. Content validity was deemed acceptable (CVC > 0.80) for all items. From the 25 items of FEED-ER, 23 loaded into the originally theorized dimensions (with factor loadings > 0.50). Additionally, the five feedback domains were found to be statistically distinct, with correlations between 0.40 and 0.60. The final survey has 23 items. CONCLUSIONS: This is the first study to develop and provide validity evidence for an EM-specific feedback tool that has strong psychometric properties, is reproducible and reliable, and provides an objective measure for assessing the quality of feedback in the ED.
BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) emphasizes constructive feedback as a critical component of residency training. Despite over a decade of using competency-based milestone evaluations, emergency medicine (EM) residency programs lack a standardized method for assessing the quality of feedback. We developed two novel EM-specific feedback surveys to assess the quality of feedback in the ER (FEED-ER) from both the resident and the faculty perspectives. This study aimed to evaluate the surveys' psychometric properties. METHODS: We developed FEED-ER using a Likert scale with faculty and resident versions based on the ACGME framework and a literature review. The preliminary survey consisted of 25 questions involving the feedback domains of timeliness, respect/communication, specificity, action plan, and feedback culture. We conducted two modified Delphi rounds involving 17 content experts to ensure respondent understanding of the items, item coherence to corresponding feedback domains, thematic saturation of domain content, and time duration. A multicenter study was conducted at five university-based EDs in the United States and one in Thailand in 2019. We evaluated the descriptive statistics of the frequency of responses, validity evidence, and reliability of FEED-ER. RESULTS: A total of 147 EM faculty and 126 EM residents completed the survey. Internal consistency was adequate (Cronbach's alpha > 0.70) and test-retest reliability showed adequate temporal stability (ICC > 0.80) for all dimensions. Content validity was deemed acceptable (CVC > 0.80) for all items. From the 25 items of FEED-ER, 23 loaded into the originally theorized dimensions (with factor loadings > 0.50). Additionally, the five feedback domains were found to be statistically distinct, with correlations between 0.40 and 0.60. The final survey has 23 items. CONCLUSIONS: This is the first study to develop and provide validity evidence for an EM-specific feedback tool that has strong psychometric properties, is reproducible and reliable, and provides an objective measure for assessing the quality of feedback in the ED.
Authors: Shalini T Reddy; Matthew H Zegarek; H Barrett Fromme; Michael S Ryan; Sarah-Anne Schumann; Ilene B Harris Journal: J Grad Med Educ Date: 2015-06
Authors: Lalena M Yarris; Judith A Linden; H Gene Hern; Cedric Lefebvre; David M Nestler; Rongwei Fu; Esther Choo; Joseph LaMantia; Patrick Brunett Journal: Acad Emerg Med Date: 2009-12 Impact factor: 3.451
Authors: Noelle Junod Perron; Mathieu Nendaz; Martine Louis-Simonet; Johanna Sommer; Anne Gut; Anne Baroffio; Diana Dolmans; Cees van der Vleuten Journal: Adv Health Sci Educ Theory Pract Date: 2012-11-30 Impact factor: 3.853
Authors: Lalena M Yarris; Rongwei Fu; Joseph LaMantia; Judith A Linden; H Gene Hern; Cedric Lefebvre; David M Nestler; Janis Tupesis; Nicholas Kman Journal: Acad Emerg Med Date: 2011-05 Impact factor: 3.451