Musab S Hommos1, Ethan F Kuperman, Aparna Kamath, Clarence D Kreiter. 1. M.S. Hommos was clinical assistant professor, Department of Internal Medicine, Carver College of Medicine, University of Iowa, Iowa City, Iowa, at the time this work was done. He currently is a fellow physician, Division of Nephrology and Hypertension, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota. E.F. Kuperman is clinical assistant professor, Department of Internal Medicine, Carver College of Medicine, University of Iowa, Iowa City, Iowa. A. Kamath was clinical assistant professor, Department of Internal Medicine, Carver College of Medicine, University of Iowa, Iowa City, Iowa, at the time this work was done. She currently is assistant professor, Department of Internal Medicine, Duke University Health System, Durham, North Carolina. C.D. Kreiter is professor, Department of Family Medicine and Office of Consultation and Research in Medical Education, Carver College of Medicine, University of Iowa, Iowa City, Iowa.
Abstract
PURPOSE: To develop and determine the reliability of a novel measurement instrument assessing the quality of residents' discharge summaries. METHOD: In 2014, the authors created a discharge summary evaluation instrument based on consensus recommendations from national regulatory bodies and input from primary care providers at their institution. After a brief pilot, they used the instrument to evaluate discharge summaries written by first-year internal medicine residents (n = 24) at a single U.S. teaching hospital during the 2013-2014 academic year. They conducted a generalizability study to determine the reliability of the instrument and a series of decision studies to determine the number of discharge summaries and raters needed to achieve a reliable evaluation score. RESULTS: The generalizability study demonstrated that 37% of the variance reflected residents' ability to generate an adequate discharge summary (true score variance). The decision studies estimated that the mean score from six discharge summary reviews completed by a unique rater for each review would yield a reliability coefficient of 0.75. Because of high interrater reliability, multiple raters per discharge summary would not significantly enhance the reliability of the mean rating. CONCLUSIONS: This evaluation instrument reliably measured residents' performance writing discharge summaries. A single rating of six discharge summaries can achieve a reliable mean evaluation score. Using this instrument is feasible even for programs with a limited number of inpatient encounters and a small pool of faculty preceptors.
PURPOSE: To develop and determine the reliability of a novel measurement instrument assessing the quality of residents' discharge summaries. METHOD: In 2014, the authors created a discharge summary evaluation instrument based on consensus recommendations from national regulatory bodies and input from primary care providers at their institution. After a brief pilot, they used the instrument to evaluate discharge summaries written by first-year internal medicine residents (n = 24) at a single U.S. teaching hospital during the 2013-2014 academic year. They conducted a generalizability study to determine the reliability of the instrument and a series of decision studies to determine the number of discharge summaries and raters needed to achieve a reliable evaluation score. RESULTS: The generalizability study demonstrated that 37% of the variance reflected residents' ability to generate an adequate discharge summary (true score variance). The decision studies estimated that the mean score from six discharge summary reviews completed by a unique rater for each review would yield a reliability coefficient of 0.75. Because of high interrater reliability, multiple raters per discharge summary would not significantly enhance the reliability of the mean rating. CONCLUSIONS: This evaluation instrument reliably measured residents' performance writing discharge summaries. A single rating of six discharge summaries can achieve a reliable mean evaluation score. Using this instrument is feasible even for programs with a limited number of inpatient encounters and a small pool of faculty preceptors.
Authors: Danielle E Weber; Justin D Held; Roman A Jandarov; Matthew Kelleher; Ben Kinnear; Dana Sall; Jennifer K O'Toole Journal: J Gen Intern Med Date: 2020-01-28 Impact factor: 5.128