Jordan D Bohnen1, Brian C George2, Reed G Williams3, Mary C Schuller4, Debra A DaRosa4, Laura Torbeck3, John T Mullen5, Shari L Meyerson4, Edward D Auyang6, Jeffrey G Chipman7, Jennifer N Choi3, Michael A Choti8, Eric D Endean9, Eugene F Foley10, Samuel P Mandell2, Andreas H Meier11, Douglas S Smink12, Kyla P Terhune13, Paul E Wise14, Nathaniel J Soper4, Joseph B Zwischenberger9, Keith D Lillemoe5, Gary L Dunnington3, Jonathan P Fryer4. 1. Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts. Electronic address: jbohnen@partners.org. 2. Department of Surgery, Harborview Medical Center, University of Washington, Seattle, Washington. 3. Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana. 4. Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois. 5. Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts. 6. Department of Surgery, University of New Mexico School of Medicine, Albuquerque, New Mexico. 7. Department of Surgery, University of Minnesota Medical School, Minneapolis, Minnesota. 8. Department of Surgery, University of Texas Southwestern Medical Center, Dallas, Texas. 9. Department of Surgery, University of Kentucky College of Medicine, Lexington, Kentucky. 10. Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin. 11. Department of Surgery, State University of New York Upstate Medical University, Syracuse, New York. 12. Department of Surgery, Brigham and Women׳s Hospital, Boston, Massachusetts. 13. Department of Surgery, Vanderbilt University Medical Center, Nashville, Tennessee. 14. Department of Surgery, Washington University School of Medicine, St. Louis, Missouri.
Abstract
PURPOSE: Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS: Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS: A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS: SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed. Copyright Â
PURPOSE: Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS: Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS: A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS:SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed. Copyright Â
Keywords:
Interpersonal and Communication Skills; Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; Professionalism; SIMPL; autonomy and Zwisch scale; intraoperative feedback; mobile technology; operative performance assessment; smartphone
Authors: Jenny X Chen; Edward H Chang; Francis Deng; Shari Meyerson; Brian George; Elliott D Kozin; Stacey T Gray Journal: J Grad Med Educ Date: 2021-10-15
Authors: Patrick D Kelly; Aaron M Yengo-Kahn; Steven G Roth; Scott L Zuckerman; Rohan V Chitale; John C Wellons; Lola B Chambless Journal: Neurosurgery Date: 2021-10-13 Impact factor: 5.315
Authors: Joceline V Vu; Brian C George; Michael Clark; Samantha J Rivard; Scott E Regenbogen; Gifty Kwakye Journal: J Surg Educ Date: 2021-01-09 Impact factor: 3.524
Authors: Kristopher M Day; Jillian K Scott; Lani Gao; Tara M Lee; Jimmy L Waldrop; Larry A Sargent; J Woody Kennedy; Jason P Rehm; Mark A Brzezienski Journal: Plast Reconstr Surg Glob Open Date: 2017-05-04
Authors: Elliott D Kozin; Jordan D Bohnen; Brian C George; Natalie Justicz; C Alessandra Colaianni; Maria Duarte; Stacey T Gray Journal: OTO Open Date: 2017-01-18
Authors: F Huettl; H Lang; M Paschold; F Watzka; N Wachter; B Hensel; W Kneist; Tobias Huber Journal: Int J Colorectal Dis Date: 2020-03-28 Impact factor: 2.571