Megan Eaton1, Rebecca Scully2, Mary Schuller3, Amy Yang3, Douglas Smink2, Reed G Williams4, Jordan D Bohnen2, Brian C George5, Jonathan P Fryer3, Shari L Meyerson6. 1. University of Texas Health San Antonio, San Antonio, Texas. 2. Massachusetts General Hospital, Department of Surgery, Boston, Massachusetts. 3. Northwestern University, Department of Surgery, Chicago, Illinois. 4. Indiana University, Department of Surgery, Bloomington, Indiana. 5. University of Michigan, Department of Surgery, Ann Arbor, Michigan. 6. University of Kentucky, Department of Surgery, Lexington, Kentucky. Electronic address: shari.meyerson@uky.edu.
Abstract
OBJECTIVE: The System for Improving and Measuring Procedural Learning (SIMPL) is a smart-phone application used to provide residents with an evaluation of operative autonomy and feedback. This study investigated the perceived benefits and barriers to app use. DESIGN: A database of previously performed SIMPL evaluations was analyzed to identify high, low, and never users. Potential predisposing factors to use were explored. A survey investigating key areas of value and barriers to use for the SIMPL application was sent to resident and faculty users. Respondents were asked to self-identify how often they used the app. The perceived benefits and barriers were correlated with the level of usage. Qualitative analysis of free text responses was used to determine strategies to increase usage. SETTING: General surgery training programs who are members of the Procedural Learning and Safety Collaborative. PARTICIPANTS: Surgical residents and faculty. RESULTS: At least 1 SIMPL evaluation was created for 411 residents and 524 faculty. Thirty percent of both faculty and residents were high-frequency users. Thirty percent of faculty were never users. One hundred eighty-eight residents and 207 faculty (response rate 46%) completed the survey. High-frequency resident users were more likely to perceive a benefit for both numerical evaluations (76% vs 30%) and dictated feedback (92% vs 30%). Faculty and residents commonly blamed each other for not creating and completing evaluations regularly (87% of residents, 81% of faculty). Suggested strategies to increase usage included reminders and integration with existing data systems. CONTRIBUTIONS: Frequent users perceive value from the application, particularly from dictated feedback and see a positive impact on feedback in their programs. Faculty engagement represents a major barrier to adoption. Mechanisms which automatically remind residents to initiate an evaluation will help improve utilization but programs must work to enhance faculty willingness to respond and dictate feedback.
OBJECTIVE: The System for Improving and Measuring Procedural Learning (SIMPL) is a smart-phone application used to provide residents with an evaluation of operative autonomy and feedback. This study investigated the perceived benefits and barriers to app use. DESIGN: A database of previously performed SIMPL evaluations was analyzed to identify high, low, and never users. Potential predisposing factors to use were explored. A survey investigating key areas of value and barriers to use for the SIMPL application was sent to resident and faculty users. Respondents were asked to self-identify how often they used the app. The perceived benefits and barriers were correlated with the level of usage. Qualitative analysis of free text responses was used to determine strategies to increase usage. SETTING: General surgery training programs who are members of the Procedural Learning and Safety Collaborative. PARTICIPANTS: Surgical residents and faculty. RESULTS: At least 1 SIMPL evaluation was created for 411 residents and 524 faculty. Thirty percent of both faculty and residents were high-frequency users. Thirty percent of faculty were never users. One hundred eighty-eight residents and 207 faculty (response rate 46%) completed the survey. High-frequency resident users were more likely to perceive a benefit for both numerical evaluations (76% vs 30%) and dictated feedback (92% vs 30%). Faculty and residents commonly blamed each other for not creating and completing evaluations regularly (87% of residents, 81% of faculty). Suggested strategies to increase usage included reminders and integration with existing data systems. CONTRIBUTIONS: Frequent users perceive value from the application, particularly from dictated feedback and see a positive impact on feedback in their programs. Faculty engagement represents a major barrier to adoption. Mechanisms which automatically remind residents to initiate an evaluation will help improve utilization but programs must work to enhance faculty willingness to respond and dictate feedback.