Community organization feedback about an undergraduate medical education service learning program Rétroaction d'un organisme communautaire sur apprentissage en milieu de travail d'un programme d'études médicales de premier cycle.

BACKGROUND: In 2016, Service Learning (SL) became a curricular requirement for undergraduate medical education (UGME) students at the University of Manitoba. Students partner with a community-based organization for two years to engage in non-clinical activities in community settings. Significant feedback has been collected from students re: their SL experiences. This project specifically collected feedback from community organizations involved with SL.
METHODS: In June 2019, an electronic survey was distributed to the 36 community organizations involved with SL.
RESULTS: Twenty-seven organizations completed the survey. Feedback was grouped into two main themes: 1) Logistics and 2) The SL Experience. About half (52%) of respondents indicated it was "easy" to schedule students for SL; however, students' busy schedules and differences between hours of organization programming and students' availability were highlighted. Most respondents described students as "engaged" (70%); respondents indicated SL raised students' understanding of power and privilege (56%) and systemic oppression (63%).
CONCLUSIONS: Community organizations shared valuable insights to inform the SL program. Results identified specific aspects of the SL program to address moving forward, such as sharing learning objectives with community partners. Ensuring processes are in place to obtain feedback from community partners is an essential step to improve SL programs, and to strengthen reciprocal community-university partnerships.

Introduction

The Committee on Accreditation of Canadian Medical Schools (CACMS) defines Service Learning as “a structured learning experience that combines community service with preparation and reflection” (p. v).[1] CACMS standard 6.6 outlines that “the faculty of a medical school [should] ensure that the medical education program provides sufficient opportunities for, encourages, and supports medical student participation in a service-learning activity” (p. 11). [1] While there is not one standardized approach to organizing and implementing Service Learning programs for Canadian medical students, a common thread is many Service Learning programs depend on the involvement of community partners to provide these opportunities.

In 2016, Service Learning (SL) became a curricular requirement for undergraduate medical education (UGME) students in the Max Rady College of Medicine at the University of Manitoba (UM). Service Learning programs can be an application of social accountability in medical schools; social accountability involves multiple considerations such as educational programming, service, research, student recruitment, and preparing students for future practice.[2-4] A socially accountable approach to medical education requires institutions to partner with communities to identify and respond to priority health needs, particularly for groups experiencing health disparities, at individual and community levels.[5-7]

While Service Learning fits under the larger umbrella of social accountability, more specifically, the SL program at UM is informed by principles of critical service learning[8] and recommended best practices in community-university engagement.[9-12] Grounded in reciprocal and active relationships, these principles include working to identify and challenge power asymmetries in community-university partnerships, responding to community-identified priorities, and self-reflection and action on the part of university partners to engage in critical allyship and system-level change.[8,11,13]

In the UM program, UGME students are matched with a community organization at the outset of their first academic year; students develop longitudinal relationships and work with these organizations for 46 hours throughout their pre-clerkship program. Organization mandates are diverse and include programs related to food security, homelessness, and education/recreation programs for children and youth. Since the launch of the SL curriculum, regular feedback has been collected from students to inform and improve this educational experience. Literature about critical service learning and community-university engagement indicates the importance of seeking involvement and feedback about SL programs from community partners, though this is less common.[8,11,15] At UM, many contacts from community organizations involved with SL regularly provide informal feedback to members of the SL Team, composed of university faculty, senior administration, and administrative staff. In recognition of community organizations as active partners and co-educators in the SL Program,[8] the SL Team undertook an initiative to seek formal feedback from community organizations in the summer of 2019. Ensuring the university has processes in place to seek out feedback is an important part of strengthening reciprocal community-university relationships, working to implement critical service learning principles, and improving the SL experience for both students and community partners.[8,11,15]

Methods

In June 2019 the SL Team emailed a 15-question survey to the 36 organizations involved with SL to seek feedback about their experiences working with UGME students (Appendix A). The survey included a combination of Likert scale and narrative questions to seek feedback about multiple facets of the program including the process of matching students with organizations, number of hours students engage in SL, communication, and positive program aspects/areas for improvement. In addition, we included a number of specific questions about the program learning objectives (Appendix B) to better understand if opportunities to address these objectives took place, and if the organizations observed student development in these areas. The survey was available via Survey Monkey for two weeks; a phone option was also available. Survey participation was voluntary, and organizations could choose to participate anonymously.

Following data collection, de-identified aggregate data was exported from the online survey platform for analysis. Though the scope of this initiative was a program evaluation and not a more robust research project, we sought Health Research Ethics Board approval to publicly share the results (University of Manitoba Health Research Ethics Board Registry Number: H2019:083 [HS22616]).

After data collection, we formed a working group consisting of community organization and university representatives (authors RB, NC, FR, KC and CJ). Three community leaders were purposefully recruited from organizations actively involved with SL to reflect a variety of characteristics such as size of organization, types of programming offered, and duration of SL involvement. This step recognized the importance of community involvement not only as data sources but also data interpreters and knowledge disseminators. Drawing on principles of community-university partnerships, several considerations were intentionally taken into account during the formation of the working group. The working group was composed of more community organization representatives (n=3) than university representatives (n=2) in an effort to center community perspectives. In acknowledgment that community-university meetings often privilege the university schedule and location,[16] meeting times and locations were negotiated with working group members. This working group provided direction about the following:

Data analysis and interpretation

Development of an electronic summary report for all 36 community organizations (including hard copies by request)

Identification of other knowledge translation methods to share findings with various stakeholders:

Oral presentations to various UGME governance committees at UM

Posting the summary report on the university website[17]

Development of a peer-review article, including negotiation of various roles in manuscript preparation (e.g., reviewing literature, writing the first draft, providing feedback on drafts)

Submission of abstracts to various conferences

To conduct analysis, percentages for Likert scale answers were generated by Survey Monkey. Working group members independently reviewed survey results and narrative comments to identify emergent key themes.[18] Subsequently, working group members engaged in dialogue to compare and negotiate key themes until consensus was reached.

Results

In total, 27 of the 36 organizations completed the survey. Our analysis of the results identified two main themes:

Theme 1

This feedback addressed program logistics such as the process of matching students with organizations, scheduling times for students to complete their SL hours, and organizations’ communication with students and the university.

At the outset of year one students receive a list of all possible SL opportunities compiled from the SL Team’s relationships with community organizations whose mandates address social determinants of health. Students complete an online survey to rank their SL site preferences. As much as possible students are matched with their highest ranked sites. Students remain working with the same organization longitudinally for two years. Respondents indicated a high level of satisfaction with the matching process (93% very satisfied).

Students are responsible to complete ~2-4 hours of SL per month (October – April). First year students must complete 24 hours in the academic year; second year students must complete 22 hours. Curricular time is allotted to SL, however, in many cases the scheduled time is not when students complete their hours. For example, it may not be feasible for students to complete SL between 10-11 a.m. if they are matched with an after-school program. This scheduled hour during the day is time “in lieu of” the hour they will spend at SL. Over half of respondents expressed that scheduling students to complete SL hours was “easy” (52%); a third of respondents indicated it was “somewhat easy” (33%) while 15% indicated it was “somewhat difficult”. Greater ease of scheduling was facilitated by organizations who have the infrastructure to support structured scheduling processes (e.g., an online platform where students can sign up for shifts).

Most respondents expressed it was “very easy” to communicate with members of the university SL Team (85%). Communicating with students was characterized as slightly more challenging with 74% of organizations describing student communication as “easy”, 19% as “somewhat easy”, and seven percent as “somewhat difficult”. Respondents’ narrative comments expressed that any communication challenges tended to center around difficulty communicating with specific individuals, not necessarily from challenges in the overall communication structure.

Theme 2

This feedback addressed the learning objectives, students’ engagement, and any specific challenges or suggestions related to SL.

During their first year SL orientation students receive a syllabus with five learning objectives to guide the SL program (Appendix B). One of the survey questions provided a list of these learning objectives and asked if the program provided opportunities to meet them. In response, 89% of respondents indicated “yes” and 11% indicated “somewhat”. Several responses highlighted the importance of providing the learning objectives to the organizations prior to students’ SL engagement. As one respondent explained, “...This would ensure that the site is also observing that objectives are completed in the required time frame and so the site can provide any necessary assistance to the students to the best of our abilities.”

Narrowing the focus, several questions on the survey asked if students’ participation in SL increased their awareness of specific topics outlined in the learning objectives such as systemic oppression, critical reflection, and power and privilege. In general, respondents indicated these specific objectives were met (see Table 2). As one example, regarding awareness of systemic oppression, one respondent expressed, “[The student] came with a good understanding of this and her awareness has allowed her to see more deeply and in a hands-on way how to work with people who are oppressed with compassion and building rapport.”

Table 2

Select survey results Theme 2 – Service Learning Experience

	Skills were demonstrated	Skills were somewhat demonstrated	Skills were not demonstrated		No response
Skills: Did your SL students demonstrate skills to identify and address the needs of the individuals and communities who access your organization?	67% (n=18)	33% (n=9)	0%		0%
	Responsive	Somewhat Responsive	Somewhat Unresponsive	Unresponsive	No response
Responsive: To what extent do you think the SL program is responsive to the needs and priorities of your community organization?	70% (n=19)	26% (n=7)	0%	4% (n=1)	0%
	Engaged	Somewhat Engaged	Somewhat Disengaged	Disengaged	No response
Engagement: How would you describe the students’ level of engagement while participating in SL at your organization?	70% (n=19)	26% (n=7)	4% (n=1)	0%	0%
	Yes, this amount of hours is sufficient	A slight increase in required hours would be beneficial (5-9 hours more)	A significant increase in required hours would be beneficial (10+ hours more)	A slight decrease in required hours would be beneficial (5-9 hours less)	A significant decrease in required hours would be beneficial (10+ hours less)
Hours: Do you think the number of required hours is beneficial to both your organization and the students?	56% (n=15)	19% (n=5)	26% (n=7)	0%	0%
	Yes	Somewhat	No		No response
Objectives: Did the SL program facilitate students’ increased awareness of systemic oppression?	63% (n=17)	30% (n=8)	0%		7% (n=2)
Objectives: Did the SL program facilitate students’ increased awareness of power and privilege?	56% (n=15)	37% (n=10)	0%		7% (n=2)
Objectives: Did the SL program facilitate students’ increased awareness of critical reflection?	59% (n=16)	30% (n=8)	4% (n=1)		7% (n=2)
Objectives: Do you think the SL program provides opportunities to meet program objectives?	89% (n=24)	11% (n=3)	0%		0%

Respondents rated students’ levels of engagement quite high; 70% of respondents described students as “engaged” and 26% described students as “somewhat engaged”. The majority of the 11 narrative survey comments for this question were very positive such as, “Given their busy schedules, [the students’] level of involvement and support was impressive.” A few respondents indicated that encouragement to engage was required for some students. For example, one respondent wrote “Some students needed some prompting to get engaged with the kids.”

In response to a narrative question, when asked about specific SL challenges for their organizations, 89% indicated they did not experience any challenges while 11% indicated there were some challenges particularly related to communication.

In addition to the specific question about quantity of hours, organizations’ narrative responses to several different questions included recommendations for more SL hours for students. For example, one respondent indicated, “...These students aren’t here long enough for us to truly get to know them, and it doesn’t allow us to really make sure that they are really getting into the community participation. That being said, I believe the time spent is still making an impact on the students.”

Another emergent theme, and something for universities to consider, is the resources required by the community organizations. One respondent shared that while they value working with SL students, resource restraints can impact organization service delivery and therefore the ability to engage with students: “[It’s] so great to have this relationship, but our real ‘needs and priorities’ come from how stretched we are in terms of even being able to support students in their service learning. Additional resources for CBOs [Community Based Organizations] to help with this would be a welcome support.”

Discussion

The survey responses shared by the community organizations were very valuable and informed a number of practical changes implemented by the SL Team.

Action 1 - Share program objectives with community organizations (implemented in fall 2019)

While some aspects of the syllabus had been shared with community organizations, such as student hour requirements, the survey results brought to light that the SL Team had not actively shared the program learning objectives with community organizations. Ensuring all individuals involved in delivering the SL program have access to the learning objectives is an important foundational step for building communication between community partners and the university, and for identifying shared program goals.[15,16] In fact, Weerts found that community participants identified communication and mutual respect as the two most important factors in building community-university partnerships.[10] Communication-related considerations such as the avoidance of acronyms and jargon, and also seeking direction from community partners regarding the preferred formats in which ongoing program-related communication takes place are also important factors.[10,11,15]

Furthermore, frameworks of critical service learning and community-university partnerships require that existing power relationships between community organizations and the university must be deliberately considered and asymmetries challenged.[19] This includes identifying the inherent power imbalance if the university exclusively designs the learning objectives, program delivery, assessment measures, and indicators of program success.[9,11,12,19] Gelmon et al. described that, “community partners need to be seen and involved as active participants in the design, implementation and evaluation of community-based learning experiences…” (p. 105).[9] Existing literature points to an ongoing practice of universities engaging with community in order to fulfill an institutional agenda, with university determined outcomes, learning objectives, and terms of engagement.[8,13,19] We see examples of this within the scope of this project; university members developed all of the survey questions sent to organizations, and had developed all of the learning objectives for the SL program. In reflecting upon these practices through the lenses of critical service learning and community-university partnerships, collaborating with community members earlier, and on an ongoing basis, is optimal. In this case, the survey results informed changes to more effectively communicate expectations about the SL objectives, activities, and assessment with community organizations. To continue to move towards more reciprocal partnerships with community organizations, the pursuit of collaborative community-university dialogue to review, revise and/or re-imagine components of the SL program, such as the program learning objectives, will be facilitated by the SL Team.

Action 2 - Present a summary of community organization feedback to relevant university committees who oversee UGME programs, including Service Learning

During the 2019/2020 academic year members of the SL Team presented key findings to committees such as the Pre-Clerkship Curriculum Committee, the Department of Community Health Sciences UGME and Department Council Committees, Max Rady College of Medicine Department Head Council, and the Rady Faculty of Health Sciences Social Accountability Committee.

Furthermore, we are pursuing additional opportunities for knowledge translation. National and international opportunities for presentation are desirable; however, to involve all members of the working group, limited university fiscal resources to cover costs like travel/conference registration fees, and the full schedules of organizational representatives undertaking essential community work, are factors that must be considered and navigated.

Conclusion

Seeking feedback from community partners is not an endpoint. Rather, this feedback should propel further discussions and improvements for the SL program. Survey results identified specific actions to improve the SL program, and the process of engaging in this survey also highlighted additional points to consider and do differently to strengthen community-university partnerships. To work towards meaningful and responsive collaborations, universities must acknowledge and disrupt practices that privilege university priorities over community.[11,13,19] Even when community has not been involved from the initial stages, opportunities can be created for community and university to engage in meaningful dialogue, development, evaluation, and revision at a variety of stages in educational projects and programs. As universities continue to learn, reflect, respond, and improve in building collaborative partnerships with community, these actions will move towards strengthening the essential community-university relationships that are at the core of many SL programs.

Table 1

Select survey results Theme 1 – Logistics

	Very Satisfied	Somewhat Satisfied	Somewhat Dissatisfied	Dissatisfied	No response
Matching: Please rank your level of satisfaction with the process of placing service learning students with your organization.	93% (n = 25)	4% (n = 1)	4% (n = 1)	0%	0%
	Easy	Somewhat Easy	Somewhat Difficult	Difficult	No response
Scheduling: How easy/difficult was it to coordinate scheduling with your students so they could complete their SL hours with your organization?	52% (n = 14)	33% (n = 9)	15% (n = 4)	0%	0%
Communication: How easy/difficult was it to communicate with members of the university SL team?	85% (n = 23)	4% (n = 1)	0%	4% (n = 1)	7% (n = 2)
Communication: How easy/difficult was it to communicate with your SL students?	74% (n = 20)	19% (n = 5)	7% (n = 2)	0%	0%