Assessing Indigenous Cultural Safety Learning using Modified Reflexive Visual Arts.

Objective: Assessing Indigenous cultural safety learning in pharmacy students using modified reflexive visual arts Innovation: Traditional quantitative assessment methods are often ineffective and impractical for the evaluation of Indigenous cultural safety learning. Existing qualitative assessment methods have shown potential in small-class and experiential environments, but evidence to guide the scalability and use in a large lecture format is sparse. An innovative, visual arts-based qualitative assessment of cultural safety learning was developed and deployed to 223 first-year pharmacy students. The assessment was deployed in a pre- and post-term style in a foundational pharmacy module that included content on Indigenous health and cultural safety. The pre-term assessment included two activities for students: 1) a visual art self-reflection requiring students to use any visual art medium to depict what they think it means to practice as a pharmacist with cultural safety, accompanied by a brief written description and (2) an in-class session with a brief lecture component, small and large group sharing of reflections, debrief of experiences, and student peer review of the visual reflections. The post-term portion included a similar self-reflection activity and an in-class session that now asked students in their small groups to: 1) compare their pre/post reflections, and 2) collectively create a new summative visual that depicts the entire group's thoughts. Surveys and a focus group were used as an additional source of data. Critical Analysis: Survey responses (n=215) indicated that feedback for the assessment was highly positive, with 77% of students recommending the activity for future first-year pharmacy students. Students also validated the utility of the assessment, with 70% strongly or somewhat agreeing that the activity was valuable for their learning, 80% strongly or somewhat agreeing that the activity stimulated their thinking, and 81% strongly or somewhat agreeing the activity accurately reflected their true feelings on practicing cross-culturally. © Individual authors.

DESCRIPTION OF THE OPPORTUNITY

In response to the Truth and Reconciliation Commission of Canada: Calls to Action 23-24[1], health sciences programs in Canada have worked to increasingly prioritize Indigenous[2] content on cultural safety. Cultural safety is an outcome that results from the shared respect, meaning, knowledge, and experience of learning together with dignity, free of racism and discimination.[1] Our work has been guided by various Faculty and Institutional strategic plans, including Shaping UBC’s Next Century and the Faculty of Pharmaceutical Sciences Catalyst for Change: 2017-2022. Meeting these guidelines for content inclusion has created the opportunity to develop standards and authentic assessments that accurately reflect students’ cultural safety learning. When applying typical Western quantitative standards and assessments to cultural safety learning, we risk only superficially addressing the learning complexities, knowledge, and behaviors of the subject. In addition, these methods are often not led by Indigenous scholars nor aligned with Indigenous views of learning, perpetrating one of many challenges to naturalizing Indigenous knowledge.[2] Using these colonially influenced standards and assessments potentially undermines the very learning they are intended to capture.[3]

To overcome the limits of traditional assessments, a variety of techniques have been used in health professions education to document student learning of culture safety.[4,5] For example, portfolio assessments can showcase how learning unfolds longitudinally, using multidimensional sources of understanding.[6] Group and individual reflections can help students understand how the “hidden” curriculum impacts their developing professionalism and identity,[7] and photo stories can allow students to uncover stigmatization and build patient-focused awareness.[8] However, we sought a method for assessing health professions students’ learning of these topics in large group, lecture-style formats.

One method for a scalable assessment of cultural safety is reflexive photography. Reflexive photography is a qualitative research technique requiring participants to explore and react to interactions with their environment through photographs and personal reflections.[9] This technique was preferred by Indigenous experts and partners in our work, who valued interactions with self and environment, over assessments involving written reflections or exams. In recent years, the technique has been used to assess cultural competency in higher education, including health professions programs such as nursing.[10] However, literature on the use of reflexive photography for assessing cultural safety is limited to small student cohorts. The need to examine this assessment in large lecture formats is underpinned by the growing importance of cultural safety topics in pharmacy education.

STATEMENT OF THE INNOVATION

To address the limitations of traditional assessment methods of student cultural safety learning and scalability to a large class setting, a novel assessment using a visual arts reflection approach was developed (JM & LL). This methodology is similar to reflexive photography and PhotoVoice; in our model students were able to choose any visual art medium. This was developed after extensive consultation with Indigenous partners and community members highlighted the need for better and more authentic assessment in this area. Particular attention was given to a sustainable approach, creating supporting resources for partnering Indigenous communities and other faculty who may be interested in using this method, and the goal of easy implementation with other health professions programs.

THE EDUCATIONAL INNOVATION

To inform the initial planning phase, the project team connected with local Indigenous and non-Indigenous experts on arts-based pedagogy and assessment and conducted a literature review and environmental scan related to arts-based qualitative assessments and methods for assessing cultural safety. In addition to this external engagement, the instructional team was led by non-Indigenous pharmacy faculty (JM & LL) and supported by a small group of five Indigenous and non-Indigenous experts in curriculum, art, and qualitative design. Pharmacy students, who had recently completed Year 1 of the Entry-to-Practice PharmD program (PharmD program) at the University of British Columbia, were also consulted throughout. Experience and materials from a visual art reflection used in an existing Indigenous health elective course in the PharmD program were also captured. Given the topic, instructor preparation focused on the potential for triggering or unsafe student conversations during the session and included consultation with colleagues who have implemented mandatory Indigenous content in other programs. Finally, with the goal of scaling this assessment to other programs, interprofessional consultation on the project included faculty from medicine, nursing, and midwifery. Collectively, a new assessment protocol, activity document, session material, and rubric were created and reviewed by Indigenous partners that had self-identified as having personal interest in this area. An Indigenous advisory committee also reviewed and had oversight on the implementation of this activity.

The mandatory assessment spanned the 13-week period of Term 1 for 223 first year students in the PharmD program. This term was chosen to capture incoming first year pharmacy student perspectives and to coincide with scheduled introductory content on Indigenous health and cultural safety. The assessment was deployed in a pre- and post-term style for a total of four distinct activities. The pre-term assessment included: 1) a visual art self-reflection requiring students to use any visual art medium to depict what they think it means to practice cultural safety as a pharmacist, accompanied by a brief written description and 2) a 2 hour in-class session with a brief lecture component, small and large group sharing of reflections, debrief of experiences, and student peer review of the visual reflections. The purpose of the peer review was to generate a sense of depth in the reflections, offer constructive feedback, and to allow students to communicate their thinking and understanding in a safe way. Peer review was anonymous using the iPeer platform (http://ipeer.ctlt.ubc.ca) and followed a 4-point rubric on 3 criteria: 1) Significance (did this image contribute significantly to my/our understanding of social or cultural life?) 2) Reflexivity (is there adequate self-awareness and self-exposure on behalf of the student for the viewer to understand the point of view?) and 3) Resonance (did this image affect me, emotionally and/or intellectually?). The post-term portion included a similar self-reflection activity, with the addition of asking students to compare with their pre-term art. During the 2 hour in-class session, students returned to the same small groups and were asked to co-construct a new summative visual that explores the entire group’s emerging understanding. Chart paper and markers were provided to all groups. A summary of the time allotment for these activities are provided in Table 1.

Table 1.

Visual Arts Reflection Activity Timeline

Stage	Activity	Time allotted
Pre-Term Activity 1 (Week 2)	Online individual assignment	2 weeks to complete
Pre-Term Activity 2 (Week 3)	Introduction lecture	15 min.
	1st small group discussion	20 min.
	1st large group discussion	20 min.
	2nd small group discussion	20 min.
	2nd large group discussion	20 min.
	Wrap-up	15 min.
Post-Term Activity 1 (Week 12)	Online individual assignment	2 weeks to complete
Post-Term Activity 2 (Week 13)	Introduction lecture	15 min.
	Small group discussion	20 min.
	Collaborative reflection	30 min.
	Large group discussion	30 min.
	Wrap-up	15 min.

Introductory content on Indigenous health and cultural safety was taught throughout the term. A 3-hour lecture series was embedded in PHRM 100 – Foundations of Pharmacy, introducing cultural safety, stereotypes, social determinants of health, and anti-racism. An additional 8 hours of interprofessional activities followed, covering personal relationships with land and health, colonialism, and residential schools. These interprofessional activities were deployed across 3 online modules and 2 face-to-face small group activities with students from medicine, nursing, dentistry, and others. In total, students received 11 hours of mandatory Indigenous health content between the pre- and post-term visual art reflections.

To better understand perspectives of this pilot assessment, students were asked to complete a survey with the pre-term reflection and a survey after the post-term reflection. Survey questions were presented using a combination of 5-point Likert scales and open text boxes to identify strengths, weaknesses, and overall impressions of the assessment. After completion of the term, two focus groups were conducted (n=7) to further explore themes and student opinions identified in the surveys. Focus groups were 30-minutes each and were conducted by a graduate student (KM) using a semi-structured approach developed by the project team. Focus groups were audio recorded, de-identified, and transcribed by KM, with final review of the data by the project team.

To better understand faculty perspectives, 4-6 members of the project team were present at one or both of the reflection sessions, offered tailored feedback to support group discussions, and noted observations to share back with the team. The study design, surveys, and project documents were reviewed and approved by the university’s Behavioural Research Ethics Board.

Findings

Student art reflections represented a diverse range of perspectives. Common motifs included icebergs, tree roots, or handholding. Unique depictions included music as language, the construction/deconstruction of culture using building blocks, and drawings incorporating different languages. Differences in pre- to post-term submissions showed the evolution of understanding and awareness of cultural safety. An example of a student’s pre- and post-term submissions are provided in Figure 1.

Figure 1:

Example of Activity Evolution and Marking Rubric

Assignment Caption: In my first reflection, I focused on going beyond the structure of [care] by asking questions that aren’t involved in the typical style of flowsheet. However, through this term I have learned how deeply engrained colonialism is in every societal structure and how this poses vast barriers for [I]ndigenous peoples. Therefore, for my second reflection I cut up the flowsheet from my first reflection and created a new shape with it. This represents how we don’t just need to go beyond our current structure, we need to break it down and recreate it.

Student feedback of the assessment was overwhelmingly positive. Seventy-seven percent (77%) of students stated that they would recommend the activity for future first-year pharmacy students and 70% agreed or strongly agreed that doing the activity was valuable for their learning. Eighty percent (80%) of students also agreed or strongly agreed that the activity stimulated their thinking; 75% agreed or strongly agreed that the activity gave them the chance to show what they learned; and 81% agreed or strongly agreed that the activity accurately reflected their true feelings on practicing cross-culturally.

Student comments from the focus groups were generally positive. Independent of artistic skill level, many students enjoyed the opportunity to deviate from the more typical activities of the program by participating in this unique modality. Students commonly noted that completing the reflection made them feel uncomfortable and sharing their artwork was intimidating. Finally, students also identified that greater Indigenous health content would have been helpful to better educate and inform on final reflections.

CRITICAL ANALYSIS

It was unclear how pharmacy students with experience in predominantly traditional assessment methods would receive an assessment emphasizing arts-based reflection.

Implementation of each component of the assessment was simple and cost-effective despite the 223-student class size. All required software was already in use in the program, setting a low technology barrier for students and the instructor. The only costs incurred were the purchasing of markers and chart paper for the post-term activity. The instructor and facilitators were enthusiastic about the assessment, remarking that reflections were overall of high quality despite the small grade incentive, and that students were generally engaged and on task during the large-group sessions. Informal feedback and conversations with Indigenous partners were also positive and supportive of expanding this method.

After analyzing the submissions, many students were found to use images from the Internet for their narrative write-ups. Although this was not initially a concern, the overwhelming majority of submissions noted by the project team as being “lower effort” were created using internet images. It is apparent that mitigating the superficiality of copying an existing image will be important in achieving the assessment’s fullest potential. As such, future iterations of the assessment will prioritize the act of creating art by requiring students to submit only original works. While determining “effort” in an arts-based reflection is an imprecise judgement, factors including depth of reflection, intentionality, and the accompanying written description were all considered. When reviewing only those submissions deemed “lower effort”, differences between pre- and post-term reflections were difficult to discern. Any future expansion of this assignment will require careful attention to intentional and unintentional perpetration of stereotypes that “lower effort” submissions may be particularly susceptible to and should be paired with conversations on positionality, power, and ethical engagement.

It is important to note that insecurity in artistic ability was a significant concern expressed in the focus groups, and as such, efforts should be made to have students feel safe to express themselves in future iterations of the assessment. This should include providing a range of choices of representation (e.g., 3-dimensional art mediums such as clay or building blocks; create a mask, etc) to limit the risk of students feeling pressure to demonstrate artistic proficiency, and reminders from the instructor that artistic ability is not a focal point of the assessment. This was a notable concern given that the activity was assessed only for completion.

Faculty provided feedback that will be valuable for future iterations of the assessment. The most significant suggestion was consideration of a longer gap between the pre- and post-term reflections. Although explicit instruction related to cultural safety and Indigenous health content was only available in this one term of the program, it is likely that students would have even greater insight into their own developing competencies as they advance in the program, providing for a potentially more drastic contrast between reflections. As Indigenous and cultural safety teaching expands in the curriculum, this opportunity can be explored as a longitudinal method of assessment. A longer gap between the reflections risks confounding the extent to which students’ development is a result of effective Indigenous-related content, or a result of unrelated interpersonal growth throughout the program. Regardless, this suggestion may emphasize student-perceived limitations of the program’s current cultural safety and Indigenous health offerings, and presents an argument for broader curricular implementation of this content that may not have been identified without this assessment.

NEXT STEPS

To meet institutional and national guidelines for Indigenous health and cultural safety content, it is essential that health professions programs integrate scalable and effective assessments of this content into their curricula. Although, the best assessment of cultural safety in students should come directly from Indigenous partner participation or those receiving care, this is difficult to achieve or sustainably implement in a large classroom setting. This assessment was positively received by students, faculty, and Indigenous partners while demonstrating ease of implementation in a large-classroom format. Most importantly, the assessment provided valuable insights into first-year pharmacy students’ understandings of cultural safety and Indigenous health, and their perceptions of the program’s delivery of this content. This text provides a valuable model for pharmacy programs and other health professions institutions to efficiently implement an assessment of cultural safety into their own curriculums. Feedback received will be used to refine the assessment for future iterations in pharmacy and will help guide implementation into other health professions programs in the institution.