Virtual Reality: An Immersive Tool for Social Work Students to Interact with Community Environments.

Research supports various benefits of using virtual reality (VR) within social work education. As a pilot study, this paper describes the impact of a 360 VR simulation designed to immerse students at a New York school of social work in a typical New York City neighborhood, with the goal of helping them learn about how its history, resources, demographics, and physical space impacts its inhabitants. This, in turn, is intended to support novice students in gaining familiarity with new social contexts and communities, and in connecting macro and micro nuances with practice. An overview of the role of VR in social work education is provided, followed by a description of the pilot 360 VR simulation developed by the authors of this paper, including the rationale for that development and the theoretical framework for its design. Using a mobile device and Google Daydream headset, students are guided through a panoramic urban environment by a pre-recorded voiceover that promotes reflective and analytical thinking as they observe the community through the lens of a social worker. Independent sample t-tests showed statistically significant changes in average scores between pre-and-post tests. Results of the pilot as indicated by pre- and post-survey of student perceptions and test of their knowledge are provided. Pedagogical and clinical practice implications for the 360 VR simulation are identified and discussed.

Introduction

Social work educators and professional accrediting bodies have written for a number of years about the importance of integrating macro and micro perspectives in social work practice (Asakura et al., 2018; Bogo, 2015; Boitel & Fromm, 2014; Rothman & Mizrahi, 2014; Wayne et al., 2006). In order to more effectively help students, make connections between macro and micro concepts, social work education could employ more authentic forms of practice that bridge classroom learning and professional application. Virtual simulation is one teaching and learning scaffolding strategy aimed at enhancing this connection and better preparing students for real professional practice. Virtual simulation may be used to improve social work education in order “to stimulate new ways to think about social issues and to provide a safe practice arena for skill development” (Huttar & BrintzenhofeSzoc, 2020, p. 139). However, there is a shortage of research on the use of virtual simulations to help social work students integrate micro and macro social work concepts in practice. To broaden this research direction, this pilot study aims to investigate learning outcomes and student perceptions of a 360 VR simulation that immerses the student in a physical setting to connect the micro and macro concepts within social communities and contexts.

Literature Review

Virtual Simulation Based Learning

Virtual simulations are computer-generated or 3D programmed environments that immerse the viewer, allowing them to experience content visually in the same way one experiences it in the real world (Reinsmith-Jones et al., 2015). Technology used to experience virtual simulations include computers, web browsers, mobile phones, and virtual reality (VR) and augmented reality (AR) headsets, and can be built using software platforms such as Second Life, Unity, and Wonda VR, to name a few. VR is a type of technology used to experience a simulated world through desktop computers, mobile devices, and headsets with or without hand controllers, or in combination with these other technologies. The use of VR headsets to interact with the virtual environment provides an even more immersive experience than animated 3D virtual environments such as Second Life or static 3D video viewed on a desktop or through a browser. Some virtual simulations are designed to be accessible day or night and are therefore able to accommodate multiple student connections at one time (Huttar & BrintzenhofeSzoc, 2020). According to Wann and Mon-Williams (1996), VR environments “capitalize upon natural aspects of human perception by extending visual information in three spatial dimensions,” while also supplementing information with other stimuli and having the user interact with various data sets (p. 833).

Virtual environments, and simulations generally, grant students “safe spaces” to make mistakes and to hone skills sets that they would not ordinarily have time to process in “real” settings (Boulos et al., 2007; Huttar & BrintzenhofeSzoc, 2020). Research confirms that students feel more comfortable and less under pressure to perform in virtual settings than in face-to face role plays, a more traditional method of simulation (Fitch et al., 2016). To facilitate learning, students can pause the simulation in order to revisit a scenario multiple times or attempt to answer questions more than once. These systemic techniques improve reflective skills (Boulos et al., 2007), which ultimately could lead to better practice processes and understanding.

Simulation training, particularly when incorporating standardized patient information and internal rubrics, can be used to measure and assess a student’s proficiency level in the areas of practice and field learning (Huttar & BrintzenhofeSzoc, 2020; Wilson et al., 2013). This is an improvement over unstandardized assessment modes (e.g., via field instructors) that are often used to evaluate student competencies (Washburn et al., 2016). Social work techniques are not always as clear or easily defined as those in other health-related disciplines because the delivery and operationalization of certain interpretive and evaluative skill sets are more abstract (Rishel and Majewski, 2009).

Virtual Simulations in Social Work Education

The use of virtual simulations is steadily increasing within social work education and holds promise for advancing pedagogy in ways that bring students closer to direct practice (Huttar & BrintzenhofeSzoc, 2020; Trahan et al., 2019). Indeed, the use of technology and the value of virtual communities are highlighted within the standards maintained by social work accreditation bodies (NASW, ASWB, CSWE, CSWA, 2017). More broadly, simulation-based learning in social work has demonstrated effective outcomes for adult students because it provides problem-focused, self-directed learning opportunities and relevant applicability to real agency-based work (Huttar & BrintzenhofeSzoc, 2020; Washburn et al., 2016). The flexibility of virtual simulations allows for greater diversity and variability in field settings and in client demographics than could consistently be found in “real” field placement outlets (Wilson et al., 2013). It can also provide students with “immediate” feedback and knowledge from their interactions within an assortment of social work experiences (Huttar & BrintzenhofeSzoc, 2020). Admittedly, some virtual designs are limited; in particular, third-party tools like Second Life or other 3D computer-generated systems may lack interactional content or the functionality to achieve a more customizable and immersive space like virtual reality (Doel & Cooner, 2002; Levine & Adams, 2013; Reinsmith-Jones et al., 2015; Tandy et al., 2016; Wilson et al., 2013). Nonetheless, virtual simulations, in general, provide students the advantage of gaining field/practice perspectives before entering, or simultaneous to, the practicum phase of the curriculum.

Research shows virtual simulations are beneficial in preparing social work students (undergraduate and MSW) for direct social work practice (e.g., client interviews, case management skills, identification of discrimination and oppression) (Huttar & BrintzenhofeSzoc, 2020). Through scaffolded evaluatory methods, virtual simulation can enable social work students to obtain a clearer understanding of the intersectionality of the social work curricula and bridging abstract concepts to real practice through a simulated experience, but also by assisting in the evaluation of subtle skills (Doel & Cooner, 2002; Huttar & BrintzenhofeSzoc, 2020). However, these findings come from just seven studies included in a systematic review conducted by Huttar and BrintzenhofeSzoc (2020). Further, the authors found that only four out of the seven studies evaluated student learning effectively. Huttar and BrintzenofeSzoc (2020) note that none of the studies addressed how virtual simulations could be used in “macro-level social work practice such as advocacy or community engagement” (p. 135). In their conclusion, they suggest that more research be conducted to investigate how virtual simulations improve knowledge, skills, and attitudes.

A 360 VR Simulation

Rationale for Development

To utilize the strengths of virtual simulations identified by prior research in social work education as well as address some of the educational gaps in the literature, the authors of this paper created and evaluated a 360-simulation using Wonda VR™ and the Google Daydream headset with a mobile device. Specifically, this project aims to more effectively support novice students to gain familiarity with new social contexts and communities (Felton & Harrison, 2017; Harrison & Ip, 2013; Ross et al., 2019). While it is possible to bring students physically into urban settings such as the Lower East Side in New York City, it is challenging to provide instruction and practice while guiding novice students individually through the process of observation of and reflective thinking about community nuances. This VR simulation can be used as a strategy to scaffold social work concepts, while also guiding novice students in navigating the unfamiliar community environments in which they study and work. For example, social work educators utilize biopsychosocial and ecological perspectives to help students contextualize how macro and meso circumstances affect individuals in a multitude of ways (Berzoff, 2011). While classroom instruction on such perspectives may be highly abstract, this VR simulation aims to scaffold didactic instruction received in the classroom with concrete knowledge of the complex urban settings where individuals and communities exist. During internships in community agencies, the gap between academic preparation and real-world practice can become obvious; macro and micro topics are often treated separately with diminishing attention on macro nuances, and community practice, and skills development (Boitel & Fromm, 2014; Liddell & Lass, 2019; Rothman & Mizrahi, 2014). Virtual simulations can help build students' cultural understanding of the community and prepare learners for community-based work (Huttar & BrintzenofeSzoc, 2020).

Project Description

The 360 VR simulation takes place in the Lower East Side (LES) of Manhattan, New York. It was chosen due to its demographic diversity, immigration history, social resources, and connection to the settlement house movement, each an important facet within social work (Citizens’ Committee for Children of New York, 2015). The 360 VR simulation was developed by an internal media design and development team who videotaped each scene of the 360-video using a Nikon KeyMission™ 360 camera and standard tripod. Each scene of video footage was stitched together using Adobe Premiere software, and an actor provided a recorded voice-over audio. The media team adopted Wonda VR™, a prosumer-oriented 360 video-editing software, which presented a robust alternative to other cost-prohibitive and less user-friendly professional development software platforms on the market. Wonda VR™ enabled developers to add gaze-activated hotspots in each 360 scene in order to provide the viewer with historical and demographic information about the observed environment.

The learning objectives and goals of the VR learning experience include identifying key terms (e.g., gentrification, territoriality) and understanding historical contexts, demographic information, and social risk and protective factors. This 360 VR simulation is approximately 12 min in duration, and includes an instructional guide, learning objectives, content chapters, an introduction, and 17 navigable and interactive panoramic scenes (environments). Each panoramic scene contains audio and textual dialogues, and a situational map.

The 360 VR simulation starts with the disembarkation from the F train at the Forsythe subway station (Fig. 1) in the Lower East Side of New York City. A female narrator guides the student with audio and poses questions to focus their attention and prompt critical reflection on the environment as they navigate the space. In one scene the narrator poses this question to the student observer: “As you stand on the train platform, what do you notice about the people that are around you? Think about the transportation resources that are available for your client in this neighborhood.” In another scene the narrator poses this question to the student: “A sociopetal space encourages social interactions. Identify and name two such spaces on Grand Street. How might these be used for young and older clients?”.

Fig. 1

First scene from 360 VR of Forsythe train station with individuals on the train platform. “Hotspot” is shown with the map of the Lower East Side

The narrator also guides the student to interactive "hotspots" (Fig. 2) that provide additional written information about the community landscape, including data points on demography, mental health, and housing. Additional reflective exercises are embedded in these interactive hotspots. One clickable hotspot asks the student to “Review the statistics on both poverty and median incomes. In your opinion, how does the community reflect the statistics, particularly as it relates to the current photo clip?” These “hotspots” are presented throughout the 360 VR simulation to further encourage students to think critically about the population inhabiting the Lower East Side.

Fig. 2

Street scene from 360 VR of the Lower East Side. “Hotspot” indicates the number of police stations, fire stations, and EMS resources in the area

Theoretical Framework

The design of the 360 VR simulation is grounded in situated learning theory (Brown et al., 1989; Lave & Wenger, 1991) and in interrelated factors that directly and indirectly influence learning outcomes in VR environments, such as immersion, presence, emotion, motivation, and cognition (Dengel & Magdefrau, 2018; Witmer & Singer, 1998). The implications of situated learning theory include the importance of context in learning, the need for students to build strong mental models and transferable knowledge to function in real clinical settings, and the role of prior knowledge in decision-making and problem solving (Brown et al., 1989; Lave & Wenger, 1991).

Situated Learning Theory

Situated learning theory views learning as connected to real context in a community of practice (Lave & Wenger, 1991). Within this theory, legitimate peripheral participation (LPP) is seen as the first step for novices; LPP means situating themselves in the periphery of a community of practice, and from there observing the rules of communication in order to understand them. LPP provides a way of understanding how learners move from that periphery to a more central role with greater professional responsibility (Lave & Wenger, 1991). Understanding how students assimilate and process knowledge helps designers of educational environments prepare situated learning domains. There are two basic principles for situated learning: (1) knowledge needs to be presented in an authentic context that would normally involve that knowledge, and (2) learning requires social interaction and collaboration (Lave & Wenger, 1991). Both principles are applied to the design of this 360 VR experience. It directly employs the first principle insofar as it immerses the learner in an authentic community setting and then asks them to think like a practitioner, using a set of cognitive tools to discern challenges their clients may face in that environment. This kind of immersion allows the learner to have an inherently coherent and meaningful experience, in contrast to other methods of instruction which present skills and concepts as abstract and disconnected (Lave & Wenger, 1991).

Indirectly, the VR design also makes room for the second principle of situated learning, insofar as it is intended to initiate the social worker’s process of enculturation into the community. Once situated and immersed in the environment, the learner engages cognitively through focused observation, mental note taking, and reflective thinking, as they build and connect more knowledge about the community, its inhabitants, and the role of a social worker within it. As the learner actively controls the direction of their observation in the 360 environment, the audio guided questions help the learner maintain focused and purposeful attention, and model reflective questions a professional might ask of themselves in the field. These processes ground students’ learning in layers of social consciousness. Further, this experience is intended to be used in a physical or remote classroom followed by group discussion to facilitate continued reflection, meaning-making, and assessment of learning outcomes in a collaborative context. Simulations should be integrated into a course plan as supplementary material, as scaffolds, rather than replace traditional instructional models (Smetana & Bell, 2012).

Immersion and Presence

As authenticity is central to situated learning theory, the role of presence and immersion are important to creating authentic social work environments. Presence and immersion enhance learning and performance (Witmer & Singer, 1998). More specifically, it is the meaningfulness and coherence of an environment that promotes learning (Underwood & Schulz, 1960), and because the users perceive themselves to be situated in the virtual world, they experience that world directly, making the experience more meaningful (Witmer & Singer, 1998). VR creates a deep sense of immersion and presence through high perceptual fidelity, focused attention, and naturalness, a sense that is not similarly achieved when information is absorbed through a book, movie, or video games (Witmer & Singer, 1998). Research shows that a head-mounted-display (HMD) immersive experience results in greater learning compared to the level of immersion experienced through a desktop computer setting (Mania & Chalmers, 2001; Mikropoulos, 2006). Presence, the feeling of being there (Lee, 2004; Nowak & Biocca, 2003) and immersion (Witmer & Singer, 1998), have a positive effect on memory and learning (Lee et al., 2010; Lin et al., 2002). When other subjective measures such as engagement and enjoyment are taken into account, results are even more striking (Lin et al., 2002). Immersion and presence are central to meeting the learning objectives of this VR environment, and that is why 360 panoramic video of a typical New York City community was used in design.

Motivation, Cognition, and Emotion

Other important factors influencing the success of VR educational environments on learning outcomes are motivation, cognition, and emotion (Dengel & Magdefrau, 2018). With respect to motivation, providing clear goals and objectives helps drive behaviors aimed at learning and exploring (Dengel & Magdefrau, 2018). To foster motivation, Dweck (1986) suggests setting challenging but proximal goals that are directed toward learning and not performance. The design of this 360 VR experience includes an instructional guide, learning objectives, and reflections to help cognitively support and motivate the learner through the experience from beginning to end. There are no embedded performance measures in the simulation so as to intrinsically encourage learning through this guided discovery platform.

Engaging students cognitively by using reflective prompts can also influence learning outcomes (Dengel & Magdefrau, 2018). Research on metacognition suggests that the incorporation of reflective exercises into learning experiences improves the potential for knowledge transfer to future practice (Bransford & Schwartz, 1999; Brown, 1989; Flavell, 1976). The main goal of this 360 VR simulation is to prepare students for future practice, consequently the use of the reflective prompts are embedded throughout this 360 VR simulation to support this goal. They encourage the learner to pause, observe surroundings, identify key characteristics, reflect on the environment, and make connections between micro and macro nuances in the community to better understand their clients’ needs. This practice of observation, reflection, and connection is a common social work activity in the field (Bogo, 2015).

Positive emotions (e.g., enjoyment, pride, and excitement) or negative feelings (e.g., frustration, indifference, stress, anxiety, annoyance, and confusion) can influence cognitive engagement levels and thus learning outcomes (Dengel & Magdefrau, 2018; Domagk et al., 2010). For example, poorly designed learning environments, faulty equipment or technology may cause stress, anxiety, or frustration, resulting in cognitive disengagement and thus poor learning outcomes. Learning experiences that foster positive emotional factors, conversely, can beneficially impact learning outcomes (Villavicencio & Bernardo, 2013). The design of this VR experience aims to mitigate negative feelings and promote positive emotions by employing various evidence-based multimedia design principles, such as the modality principle (Mayer, 2009), signaling principle (Mayer, 2009), voice and personalization principles (Mayer, 2009), and guided discovery principle (Moreno & Mayer, 2005), to name a few. These principles have been shown to optimize cognitive processing of the information in the learning environment, render a positive experience, and promote learning.

The following results from the pilot study will show broadly the impact of the design on student learning and perceptions.

Methods

Pilot Study Design

A one group pretest/posttest design guided this pilot study, which investigated the research question “Does a virtual reality simulation impact student learning of a community environment?” Pre-post survey questions also broadly investigated student perceptions and attitudes about the experience. The Formal Internal Review Board at the university approved this pilot study and deemed the procedures and content met ethical standards.

Recruitment

Participation was voluntary for all first-year MSW students enrolled in the generalist Practice course during the 2019 academic term without any other form of exclusion. The Chair of the Practice curriculum and Associate Dean of the MSW program approved and facilitated email communication about the pilot study to all first-year MSW students. Interested students contacted the principal investigator who scheduled their appointment time with the VR lab. Students were incentivized to participate with a $20 gift receipt to the school's bookstore, but their choice did not impact their course grades.

Data Collection

Over the course of the spring and fall semesters in 2019, thirty first-year graduate students volunteered to participate in the project. Data collection included a pre-post survey/test designed in Qualtrics™ and delivered via a laptop browser. All consenting participants were assigned a unique identifier in order to anonymize their data and protect their identity. Analyzed data resided in the university’s approved secure server.

Instruments and Measurements

Instruments and measurements used in this pilot study included a 360 VR simulation, a pre-test and survey, and a post-test and survey. As previously described, the 360 VR simulation is approximately 12 min in duration. It includes supportive material in the form of an instructional guide, learning objectives, content chapters, an introduction, and 17 navigable and interactive panoramic scenes (environments). Each panoramic scene contains audio narration, textual information, and a situational map.

The goal of the pre-survey, crafted by the authors of this study, was to determine if participants’ prior experience influenced their performance. Questions covered prior experience with VR technology, familiarity with the Lower East Side, and years lived in NYC. There were three demographic questions, five questions about the participants’ prior technology experience, one question about assistive technology needs, one question about prior experience with the Lower East Side, and one question about years living in New York City.

The post-survey contained twenty-eight questions divided into three parts (Learning Experience, Technology Experience, General Attitudes), and its goal was to gauge student perceptions and attitudes about their learning experience, as well as determine the success of the design supported by learning theory and principles. It contained questions appropriated from a survey provided and crafted by Dr. Yan Shen at North Carolina State University's Distance Education and Learning Technology Applications program, who designed it partly on the work of Cheng et al. (2014). All other survey questions were crafted by the researchers of this pilot study. While the constructs and questions used in this post-survey were not adopted from previously tested scales nor qualitatively tested by expert judges prior to use, they were chosen for their relevance to the learning theory and principles which informed design of the 360 VR simulation.

Part One of the post-survey, under the category learning experience, contained thirteen questions. Four focused on immersion in VR and eight on the VR program’s guided reflective questions. The post-survey questions asked participants to rate their agreement or disagreement with statements using a seven-point scale (1 = strongly disagree, 2 = disagree, 3 = somewhat disagree, 4 = neither disagree or agree, 5 = somewhat agree, 6 = agree, 7 = strongly agree). The last scale question in this category asked a question about participants’ preference for the virtual reality experience versus visiting a real-world environment to complete the same learning activity using a five-point scale (1 = Definitely prefer visiting the real-world environment, 2 = Leaning toward visiting real-world environment, 3 = No preference, 4 = Leaning toward using a 360-virtual environment, 5 = Definitely prefer using a 360-virtual environment).

Part Two of the post-survey, under the category technology experience, asked nine Likert scale questions. Four questions surveyed participants about their agreement or disagreement with statements about the navigation and vividness of the 360 virtual environment using a seven-point scale (1 = strongly disagree; 7 = strongly agree). Five questions in this category asked participants to rate their response to experiencing any physical discomfort during or after the 360 virtual experience using a five-point scale (1 = never, 2 = rarely, 3 = sometimes, 4 = often, and 5 = every time).

Part Three of the post-survey, under the category general attitudes, asked nine questions about overall impressions of the 360 virtual simulation. Participants rated responses to six questions eliciting their feelings about using the 360 VR simulation to learn the material using a seven-point scale (0 = does not describe my feelings at all, 3 = somewhat describes my feelings, 6 = very much describes my feelings). Three additional questions asked participants about their agreement or disagreement with a statement about their attitudes toward using the 360 virtual simulation using a seven-point scale (1 = strongly disagree; 7 = strongly agree).

At the end of each categorized part of the survey, participants were asked to complete answers to open-ended questions. In the learning category, participants were asked to describe an aspect of the simulation that helped their learning, and an aspect that was not helpful to their learning. In the technology category, participants were asked about any negative experiences or challenges with the technology. In the attitudes category, participants were asked to explain their likes and dislikes about the simulation, as well as areas for improvement they perceived.

Participants also completed twenty-eight item pre- and post-diagnostic knowledge tests (Appendix: Table 1) to address the research question, “Does a virtual reality simulation impact student learning of a community environment?” The pre-post diagnostic tests were designed by the authors of this study. Questions were identical in each test. Administration of these tests aimed to assess success of learning objectives supported by the content in the 360 VR simulation. Fifteen multiple-choice questions probed student knowledge about the Lower East Side and their ability to identify features of the community and its inhabitants. Two client vignettes contained seven questions covering each case, questions which required students to apply micro and macro concepts based on the 360 VR simulation.

Table 1

Pre-Post test by question (Chi-square)

#	Question	Pre-test (N = 30)Prop correct	Post-test (N = 30)Prop correct	DiffPre-Post	Chi-squarep-value
1	Identify one event that makes the Lower East Side as a historically important location	0.733	0.433	− 0.3	0.036
2	Identify a sociopetal space that you observed on the Lower East Side. (Select all that apply.)	0.033	0.333	0.3	0.008
3	Most of the people living on the LES are Caucasian and earn a substantially high income	0.667	0.867	0.2	0.127
4	Unlike many parts of Manhattan and the outer boroughs, the Lower East Side is not affected by gentrification	1	0.967	− .033	1
5	What is territoriality?	0.767	0.933	0.167	0.148
6	What is a NORC?	0.5	0.9	0.4	0.002
7	The Lower East Side has various food outlets and farmer’s markets	0.867	1	0.133	0.121
8	Residents on the Lower East Side have fewer psychiatric hospitalizations than other areas of Manhattan	0.8	0.9	0.1	0.47
9	The LES has many busy thoroughfares and avenues	0.8	1	0.2	0.031
10	There are fewer than 10 schools on the Lower East Side but it has over Charter schools	0.5	0.967	0.467	0
11	The LES has a bustling business community and diverse types of products are offered to its residents	0.767	0.9	0.133	0.299
12	Residents of the Lower East Side have lower rates of substance/alcohol misuse than the rest of NYC	0.8	0.867	0.067	0.729
13	What are some signs of gentrification taking place on the Lower East Side? (Select all that apply.)	0.467	0.667	0.2	0.193
14	The Lower East Side has between 120 and 140 New York City Housing Authority properties	0.8	0.867	0.067	0.729
15	Due to its affluent population, the Lower East Side has less than five homeless shelters	0.767	0.833	0.067	0.747
16	Juan identifies as a Latino male. The Latinx population accounts for the majority ethnic group on the Lower East Side	0.633	0.833	0.2	0.144
17	According to stimulation theory, moderate levels of stimulation at one time is more conducive to behavior (e.g., steady, even). If you were accompanying Juan for a walk what location of the Lower East Side would you want to avoid given his PTSD diagnosis? (Select all that apply.)	0.267	0.4	0.133	0.411
18	The NORC that Juan lives in is part of New York City’s Housing Authority	0.633	0.733	0.1	0.579
19	You need to accompany Juan to a doctor’s appointment and Access-A-Ride vehicles are not available. His doctor’s office is near Rockefeller Center. What should you do if the train is the only other feasible option?	0.267	0.3	0.033	1
20	To assist with Juan’s mobility, you will need to monitor his movement outside of his apartment. The sidewalks on the Lower East Side are well maintained and there is little construction, therefore he will have few challenges in getting around the community	0.733	0.9	0.167	0.182
21	Juan’s challenged socio-economic situation is unusual in the Lower East Side, which is known to be the playground of the rich	0.633	0.9	0.267	0.033
22	As Chinese immigrants to the Lower East Side, Mei and her parents are an underrepresented demographic in that community	0.767	0.933	0.167	0.148
23	Mei and her brothers can attend a public school on the Lower East Side because that area of Manhattan has over 40 public school and 5 charter schools to choose from	0.633	0.9	0.267	0.033
24	Sometimes after school, Mei and her brothers wait for their mother at this street’s open seating mall, which once was the site of elevated train tracks. What’s the name of the street?	0.5	0.7	0.2	0.188
25	On immigrating to the Lower East Side, Mei and her parents found food markets, retailers, and restaurants that catered to the Asian community. This is an example of ___	0.533	0.7	0.167	0.288
26	An example of a sociopetal space on the Lower East Side that Mei’s brothers frequent is __	0.867	0.933	0.067	0.667
27	Mei is working part-time in a coffee shop. The rise of expensive coffee shops and clothing boutiques along the Lower East Side is a signal for what societal process?	0.967	1		1
28	Mei’s parents’ annual income fluctuates above and below the Federal Poverty Line. This is unusual considering almost 90% of the residents of the Lower East Side live above the poverty line	0.733	0.833	0.033	0.531

Pilot Study Procedures

Each participant was individually scheduled by a researcher to meet in a lab setting. Only one participant engaged in the pilot study at a time. The lab was equipped with a laptop computer for collecting consents as well as pre- and post-survey and test responses. The lab also provided a Google Daydream headset with mobile phone for viewership of the 360 VR simulation. In the lab, each participant completed an online consent form and pre-survey and diagnostic test using Qualtrics. Next, each participant received instructions on what to expect from the virtual reality technology. VR may induce physiological symptoms such as dizziness, nausea, and sweating in some users (Groen & Bos, 2008). These reactions usually result from the mismatch or discrepancy between visual and vestibular systems (Park et al., 2018). Consequently, participants were advised to communicate with the researcher if they felt dizzy, nauseous, or in any discomfort of any kind with the software or technology. Each participant then interacted with the virtual reality environment for about twelve minutes, until the simulation was complete. Finally, each participant completed a post-survey and diagnostic test using Qualtrics. Statistical computing of collected data used R software.

Results

Pre-survey

Demographics

Of the thirty consenting participants, 80% self-identified as female, 97% were enrolled in the two-year track MSW graduate program, and 93% were from the United States. While more voluntary participation from international students was preferred, the cohort of mainly U.S. students sufficed for this pilot study.

Prior Experience

Participants reported feeling comfortable using technology in general (M = 4.3, SD = 1.09), but only about half answered “yes” to having prior experience using a VR headset or Google cardboard set (M = 0.53, SD = 0.51). Most reported moderate familiarity with virtual reality experience (M = 2.3, SD = 0.99). Despite some familiarity with the VR headsets, on average, participants reported less than 30 min to never in terms of time spent using virtual reality simulations (M = 1.80, SD = 0.81). Only a few reported using a desktop computer to view VR simulations, none had used a mobile device with Google cardboard or a VR headset headset (M = 2.78, SD = 0.65). When asked about needing assistive technology or not, only one participant selected yes but specified in a follow-up open response that they may need to use glasses with the headsets. Participants reported a mean average of 2.67 years (SD = 1.42) living in New York City and, on a scale of 1 to 5 with 1 being least familiar, a 2.83 mean average (SD = 0.79) familiarity with the Lower East Side.

Post-survey

Quantitative Results

Overall, perceptions related to immersion scored positively above average (M=5.79, SD=1.2), and perceptions related to the guided reflections embedded in the experience also showed above average positive agreement (M=5.72, SD=1.21), suggesting participants found the experience beneficial to their learning. In relation to technology experience, perceptions pertaining to navigation and vividness scored positively above average (M=5.72, SD=2.04) and perceptions related to discomfort were below average (M=2.21, SD =1.21), suggesting the technology and design did not significantly impair the experience. Overall, attitudes about the experience scored above average (M=5.19, SD=1.74) and attitudes related to use were also above average (M=5.37, SD=1.47), suggesting the experience was exciting and motivating.

Statistical tests of non-zero correlations showed significant findings for questions grouped in categories related to immersion, guided reflection, and attitudes-feelings. Findings were positively correlated for immersion and guided reflection (r (28) =.556, p=.001), immersion and attitudes-feelings (r (28) =.793, p=0), and guided reflection and attitudes-feelings (r (28) =.358, p=.036).

Qualitative Results

Two main themes emerged from positive responses for each category (learning, technology, and attitudes): the use of reflective questions was widely considered helpful to learning, and participants widely noted the simulation’s immersive feeling. The following quote reflects the intersection of these themes: “It was helpful to be in a setting while being asked how a client with PTSD may react. You could easily pick up the small details that could affect the client which I may not have thought of otherwise.” Counts of positive responses showed participants found their learning most enhanced by the simulation’s making connections to theory through the use of guided reflective questions (n=13) and its using visual examples of concepts (n=9). Counts of positive attitudes towards the experience were highest for the immersive quality (n=18). These written comments suggest how the students connected the experience of being immersed in the environment to social work practice concepts through use of the guided reflection questions.

The main negative theme to emerge centered around the VR headsets causing blurriness and lacking visual clarity. The following quote exemplifies this theme: “The visual was a bit blurry. A better fitting headset might help.” While counts of negative responses such as blurriness and dizziness were low for qualities not helpful to learning (n=2), counts were high for aspects of the technology related to blurriness (n=11). Lastly, attitude responses were highest for making improvements to the clarity of the visuals (n=11). These written statements about experience with the technology (e.g., Google Daydream headsets) suggest how technology can detract from the experience, but does not appear to have severely impacted learning.

Pre-Post Test

To observe differences between the pre- and post-test mean (M), standard deviation (SD), Wilcoxon signed-rank tests, and t-tests were performed on performance measures and showed statistically significant results. Participants scored on average 18.43 (SD = 3.2) on the pre-diagnostic test compared to an average of 22.5 (SD = 2.8) on the post-diagnostic test. To check the assumptions of the t-test, we applied the Shapiro-Wilks test to assess the normality of the total score and percentage correct variables. No significant deviations from a normal distribution were detected (pre-test W = 0.97, p = 0.56, and post-test W = 0.95, p = 0.13). A t-test comparing the pre- and post-test average outcomes revealed a statistically significant difference, t (57) = 5.23, p < 0.001, suggesting the 360-virtual reality experience positively impacted learning. Wilcoxon signed-rank tests showed similar results, W = 143, p < 0.001.

An additional regression analysis examining the pretest factors of years lived in New York City and familiarity with the Lower East Side did not significantly statistically impact the post-test scores. Correlations between the post-test score, immersion, guided reflection, and attitudes-feelings were moderate and some relationships among categories are stronger. However, each of those variables had an insignificant (p > 0.05) correlation with the total score (although a few were close). Findings were moderately negatively correlated for post-test score and immersion (r (28) =  − 0.34, p = 0.06), post-test and guided reflection (r (28) =  − 0.33, p = 0.07), and post-test score and attitudes-feelings (r (28) =  − 0.21, p = 0.27). Linear regression controlling for immersion, guided reflection, and attitudes-feelings scores also did not show statistically significant results in any of these areas on the post-test scores.

Table 1 (Appendix) provides an item-by-item comparison of the scores on the pre- and post-surveys. The analysis implements a form of a Chi-squared test. When searching for areas of difference, an adjustment for multiple comparisons would be required. An overall significance level of 0.05 would imply a threshold of 0.0018 for any p-value. A few questions appear to demonstrate meaningful improvements in effect size, such as the knowledge of the students, the definition of a naturally occurring retirement community, identification of sociopetal spaces, a knowledge of the balance between public and charter schools, and the ethnic and demographic characteristics of the community.

Discussion and Implications

VR simulations are emerging to improve student learning in social work education (Huttar & BrintzenofeSzoc, 2020). The main benefit sought is an immersive and authentic experience that allows students to safely and repeatedly practice with immediate guidance and feedback as a scaffold to prepare for more complex and real community environments (Huttar & BrintzenofeSzoc, 2020). This VR simulation project was designed and developed to overcome challenges to educating novice students in an urban environment about new social contexts and communities, and to bridge classroom didactic instruction to authentic application. As a pilot study with only one group, the generalizability of these results is limited but the results do support the outlined theory and principles for effective simulation design. Specifically, this pilot investigated which features were most helpful to learning, the impact of the technology on participant experience, participants’ general attitudes towards the experience, and participants’ assessment of learning. Results and feedback show many of the benefits and challenges of VR simulations as a learning activity.

The features students found most useful to their learning about the community living in the Lower East Side of NYC were the guided reflective questions combined with immersion in this authentic community context. This was shown by high agreement scores for the efficacy of guided reflective questions and feeling immersed. Participants agreed these features made them feel engaged and thoughtful, and able to better learn social work concepts. Written statements by students also highlighted the benefits of feeling immersed in the environment, and noted that the guided reflective questions helped them apply social work practice concepts. Not only did students find being immersed and cognitively active during the experience beneficial to their learning, they also used positive emotional descriptions in describing the beneficial impact on their learning. As a result, the design of VR simulations for learning should provide and align objectives with cognitive activities and instructional content within an authentic environment in a way that closely mirrors the way experts in the field think.

Aspects that hindered the experience for participants were mainly rooted in the Google Daydream headsets. Specifically, respondents recommended making imagery and audio clearer, which also reduces the likelihood of motion sickness. As it happens, a new Wonda VR™ platform called Wonda VR Spaces™ was released in 2020; this platform will allow users to launch the application from a laptop or desktop web browser instead of mobile device, still while wearing headsets. These changes should improve visual and audio performance. Additionally, participant feedback suggested that adding more interactivity to the 360 VR simulation and making the hotspots more user-friendly would improve simulation design. Currently, the 360 VR project is under revision to address this feedback and changes will be delivered in the next version using Google Oculus Quest with intent to improve the experience. Comments also pointed towards including discussion before and/or after the experience to expand upon individual reflections to the question prompts conjured during the experience. Expanding the use of the 360 VR tool to an actual course will allow for more social interactions (e.g., discussion) between students and between students and faculty, thus adhering to principles of situated learning theory (Brown et al., 1989; Lave & Wenger, 1991).

Beyond perceptions and feedback from participants, pre- and post-test results demonstrated the impact of the VR experience on student learning. Since the participants were first-year students with little experience of New York, the Lower East Side, or social work practice concepts, the test was designed at an appropriate difficulty level for novices. Pre-and post-test differences showed statistically significant gains in scores, confirming students learned more about social work practice concepts as a result of the 360 VR simulation.

The overall efficacy of the project design speaks to the solidity of applied design principles: to situate the learner in an authentic context (Lave & Wagner, 1991); immerse the learner in direct experience of a world in order to make it meaningful (Witmer & Singer, 1998); engage the learner cognitively with relevant application and practice of skills (Dengel & Magdefrau, 2018); and evoke positive emotion and motivation (Domagk et al., 2010) to foster deep learning. This 360 VR simulation initiated students into the world they will occupy professionally in a way that could not be achieved in a classroom or easily in the real community, using an immersive learning environment that engaged them in cognitive activity. As novice students gain more experience with the VR environment, they will develop their schematic knowledge and confidence, allowing them to simulate more complex and realistic environments. As more designs of VR simulations become part of the toolbox of social work teaching and learning, evaluating specifically which features, conditions of use, and target users result in greatest learning most will require ongoing research.

Curricular Applications

Pelden and Banham (2020) contend that students need access to field environments to practice skills, and that such environments can and should be embedded in the classroom curriculum. This VR simulation can be integrated into a course curriculum, such as the generalist practice year, as a scaffold to bridge the didactic instruction of micro and macro concepts to authentic application. When it is used as a formative assessment activity in the classroom, faculty can further the experience by planning for focused follow-up discussion, continued reflective thinking exercises, and summative assessments. The sequence of instruction from processing instructional content to authentic application to deeper reflective exercises and assessment will prepare students for more complex cases and environments. Specifically, Human Behavior in the Social Environment I (HBI), a core course that all graduate students are required to take in their first year of study, examines the different systems (individual, family, and communities) in the environment and the relationship that exists between them. Lessons within HBI courses are specifically centered on the social environment, with topics that include risk and resilience factors within communities that impact human behavior, and social work interventions designed to develop and promote community resources. Faculty can use this VR simulation to help students better understand how to recognize the resources that are both present and deficient within the community. Students can critically think about the implications of their presence or absence for community residents and discuss these nuances further in small or large groups in the classroom. This will better prepare students to tour unfamiliar or inaccessible neighborhoods as assigned in internships or agency work, integrating what they are observing to greater social work concepts and theories while thinking critically about the community they are serving.

Wayne et al. (2006) affirmed that the literature was saturated with calls to modify field education (e.g., creating hybrid models with simulation laboratories) in a manner reflective of the changing social work landscape. VR simulations are one learning method well-positioned to respond to that call. VR tools allow exploration of environments without the complication of conflicts that are at times encountered within in-person field situations (Wilson et al., 2013). They can also serve as a bridge for students with disability, as well as the many students with access issues during mandatory stay-at-home orders and travel restrictions. In her call to action for the social work profession in a post-COVID world, Nissen (2020) identifies VR technology as an emerging area that can share and improve the larger social work practice structure. To broaden that educational ecosystem, schools could place the 360 VR tool within field preparatory courses, where novice students would benefit from obtaining skills and knowledge that can facilitate their orientation to new social environments (Harrison & Ip, 2013; Ross et al., 2019). 360 VR can guide a student and familiarize them with the community they are assigned to serve through their agency work. Within the VR simulation, reflection prompts can help students to identify macro and micro concepts within the neighborhood environment, the synthesis of which may benefit their work with clients who often live in those very same locations. They can gain a better perspective about the housing, medical, and educational resources that exist for their clients outside the agency setting. They can also obtain a historical understanding of the community, which will increase their knowledge of immigration and gentrification practices within the said environment. Classroom discussion and assessment of learning outcomes can follow this simulated formative assessment to further enhance learning and prepare for more complex cases and social structures.

Although the authors did not originally create this VR simulation with the intention of contending with circumstances created by COVID-19, the results of the pilot suggest that VR simulations may also be beneficial to social work students for remote learning. Pandemic conditions have required schools of social work, and particularly field education, to respond in unprecedented ways to accommodate the ever-evolving field landscape (Morris et al., 2020; Nissen, 2020). Restrictive measures on movement and mandated social distancing beckons social work schools to consider alternative methods for imparting field training that students might otherwise achieve in-person.

Limitations and Future Research

A few limitations are highlighted and contextualized to inform future studies. This pilot study used a one group pre-test, post-test design. As a result, it is prone to threats to internal validity, including testing effects. Future studies will aim to conduct a true experimental design that randomizes participants into more than two groups to compare learning design features or methods. The study was further limited by a laboratory setting rather than a true academic classroom or field learning setting. In the future, the 360 VR simulation will be embedded in course curricula, and studies will be conducted on its use in a live classroom with other students. We may also test the VR simulation with postgraduates to enhance field training. In this pilot, students recruited from the generalist practice year of the MSW program self-selected to participate and received remunerations, therefore they may not have been intrinsically motivated to participate. Future studies will aim to include students who are part of a live course or field training exercise, which may optimize their motivation. Future studies may also investigate use for remote learning. Finally, future studies will aim to include more international students to understand if the 360 VR simulation addresses specific educational challenges for these students.

Another limitation to this pilot study centers on the survey questions that were not previously tested for reliability and validity and therefore may prove inconsistent when used with other cohorts of students. Future studies will use standardized surveys, particularly around immersion and presence.

Conclusion

This pilot study evaluated the effects of a novel 360 VR tool on student learning, perceptions, and attitudes. Pre- and post-performance results of the study indicated that students who engaged with the VR environment showed statistically significant learning gains. Survey results showed positive responses; in particular, the 360 VR simulation was considered helpful to students’ efforts to learn social work practice concepts, and was found enjoyable, exciting, fun, and motivating. These results suggest that VR may be a useful ancillary tool to assist student learning about community environments. Findings from this study provide guidelines and questions for further research studies.