Using Evidence-Based Learning Theories to Guide the Development of Virtual Simulations.

Technology-enhanced training such as virtual simulations can be effective only to the extent evidence-based principles of learning are integrated into their training platforms. Assuming skill acquisition is the target of training, programs should include time and space for repeated practice opportunities structured through evidence-based learning theories (Amodeo et al. in Adm Soc Work 33:423-438, 2009). Essential learning attributes derived from Cognitive Load Theory (Sweller in Cognit Sci 12:257-285, 1988) and the Theory of Deliberate Practice (Ericsson et al. in Psychol Rev 100:363-406, 1993) can effectively integrate into virtual simulations intended to build expertise. Virtual Home Simulation (VHS) and Virtual Motivational Interviewing (VMI) are presented in this paper as an approach to develop virtual simulations that provide child welfare workers a means to deliberately practice essential skills toward competence before entering the workforce. This paper reviews the development process, specific design aspects, and lessons learned as a guide on how to integrate effective learning attributes. Implications for using virtual simulations, including cost-benefits, measuring performance over time, and addressing training complications due to Covid-19 or similar obstacles, are also provided.

Simulated training environments are designed to enhance the learning of complex skills prior to on-the-job performance. Because social work can involve stressful, high-stakes situations for both a worker and a client or family, simulation-based training methods can allow workers to practice targeted skills in a safe and controlled learning environment. Despite literature supporting the efficacy of simulation training (Bogo et al., 2014; Cooper et al., 2012; Craig et al., 2017; Haji et al., 2015; Logie et al., 2013), in our experience, skills training in social work relies heavily on traditional, less effective methods of learning, such as lectures, PowerPoints, videos, and role-playing activities, with relatively less time devoted to structured practice opportunities for learners. In other professional domains, such as sports (Pagé et al., 2019), medicine (Fu et al., 2014; Gaba, 2007; Hashimoto et al., 2015; Lau et al., 2018; Zhu & Wu, 2016), and aviation (Hays et al., 1992), virtual simulations, including virtual reality (VR), have been used extensively in training. Virtual simulations, including VR, can provide an effective learning experience through the ability to create realistic experiences that are immersive (Jerald, 2015). However, research across a wide range of skill development approaches has shown that merely providing realistic simulations, whether in-person or virtual, does not reliably lead to skill acquisition or competence (Clark, 2019). Instead, the efficacy of any skills training depends on the degree to which evidence-based learning theories drive the development and delivery of the training (Ericsson, 2006; Mayer, 2009). More broadly, learning experts have concluded that technological innovations in training, such as virtual simulations, will only lead to better learning outcomes if these innovative technologies explicitly incorporate the psychology of how humans learn (Clark, 2019). Without the guidance of effective learning theories, simulated training is likely to fail to produce better skilled social workers. As Clark argues:This paper demonstrates how two learning theories are integrated into the design of virtual simulation tools in the Virtual Social Work Trainer (Social Research Institute & Games and Applications Lab, 2021), which is a suite of tools developed to promote child welfare competency. The key aspects of these two learning theories, the Theory of Deliberate Practice (DP; repeatedly practicing intentional behaviors under the guidance of a coach) and Cognitive Load Theory (CLT; chunking learning activities into related, manageable segments), are discussed in greater detail below, along with research supporting effectiveness from various professional domains. We argue that virtual simulation efforts that explicitly incorporate these theories will enable child welfare educators to create simulations that reliably support a more competent workforce.

For more than 70 years, instructional scientists have attempted to prove the superiority of each new technology over old-fashioned classroom instruction. When we plan instruction solely to leverage the latest technology, we ignore the psychology of human learning … That’s because the psychological active ingredients of your lessons are what cause people to learn, regardless of what media you are using.” (2019, p. 182)

Virtual Simulations

Subject matter experts develop VSWT at the University of Utah, College of Social Work, in partnership with the Games and Applications Lab to provide current or prospective child welfare workers an opportunity to practice core tasks as needed to increase proficiency. While VSWT is in early development, it is being utilized across numerous public child welfare agencies and partner universities in the United States. Lessons learned from these two VSWT skills modules that are currently used are discussed below. One is a VR-based platform, and the other is a Microsoft Windows/Android application. Our definition of VR follows Jerald’s as being “a computer-generated digital environment that can be experienced and interacted with as if that environment were real” (2015, p. 9). A fundamental distinguishing feature of VR is presence, which may be characterized as the experience of a deep psychological involvement to the point where the user's perceptual system “fails to recognize the role of the technology in the experience” (International Society for Presence Research, 2000). Beyond the requirement of presence, VR simulations greatly vary in their content, how they are designed, and the format in which they are presented (Jerald, 2015). In this paper, references to virtual simulations include both VR platforms and simulation applications on a computer or tablet designed to simulate situations in child welfare practice.

Having to depend upon instructors' regular presence and expertise can place heavy resource burdens on child welfare jurisdictions. Virtual simulations can increase accessibility and create consistent feedback during repeated practice sessions. And although exposure, feedback, and repeated practice contribute to skill development (Ericsson, 2006), these design attributes in isolation are insufficient to produce experts reliably. Neglecting to incorporate learning theories directly into the technology may relegate vital learning elements to occur in less consistent settings after the simulation, for example, through semi-structured feedback discussions.

An informal review of other VR platforms intended for use in child welfare practice and social work generally revealed that the majority focus primarily on exposure, vicarious experience, and/or cue response (Accenture, 2019; Beal, 2017; GLC Team, 2018; Hogan & Ward, 2015; Lanzieri et al., 2020). Distressingly, we saw no explicitly defined learning theory underlying the technology. While VR has the innate capacity to offer realistic environments optimal for exposure and vicarious experiences, skill acquisition in these technologies has primarily been viewed as either the objective response to a stimulus (Bordnick et al., 2012; Washburn et al., 2020) or an increased understanding of an environment (Lanzieri et al., 2020). VR characteristics such as exposure (Beal, 2017; Lanzieri et al., 2020), feedback (Hogan & Ward, 2015), and consistent, repeated experiences (Bordnick et al., 2012; Washburn et al., 2020) can increase skill. However, to more reliably produce experts in the field, there should be intentional incorporation of a viable learning theory and its components within the technology.

While a detailed overview of learning theory is outside the scope of this article, we make the argument that innovative, technologically-supported training developments, such as virtual simulations, need to be guided by specific evidence-based learning theories. To better understand the practical application of DP and CLT, we will first briefly review their main components.

The Theory of Deliberate Practice

In many professional domains, such as music, chess, and sports, skilled performance is acquired primarily through targeted skills practice. For example, musicians repeat scales (Hayes, 1981), chess masters study opening moves (Krogius, 1976), and professional basketball players shoot endless foul shots (Ericsson, 2016; Pagé et al., 2019). The number of hours a trainee in these fields practices overwhelmingly outnumbers the number of hours they perform. Additionally, those who reach expert performance levels spend up to five hours a day practicing in a very structured way (Ericsson, 2016). Namely, they target specific skills under the guidance of a coach or teacher who provides frequent feedback. This approach to skill acquisition has been termed deliberate practice (Ericsson et al., 1993).

Within the health sciences, DP-based simulation is more developed and is therefore suggested as a framework in a wide range of medical simulations (Clapper & Kardong-Edgren, 2012; Harris et al., 2013; Mitchell & Boyer, 2020). Simulations incorporating DP have been found effective for acquiring medical skills (Burden et al., 2014; Gross et al., 2019).

DP is similar to experience-based models of expert skill acquisition because highly skilled performance is viewed in part as a product of extensive practice. However, under the DP model, the structure of the practice is crucial. According to Ericsson, practice often fails to produce expert performance levels because most learners reach a premature plateau of skill development at which the skill set becomes “automated” to the point the learner can devote cognitive resources to other, unrelated tasks, simultaneously (2006, p. 685). Further practice does not bring proportional increases in skill because automation results in the loss of “the ability to control the execution of those skills, making intentional modifications and adjustments difficult” (2006, p. 684). In other words, learning requires conscious control, and when any task becomes automatic, learning will slow considerably.

Ericsson et al. (2018) suggest that those seeking to reach expert performance levels can avoid prematurely plateauing by using training methods that extend skill development and therefore delay automation. This is done through carefully designed practice opportunities that:

target specific, limited aspects of a skill or performance;

target skills that are just beyond the current level of performance;

utilize the guidance of coach or teacher;

place limited cognitive demands on the learner;

require problem solving and exploration of alternatives;

provide frequent feedback and opportunities for reflection;

provide repetition and frequent practice, usually daily.

The impact of training applications using a DP approach is effective across fields as diverse as aviation (Aguinis & Kraiger, 2009), sports (Ericsson, 2016; Pagé et al., 2019), chess (Ericsson et al., 1993; Krogius, 1976), music (Hayes, 1981), and medicine (Hashimoto et al., 2015; Palter & Grantcharov, 2011).

Despite the evidence of DP within in-person simulations (Bogo et al., 2014), the body of literature for DP-based virtual simulations is minimal, particularly in child welfare practice. In a recent scoping review of simulations in social work education (Kourgiantakis et al., 2020), three studies referenced using DP. Of these three studies, however, only one used a DP framework (Kourgiantakis et al., 2019). As of this writing, we are not aware of any literature to date using DP-based virtual simulations within child welfare training and education.

Cognitive Load Theory

Although DP identifies the components to incorporate into skills training, a guiding structure for developing virtual simulations has not been formally operationalized within the theory itself. Some researchers have suggested that multimedia training approaches based on CLT could be used in developing DP-based instructional designs (Van Gog et al., 2005). When combining DP with CLT, the flow of new material in the learning environment must be controlled so that the learner is challenged but not overwhelmed. CLT focuses on how instructional designs can achieve this balance by designing with human cognitive limitations in mind, such as the severe constraints of working memory (Paas et al., 2003).

A significant task of designing effective virtual simulations is to reduce the amount of extraneous cognitive load, reducing elements that may require attention but are not part of the task the learner is trying to master (Chandler & Sweller, 1991). For example, technological platforms can be complex, requiring a user to manipulate multiple navigational controls, which likely reduce the learner’s ability to focus on and retain information when practicing skills. In addition to reducing extraneous cognitive load, intrinsic cognitive load, which is the load that stems from the subject matters difficulty, can be reduced through “chunking,”—a process combining “elements that have a strong association with one another, and relatively weaker “associations of elements within other chunks” (Gobet et al., 2001, p. 236). This allows basic foundational information and skills to be learned first and then combined into more complex repertoires or schemas. Instructional designs based on CLT effectively develop skills in complex performance domains (Mayer, 2009).

CLT-based training approaches' effectiveness has been measured in many fields, with education the most common (see Sweller et al., 2011 for a review). As with simulations using DP, efforts to implement CLT-based simulations are more developed in the health sciences. Best practice guidelines for developing technology-based learning materials in medicine have been suggested to incorporate CLT principles (Grunwald & Corsbie-Massay, 2006). CLT-informed virtual simulations show support for increasing medical skills (e.g., Andersen et al., 2016, 2018; Bharathan et al., 2013). Recently, a framework has been put forth that holds promise for creating VR-simulations that can individualize instruction based on a trainee's in-the-moment cognitive load (Rodenburg et al., 2018). However, we have found no simulations in child welfare that have been explicitly developed using CLT learning principles.

A Case Example: Virtual Social Work Trainer (VSWT)

This section gives examples of how DP and CLT shape the development of the VSWT. Our examples are meant to be an illustrative rather than exhaustive account of how these learning theories guide our approach.

VSWT is being developed by a team of faculty researchers, subject matter experts, and engineering students at the University of Utah (Social Research Institute & Games and Applications Lab, 2021). The individual skills modules are intended to supplement current approaches to training in child welfare practice by providing a method for trainees and child welfare workers to engage in frequent, immersive practice. We report on the development of two of these modules.

The first module, Virtual Home Simulation (VHS), is a VR-based simulation tool that offers both a fully-immersive VR experience and a desktop approach for repeated skills practice for child welfare workers. VHS allows prospective caseworkers to practice key skills often needed during a home visit. Specifically, it tests their ability to evaluate a home environment for evidence of risk and protective factors and appropriately document their observations. Beyond the ability to correctly mark items in a home as being protective or indicating possible risk, VHS aims to provide users with the schema development necessary to understand the decision rationale behind the real-world skills emulated in the tool. Preliminary evidence suggests VHS may also serve as a realistic job preview for social workers interested in casework (McDonald & Davis, 2019). The tool's immersive and realistic nature allows the user to assess a home environment for possible indicators of risk or protection while also increasing their awareness regarding their professional or personal fit in the field (Fig. 1).

Fig. 1

Virtual Home Simulation (VHS)

The second VSWT skill module, Virtual Motivational Interviewing (VMI), is a mobile simulation app for prospective child welfare workers to solidify and improve their motivational interviewing (MI) skills. VMI also follows a scaffolded learning approach where the user incrementally increases skills at their own pace as their competency increases. The app enables the user to narrowly practice specific skills and aspects of MI that they wish to target while receiving expert feedback and guidance along the way (Fig. 2).

Fig. 2

Virtual Motivational Interviewing (VMI)

Simulation Components Informed by the Theory of DP

The Role of Experts—Guidance and Comparison

Expert guidance plays an essential role in DP (Ericsson, 2006), and as such, has been incorporated into the development of both tools illustrated here. Experts are used in both a guidance and comparison capacity. With each practice session, participants receive feedback and instruction from experts in the field and obtain detailed feedback on their performance compared to a consensus model of expert performance.

VHS incorporates an expert algorithm in the scoring and feedback for all users. The algorithm is derived from over 70 experts and continuously evolves as more experts contribute their data using the tool. Extrapolating from Chi’s chapter on approaches to studying experts’ characteristics (2006), we define child welfare expertise utilizing an abundance of criteria including organizational role, type of experience, years of experience, and VHS performance attributes. The feedback a user receives on their performance during each practice session is derived from performance data of those meeting these expert criteria when they carry out VHS's tasks. The feedback allows a user to measure their performance compared to the consensus of these experts. Furthermore, VHS users receive qualitative feedback derived from a thematic analysis of the experts’ rationale for their decisions.

Similar to VHS, VMI uses experts to provide structured feedback, guidance, and comparison. VMI incorporates a scoring algorithm provided to users that tracks progression, skill acquisition, and adherence to the expert consensus of MI techniques. But unlike VHS, VMI does not use an expert algorithm to provide feedback or guidance. Instead, the content in VMI is constructed using a manual for MI (Miller & Rollnick, 2012) and quality checked by a faculty member, Dr. Lundahl, at the University of Utah who practices, trains, and publishes on the use of MI in child welfare practice for over a decade. This expert guidance and comparison model enables participants to have clear objectives and direction while visualizing their progress from a quantitative standpoint.

Notably, a clear goal or milestone of expertise allows researchers and trainers to objectively determine when participants have reached a benchmark or target skill acquisition level. Accordingly, both VHS and VMI provide performance scores that summarize a learner’s progress during each session as well as over many sessions. Performance scores fall in one of five levels: (1) fundamental awareness, (2) novice, (3) intermediate, (4) advanced, and (5) expert.

Skills-Scaffolding Practice

Using a DP approach, skills training starts with practicing simple skills. Increasingly complex skills or combinations of simple skills are then introduced to allow the trainee to focus on skills that are just beyond current competency levels (Ericsson & Pool, 2016). This practice is analogous to a baseball player hitting balls in batting practice. The focus is on mastering the single task: hitting the ball. Then, it progresses to more complex skill combinations such as hitting a ball during a game or to a specific location on the field. An additional method for scaffolding practice is to adjust the learning material's difficulty so that it is just out of reach of the learner’s current level of mastery (Ericsson, 2006). To further use the baseball analogy, if a ten-year-old baseball player hoped to enhance their skills to a professional level, it would not benefit the young player as much to compete against a professional. Instead, the young player would benefit more by competing with others that were just slightly ahead of their own skillset to push their skills beyond their current limits and incrementally model that of others on higher skill levels.

To illustrate how this type of scaffolding works in teaching child welfare practice, Fig. 3 shows the VMI learning experience process. As shown, the competency is divided into two complex skills that are further divided into eight simple skills. Each simple skill is practiced to mastery using repeated exercises that require practicing the skill during an interaction with a simulated client. Once all simple skills have been mastered, the learner can then practice using combinations of the simple skills to master a complex skill. For example, a learner would be presented with interactions that require identifying and magnifying client change talk. The learner must perform one or more of the simple skills related to identifying and magnifying change talk to show mastery of this more complex skill.

Fig. 3

Virtual Motivational Interviewing (VMI) Scaffolding

These skills modules aim to progress learning at a challenging pace but within reach of the learner. The algorithms of these skills modules allow users to move at an individualized pace and receive customized feedback related to their progress and performance over time as they continue to practice deliberately.

Immediate and Specific Feedback

For the participant to most effectively gain the desired skills, immediate feedback is crucial (Ericsson & Pool, 2016). Both VHS and VMI provide multiple methods for giving immediate feedback to participants. Similar to narrowing the user’s focus to specific and important tasks, the user's feedback only focuses on the skill module's target behaviors. Feedback on performance not directly related to the target behavior or skill only distracts the user from clearly understanding how to improve their current skill level. Accordingly, feedback must be given to be simple enough for a universal user to understand and apply.

Both VHS and VMI provide immediate and specific feedback in the form of thought questions and expert consensus feedback, including the particular rationale for assessment decisions that best support the target skill. In both VHS and VMI, users can receive specific feedback with each interaction and choice they make. For example, in VHS, a user who selects an item in a home and marks it as a risk can immediately receive feedback about whether the item was considered a risk by experts in general and what the reasoning was behind their assessment decision. VMI follows a similar format when making decisions for MI responses during client engagement that are congruent with the target MI skill. Additionally, both VHS and VMI focus feedback on the micro-skill at hand. They offer an overall evaluation of performance over time, essentially aggregating each user's practice sessions to understand where they are improving and what areas may need additional practice. Ultimately, the thought questions, feedback, and rationale in each of the tools aim to empower the user to deliberately increase their skills during practice by helping participants understand why they are doing the skill, including in what contexts the specific skills may be warranted.

Simulation Components Informed by CLT

Limiting the Amount of Material Covered

Working, or short-term, memory is the cognitive process of intentionally holding and processing information (Baddeley et al., 1974) and as such is foundational to all cognition, including learning (Cowan, 2010). Given the severe constraints of short-term or working memory, which CLT is designed to address (Sweller et al., 2019), each learning experience in VHS and VMI is designed to teach a single skill. For example, VHS's learning task is focused on one foundational skill, namely identifying environmental indicators of risk or protective capacities of the caregivers, even though there is a myriad of additional skills required to carry out a home visit expertly. Providing practice experience and evaluating all the possible skills used during a home visit would be overwhelming at best for a new learner. It could also cause cognitive overload, further stifling learning.

The amount of material covered is also limited by encouraging short practice sessions, usually less than 15 min. For example, VMI consists of chapters that are combinations of skills falling under a particular MI component (i.e., Engaging, Focusing, Evoking, and Planning). Each chapter contains up to five sub-skills. For each sub-skill, there are up to 15 practice exercises that make up a session. A participant is encouraged to do no more than one session per day, which should take a participant about 10–15 min.

Chunking and Focusing on the Most Important Tasks

Chunking refers to the concept of combining information into conceptually related “chunks” that allow working memory limits to be expanded and thereby enhance the ability to learn complex tasks or schemas (Gobet et al., 2001). From a learning perspective, this means learning is done in short segments starting with the most basic skills that build upon one another. Over time, these skills are connected into complex schemas (Sweller, 1988). For example, in VMI, a user may be attempting to learn the overall task of getting a caregiver to attend drug treatment. This complete task would be overwhelming and challenging to learn for someone just entering child welfare employment. Therefore, breaking this task into smaller chunks, such as engaging the caregiver and validating their concerns, will help the learner build towards the larger goal of assessing motivations for change and creating a plan of action. Chunking helps a learner master a specific skill before moving on.

CLT suggests that we cannot overwhelm users with learning or analyzing many tasks at once. Focus on irrelevant or extracurricular stimuli deters a learner from learning the skill at hand (Kalyuga, 2009). Focusing only on the target behaviors also allows researchers to have more fidelity and statistical validity when measuring skill competence because the tasks are free from unnecessary confounding variables.

Implications for Clinical Education

Evidence-based learning theories have been used to reliably train workers to expert performance levels in many professional domains (Ericsson, 1996). We believe that skills-focused virtual simulations may assist learners in mastering many of the core skills of child welfare practice. To become experts, workers need ample opportunities to practice target skills (Ericsson et al., 2006). Workforce development plans should include an overarching approach to providing their workers with these practice opportunities and should consist of specific feedback elements, which leads to improved performance. Technologically-supportive platforms that can measure performance over time can help meet this need and significantly reduce coaches and mentors' burden. VHS and VMI can expand the reach of a jurisdiction's current mentoring and coaching infrastructure, essentially multiplying their impact without the demand for more resources to support their new workers the requisite amount of time. This type of support along a social worker’s professional development trajectory may be particularly helpful if the new worker is in an area with limited resources and demand for ongoing coaching is needed. It can also facilitate more frequent and effective learning experiences for jurisdictions that cover large geographical areas or have training programs that are remote/online.

Compared to in-person lecture-based learning or online video conferencing, mobile platforms enable users to practice as often as they need and provide a greater cost–benefit over traditional training costs. Such individualized practice sessions designed as outlined in this article may be more effective than commonly used training methods, particularly in jurisdictions that struggle to maintain training quality due to restrictions or other circumstances created by the Covid-19 pandemic. Structured virtual simulations that provide consistent opportunities for repeated practice set the stage for research in child welfare practice skills development to better explore the efficacy of skill development and the transfer of learning to the field.

Future Research

Fully structured skills practice experiences through virtual simulations can facilitate rigorous research around competency through child welfare skills practice. The learning components discussed in this paper, coupled with a consistent learning environment, allows research to move beyond discussing competency from a largely subjective standpoint of perceived competence in child welfare and creates more pathways to understand actual competence. Combining these technologically-supported practice experiences with the broader coaching and mentoring programs specific to each jurisdiction can help create valid learning environments that measure skill levels. This could support research efforts to understand better the particular skill sets most strongly correlated with positive outcomes for the families served.

More research is needed to measure how effectively such technologically-driven learning platforms build skills in the child welfare workforce. As research continues to explore how performance during practice translates to performance on the job, we can continue to improve the learning platforms with a precision not yet evident in child welfare workforce development.