Better together: Effects of four self-efficacy-building strategies on online statistical learning.

The goal of this study is to test the individual and combined effects of supplementing an online statistics lesson with four motivational strategies corresponding to Bandura's (1997) four sources of self-efficacy (anxiety coping, modeling, mental practice, and effort feedback) on cognitive, motivational, and affective outcomes. Internet participants (N = 279) completed an online statistics module in one of six conditions with one or all four self-efficacy-building strategies (5 treatment conditions) or none of these strategies (control condition). The results indicated that the four strategies worked effectively in combination, significantly improving transfer test scores (d = 0.608), increasing self-efficacy ratings (d = 0.696), and reducing task anxiety ratings (d = -0.534), as compared with the control condition. By contrast, no motivational strategy alone was effective. The results suggest the importance of taking advantage of the power of all four sources of self-efficacy information in combination when designing motivational interventions for online mathematical lessons.

Introduction

The goal of this study is to test the effects of supplementing an online statistics lesson with individual and combined motivational strategies corresponding to Bandura’s (1997) four sources of self-efficacy (i.e., anxiety coping, modeling, mental practice, and effort feedback) on cognitive, motivational, and affective outcomes (i.e., test scores, self-efficacy ratings, and task anxiety ratings, respectively). Low self-efficacy (i.e., perceived capability to complete a task) in mathematical learning is a common problem in P-16 education, leading to many undesirable consequences, such as low performance or negative attitudes towards math-related courses or careers (Ashcraft, 2002, Toland and Usher, 2016).

Addressing this low self-efficacy problem in mathematical learning is particularly critical in online learning environments (Wadsworth, Husman, & Duggan, 2007), as instructor-based traditional self-efficacy supports are not always readily available. Online learning has become more commonplace over the past decade. A recent report found that in Fall 2017, more than 3 million postsecondary students enrolled exclusively in distance learning courses, and another 3.5 million took some combination of distance and in-person courses (Ginder, Kelly-Reid, & Mann, 2018). The COVID-19 pandemic has resulted in an even larger-scale move to online instruction, which could have a long-lasting impact on institutes of higher education. Despite the growing trend of online learning, research focusing on developing self-efficacy in online mathematical learning environments is scarce (Huang & Mayer, 2019).

In face-to-face learning environments, researchers have tested interventions focusing on self-efficacy development in learning mathematical content, showing the potential of theory-based interventions to increase self-efficacy and performance (e.g., Cordero et al., 2010, Schunk and Hanson, 1985, Schunk and Hanson, 1989). Nevertheless, limited research has taken an integrated approach that addresses all four sources of self-efficacy information hypothesized by Bandura (1997)—mastery experience, vicarious experience, social persuasion, and physiological states—in one intervention, let alone in technology-based online environments. Such an approach could maximize the effect of efficacy-supportive strategies on various learning, motivational, and affective outcomes. For example, one recent study demonstrated the effectiveness of an online intervention aligned with Bandura’s four sources of self-efficacy. The intervention increased learning performance and self-efficacy while reducing anxiety (Huang & Mayer, 2019). Nevertheless, the four strategies were investigated as a package to test their combined effect. This approach did not permit the authors to assess whether any particular strategy contributed more to the positive outcomes. The present study was intended to provide a more nuanced examination of the mechanisms of self-efficacy development by testing these self-efficacy-building strategies individually and as an integrated set. The central research question is: What is the individual and combined effectiveness of four motivational strategies aligned with Bandura’s four sources of self-efficacy for improving transfer test performance and self-efficacy ratings and for lowering task anxiety ratings in an online statistical learning environment? In short, does an intervention need to feature all four strategies in combination or would just one be enough to improve online learning and motivational outcomes?

This study was guided by Bandura’s (1997) social cognitive theory, in which self-efficacy, or beliefs about one’s capabilities to perform given tasks, plays a central role in human learning. We focus specifically on self-efficacy development within an example-based learning environment involving statistics problem solving.

Example-based learning environments

Example-based learning values the use of a large number of examples to facilitate learning for novice learners, with an emphasis on reducing unnecessary cognitive load induced by conventional problem-solving (Tuovinen & Sweller, 1999). Numerous studies have demonstrated the benefits of studying examples, including learning efficiency, effectiveness, and the development of confidence and positive attitudes (Mayer, in press, Renkl, 2011). Particularly, the strategy of presenting learners with an example followed by a similar practice problem (i.e., example-problem pairing) is more effective than presenting them with examples only (Renkl, 2011, Renkl, 2014, Van Gog et al., 2011).

Two types of examples often integrated into online multimedia instruction include written worked examples — step-by-step solutions to a problem in a written format — and video modeling examples — demonstration of others solving a problem in steps in a video format (Biesinger and Crippen, 2010, Crippen and Earl, 2007, Huang, 2017, Huang and Mayer, 2019). As pointed out in Van Gog and Rummel (2010), although both types of examples concern example-based learning, worked examples were studied mostly from a cognitive perspective (Tuovinen and Sweller, 1999, Ward and Sweller, 1990), whereas modeling examples were studied mostly from a social cognitive perspective (Bandura, 1997; as detailed in Section 1.2.2). A limited number of empirical studies involving online multimedia learning have studied example-based learning environments by integrating the cognitive perspective and the social cognitive perspective. For example, Huang (2017) documented the effectiveness of both expert modeling and peer modeling of statistics problems solving in increasing students’ learning and self-efficacy.

Empirical research has also investigated how example-based mathematical or science learning environments can be used to support other instructional strategies in online multimedia instruction, such as self-explanations (Crippen & Earl, 2007), feedback protocols (Biesinger & Crippen, 2010), anxiety-reducing features (Huang & Mayer, 2016), or a set of self-efficacy-supportive features developed in line with Bandura’s (1997) social cognitive theory (Huang & Mayer, 2019). Specifically, these studies investigated how the addition of supportive elements in an example-based online learning environment could influence student learning and self-efficacy. Two of these studies showed that using example-problem pairs in combination with other instructional strategies increased mathematical content learning and/or self-efficacy (Huang and Mayer, 2016, Huang and Mayer, 2019).

Strategies to enhance self-efficacy via four hypothesized sources

Within the online example-problem pairs learning environment, four strategies related to the four sources of self-efficacy information (Bandura, 1997) were investigated by Huang and Mayer (2019). The first strategy was anxiety coping, which aligns with the physiological states source of self-efficacy. The second strategy, expert modeling, aligns with the vicarious experience source. The third strategy was mental practice, which aligns with the mastery experience source, and the fourth strategy was effort feedback in line with the social persuasion source of self-efficacy (Huang & Mayer, 2019). Each strategy and the corresponding source of self-efficacy information is described below.

Physiological states: Anxiety coping

Physiological states, such as anxiety, have been shown to negatively affect self-efficacy; therefore, one way to strengthen efficacy beliefs is by mitigating anxiety levels or negative emotional states in stressful learning situations (Bandura, 1997, Usher and Pajares, 2008). Math anxiety has been identified as a ubiquitous phenomenon in academic settings (Ashcraft, 2002), and extensive empirical research has demonstrated a negative relationship between math anxiety and math self-efficacy (e.g., Griggs et al., 2013, Huang et al., 2019, Pajares and Miller, 1994).

Accordingly, a number of research studies have focused on developing psychological interventions to reduce math anxiety (Feng et al., 2014, Gan et al., 2016, Park et al., 2014, Sharp et al., 2000, Zettle, 2003). These interventions can be categorized into behavioral, cognitive, or cognitive-behavioral types, with behavioral interventions focusing on the emotionality component of math anxiety, cognitive interventions focusing on the worry component of math anxiety, and cognitive-behavioral interventions including both emotionality and worry components (Hembree, 1990). Empirical studies in computer-based learning have documented the benefits of affective coping messages delivered by pedagogical agents in reducing math anxiety or improving performance (Huang and Mayer, 2016, Im, 2012, Shen, 2009). There is also evidence that cognitive anxiety coping techniques, such as growth mindset development, can improve outcomes (Huang and Mayer, 2019, Im, 2012). For example, previous research has shown a growth mindset intervention to be effective in reducing students’ anxiety in learning statistics (Smith & Capuzzi, 2019). A growth mindset reflects a belief that one’s ability is not fixed, but changeable and improvable through continued efforts (Dweck, 2008a). A coping message conveying this belief may likewise reduce math anxiety and positively influence perceived self-efficacy.

Vicarious experience: Modeling

Vicarious experience is defined as experience gained through observation, which includes modeled performance (Bandura, 1997). In most activities, people judge their capabilities based in part on the accomplishments of others as “there are no absolute measures of adequacy” (Bandura, 1997, p. 86). Two types of modeling that are frequently researched differ on the basis of learner-observer similarity, for example, peer or coping models versus expert or mastery models that demonstrate differing learning processes (Bandura, 1997). On one hand, when people see someone similar to themselves fail or succeed on a certain task, their self-efficacy can be weakened or strengthened accordingly as a result of the social comparison; on the other hand, people look for competent models who possess the expertise they aspire to attain (Bandura, 1997). Either modeling type could be effective in increasing learning or self-efficacy (e.g., Cumming and Ramsey, 2011, Schunk and Hanson, 1985, Schunk and Hanson, 1989). Nevertheless, according to Bandura (1997), “The instructive contribution of modeling is especially important when perceived inefficacy reflects skill deficits rather than misappraisals of the skills already possessed” (p. 88). He also emphasized that the influence of positive models could be obtained by “maximizing modeling’s instructive function and minimizing its comparative evaluative function” (p. 92).

In technology-supported learning environments, modeling can be presented by a pedagogical agent – a virtual character who guides the learner for instructional purposes (Veletsianos, 2010). More specifically, a pedagogical agent can be used to demonstrate and verbalize covert thought processes involved in a problem-solving process. Previous empirical studies have included pedagogical agents who model a mathematical problem-solving process. Researchers then evaluate the benefits to cognitive, motivational, or/and affective learning (Huang, 2017, Huang and Mayer, 2019).

Mastery experience: Mental practice

Mastery experience refers to one’s perception of previous successes and has been identified as “the most influential source of efficacy information” because it provides “the most authentic evidence” of whether one has the capabilities to succeed (Bandura, 1997, p. 80). It is how people evaluate their performance, rather than the performance per se, that strengthens or weakens self-efficacy beliefs (Usher, Ford, Li, & Weidner, 2019). Bandura discussed both mastery-based and mastery-oriented strategies to support mastery performance. Mastery-based strategies focus on actual mastery experiences, whereas mastery-oriented strategies may rely on experiences generated from virtual performance, such as via virtual reality technology rather than actual performance in the real world. Transfer of perceived mastery is then possible from virtual reality to actual reality (Bandura, 1997).

Just as people can gain a sense of mastery from virtual reality experiences, they can also gain a sense of mental mastery from imagined cognitive performances. Mental practice (i.e., mentally rehearsing the process of a successful performance) can enhance a sense of mastery (Bandura, 1997, Fiorella and Mayer, 2015). However, research on mental practice for cognitive tasks is limited despite its potential (Leahy and Sweller, 2008, Leopold and Mayer, 2015). Some have shown that mentally practicing the procedure in an example after studying facilitates learning (Cooper, Tindall-Ford, Chandler, & Sweller, 2001). However, little empirical research has examined whether this strategy would increase self-efficacy and other desired learning outcomes (Huang & Mayer, 2019).

Social persuasion: Effort feedback

Social persuasion serves as another means of developing self-efficacy beliefs. People who are convinced by others that they are capable of mastering a certain task are more likely to develop a sense of self-efficacy (Bandura, 1997). For social persuasion to be effective in fostering self-efficacy, messages should be perceived as an authentic evaluation of one’s capabilities (Bandura, 1997).

Attributional feedback, that is, feedback that pertains to the causes of one’s performance (Schunk & Rice, 1986), is a common form of social persuasion that affects self-efficacy (Bandura, 1997). Attributional feedback linking achievement outcomes to effort or to ability has shown different effects on motivation and performance (Schunk, 1983). Research conducted by Schunk and his colleagues indicated that ability feedback enhanced self-efficacy and performance during initial skill development (e.g., Schunk, 1983, Schunk and Rice, 1986). However, more recent research on growth mindset has shown that effort feedback, which attributes one’s performance to modifiable causes (e.g., effort) increases self-efficacy and persistence (Dweck, 2008a, Mueller and Dweck, 1998, Romero et al., 2014). Research on attributional feedback has been primarily carried out in face-to-face settings, with only limited work conducted in online settings (e.g., Huang and Mayer, 2019, Zhao and Huang, 2020). In an online example-based learning environment, customized effort feedback can be integrated into paired problems with the goal of strengthening learner self-efficacy.

Relative strength of four primary self-efficacy sources

A growing body of research has shown that the four hypothesized sources of self-efficacy are core elements in the development of academic self-efficacy, but the relative strength of each informational source varies according to contextual demands and individual differences (Usher & Weidner, 2018). Mastery experience is generally considered the most influential source as it involves individuals’ direct experience, but when information from all four sources is available, mastery experience may not be the sole basis on which individuals adjust their self-efficacy. For example, Bandura (1997) acknowledged that vicarious experience, the indirect experience of observing others, typically has a weaker effect on self-efficacy than direct experience, but under certain conditions observing others can be more impactful. Ultimately, the influence of the four sources on self-efficacy depends on how people interpret the information, contextual factors (e.g., learning domain), and individual factors (e.g., gender, ethnicity, ability level; Bandura, 1997, Butz and Usher, 2015, Joet et al., 2011, Usher and Pajares, 2008).

Survey-based research on the sources of self-efficacy in K-12 mathematics has generally shown that mastery experience is a strong source of students’ self-efficacy (Butz and Usher, 2015, Joet et al., 2011, Usher and Pajares, 2009). In the context of college students learning algebra in a primarily asynchronous online environment, however, vicarious experiences and physiological states have been identified as significant sources of math self-efficacy (Hodges & Murphy, 2009). In short, although the previous literature generally supports the four hypothesized sources of self-efficacy, the relative strength of the relationship between these sources and self-efficacy has been less consistent.

One reason for these inconsistencies might be variation in how people integrate multiple sources of self-efficacy information can vary. As Bandura (1997) explained,

Some may combine efficacy-relevant factors additively - the more indicants there are, the stronger is the belief of personal capability. Others may use a relative weighting rule in which some factors are weighted more heavily than others. Still others may use a multiplicative combination rule. Here the conjoint impact of factors on efficacy beliefs is greater than simply their additive effect. (p. 114)

It would be worthwhile to investigate the individual and combined effectiveness of instructional strategies connected to each of the four sources of self-efficacy when applied to an online mathematical learning environment. To date, little experimental evidence has focused on self-efficacy development. Even less has focused on assessing self-efficacy-supportive interventions for online adult learners. Examining how targeted efficacy-relevant experiences affect self-efficacy individually and in combination in asynchronous online learning environments via the four hypothesized informational sources could provide helpful tips for optimizing instructional design.

Predictions for the present study

This study is intended to extend Huang and Mayer’s (2019) research by analyzing the integrated effect of four self-efficacy-supportive strategies, which include (1) an agent-delivered anxiety coping message informed by the physiological states source of self-efficacy, (2) expert modeling examples informed by the vicarious experience source, (3) mental practice of mathematical examples informed by the mastery experience source, and (4) effort feedback to the paired practice problems informed by the social persuasion source. Our goal is to test these four self-efficacy-building strategies, both individually and as an integrated set, to better understand the mechanisms of self-efficacy development. Using Bandura’s (1997) theory of self-efficacy and previous findings from the literature, we predict that adding all of the four self-efficacy-building strategies to the online statistical lesson will lead to

increased performance scores on practice, retention, and transfer tests (Prediction 1),

enhanced task-specific self-efficacy ratings (Prediction 2), and

reduced task anxiety ratings (Prediction 3).

We have elected not to make predictions concerning the individual effectiveness of each strategy for several reasons. First, there is a lack of conclusive evidence concerning the individual contribution of each strategy, particularly in this novel context of adult learners studying mathematical content in an online asynchronous environment. Second, according to Bandura’s (1997) theorizing, the single or combinatory influence of efficacy-relevant informational sources depends on multiple factors such as how people interpret the information, the particular context, and individual differences. It is therefore possible that any individual strategy alone will not be sufficient to change the targeted outcomes. It may be that the combined effect of the four strategies has the strongest influence. Therefore, it is an open question as to whether one or more of these efficacy-building strategies will be more effective than no strategy (i.e., a control group) in improving learning and self-efficacy and reducing anxiety in mathematics. If certain strategies are effective, we can promote their use in learning interventions. If the combined strategies are effective, we can conclude that the effectiveness of a treatment aimed at promoting self-efficacy and learning in an online lesson rests in the mutually-reinforcing power of combining information from all four sources of self-efficacy.

Method

Participants and design

The participants consisted of 296 adults recruited from Prolific, an online crowdsourcing platform for research where Internet participants can be filtered based on their demographic screeners. Seventeen participants were excluded from the data analyses due to their failure to meet the attention check criteria included in the study as described in the Procedure section (n = 16) or in the recruitment criteria (n = 1), leaving a total of 279 participants in the final data set (151 women, 128 men; M age = 23.23 years, SD = 6.24). Recruitment criteria for the study included that the participant (a) was at least 18 years old or older, (b) spoke English as their first/primary language, (c) had a current education level between, and including, a secondary school diploma and an undergraduate degree, (d) had no prior knowledge of the learning topic, and (e) had received a 90% approval rate or higher for their previous research participation via the study platform. Most of the participants were White (74.2%), followed by Asian/Pacific Islander (7.5%), Black (5%), Latino or Hispanic (5%), mixed race (5%), and other race (3.3%). Participants were randomly assigned (stratified by gender) to one of five treatment conditions (T1-T5) or to a control group (C). In a between-subjects design, 48 participants (26 women, 22 men) were assigned to the anxiety coping group (T1), 49 (26 women, 23 men) were assigned to the modeling example group (T2), 49 (28 women, 21 men) were assigned to the mental practice group (T3), 45 (26 women, 19 men) were assigned to the effort feedback group (T4), 47 (25 women, 22 men) were assigned to the integrated strategies group (T5), and 41 (20 women, 21 men) were assigned to the control group (C).

Instructional materials

The instructional materials consisted of six versions of an online, self-paced instructional module designed to teach two statistical rules: the empirical rule and Chebyshev’s rule. The module materials were adapted from those used by Huang and Mayer (2019). The overall module structure was: (1) demographic survey and self-efficacy pretest; (2) review of prerequisite skills (how to define and calculate a mean score); (3) introduction to the two statistical rules; (4) practice activity consisting of five condition-dependent example-problem pairs; (5) task anxiety and self-efficacy measures; and (6) posttest. All six versions of the module were identical except for the practice activity (i.e., Part 4 of the module described above). The practice activity in the treatment versions included one or all of the four strategies aligned with Bandura’s (1997) four sources of self-efficacy, whereas the control version did not include any of the four strategies, as described below.

Control version

In the practice activity of the control version of the module, participants received five step-by-step worked examples (as exemplified in Fig. 1 ). Each worked example was followed by a paired problem for participants to solve (as exemplified in Fig. 2 ). Participants received knowledge-of-correctness feedback on their performance for each problem (e.g., “You received 3 out of 4 points”).

Fig. 1

An example of written, worked example in the practice activity.

Fig. 2

An example of a paired problem in the practice activity.

Treatment 1: Math anxiety coping strategy

The T1 version of the module included a practice strategy that was aligned with the physiological states source of self-efficacy. Before receiving the practice activity, participants in this condition received an anxiety coping message delivered by an animated pedagogical agent in a video (as exemplified in Fig. 3 ). The purpose of the anxiety coping message was to reduce participants’ anxiety level by encouraging a growth mindset (Dweck, 2008b). The pedagogical agent delivering the message was a female character serving the role of a motivator. The video lasted for 83 s. The complete transcript is presented in Appendix A (taken from Huang & Mayer, 2019, pp. 1014–1015). The rest of the practice activity was identical to that in the control version of the module.

Fig. 3

A screenshot of the anxiety coping video embedded at the beginning of the practice activity.

Treatment 2: Modeling examples

The T2 version of the practice activity included a strategy aligned with the vicarious experience source of self-efficacy. Instead of receiving step-by-step worked examples presented in the control version, participants in this treatment condition received five expert modeling examples. The modeling examples were delivered via video by an animated pedagogical agent rendered as a male instructor with white hair (as exemplified in Fig. 4 ). The pedagogical agent demonstrated and verbalized the problem-solving process for each example problem, with a voice that was intended to sound confident and authoritative. Key points of the problem-solving process were displayed synchronously as text information, which appeared next to the expert model. The paired problems and corresponding knowledge-of-correctness feedback were the same as in the control group.

Fig. 4

A screenshot of a modeling example video in the practice activity.

Treatment 3: Mental practice

The T3 version included a strategy aligned with the mastery experience source of self-efficacy. The worked examples and paired problems were identical to those in the control condition. However, participants in this treatment received a mental practice activity after each worked example. Specifically, participants were asked to mentally practice the problem-solving procedure presented in the example before they moved to the paired problem. The mental practice instruction was presented by the same female motivational pedagogical agent who delivered the anxiety coping message in T1 version of the module, but in this case, she appeared as a static image accompanied by textual information (as shown in Fig. 5 ). Participants were guided to refer to the example if they encountered any difficulty while they were mentally performing the procedure demonstrated in the example. This framing was used to ensure that this mental practice activity would serve as a mastery aid by enabling learners to experience a sense of success in their skill development (i.e., avoid failure in their imagined experience; Huang & Mayer, 2019).

Fig. 5

An example of mental practice instruction displayed after each example in the practice activity.

Treatment 4: Attributional feedback

The T4 version included a strategy aligned with the social persuasion source of self-efficacy. Participants in this treatment received the same example-problem pairs as the control group. The only difference was that they received effort feedback at the end of each practice problem following the paired worked example, in addition to the knowledge-of-correctness feedback (as shown in Fig. 6 ). Effort feedback messages were integrated in the design corresponding to each of the five problems in the practice activity, so no participant would receive the same feedback message twice. For each problem, there was a message for a correct response (e.g., Good work. Your effort paid off. Keep it up!) and an incorrect response (Your answer is not 100% correct. Don't give up. Focus on the next example-problem pair. Study the example carefully. With hard work, you can improve your performance.). In other words, participants received one of the two messages in each effort feedback pair for each problem, depending on whether they solved the problem correctly or incorrectly. Appendix B includes the complete sets of the effort feedback messages used in the T4 condition.

Fig. 6

An example of effort feedback displayed after participant submission of a problem in the practice activity.

Treatment 5: Integrated set of strategies

The T5 version of the module included all of the previously mentioned four strategies in the T1 – T4 versions, which were presented as an integrated set. That is, participants in this condition received the anxiety coping message (Strategy 1) at the beginning of the practice activity followed by five pairs of modeling examples (Strategy 2) and corresponding problems. For each modeling example-problem pair, they were asked to mentally practice the problem-solving procedure demonstrated in the example (Strategy 3) before moving to the paired problem. In addition, after each problem, they were presented with an effort feedback message (Strategy 4) either praising their effort (correct response) or encouraging them to keep on trying (incorrect response). The top portion of Fig. 7 shows the study procedures once participants started the module; the bottom portion of the figure shows the similarities and differences among the six versions of the module.

Fig. 7

Structure of the different versions of the instructional material. Differences among the conditions are highlighted.

Assessment materials

Assessment materials1, which were adapted from Huang and Mayer (2019), included a demographic survey, self-efficacy questionnaire, task anxiety scale, practice problems, and retention and transfer posttests.

The demographic survey consisted of questions on participants' basic characteristics, such as age, gender, ethnicity, comfort level with computer-based instruction (1 = extremely uncomfortable; 5 = extremely comfortable), and skill level of basic math calculations (1 = extremely unskilled; 5 = extremely skilled).

A 6-item, task-specific self-efficacy measure was used to assess individuals’ beliefs in their capability to perform the statistical content covered in the online module. Participants were given a 101-point rating scale (0 = no confidence at all; 100 = extremely confident; Bandura, 2006) on which to judge their perceived confidence for performing six tasks related to the two statistical rules (e.g., “Distinguish the conditions for applying the Empirical Rule and Chebyshev’s Rule”). To maximize face validity, the measure was developed based on the specific learning objectives of the online module. Participants’ task-specific self-efficacy was measured twice: before the start of the instruction (α = 0.97) and at the end of the instruction – specifically, after the practice activity but before the performance tests (α = 0.96).

Task anxiety was assessed with a single item. Participants were asked to rate the amount of anxiety they felt when studying the example-problem pairs during the practice activity (1 = very, very low anxiety; 9 = very, very high anxiety). Task anxiety was measured at the end of the practice activity.

The practice activity included 20 questions (α = 0.91) in five word problems (e.g., “A data set with a bell-shaped distribution and size N = 400 has a mean = 3 and a standard deviation = 1.5. Find the approximate number of observations in the data set that lie between 1.5 and 4.5”). Participants received 1 point for each correctly answered question or 0 points for each incorrect response. The maximum total score of the practice problems was therefore 20 points.

The retention posttest included six questions (α = 0.81) that required recalling/recognizing statements concerning the two statistical rules (e.g., “Per Chebyshev's Rule, at least what percentage of the data in a distribution falls within 2 standard deviations from the mean?”). The maximum score on the retention test was 6 points, with 1 point assigned to each correct response and 0 points to each incorrect response.

The transfer posttest test (11 word problems, α = 0.94) included questions of both near and far transfer. The near transfer test included six word problems (with 18 questions) similar to those presented during the practice, requiring the application of one of the two rules [e.g., “A data set of size N = 100 with a bell-shaped distribution has a mean = 5 and a standard deviation = 1.5. Please determine the following: How many standard deviation(s) are 3.5 and 6.5 away from the mean?”]. The far transfer test included five real-life problems (with 13 questions) that were different than the practice problems, although the underlying principles of the problems were similar (e.g., “A total of 200 students took an IQ test. The scores showed a bell-shaped distribution with a mean of 100 and a standard deviation of 17. Please determine the following: Approximately how many people have an IQ score between 83 and 117?”). The maximum total score for the transfer test was 66 points (i.e., the sum of the scores for the 31 individual question items pertaining to the 11 word problems; the score for a correct response ranged from 1 to 3 points per question).

The retention test and the transfer test were presented as a posttest at the end of the online module. We did not include a pretest on statistics performance because research on the testing effect shows that the act of taking a test is itself an instructional experience that can affect subsequent posttest performance (Brown, Roediger, & McDaniel, 2014). Instead, we sought to ensure that our participants did not have previous statistics instruction in the topics in the lesson by asking two screening questions, which asked potential participants whether they had learned about the Empirical rule or Chebyshev’s rule prior to the study. Only participants who responded “no” to both questions were eligible for the study.

Procedure

Participants voluntarily completed the study through the Prolific online crowdsourcing platform where they were recruited. Participants were informed that they would be learning basic statistics content as well as completing related surveys, practice problems, and tests if they decided to participate in the study. A link to the study was created in the platform, and participants clicked the link when they were ready. Consent information was presented first, followed by instructions on how to complete the study (e.g., find a quiet place, minimize distractions, have a calculator and scrap paper ready). After responding to the demographic question on gender, participants were randomly assigned (stratified by gender) to one of the six versions of the module. The average module completion time was about one hour. Each participant was paid $15 after completion of the study. We obtained IRB approval and followed guidelines for treatment of human subjects.

Three attention check criteria were used in the study to ensure the quality of the data: the total time on task (no less than 15 min) and two multiple choice questions embedded in two different places in the online module. The length and format of each attention check question looked similar to that of the other questions where it was embedded (Oppenheimer, Meyvis, & Davidenko, 2009). We added the first attention check question to the first self-efficacy measure (“It is important that you pay attention to this study. Please write the number 25 in the blank next to this statement at the end of the scale”), and the other in the practice activity (“It is important that you pay attention to this study. Please check the radio button next to ‘2’ below”).

Data analyses

First, a number of tests were conducted to check the equivalence of participants assigned to each group in terms of their background characteristics. Chi-square tests of independence were conducted on gender and ethnicity (ethnicity categories were collapsed to White vs. Other Ethnicities due to the small numbers for the other ethnicity groups). One-way analyses of variance (ANOVA) were conducted on age, pretest self-efficacy, comfort level with computer-based instruction, and skill level of basic math calculations. Next, as the time participants spent engaging with the online module (time on task) could be a potential confounding variable influencing the outcomes, another one-way ANOVA was conducted to compare the total time participants spent on the module among the conditions. Finally, to assess the individual and combined effectiveness of the four self-efficacy-supportive strategies, separate one-way ANOVAs with Dunnett’s post hoc tests were conducted to compare each of the five treatment conditions with the control condition on the intended outcome measures: performance (practice, retention, and transfer), self-efficacy, and task anxiety.

Results

Were the groups equivalent on basic characteristics?

As a preliminary step, we sought to determine whether the groups were equivalent in terms of their background characteristics. A chi-square test showed that there was no significant difference among the groups in the proportion of men and women, χ2 = 0.93, p = .97, or of proportion of White and other ethnicities (which we collapsed into one group due to the small numbers in the ethnic groups), χ2 = 3.65, p = .60. One-way ANOVAs showed no significant differences among the six groups in terms of participants’ age, F(5, 273) = 1.55, p = .18, comfort level with computer-based instruction, F(5, 270) = 1.04, p = .39, skill level with basic math calculations, F(5, 272) = 1.19, p = .32, or self-efficacy level prior to the instructional modules, F(5, 273) = 0.56, p = .73. We therefore concluded that the groups were equivalent on basic characteristics.

Were the groups equivalent on time on task?

Next, we sought to determine whether the groups differed in the amount of time they spent on the lesson. A one-way ANOVA indicated that there was no significant difference by condition in the amount of time participants spent on the online module, F(5, 273) = 1.68, p = .14. We thus concluded that the groups were equivalent on time on task.

Did the treatment groups perform better than the control group?

The first prediction of the study was that the treatment group receiving all of the self-efficacy strategies would perform better than the control group on the practice activity, the retention test, and the transfer test. In Part A of Fig. 8 , the first three bars for each Condition category display the means and standard deviations of participant scores on practice, retention, and transfer tests, respectively. Raw scores are presented in the first three rows of Fig. 8, Part B. Separate one-way ANOVAs with Dunnett's post hoc tests were conducted to compare each treatment condition to the control condition on the three performance outcome measures. First, a significant condition difference on practice scores was detected, F(5, 273) = 2.697, p = .021, η 2 = 0.049. Follow-up Dunnett’s tests indicated that the integrated treatment group (M = 17.83, SD = 3.14) performed significantly better during practice than the control group (M = 14.54, SD = 5.07), p = .001, d = 0.780. The condition difference on transfer scores approached significance, F(5, 273) = 2.118, p = .064, η 2 = 0.037. Follow-up Dunnett’s tests showed that the integrated treatment group (M = 53.21, SD = 15.68) performed significantly better on the transfer test than the control group (M = 42.07, SD = 20.63), p = .011, d = 0.608. No significant differences were detected between the treatment groups and the control group on retention, ps > 0.05. Consistent with the first prediction, these findings indicate that the four self-efficacy-supportive strategies—as an integrated set rather than individually—were effective in improving practice and transfer performance. In contrast, none of the four self-efficacy strategies, when presented as individual treatments, produced significantly greater practice, retention, or transfer scores relative to the control condition.

Fig. 8

Means and standard deviations of the outcome measures by condition. Note: Values in parentheses indicate maximum scores. * indicates a significant difference between a treatment group and the control group.

Did the treatment groups report higher self-efficacy than the control group?

The second prediction was that participants receiving all of the self-efficacy-enhancing strategies would report higher self-efficacy after taking part in the statistical learning module than participants in the control group. In Part A of Fig. 8, the fourth bar for each Condition category shows visually the means and standard deviations of participants’ self-efficacy levels, by condition, at the end of the module. Raw scores are listed in the fourth row of Part B of the figure. A one-way ANOVA with Dunnett's post hoc tests revealed a significant effect of condition on participants’ self-efficacy level, F(5, 273) = 2.698, p = .021, η 2 = 0.047. Consistent with the second prediction, follow-up Dunnett’s tests showed that the integrated treatment group (M = 73.64, SD = 25.21) reported significantly higher self-efficacy than the control group (M = 53.97, SD = 31.04), p = .004, d = 0.696. However, no significant difference emerged between the other treatment groups and the control group on reported self-efficacy after the learning module, ps > 0.05. These findings indicate that the four self-efficacy strategies as an integrated set, rather than individually, were beneficial in promoting participant self-efficacy. In contrast, none of the four self-efficacy strategies, when presented as individual treatments, produced a significantly greater self-efficacy rating than did the control condition.

Did the treatment groups report lower anxiety than the control group?

The third prediction of the study was that the participants assigned to the learning module with all of the self-efficacy-supportive strategies would report lower task anxiety levels than those assigned to the control group module (i.e., without those strategies). The last bar in Part A of Fig. 8 displays the means and standard deviations of participants’ self-reported anxiety levels by condition, with the raw numbers provided in the corresponding final row of Part B. A one-way ANOVA with Dunnett's post hoc tests revealed a significant effect of condition on task anxiety: F(5, 273) = 3.821, p = .002, η 2 = 0.065. Consistent with the third prediction of the study, follow-up Dunnett’s tests showed that the integrated treatment group (M = 3.66, SD = 2.23) reported significantly lower anxiety than the control group (M = 4.93, SD = 2.52), p = .034, d = -0.534. However, no significant differences were found between the other treatment groups and the control group on task anxiety, ps > 0.05. Again, these results indicate that the four self-efficacy strategies as an integrated set, rather than individually, were successful in mitigating task anxiety. In contrast, none of the four self-efficacy strategies, when presented as individual treatments, produced a significantly lower anxiety rating compared to the control condition.

Discussion

Main findings

This study compared four self-efficacy-supportive strategies embedded in an online statistical learning module, both individually (T1 – T4) and in a combined manner (T5), with a control condition that did not include any of these self-efficacy-building strategies. Overall, the results showed that the four self-efficacy strategies worked effectively as an integrated set but did not individually affect learner outcomes. Compared to those in the control group who received the online learning module with only knowledge-of-correctness feedback, participants who were randomly assigned to the integrated treatment group (T5) involving all four strategies designed to enhance self-efficacy performed better on practice and transfer tests, and reported higher self-efficacy and lower anxiety than the control group. These findings largely replicate previous results by Huang and Mayer (2019) and provide additional evidence of the effectiveness of instructional design that embeds self-efficacy strategies aligned with the four sources of self-efficacy information described by Bandura (1997). In addition, the effect sizes of the present study were noticeable, with d = 0.780 for practice, d = 0.608 for transfer, d = 0.696 for self-efficacy, and d = −0.534 for task anxiety. These values fall within the medium-to-large effect size range based on Cohen’s standards (1988), higher than those reported by Huang and Mayer (2019).

At the same time, a new finding of the present study is that when only one of the strategies was used (T1 – T4), there was no demonstrated benefit of the treatment that resulted in statistically significant differences between the treatment and control groups. In other words, when any of the four self-efficacy strategies alone was used during the statistical learning module, the effect of the individual strategy was not strong enough to significantly increase learners’ test performance or self-efficacy or to reduce their task anxiety.

Theoretical implications

This research has important theoretical implications. It is among the first studies to systematically investigate the combinatory effect of efficacy-relevant information in the particular setting of adult online mathematics learning. In this way, our findings contribute to the scarce literature in this area by documenting the effectiveness of self-efficacy interventions for online lessons. Specifically, this experiment showed that instructional elements that are systematically designed to address the four sources of self-efficacy information conceived by Bandura (1997)— vicarious experience, mastery experience, social persuasion, and physiological states—enhance both learning and motivational outcomes. No single strategy alone showed these effects, however. We take this as evidence for the mutually-reinforcing power of combining efficacy-supportive strategies in the context of online mathematics instruction.

In contrast to the approach used by Huang and Mayer (2019), the present study was designed to dissect the set of self-efficacy strategies by pinpointing the potential of each strategy to contribute to self-efficacy development and statistical content learning. The results were somewhat surprising, as they showed that alone, a design element reflecting a single source of self-efficacy was insufficient for changing learners’ outcomes. Furthermore, these findings, together with those presented in Huang and Mayer (2019), verify that mathematics self-efficacy is a malleable and dynamic construct (Bandura, 1997, Gist and Mitchell, 1992), and its development can be significantly influenced by a series of purposefully designed instructional strategies over a relatively short period of time. It is worth noting that although one of the strategies – modeling examples – served as a replacement of the written worked examples used in the module, the other three strategies – anxiety coping message, mental practice, and effort feedback – were added elements not present in the control version. For the integrated treatment group, as learners studied the online module, they continuously received new information via one strategy after another, which helped shape their self-efficacy. For the four individual strategy conditions, however, learners were only given one type of efficacy-relevant information, thus the impact of the individual strategy on self-efficacy development and related outcomes was not as impactful.

The results of this study also support Bandura’s (1997) notion of context-dependent benefits. Researchers have pointed out that “people weigh and combine information from the various sources to form self-efficacy judgments” (Schunk & Usher, 2012, p. 22), and the relative influence of each source may differ depending on contextual and circumstantial factors such as gender, ethnicity, task difficulty, and learning domain (Schunk and Usher, 2012, Usher, 2009). In this study, the combination of multiple sources of self-efficacy information was most potent. That is, in the context of studying mathematical content in an asynchronous online learning environment, adult learners seem to benefit most from integrating the sources of self-efficacy information additively, such that the more efficacy-supportive strategies are available, the stronger learners’ self-efficacy beliefs. The findings of the study may help explain why some previous research conducted in similar contexts but targeting only one source of self-efficacy information failed to increase both self-efficacy and learning (e.g., Huang & Mayer, 2016).

Practical implications

The primary practical implication of these findings is that interventions designed to build learners’ self-efficacy may be most effective when they tap all four sources of self-efficacy information as conceived by Bandura (1997). In short, when considering self-efficacy-supportive strategies in the context of learning mathematical content in an asynchronous online learning environment, researchers should make efforts to incorporate all four sources of self-efficacy information rather than focusing on only one of these sources, as the impact on cognitive, affective, and motivational learning is much stronger when the distinct types of efficacy-relevant information are made available to learners.

In addition, the design elements of the self-efficacy intervention in this study may inform other researchers and practitioners of the strategies that can be used to effectively enhance learning and self-efficacy while reducing anxiety in an online learning environment. Self-efficacy is a powerful motivational construct that connects to a number of desired educational outcomes, including enhanced learning performance and reduced anxiety. Numerous research studies have shown the linkage between self-efficacy, anxiety, and learning performance (Griggs et al., 2013, Huang et al., 2019, Huang and Mayer, 2016, Pajares and Graham, 1999). Hence, a self-efficacy-supportive intervention has positive effects beyond self-efficacy improvement. Key design elements of the self-efficacy intervention in this study included four strategies in an example-based learning environment: (a) an anxiety coping message delivered by a pedagogical agent serving a motivational role, which is aligned with the physiological states source, (b) modelled problem-solving process presented by a pedagogical agent serving an expert role, which is aligned with the vicarious source, (c) mental practice after each modelled example, which is aligned with the mastery experience source, and (d) customized effort feedback to each practice problem, which is aligned with the social persuasion source. These strategies can be easily modified and customized to fit with different learning content and in different learning contexts to improve a variety of outcomes related to self-efficacy.

In addition, the fact that we were able to test a self-efficacy-building intervention in the context of an online learning experience provides additional, and timely, practical implications. As mentioned above, online learning has become more commonplace over the past decade, and this trend has rapidly accelerated in response to the current COVID-19 pandemic. Many institutions around the world, whether in emergency response or by preventative measure, have moved traditional face-to-face courses to online delivery formats. Many have announced plans to continue completely online or hybrid instruction indefinitely. Moreover, it is likely that this instructional impact will extend beyond this historical crisis as an increasing number of institutions, instructors, and students see the need of offering more flexible learning opportunities. Nevertheless, anecdotal evidence suggests that students’ motivation and learning have suffered as the result of shifts to online instruction. As mathematical content delivery moves to online formats, instructors might want to consider how to support students’ self-efficacy for mathematical learning. The set of self-efficacy-supportive strategies tested in this study may provide a helpful beginning.

Limitations and future directions

We included the same strategies as were used by Huang and Mayer (2019) in order to dissect the set of four strategies and investigate their individual effects in addition to the combined effect of the four self-efficacy strategies. However, as noted by Huang and Mayer (2019), these four strategies “are just four of many viable forms aligned with Bandura’s four sources of self-efficacy information” (p. 1031). Other valid instructional strategies that align with Bandura’s (1997) theorized sources of self-efficacy can be examined in future research, for example, the use of live models or pedagogical agents whom the learner selects, or a breathing exercise designed to refocus the mind and lower heartrate.

Another limitation is that the present study involved mostly White learners in their early 20s in an online statistical learning environment. Future research should test these strategies in different settings, with different learning content and target learners, and examine if the results of the study can be replicated. Participants in this study may not share similarities to learners in more taxing environments, such as those having to navigate learning complex content online every day and in multiple content areas. In addition, the fact that the participants of the study were recruited from an online crowdsourcing platform presents a potential threat to the external validity of the study. Nevertheless, the context of the study resembled an asynchronous, self-paced, informal learning environment similar to those increasingly seen across college campuses where learning “anywhere, anytime” is emphasized. Future work should test these effects within an authentic learning setting, such as a university-level statistics class, where participants are not incentivized for their participation with monetary awards. This would help researchers eliminate the monetary motivation to participate that might have affected the emotional response, motives, or engagement level reported by participants in this study. Thus, it will be useful to see if the same results can be replicated with participants enrolled in an authentic online class.

Furthermore, qualitative data methods such as interviews and think-alouds may be used to explore whether or how different groups of learners differing in attributes such as ethnicity, age, and gender respond differently to these strategies. Insights gained through these types of data can inform the design of interventions that target specifically to the intended target learners.

Funding

This research was supported by a Quick-Turn-Around-Grant (QTAG), School of Teacher Education, College of Education & Behavioral Sciences, .

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.