Student Buy-In Toward Formative Assessments: The Influence of Student Factors and Importance for Course Success.

Formative assessment (FA) techniques, such as pre-class assignments, in-class activities, and post-class homework, have been shown to improve student learning. While many students find these techniques beneficial, some students may not understand how they support learning or may resist their implementation. Improving our understanding of FA buy-in has important implications, since buy-in can potentially affect whether students fully engage with and learn from FAs. We investigated FAs in 12 undergraduate biology courses to understand which student characteristics influenced buy-in toward FAs and whether FA buy-in predicted course success. We administered a mid-semester survey that probed student perceptions toward several different FA types, including activities occurring before, during, and after class. The survey included closed-ended questions aligned with a theoretical framework outlining key FA objectives. We used factor analysis to calculate an overall buy-in score for each student and general linear models to determine whether certain characteristics were associated with buy-in and whether buy-in predicted exam scores and course grades. We found that unfixed student qualities, such as perceptions, behaviors, and beliefs, consistently predicted FA buy-in, while fixed characteristics, including demographics, previous experiences, and incoming performance metrics, had more limited effects. Importantly, we found that higher buy-in toward most FA types predicted higher exam scores and course grades, even when controlling for demographic characteristics and previous academic performance. We further discuss steps that instructors can take to maximize student buy-in toward FAs.

INTRODUCTION

As the undergraduate population becomes more diverse, national reports have called for an increased use of formative assessments (FAs) (1, 2), which improve learning for all students and decrease achievement gaps for underrepresented minorities and first-generation students (3, 4). FAs can be implemented as pre-, in-, or post-class activities, and they contribute to a course structure that enables students to engage with material and revise their understandings (5). FAs are thought to promote learning through the achievement of five key objectives: 1) clarifying learning intentions and criteria for success, 2) revealing evidence of student understanding to the instructor, 3) providing feedback that moves learners forward, 4) activating students as instructional resources for one another, and 5) activating student ownership of learning (6).

While FAs can facilitate learning, student buy-in toward these methods remains an important area for investigation. Here, we define buy-in as the extent to which students perceive an activity to support their learning. Several reviews have found overall positive student perceptions toward in-class methods, such as clicker questions (7–9), but few studies have examined student perceptions toward out-of-class FAs. Our own prior work on student perceptions toward commonly used pre-class, in-class, and post-class FAs showed that most students find these methods beneficial and recognize specific ways that FAs improve their learning in a course, while a small number of students display resistance (10).

Understanding student buy-in and resistance toward FAs requires identifying extrinsic and intrinsic factors that influence student perceptions. Extrinsic causes of student resistance may include uneven distribution of workload within a group or poor instructor implementation (11). In the present study, we were interested in understanding intrinsic factors, or characteristics unique to a student, that can influence student attitudes and perceptions (11, 12). Several previous studies have investigated the effects of various demographic characteristics on student preferences and attitudes toward assessments and teaching techniques. Research on gender has produced mixed results, with some studies finding more positive attitudes in women (13, 14), others finding more positive attitudes in men (15–18), and a few finding no gender differences (19–21). Lower-division students (13, 14, 22, 23) and nonscience majors (14, 24) have been reported to have more positive attitudes toward clickers and active learning than upper-class students and science majors, respectively. While these studies provide key insights into student perceptions, most have focused on clickers and have neglected to include other important demographics, such as race/ethnicity or first-generation status.

Along with demographics, several other student characteristics can influence attitudes toward teaching techniques, including students’ prior experiences and expectations about college classrooms (11, 23). Disorientation and resistance can result when students’ prior experiences and expectations clash with novel teaching techniques (25, 26). Moreover, negative or positive prior experiences with a particular teaching technique can influence attitudes in subsequent courses (19), although past experiences do not always predict current attitudes (14). In addition to prior experiences, students’ preferred learning approaches, study strategies, and beliefs about learning represent other intrinsic factors that can influence their assessment preferences and reactions to transformed teaching practices (22, 23, 27, 28). Finally, student academic performance may also be an important factor in determining attitudes toward classroom practices. For example, students with higher expected (23) and actual (22) grades have more positive attitudes toward clickers than lower-performing students.

In the present study, we examined student buy-in toward six FA techniques used in twelve undergraduate biology courses to identify factors that influence buy-in and to determine whether high FA buy-in predicts course success. We hypothesized that intrinsic student characteristics, such as demographics, prior experiences, previous academic performance, perceptions, behaviors, and beliefs, would affect FA buy-in and that high student buy-in would lead to better course performance. While other authors have offered theory-based suggestions about how to minimize resistance and improve student attitudes toward new teaching techniques (11, 25, 29, 30), few data exist regarding what factors contribute to either buy-in or resistance and whether buy-in toward specific FAs influences student performance. By investigating the relationships among student characteristics, FA perceptions, and course performance, we hoped to identify potential ways to improve buy-in as well as to determine the importance of FA buy-in for broader academic success.

METHODS

Study context and survey administration

This study included 12 biology courses offered from fall 2014 through fall 2015 at the University of Nebraska-Lincoln, including seven introductory (i.e., 100 level) and five non-introductory (i.e., 200–400 level) courses. The non-introductory courses spanned the full range of topics offered by a general biology program, from molecular biology through ecology and evolution, and these courses were content-based, as opposed to being seminar, lab, or field courses. Each course was taught by a different instructor or instructor pair, and they utilized at least one out-of-class and one in-class FA type, with many courses utilizing three or four FA types. Descriptions of each FA type are shown in Table 1.

TABLE 1

Summary of formative assessment (FA) types used in the study.

FA timinga	FA type	Abbrev.	Description	Course sectionsb	Survey responsesc
Pre-class	Just-in-Time Teaching	JiTT	3–4 questions, typically open-ended, often including a metacognitive question	4	255
	Online textbook program pre-class assignments	OTP-pre	Electronic learning activities (e.g., video tutorials and closed-ended questions) related to the textbook chapter to be covered in class	5	498
In-class	Clicker questions	CQ	Electronic audience response systems in which students submitted answers to closed-ended questions; often accompanied by peer instruction	10	686
	In-class activities	ICA	Activities in which students worked in pairs or small groups to complete a task or set of questions electronically or on a worksheet	4	236
Post-class	Online textbook program post-class assignments	OTP-post	Electronic learning activities (e.g., video tutorials and closed-ended questions) related to the textbook chapter already covered in class	3	220
	Homework assignments/quizzes	HW/Q	Set of questions completed by students about topics already covered in class; question format varied among sections	6	464

Pre- and post-class FAs were completed by students outside of class.

Each course section used at least 2 FA types. There were a total of 12 sections included in the study.

The number of student survey responses for which we have complete data. Students answered questions about 2 FA types on the survey. We had complete data for a total of 1,182 student surveys.

Students in each course completed a survey about their FA perceptions online, outside of class for a small amount of required or extra credit during the second half of the semester. The format of the FA survey has been previously described (10), and the items used in the present study are included in Appendix 1. The FA survey included seven items related to FA buy-in and five items related to factors used as predictors of buy-in. Each student answered questions about two FA types used in their course. For courses with more than two FA types, students were randomly assigned only two of the FAs to avoid survey fatigue.

Of the 1,927 students enrolled in the courses, 75% submitted the FA survey and consented to the study (IRB exempt protocol 14314). We then excluded students with incomplete demographic data, leaving a total of 1,182 students, representing 61% of enrollment. Appendix 2 shows the numbers of students from each course and Table 2 shows the demographics of students in the study.

TABLE 2

Demographic characteristics of students in the study.

Demographic categories	n	%
Gender
Male	453	38.3
Female	729	61.7
Race/ethnicity
Non-URM (White, Asian, International)	1,072	90.7
URM	110	9.3
Generation status
Continuing-generation	842	71.2
First-generation	340	28.8
High school location
Urban or other	854	72.3
Rural	328	27.7
Major
Life Sciences	915	77.4
Non-Life Sciences (other STEM, non-STEM, or undeclared)	267	22.6
Class rank
First-year	380	32.1
Sophomore	397	33.6
Junior	257	21.7
Senior	148	12.5

URM = underrepresented minority.

Measuring FA buy-in

The FA survey included seven items used to measure FA buy-in: two covering the overall benefits of the FA and five items addressing four of the specific FA objectives (6). Each item had a five-point Likert scale of “strongly disagree” to “strongly agree.” Combining data from all FA types, we used principal axis factor analysis in SPSS to identify underlying factors (i.e., unobserved variables) affecting student responses (31). All seven items loaded onto one factor that explained 57.4% of response variance. Item loadings (i.e., the relations between each item and the underlying factor) ranged from 0.65 to 0.83, well above the commonly accepted threshold of 0.3 (32). Factor score estimates were calculated for each student, representing their relative buy-in for each FA type evaluated. For ease of interpretation, FA buy-in factor scores were renormalized to a 0 to 10 scale, with 0 being the lowest factor score and 10 being the highest score in the data set.

Predictor variables

Demographic information and previous academic performance were obtained from the institutional research office and used as predictors of FA buy-in. Demographic variables included gender, race/ethnicity, generation status, high school location, major, and class rank (Table 2). While many predictor variables had relatively balanced representation across categories, it should be noted that only 9% of students in our sample were classified as underrepresented minorities (URMs), so results for this variable may not be broadly generalizable. Previous academic performance was based on a z-score of students’ undergraduate GPA at the beginning of the semester or high school GPA for first-year students. More information about demographic and performance variables is provided in Appendix 3. The FA survey included five items addressing student experiences, perceptions, behaviors, and beliefs that may have predicted FA buy-in. One item asked students how many of their previous high school and college courses used a similar FA type, with a five-point Likert scale ranging from zero to more than eight courses. Two additional items addressed student perceptions about FA question content. Specifically, we asked what percentage of questions were relevant to course content and what percentage challenged students to think more deeply about course content. Another item addressed the frequency with which students discussed FA questions with their peers, using a five-point scale of never to always. Although peer discussion is one of the five FA objectives, we did not include this item as part of the buy-in score because it related to behavior and therefore was more appropriate as a predictor variable. A final item addressed students’ beliefs about their responsibility for learning, in which higher values represented a belief that students have more responsibility than the instructor and lower values indicated a belief that the instructor has more responsibility than the student for learning.

Statistical models

We used general linear models in SPSS to analyze the relationships between predictor variables and a continuous dependent variable (33). We followed procedures to determine that collinearity among predictor variables was not preventing detection of significant effects (Appendix 4).

The first set of statistical models examined the effect of several predictors on buy-in score for each FA type. The predictors included all the demographic variables, previous classes with a similar FA, GPA z-score, perceptions about FA content (i.e., relevant and challenging), FA discussion frequency, and belief in student responsibility for learning. We also included course section in the model to control for variation among courses and instructor implementation.

A second and third set of general linear models were created for each FA type to determine whether FA buy-in scores predicted exam grades and overall course grades, respectively. To isolate the influence of FA buy-in while controlling for other factors that may influence grades, we also included all the demographic variables, GPA z-score, and course section in the models. For exam grades, we calculated each student’s exam average in the course, excluding missed exams, and calculated z-scores within each course section to account for variation in exam averages across sections. For course grades, we converted letter grades to a standard 4.0 numeric scale (see Appendix 3). To visualize model-predicted grades of hypothetical students across the buy-in spectrum, we used the intercept and B coefficient values from the general linear models to calculate point estimates of grades for students with buy-in scores at the 95th, 50th, and 5th percentiles for each FA type.

RESULTS

Influence of student characteristics on FA buy-in

Each FA type yielded a wide range of variation in FA buy-in score (Fig. 1). Average results of student responses to questions regarding previous FA experiences, FA question content, FA discussion frequency, and responsibility for learning are shown in Table 3.

FIGURE 1

Distributions of FA buy-in scores for each FA type. Central bars represent medians, boxes represent inner quartiles, and whiskers represent the 5th and 95th percentiles. FA = formative assessment; JiTT = Just-in-Time Teaching; OTP-pre = online textbook program pre-class assignments; CQ = clicker questions; ICA = in-class activities; OTP-post = online textbook program post-class assignments; HW/Q = homework assignments/quizzes.

TABLE 3

Means (±SD) of student responses to survey questions serving as predictors of buy-in toward each formative assessment (FA) type.a

	Pre-Class		In-Class		Post-Class
	JiTT	OTP-pre	CQ	ICA	OTP-post	HW/Q
Previous classes with FA
Scale of 1 (0 courses) to 5 (more than 8 courses)	2.3 ± 1.1	2.3 ± 1.0	2.7 ± 1.1	2.4 ± 1.2	3.1 ± 1.2	3.3 ± 1.3
% FA questions relevant
Scale of 0–100%	83.7 ± 19.9	76.6 ± 20.9	87.0 ± 17.0	81.7 ± 20.9	74.8 ± 22.3	84.6 ± 18.7
% FA questions challenging
Scale of 0–100%	73.1 ± 22.5	64.5 ± 23.6	74.5 ± 21.0	69.2 ± 24.5	63.8 ± 25.5	74.2 ± 21.5
FA discussion frequency
Scale of 1 (never) to 5 (always)	2.5 ± 1.2	2.4 ± 1.1	3.6 ± 1.1	3.4 ± 1.2	2.5 ± 1.1	2.6 ± 1.2

Students also answered one global question regarding the extent to which they believed learning to be the responsibility of the student versus the instructor. On a scale of 0–100, the mean response to this question was 61.3 (± 17.2), indicating that students viewed themselves as slightly more responsible than the instructor.

JiTT = Just-in-Time Teaching; OTP-pre = online textbook program pre-class assignments; CQ = clicker questions; ICA = in-class activities; OTP-post = online textbook program post-class assignments; HW/Q = homework assignments/quizzes.

We used general linear models to determine which student characteristics influenced FA buy-in (Table 4). Most demographic characteristics, including gender, race/ethnicity, high school location, major, and class rank, were not significantly predictive of buy-in toward any of the FA types. Compared with continuing-generation students, first-generation students had lower buy-in toward Just-in-Time Teaching (JiTT) and online textbook program post-class (OTP-post) assignments, but had equivalent buy-in toward the other FA types. Students who had more experience with pre-class assignments had higher buy-in toward online textbook program pre-class (OTP-pre) assignments. Conversely, students who had more previous classes with in-class activities had lower buy-in toward those activities. Higher GPA predicted lower buy-in toward JiTT and higher buy-in toward clicker questions (CQs). Overall, these results suggest that demographic characteristics, previous experiences, and incoming academic performance only influenced buy-in toward select FA types.

TABLE 4

Results of general linear models to assess influence of student characteristics on formative assessment (FA) buy-in.a

Model predictors	Outcome variable
	Pre-Class FAs		In-Class FAs		Post-Class FAs
	JiTT Buy-In	OTP-pre Buy-In	CQ Buy-In	ICA Buy-In	OTP-post Buy-In	HW/Q Buy-In
Student demographics
Gender	-0.35 ± 0.19	0.02 ± 0.14	-0.01 ± 0.12	-0.09 ± 0.23	0.05 ± 0.19	0.14 ± 0.14
Female (ref: male)
Race/ethnicity	0.10 ± 0.29	0.06 ± 0.23	0.27 ± 0.20	−0.17 ± 0.48	−0.09 ± 0.33	−0.13 ± 0.24
URM (ref: non-URM)
Generation status	−0.44 ± 0.21	0.06 ± 0.15	−0.07 ± 0.13	−0.26 ± 0.25	−0.55 ± 0.21	0.07 ± 0.15
First-generation (ref: continuing-generation)
High school location	−0.01 ± 0.22	−0.08 ± 0.15	0.06 ± 0.13	−0.15 ± 0.23	−0.01 ± 0.20	−0.23 ± 0.15
Rural high school (ref: urban or other)
Major	−0.03 ± 0.23	−0.20 ± 0.16	−0.05 ± 0.14	0.50 ± 0.26	−0.41 ± 0.21	−0.04 ± 0.15
Non-life sciences (ref: life sciences)
Class rank	0.14 ± 0.12	−0.04 ± 0.09	−0.02 ± 0.07	−0.09 ± 0.14	−0.21 ± 0.11	−0.02 ± 0.08
Student prior experiences and incoming academic performance
Previous classes with similar FA (5-point scale)	0.11 ± 0.09	0.14 ± 0.07	0.02 ± 0.07	−0.22 ± 0.10	0.02 ± 0.08	0.10 ± 0.06
GPA (z-score)	−0.21 ± 0.10	−0.03 ± 0.08	0.16 ± 0.07	−0.13 ± 0.12	−0.12 ± 0.10	0.07 ± 0.07
Student perceptions and behaviors related to the FA and beliefs about learning
% FA questions relevant (Scale of 0–100)	0.02 ± 0.01	0.03 ± 0.004	0.03 ± 0.004	0.02 ± 0.01	0.03 ± 0.01	0.03 ± 0.004
% FA questions challenging (Scale of 0–100)	0.02 ± 0.01	0.03 ± 0.003	0.02 ± 0.003	0.03 ± 0.01	0.03 ± 0.004	0.01 ± 0.003
FA discussion frequency (5-point scale)	0.31 ± 0.09	0.05 ± 0.06	0.25 ± 0.06	0.15 ± 0.10	0.24 ± 0.09	0.23 ± 0.06
Student responsibility for learning (Scale of 0–100)	0.01 ± 0.01	0.01 ± 0.004	0.01 ± 0.003	0.003 ± 0.01	0.01 ± 0.01	0.01 ±.004
Course sectionb	F3, 252 = 0.96	F 4, 494 = 0.94	F9, 676 = 5.22	F3, 232 = 24.42	F2, 217 = 3.97	F5, 458 = 3.49

Most numbers shown are unstandardized B coefficients ± SE and should not be compared among predictors, but rather relative to the categories or scale of each predictor. For categorical demographic groups of interest, the coefficients shown represent the change in FA buy-in compared with the reference group indicated in parentheses. For predictor variables on a continuous scale, the coefficients represent the change in buy-in per one unit increase, with the total scale shown in parentheses.

Course section was included for control purposes. Since pairwise course comparisons were not of interest, F values are shown to reflect the overall ratio of the variation between sample means to the variation within samples, rather than B coefficients.

JiTT = Just-in-Time Teaching; OTP-pre = online textbook program pre-class assignments; CQ = clicker questions; ICA = in-class activities; OTP-post = online textbook program post-class assignments; HW/Q = homework assignments/quizzes; SE = standard error; URM = underrepresented minority.

Numbers in bold are statistically significant (p < 0.05). Adjusted R2 of the models ranged from 0.32–0.56.

The models also included student perceptions and behaviors associated with the FA as well as beliefs about learning (Table 4). Students who rated a higher percentage of FA questions as relevant and challenging had higher buy-in toward all FA types. For many FA types (i.e., JiTT, clicker questions, and post-class FAs), students who reported discussing FA questions more frequently had higher buy-in. In addition, students who accepted more responsibility for their own learning had higher buy-in toward pre- and post-class FAs. Overall, these results show that student perceptions, behaviors, and beliefs were more broadly important for determining FA buy-in.

Finally, course section was included in the models to account for differences among the classes, including instructor FA implementation. We found significant effects of course section on buy-in toward clicker questions, in-class activities (ICAs), OTP-post assignments, and homework (HW/Q) but not toward JiTT or OTP-pre assignments (Table 4). This suggests that for in-class and post-class FAs, instructional implementation was likely an important factor influencing student buy-in.

Influence of FA buy-in on course performance

To determine whether FA buy-in influenced course success, we used general linear models with outcome variables of exam scores and course grades. After controlling for demographics, GPA, and course section, we found that for all FA types except in-class activities, higher FA buy-in predicted higher exam and course grades (Table 5). An increase in buy-in score of one unit yielded a modest influence on grades; however, these effects were more striking when comparing students at different points of the buy-in spectrum. To illustrate this effect, we graphed model-predicted point estimates of grades for students with very high, medium, and very low buy-in (i.e., resistant), at the 95th, 50th, and 5th percentile buy-in scores, respectively, for each FA type (Fig. 2). Since the models control for demographics, GPA, and course section, these point estimates represent predictions for hypothetical students with the same values for these other variables. For FAs where buy-in significantly predicted grades, students with very high buy-in had exam grades 0.1 to 0.2 standard deviations above students with medium buy-in and 0.3 to 0.5 standard deviations above students with very low buy-in (Fig. 2A). With an average exam grade standard deviation of 12.7 percentage points, these differences translate into a boost of 1.3 to 2.5 and 3.8 to 6.4 percentage points, respectively. Additionally the model predicted that students with very high buy-in had course grades 0.1 to 0.3 GPA units higher than students with medium buy-in and 0.3 to 0.6 GPA units higher than students with very low buy-in (Fig. 2B).

TABLE 5

Parameter estimates of general linear models to assess influence of formative assessment (FA) buy-in on course performance.a

	Outcome Variable
	Exam Grade z-Score	Course Grade
JiTT buy-in	0.05 ± 0.02	0.08 ± 0.03
OTP-pre buy-in	0.04 ± 0.02	0.04 ± 0.02
CQ buy-in	0.05 ± 0.02	0.05 ± 0.02
ICA buy-in	0.03 ± 0.02	0.03 ± 0.02
OTP-post buy-in	0.06 ± 0.03	0.07 ± 0.03
HW/Q buy-in	0.10 ± 0.02	0.11 ± 0.02

Numbers shown are unstandardized B coefficients ± SE, representing the change in exam z-score or course grade (in GPA units) for a one-unit increase in FA buy-in score.

Numbers in bold are statistically significant (p < 0.05). Adjusted R2 of the models ranged from 0.40–0.54 for exam grades and 0.41–0.56 for course grades.

Separate models were generated for each FA type. Models also controlled for demographic variables, incoming GPA, and course section. JiTT = just-in-time teaching; OTP-pre = online textbook program pre-class assignments; CQ = clicker questions; ICA = in-class activities; OTP-post = online textbook program post-class assignments; HW/Q = homework assignments/quizzes.

FIGURE 2

Model-predicted exam grade z-scores (A) and course grades (B) of students with differing FA buy-in. Bars represent point estimate predictions from the general linear models for grades of three hypothetical students with very high, medium, and very low buy-in (i.e., resistant), at the 95th, 50th, and 5th percentile buy-in scores, respectively, for each FA type. Asterisks indicate FAs for which buy-in significantly influenced grades (p < 0.05; Table 5), while controlling for demographic variables, incoming GPA, and course section. FA = formative assessment; JiTT = Just-in-Time Teaching; OTP-pre = online textbook program pre-class assignments; CQ = clicker questions; ICA = in-class activities; OTP-post = online textbook program post-class assignments; HW/Q = homework assignments/quizzes.

DISCUSSION

Factors influencing student buy-in toward FAs

Identifying which student characteristics influence buy-in toward FAs is important for understanding how to help students value these methods and minimize potential resistance. While knowledge about buy-in toward FAs has been mostly limited to clickers, our study included a range of FA types and provided needed insight into other common FA activities. Within the sample collected, we found support for our hypothesis that student characteristics influence FA buy-in; however, only some characteristics had broad effects. Fixed characteristics, such as demographics, previous experience, and incoming academic performance were relatively poor predictors of buy-in, while unfixed qualities, including perceptions, behaviors, and beliefs, had more consistent influences (Table 4).

The finding that student buy-in did not differ based on most demographic traits contrasts with many prior studies. While other investigations have found that attitudes toward teaching techniques differ according to gender (13–18), major (14, 24), and class rank (13, 14, 22, 23), we found that FA buy-in was not influenced by these or other demographic factors, including race/ethnicity and high school location. This difference could have been due either to differences in study populations or the fact that we controlled for more variables in our study.

We did find that in comparison with continuing-generation students, first-generation students had lower buy-in toward select FA types. The underlying cause of differing attitudes between these groups remains unclear, but first-generation students face many obstacles that could influence their buy-in toward FAs. They frequently enter college with less preparation (34, 35) and less familiarity with university norms (36). First-generation students can struggle with time management and understanding faculty expectations (37), are more likely to live off-campus and have nonacademic (i.e., work, family) responsibilities (34, 38–40), and are less likely to be academically and socially engaged in college (39, 41, 42). While we hypothesize that first-generation students’ lower buy-in toward two out-of-class FAs may stem from external commitments or time management, these results warrant further investigation, particularly since FAs disproportionately help first-generation students (3, 4).

As with demographics, we found that prior experiences and academic performance only influenced buy-in for select FAs. For most FA types, prior experience with a similar FA had no influence on buy-in. This finding differs from previous work emphasizing the role of prior experiences in shaping student attitudes toward teaching techniques (11, 19, 23, 25, 26). In our study, previous experiences with a similar FA type predicted higher buy-in toward OTP-pre assignments and lower buy-in toward in-class activities, which could have stemmed from positive or negative carry-over from similar activities in the past (19). With respect to previous academic performance, students with higher incoming GPA had lower buy-in toward JiTT and higher buy-in toward clicker questions. Previous studies of student attitudes toward JiTT have not addressed student differences (43–45), but the clicker results parallel other studies (22, 23).

In contrast to the fixed characteristics, student perceptions, behaviors, and beliefs more consistently predicted FA buy-in. We found that students had higher buy-in toward all FA types when they perceived that the FA questions related to course content and challenged them to think more deeply. This finding suggests that students value high-quality FA questions and agrees with our prior analysis of open-ended survey responses showing that dissatisfaction with the content of FA questions was a common source of resistance (10). In addition, the present study revealed that students who discussed FA questions more frequently and accepted more ownership over their learning had higher buy-in toward many FA types. While students’ learning approaches and strategies can influence their preferences for certain types of summative assessments or other teaching techniques (27, 28), few studies have examined how students’ approaches affect their attitudes toward specific FAs. One study found that a desire to be involved and engaged in class predicted positive perceptions of clicker activities (23). Taken together, our results suggest that student perceptions, behaviors, and beliefs represent promising avenues for delineating sources of FA buy-in and resistance.

In addition to identifying factors associated with FA buy-in, we also found that, after controlling for other relevant variables, higher student buy-in toward most FA types predicted higher exam grades, which are determined independently from FA scores, and higher course grades, which include FA points (Table 5). Model predictions also indicate that FA buy-in can have modest, but potentially consequential, impacts on academic achievement (Fig. 2). Building on a recent study finding that students with high buy-in toward in-class active learning are more likely to engage in self-regulated learning and have higher course grades (46), we hypothesize that students who have higher buy-in engage more deeply with the FAs and gain more conceptual learning, while resistant students may only engage on a surface level. This explanation also follows from other work demonstrating that positive perceptions of learning environments lead to deeper study approaches for summative assessments (47–49) and that these approaches are associated with higher exam scores (50–52).

Implications for instruction

These findings have specific implications for instructors wishing to optimize their use of FAs. Our finding that unfixed student qualities were more predictive of FA buy-in than fixed characteristics suggests that instructors can make tractable changes to improve student FA buy-in. We also found that buy-in toward in-class and post-class FAs significantly varied among course sections, suggesting that instructional implementation plays an important role in shaping student buy-in. Moreover, since higher buy-in predicted improved course performance, instructors should spend time and effort cultivating student buy-in, including explaining to students how FAs are intended to promote their learning (11, 25, 29, 53). We further identified three main areas instructors can leverage to foster FA buy-in: making FA questions relevant and challenging, encouraging student discussion of FA questions, and empowering student ownership of learning.

Incorporating relevant and challenging questions represents one way that instructors can potentially improve student buy-in. To create relevant questions, instructors can align FA questions with their learning objectives and summative assessments (54, 55) through a process of backward design (56, 57). Instructors can create challenging questions by emphasizing higher order cognitive processes (58–60). To stimulate critical thinking, FA activities can explore student misconceptions (61) and ask students to connect scientific principles to real-world situations (62).

Student discussion also holds promise as a way to encourage buy-in and promote learning both during (63) and outside of (64) class. While some students may be less inclined to discuss with peers during and outside of class (22, 65), explaining how discussion supports the learning process, using best practices when forming student discussion groups, and creating a classroom culture that values student talk can each contribute to a sense of community and help students feel more comfortable participating in discussions (3, 66, 67). Instructors can spark discussion by providing complex problems that require collaboration or improve the quality of student discussions by explicitly prompting students to share their reasoning (68). Influencing student behavior related to FAs that occur outside of class time may be challenging, but instructors can reserve class time for discussion of these FAs, a key component of the JiTT pedagogy (69), or use online forums to provide a platform for out-of-class discussions (70).

Finally, instructors can work to improve student buy-in, particularly toward out-of-class FAs, by empowering students to take ownership of their learning. Students’ behaviors and beliefs associated with learning, including their sense of responsibility for learning, tend to remain stable over time (71, 72); however, these qualities can evolve and be influenced by instructors (72, 73). By using process-oriented teaching methods, such as helping students diagnose their learning patterns, explicitly modelling alternative thinking strategies, and actively encouraging students to try those strategies, instructors can help students gradually transition from teacher-regulated to student-regulated learning and adopt a conception that learning involves construction rather than intake of knowledge (74, 75). In addition, instructors can provide other opportunities both in and out of class for students to reflect on their understanding, confusion, and study habits in order to develop deeper metacognition (76), which forms an integral part of self-regulated learning (77, 78).

In conclusion, this study identifies unfixed student characteristics as promising leverage points by which instructors can foster student buy-in and demonstrates that buy-in toward particular FAs predicts course success. As with many educational studies, these results were obtained with a specific student population and particular instructional implementations, and thus, the generalizability of our results will require additional research. Furthermore, we anticipate that other important factors affecting FA buy-in will emerge as researchers continue to explore connections between student engagement with FAs and academic success.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.