Revisiting Clickers: In-Class Questions Followed by At-Home Reflections Are Associated with Higher Student Performance on Related Exam Questions.

Clicker questions are a commonly used active learning technique that stimulates student interactions to help advance understanding of key concepts. Clicker questions are often administered with an initial vote, peer discussion, and a second vote, followed by broader classroom explanation. While clickers can promote learning, some studies have questioned whether students maintain this performance on later exams, highlighting the need to further understand how student answer patterns relate to their understanding of the material and to identify ways for clickers to benefit a broader range of students. Systematic requizzing of concepts during at-home assignments represents a promising mechanism to improve student learning. Thus, we paired clicker questions with at-home follow-up reflections to help students articulate and synthesize their understandings. This pairing of clickers with homework allowed us to decipher how student answer patterns related to their underlying conceptions and to determine if revisiting concepts provided additional benefits. We found that students answering both clicker votes correctly performed better on isomorphic exam questions and that students who corrected their answers after the first vote did not show better homework or exam performance than students who maintained an incorrect answer across both votes. Furthermore, completing the follow-up homework assignment modestly boosted exam question performance. Our data suggest that longer-term benefits of clickers and associated homework may stem from students having repeated opportunities to retrieve, refine, and reinforce emerging conceptions.

INTRODUCTION

Over the past decades, science, technology, engineering, and math (STEM) classrooms have seen an increased emphasis on active learning (1, 2). When implemented effectively, active learning can improve student engagement, attitudes, and conceptual understanding and support a variety of outcomes, such as improved course performance and decreased achievement gaps (3–6). Many undergraduate instructors have adopted the active learning technique of administering clicker questions with peer instruction (7). A recommended clicker sequence (8) begins with instructors displaying a closed-ended question and students submitting individual answers via an electronic audience response system. Students are then prompted to discuss their answers in small groups and submit another answer in light of their discussions. Following answer submission, the instructor can invite students to share their reasoning and can provide further explanation regarding the answer options.

Clickers started to make their appearance as early as the mid-1990s, gaining popularity by the early 2000s (8–10). Asking clicker questions at strategic points during class helps break up the session, gives students time to process ideas, and highlights key concepts. Importantly, clickers allow the instructor to gauge student understanding and provide an opportunity for students to receive feedback. Clickers have been investigated across a range of course levels, and many reports associate clickers with a variety of benefits (11–14). Students generally express satisfaction with clickers, view them as helpful to their learning, cite specific aspects they value, and recommend their continued use (6, 15–20). The addition of clickers to a lecture, especially when coupled with low-risk participation points, increases class attendance and decreases attrition across the semester (7, 21–24). Compared to traditional lecture, course offerings that include clickers can result in greater student learning as measured on concept inventories or final exams (21, 25–28), although it can be difficult to isolate the effect of clickers from other course components and instructional interventions (7).

Student scores commonly increase from a first clicker vote to a second clicker vote that occurs after peer discussion (15, 16, 29, 30). Thus, some researchers have framed the question of clicker impact in terms of whether this improvement stems from peer discussion and associated learning versus students taking additional time to think or obtaining answers from their peers. One study addressed this question by posing an isomorphic question to students immediately after the second vote (31). They found that improved performance on the second vote persisted to the isomorphic question, suggesting that students changing to the correct answer reflected learning from peer discussion. Follow-up work in which students were exposed to different combinations of peer discussion and instructor explanation replicated these findings and showed an additional benefit of instructor explanation (32, 33).

Mixed results have been found, however, with respect to whether clicker-based improvements are retained on longer time scales. In one study, the initial boost seen from first to second vote was not maintained when students were asked similar questions on the subsequent exam (34). In another study conducted by members of our research group, we found that students in higher performance quartiles maintained clicker learning gains, whereas students in lower quartiles did not demonstrate similar benefits on later exams, and this finding held true for questions asked in either a multiple-choice (MC) or multiple-true-false (MTF) format (35). Finally, a study looking at retention after 4 months found nuanced results, namely, that clickers led to better retention of material for students from a nonmajors course but not from a majors course (17).

We sought to develop an instructional intervention that could potentially improve clicker outcomes while simultaneously providing more in-depth information on the understandings that students have after clicker activities. We implemented this intervention in the context of an online assignment based on the role of homework in reinforcing material covered during class (36). We hypothesized that having students revisit targeted clicker questions on homework assignments and explain the reasoning behind the answers would support improved performance on later exam questions addressing the same concepts. Instructional cues and course activities have been shown to affect student behaviors during clicker discussions (37–40), so we reasoned that this later homework activity would encourage students to focus on developing correct reasoning during their discussions and ensure that each student had an opportunity to express their understandings. We also viewed this homework assignment as a way for students to engage in retrieval practice, a process by which later engagement with a concept strengthens learning (41).

Here, we describe results from our approach of giving students clicker questions, followed by a reinforcing homework assignment and later isomorphic exam question. We structured our analyses around four research questions: (i) Does participation in clicker questions and completion of the homework predict performance on isomorphic exam questions? (ii) How do clicker vote patterns relate to student homework explanations? (iii) How do clicker vote patterns relate to performance on isomorphic exam questions? (iv) How do homework explanations relate to performance on isomorphic exam questions?

In each case, we conducted additional exploratory analyses to understand variation across student performance quartiles. By tracking student understanding across this sequence of events, our research aimed to better understand how clickers and homework relate to student learning within the given instructional context.

METHODS

Course context and research design

This study occurred in two equivalent sections of a high-enrollment undergraduate introductory biology course for life sciences majors at a research university (n = 346 consenting students; demographics of the class are presented in Table 1; classified as exempt from IRB review, 14314). Both sections were cotaught by two instructors who alternated the weeks in which they took the lead during class. The course consisted of roughly 2 to 4 unique clicker questions per 50-min class session (3 sessions per week), one homework assignment per week, and one exam for each of the four units. Within this broader context, a subset of 16 clicker questions (four per unit) was targeted on the follow-up homework assignment and subsequent exam. Thus, the primary data for this study consist of the pathways each student took from their clicker votes to homework explanations and to subsequent exam answers (Fig. 1).

TABLE 1

Demographics of consenting students (n = 346)

Category	Demographic	n	%
Genderb	Female	209	60.4
	Male	137	39.6
Generation statusc	Continuing generation	260	75.1
	First generation	86	24.9
Race and ethnicityd	Non-URM	304	87.9
	URM	42	12.1
Class ranke	First yr	213	61.6
	Non-first yr	133	38.4

Demographics were obtained from the institutional data office.

At the time of data collection, the institution did not collect information regarding nonbinary gender identities.

Students were considered continuing-generation if one or both parents had a bachelor’s degree. Students were considered first-generation if neither of their parents had a bachelor’s degree.

Non-underrepresented (non-URM) included white, Asian, and international students. Underrepresented minority (URM) included black, Hispanic, Hawaiian/Pacific Islander, and American Indian/Alaska Native students.

Class rank was not included in analyses but is provided here for context.

FIG 1

Diagram of the study design, showing pathways for two targeted clicker questions (of 16 total targeted clicker questions). All targeted clicker questions used the multiple-true-false (MTF) format. During class, all students saw the targeted clicker questions. For the first targeted clicker question (clicker 1), students answered individually (vote 1), discussed in small groups, answered individually again (vote 2), and the instructor then went through explanations for the correct answers. This sequence was later repeated for the second targeted clicker question (clicker 2). On the subsequent homework activity occurring at the end of the week, half of the students randomly received one of the clicker questions and the other half randomly received the other clicker question, and students explained why each of the four clicker question statements was true or false. For the exam, all students saw an isomorphic question corresponding to each targeted clicker question.

While the larger pool of clicker questions in the course consisted of MC and MTF questions, the 16 targeted clicker questions were all in the MTF format, thereby providing an efficient way to collect detailed information on student understanding of various conceptions (42–44). Each MTF question stem was followed by four statements with the possibility that one, two, or three of these statements was true for each question (see Fig. S1 in the supplemental material for an example clicker question). Students answered by selecting each statement that they deemed to be true on an electronic iClicker device. Clicker questions were presented to the class for an individual vote (i.e., vote 1), followed by small group discussion with nearby peers and then a second individual vote (i.e., vote 2). After the second vote, the instructor always showed the correct answers, sometimes had select students share their reasoning with the broader class, and always made sure to provide an explanation behind the answer for each statement (regardless of whether the statement was true or false). Clickers comprised 10% of the course grade. For each question, 0.9 points were awarded for participating and 0.1 point was given for answering the four statements correctly.

At the end of each week, two clicker questions from the same class day (i.e., targeted clicker questions) were incorporated in the form of a homework question embedded within a larger online assignment. For this homework activity, a targeted clicker question was displayed verbatim along with the correct answers, and students were directed to “provide a separate explanation for why each of the four statements is true or false.” Prior to the first assignment, an instructor told students that this activity was designed to help them synthesize their understandings and emphasized how students should deliberately structure their explanations to address each MTF statement. While all students received the larger online assignment each week, half of the students randomly received one of the targeted clicker questions and the other half received the other targeted clicker question. Homework was worth 20% of the course grade. The clicker homework question was graded by undergraduate assistants on a scale from 0 to 4 points. These points counted for the assignment grade but were not used for research analyses.

Each targeted clicker question then appeared on the subsequent exam in the form of an isomorphic MTF question, embedded within a larger MTF section. These isomorphic questions presented scenarios and true-false statements that aligned with but differed from the original clicker question, such that they covered similar concepts but required students to have correct understandings to answer the new question (see Fig. S1 in the supplemental material for an example isomorphic exam question). The four unit exams were worth a total of 40% of the course grade, and MTF questions were scored based on how many of the four statements were answered correctly.

Data analysis

While there were 16 targeted clicker questions in this study, we considered each of the four statements per targeted clicker question as a separate item for the purposes of analysis, leading to a total of 64 items. This decision mirrors how MTF questions have been analyzed in previous work and reflects the intention that each item targets a different conceptual aspect of the scenario (45–48).

Members of the research team coded student open-ended explanations for the clicker homework questions, which enabled us to examine how student understanding related to clicker participation and later exam performance. For each MTF statement, we evaluated the corresponding part of the student open-ended explanation by using a binary coding scheme (n = 9,408 total coding events). A student’s explanation was coded as “(1) demonstrating understanding” if they provided a thorough answer that gave sound reasoning as to “why” a statement was true or false. Explanations were coded as “(0) incomplete/incorrect” if they reiterated the statement without demonstrating an in-depth understanding of the “why” behind the statement or if they provided incorrect reasoning or irrelevant information.

Members of the research team initially worked in pairs to code the open-ended clicker homework explanations. For each MTF statement, we developed a more detailed codebook that provided specific information on the types of student responses reflecting each coding category. For each round, pairs coded 15 explanations independently, calculated their percent agreement, discussed disagreements to consensus, and updated the codebook based on their conversations. Reliability was considered to have been achieved when the two coders reached at least 80% agreement in two consecutive rounds for a given statement, at which point one coder finished coding the remaining explanations.

Statistical analyses were completed in R (version 3.6.3, RStudio version 1.2.5042). Models were run using generalized linear mixed models with a binomial distribution and logit link function in the lme4 package (49). Student was included as a random effect in each model. Demographic variables of gender, race and ethnicity, and first-generation status were incorporated to account for potential confounding effects (50), but these covariates were not a major focus of the current investigation and therefore are not discussed at length here. For certain analyses, students who did not turn in a homework assignment at all were removed in order to specifically compare students who did or did not receive a particular targeted question for reflection.

Model selection was conducted using a backwards stepwise selection process to obtain the model with the lowest Akaike information criterion (AIC) value (51). Starting with a full model, new models were tested by successively omitting interaction terms with the highest nonsignificant P values first and then omitting main effects with the highest nonsignificant P values. Terms were retained in the model if their removal caused a >2-point increase in AIC relative to the previous model. Additionally, each student was assigned into a single quartile for the semester based on their average performance on the other closed-ended, nontargeted questions across the four unit exams. Following model selection, exploratory analyses were conducted to examine contrasts within student quartile groups. Post hoc tests were conducted using lsmeans (52), psych (53), and multcomp (54) packages, and figures were produced using ggplot2 (55).

RESULTS

Research question 1: Does participation in clicker questions and completion of the homework predict performance on isomorphic exam questions?

We sought to determine from an overarching level whether participating in the targeted clicker questions and giving a homework explanation predicted performance on later isomorphic exam questions. Students were considered to have participated in a targeted clicker question if they submitted two votes for that question (irrespective of correctness) and were considered to have completed the associated homework component if they submitted an explanation (irrespective of correctness). At the whole-class level, we found that being present for a targeted clicker question and completing a homework explanation each had significant effects on later exam performance (Fig. 2A and Table 2). The interaction term between these variables was not retained, suggesting that there were no synergistic benefits to completing both a targeted clicker and its associated homework question.

FIG 2

Effect of clicker participation and homework completion on isomorphic exam question performance. Dots represent modeled least-squared means and bars represent 95% confidence intervals. (A) Overall effects of clickers and homework. (B) Clicker effect for each student quartile. (C) Homework effect for each student quartile.

TABLE 2

Effect of clicker participation and homework completion on isomorphic exam question performance

Factor	Estimate	SE	Z value	P value
(Intercept)	1.064	0.142	7.507	<0.001
Clicker participation	0.392	0.085	4.636	<0.001
Homework completed	0.140	0.038	3.732	<0.001
Generation status (first generation)	−0.002	0.168	−0.012	0.990
Race and ethnicity (URM)	0.032	0.212	0.151	0.880
Clicker participation*generation status	−0.221	0.157	−1.410	0.159
Clicker participation*race and ethnicity	−0.292	0.193	−1.513	0.130

Logit (exam score) was calculated as follows: clicker participation + homework completion + generation status + race and ethnicity + clicker participation*generation status + clicker participation*race and ethnicity (see Table S1 in the supplemental material for information on model selection). P values in boldface indicate significance (P < 0.05). SE, standard error.

Given the nature of these effects, we separately explored these variables across student quartiles to determine if they had selective benefits for certain students. We found a general upward trend across quartiles when students were absent for the clicker question (Fig. 2B) or did not complete the homework explanation (Fig. 2C). In comparison to those baseline conditions, we observed that being present for a targeted question was not associated with improved performance on the later exam question for lower-quartile students but was associated with higher performance for higher-quartile students (Fig. 2B; see also Table S2 in the supplemental material). Conversely, students across different quartiles experienced moderate but even benefits of submitting homework explanations (Fig. 2C; see also Table S3).

Research question 2: How do clicker vote patterns relate to student homework explanations?

The clicker question homework activity presented a unique opportunity to see how clicker vote patterns related to underlying student reasoning, expressed as open-ended explanations regarding the various MTF statements in the clicker question. This model included each clicker vote as a separate binary term (incorrect/correct) as well as an interaction effect between the vote terms. This interaction effect allowed us to identify whether vote 1 (V1) and vote 2 (V2) showed any combined effects on homework explanations. We did not observe significance for the V1*V2 interaction or the clicker vote main effects in the best-fit model, suggesting an equivalence across the various clicker patterns (Table 3). However, we noted that the interaction term was retained and approached significance in the final model (P = 0.055), which led us to visualize how each of the different clicker patterns related to homework explanations (Fig. 3). We saw a trend where students who answered both votes correctly were more likely than any of the other clicker vote patterns to provide a homework explanation that demonstrated understanding, but we did not conduct post hoc significance testing because the interaction term was not previously significant.

TABLE 3

Effect of clicker vote patterns on homework explanations

Factor	Estimate	SE	Z value	P value
(Intercept)	0.385	0.103	3.746	<0.001
Vote 1 correct	−0.176	0.145	−1.214	0.225
Vote 2 correct	−0.073	0.104	−0.696	0.487
Gender (male)	−0.389	0.152	−2.555	0.011
Generation status (first generation)	−0.244	0.110	−2.229	0.026
Vote 1*Vote 2	0.294	0.153	1.918	0.055
Vote 1*gender	0.228	0.120	1.901	0.057
Vote 2*gender	0.234	0.152	1.545	0.122

Logit (homework explanation) was calculated as follows: V1 + V2 + gender + generation status + V1*V2 + V1*gender + V2*gender (see Table S4 in the supplemental material for information on model selection). P values in boldface indicate significance (P < 0.05).

FIG 3

Effect of clicker vote patterns on homework explanations. Dots represent modeled least-squared means and bars represent 95% confidence intervals.

Although we did not detect a formal effect of clicker vote patterns on homework explanations, we wanted to determine whether clicker participation was associated with any benefit on later homework explanations compared to being absent. Since they were statistically equivalent, we collapsed all four clicker vote patterns into a single group, representing when students were present for the targeted clicker question. We found that being present for the clicker question predicted a higher likelihood of providing a homework explanation that demonstrated understanding (Fig. 4A; see also Table S5 in the supplemental material). When we explored this result by quartile, we saw that the lowest quartile being present in class did not significantly improve their later homework explanations (Fig. 4B; see also Table S5). Conversely, the top three quartiles (Q2, Q3, and Q4) appeared to experience a larger benefit of being present, but this only reached statistical significance for Q2 and Q4.

FIG 4

Effect of clicker participation on homework explanations overall (A) and for each student quartile (B). Dots represent modeled least-squared means and bars represent 95% confidence intervals.

Research question 3: How do clicker vote patterns relate to performance on isomorphic exam questions?

We next investigated how clicker vote patterns related to student performance on isomorphic exam questions. This model predicted student performance on isomorphic exam questions as the outcome variable and again included each clicker vote along with an interaction term. In this case, we found that the V1*V2 interaction term had a significant effect (Table 4), so we visualized and conducted post hoc analysis of the various pathways that students could take through the clicker question sequence (Fig. 5; see also Table S7 in the supplemental material). We found that answering both votes correctly (i.e., correct-correct) was associated with a student scoring significantly higher on the corresponding exam question compared to all other vote patterns (i.e., incorrect-correct, incorrect-incorrect, correct-incorrect), while none of these other patterns predicted significantly higher exam performance relative to each other.

TABLE 4

Effect of clicker vote patterns on isomorphic exam question performance

Factor	Estimate	SE	Z value	P value
(Intercept)	1.265	0.091	13.950	<0.001
Vote 1 correct	−0.227	0.104	−2.175	0.030
Vote 2 correct	−0.086	0.071	−1.207	0.228
Homework completed	−0.062	0.031	−1.974	0.048
Gender (male)	−0.048	0.094	−0.511	0.610
Generation status (first generation)	−0.049	0.126	−0.387	0.699
Race and ethnicity (URM)	−0.231	0.118	−1.959	0.050
Vote 1*Vote 2	0.535	0.110	4.849	<0.001
Vote 1*gender	0.166	0.081	2.043	0.041
Vote 2*generation status	−0.158	0.110	−1.443	0.149

Logit (exam score) was calculated as follows: V1 + V2 + homework completion + gender + generation status + V1*V2 + V1*gender + V2*generation status (see Table S6 in the supplemental material for information on model selection). P values in boldface indicate significance (P < 0.05).

FIG 5

Effect of clicker vote patterns on isomorphic exam question performance. Dots represent modeled least-squared means and bars represent 95% confidence intervals.

We then analyzed how these clicker patterns compared when a student was absent for a targeted clicker question. For this analysis, we collapsed the three vote patterns that did not significantly differ from each other (i.e., incorrect-correct, incorrect-incorrect, correct-incorrect). We discovered that students answering both votes correctly scored significantly higher than either of the other groups and that students with any other clicker pattern also performed significantly higher on the exam questions than students who were absent (Fig. 6A; see also Table S8).

FIG 6

Effect of clicker vote patterns on isomorphic exam question performance compared to being absent overall (A) and for each student quartile (B). Dots represent modeled least-squared means and bars represent 95% confidence intervals. Absent, did not submit clicker votes; correct-correct, answer both votes correctly; any other pattern, submitted incorrect-correct, incorrect-incorrect, or correct-incorrect votes.

We saw a more nuanced trend emerge when we explored student quartiles (Fig. 6B; see also Table S9). Compared to being absent, the lowest quartile saw no significant benefits of either answering correctly on both votes or having any other clicker pattern. Conversely, the top three quartiles saw an apparent benefit of answering both votes correctly, reaching statistical significance for Q2 and Q4. For the three top quartiles, having any other clicker pattern was not significantly better than being absent.

Research question 4: How do homework explanations relate to performance on isomorphic exam questions?

While the clicker pattern analysis provided insight into how student experiences during class related to their later understandings, we also wanted to see how student homework explanations related to their performance on isomorphic exam questions. Students who gave a homework explanation that demonstrated understanding were more likely than students who gave an incomplete or incorrect explanation to answer the isomorphic exam question correctly (Fig. 7A and Table 5). Furthermore, this relationship was maintained across all quartiles, suggesting that a student’s homework explanation generally predicted how well they performed on the related exam question (Fig. 7B; see also Table S11 in the supplemental material).

FIG 7

Effect of homework explanation on isomorphic exam question performance overall (A) and for each student quartile (B). Dots represent modeled least-squared means and bars represent 95% confidence intervals.

TABLE 5

Effect of homework explanation on isomorphic exam question performance

Factor	Estimate	SE	Z value	P value
(Intercept)	0.728	0.103	7.070	<0.001
Clicker participated	0.266	0.091	2.915	0.004
Homework explanation (demonstrated understanding)	0.621	0.068	9.158	<0.001
Gender (male)	−0.060	0.092	−0.654	0.513
Generation status (first generation)	−0.140	0.087	−1.615	0.106
Homework explanation*gender	0.206	0.110	1.867	0.062

Logit (exam score) was calculated as follows: clicker participation + homework explanation + gender + generation status + homework explanation*gender (see Table S10 in the supplemental material for information on model selection). P values in boldface indicate significance (P < 0.05).

DISCUSSION

This study investigated various associations between student participation in clicker questions, their opportunity and ability to explain concepts on related homework activities, and their subsequent performance on isomorphic exam questions. Overall, we detected significant benefits of participating in clicker questions and completing the homework activity (Fig. 1A and Table 2), suggesting that each of these instructional approaches had a net positive influence on student outcomes. This data set also provided a means by which to better understand the different ways that students progressed through these different stages.

We analyzed how clicker vote patterns related to homework explanations and isomorphic exam questions, two delayed measures of the learning that students might have achieved as a result of the clicker process. We found that being present for a clicker question generally predicted whether a student would demonstrate understanding on their later homework explanation (Fig. 4A; see also Table S5 in the supplemental material). Our statistical analyses do not allow us to attribute this effect to any particular vote pattern (Table 3), but we noted that students answering both clicker votes correctly had a tendency to perform better on the homework (Fig. 3). Given that all students who attended class heard an explanation for each MTF statement and could take notes that they might then use for the homework question, we can see how being present could have given a general boost to students and how the shared experience of hearing an explanation might have made students more similar to each other in terms of how they described the clicker answers. For isomorphic exam questions, we again saw a general benefit of being present for a clicker question, but the improved performance occurred predominantly when students answered both clicker votes correctly (i.e., correct-correct) (Fig. 5; see also Table S7). Interestingly, in both cases, students who changed from an incorrect to a correct clicker vote did not perform any better than students who were incorrect on both votes, suggesting that the improved performance they experienced as a result of peer discussion could not be recapitulated at later time points.

Finally, we explored how the effects varied by student quartile. Unfortunately, the lowest-quartile (Q1) students saw little benefit from participating in clicker questions compared to being absent. Students in the higher quartiles saw more positive outcomes, showing various effects of participating in clicker questions on subsequent homework explanations and isomorphic exam questions, culminating with students in the top quartile (Q4) demonstrating the greatest overall benefits of clicker participation on exam performance (Fig. 2B; see also Table S2). Conversely, the positive experience of providing a homework explanation had a more even effect (Fig. 2C; see also Table S3), and students who were able to provide a correct homework explanation were more likely to perform well on the related exam question across quartiles (Fig. 7; see also Tables S10 and S11 in the supplemental material).

Altogether, our findings suggest a potential role of clickers and associated homework explanations in reinforcing nascent conceptions. Indeed, answering both clicker votes correctly and providing a homework explanation that demonstrated understanding each predicted a higher likelihood that a student would perform better on the later exam question. While this could simply represent a selection effect for concepts that students already knew, we noted that the effect required students to participate in clicker questions (i.e., it was greater than the knowledge they had when absent, even for higher-performing students) and that the effect specifically occurred for the correct-correct pattern (i.e., just being correct on the first or second vote did not have the same effect). This result resonates with a prior study that examined student reasoning before and after peer discussion and found that explanation quality improved for students who correctly answered both clicker votes, suggesting that peer discussion specifically enabled these students to enhance their understandings (56). With respect to homework, students in our study were randomly assigned to address only one of the two targeted clicker questions that occurred in a given week, so the homework explanation effect can be linked more directly to the act of seeing and explaining a particular question. Additional research would be needed to distinguish potential effects due to cuing students toward test content, spending more time with certain concepts, and writing explanations to targeted questions.

Our finding that the highest-quartile (Q4) students experienced the most consistent benefits of clicker participation resonates with our previous clicker research (35) and other studies finding that active learning differentially benefits top students. This phenomenon has been observed in contexts utilizing small group work, such as class periods replaced with small group exercises (57) or courses taught using the SCALE-UP approach (58). Higher-performing students have also been seen to benefit more from retrieving versus copying, suggesting that the nature of the task can influence how different groups learn (41). We speculate that higher-performing students in our study may have driven their peer discussion groups, and this experience enabled them to articulate their reasoning and refine their understandings during the subsequent instructor explanation. In turn, lower-performing students may have had less opportunity to explain their ideas during peer discussion, thus limiting their ability to retrieve, formulate, and advance their conceptual understanding.

One limitation of our study was that we were unable to distinguish clicker participation from other class activities occurring that day, such as lecturing on the given topic. Previous research has attempted to disentangle these effects by comparing targeted exam question performance to alternative course offerings that did not include the corresponding clicker question or to various types of nontargeted exam questions and has generally attributed the resulting benefits to clicker questions (17, 24, 59–61). Our research here focused more directly on understanding clicker vote patterns and differential benefits of the various course structures.

This research reflects the challenges associated with implementing and studying activities embedded within a complex course environment, and we recognize how the instructional context shaped student engagement in ways that might differ from other courses (38, 40, 62). Our statistical models also revealed effects of gender and generation status, and while we did not examine these effects in detail, these findings further underscore the complex social dimensions underlying course activities and performance. Overall, our results provide support for the use of clicker questions and associated homework to guide and reinforce emerging understandings. However, additional work is needed to identify clicker approaches that provide benefits for lower-performing students and in cases where students switch from incorrect to correct votes. This can potentially be achieved through more deliberate attention to the composition, norms, or procedures of peer discussion groups. For example, previous research has suggested that lower-performing students may benefit from participating in more homogeneous groups because this allows them to work through activities without the influence of a higher-performing peer (63). Students engage in a variety of behaviors during peer discussions (64), and so modeling and monitoring discussion behaviors more explicitly might also help students engage with incorrect ideas and develop rationales for correct answers.