Student Thinking in the Professional Development of College Biology Instructors: An Analysis through the Lens of Sociocultural Theory.

An important facet of inclusive, student-centered science teaching is for college instructors to reveal and respond to student thinking. Professional development (PD) provides formal settings for instructors to develop skills attending and responding to student ideas in their teaching. Using the lens of sociocultural theory, the purpose of this study was to explore the learning experiences of college instructors in long-term faculty learning communities (FLCs) that focused on student thinking. This study employed a qualitative design using semistructured interviews, analyzed through qualitative thematic analysis. We investigated the ways that social interactions focused on artifacts of student learning facilitated college instructors' internalization of knowledge about teaching and learning. We found that participants valued the social space of the FLC for the camaraderie and diverse perspectives it facilitated and that participants internalized the discussions from their FLCs in the form of new insights into student thinking and plans for improving teaching. Our data support the idea that PD for college science teaching that includes social space focused on artifacts of student learning will lead to instructor learning. Further, our data point to the fruitfulness of new research to expand our knowledge of the implications of sociocultural theory for college science PD.

INTRODUCTION

The 2011 Vision and Change report called for inclusive, student-centered, and evidence-based undergraduate biology instruction (American Association for the Advancement of Science, 2011). This type of teaching involves college instructors discovering and responding to student thinking. We define student thinking as the ideas and thought processes, both canonical and noncanonical, that students bring to the classroom (National Research Council, 2001). Few professional development (PD) programs in higher education center around data on student thinking. In this paper, we present an investigation of the impact of one PD program that privileged data on student thinking.

Prioritizing student thinking leads to better student learning, because instructors who attend to student thinking expand their ability to make informed pedagogical decisions (Ball, 1993; Black ; Franke and Kazemi, 2001; VanSledright, 2002; Monte-Sano, 2011; Levin ). Instructors who pay attention to what students say ascertain clues about student reasoning. By intentionally eliciting and evaluating student reasoning, instructors can determine the gap between the learning goals and students’ current state of understanding (Furtak and Ruiz-Primo, 2008). Understanding this gap enables instructors to make informed decisions about their next pedagogical steps (Levin ). Thus, missing opportunities to elicit student thinking leads to more missed teaching opportunities. College instructors need to hone their teaching practices, including the practice of soliciting their students’ ideas and subsequently teaching in ways that build on those ideas (Ball and Cohen, 1999; Levin ).

PD provides formal settings for instructors to develop skills in attending and responding to student ideas in their teaching. Opportunities for PD in higher education have grown over the past several years. For example, many science, technology, engineering, and mathematics (STEM) disciplines now offer workshops, such as the Summer Institutes for Scientific Teaching (Pfund ). Thousands of STEM faculty have gained familiarity and practice with evidence-based pedagogies through these workshops (Baker ; Manduca ). Faculty learning communities (FLCs) are also now widespread (Cox, 2004; Elliott ). These small groups of instructors meet regularly to exchange ideas and thereby establish a community of learners. FLCs tend to result in a product, such as a lesson (Elliott ). Workshops, FLCs, and other PD models hold an important place in higher education, yet they often focus on pedagogies without considering the importance of building on students’ prior knowledge (Henderson ). There is a need for more PD that focuses on student thinking.

The current research investigates the PD model offered by the Automated Analysis of Constructed Response (AACR) project, a long-term initiative explicitly focused on student thinking as revealed in student writing (Urban-Lurain ; Beyond Multiple Choice, n.d.). The AACR project develops constructed-response assessments of core STEM concepts that capture students’ knowledge of the concepts. AACR assessments require students to respond in their own words, so the assessments reveal students’ conceptual understanding in greater detail than multiple-choice assessments (Haudek ). The AACR project analyzes student writing using computer-automated methods and generates reports within minutes on the categories of student ideas present. Thus, the AACR project provides college instructors with insight into student thinking. Research on student thinking from AACR questions and reports corroborates the notion that students possess heterogeneous ideas, including mixtures of canonical and noncanonical conceptions (Beggrow ; Haudek ; Nehm and Schonfeld, 2008; Parker ; Prevost ; Schmiemann ; Weston ). The AACR project formed FLCs in which instructors reviewed data on their own students’ thinking from AACR reports, discussed teaching and educational issues in undergraduate science education (McCourt ; Urban-Lurain ), and created and implemented a lesson based on student-thinking data (Pelletreau ). Prior research on AACR FLC participants showed that instructors became motivated to attend to student thinking (Zagallo ). Here, we report on the learning experiences of college biology instructors who engaged in this FLC PD program that prioritized social interactions and the review of artifacts of student learning.

Theoretical Framework

We utilized Vygotsky’s sociocultural theory of learning and development as the theoretical framework in this study (Vygotsky, 1978). Sociocultural theory is widely recognized for its foundational role in shaping modern views of education and is sometimes viewed synonymously with derivative terms, including social constructivism (Amineh and Asl, 2015), socially shared cognition (Resnick ), and socially distributed cognition (Hutchins, 1991). However, we use the term “sociocultural theory,” because our theoretical framing focuses on claims directly from Vygotsky’s work, expounded by contemporary researchers (e.g., Wertsch and Tulviste, 2013). Broadly speaking, Vygotsky (1978) posited that cognition stems from social interactions, which are mediated by artifacts of culture, such as materials, language, and symbols. Vygotsky’s views emerged from his studies of children learning from adults in authentic settings. He focused on two practical issues in educational psychology: the assessment of children’s cognitive abilities and the evaluation of instructional practices (Wertsch and Stone, 1985). In recent years, there has been growing interest in applying a sociocultural lens to learning in the K–12 context. This has involved studies of both student learning and instructor learning, including instructor learning during PD (Eun, 2008; Shabani ; Rigelman and Ruben, 2012; Shabani and Ewing, 2016). In this study, we applied Vygotsky’s sociocultural lens to investigate instructor learning in the context of higher education PD. To assist readers in understanding our approach, we discuss three tenets of sociocultural theory that are most relevant to exploring the learning of college instructors in our study.

First, sociocultural theory asserts that learning is social. Learning occurs as individuals repeatedly interact within cultural contexts, including peer groups, schools, and professional learning communities (Lantolf ). These communities provide time and space for creating and exchanging knowledge via interpersonal interactions and collaboration (Renshaw, 2003; Hord and Hirsh, 2008; Klar, 2012). Furthermore, social interaction among two or more individuals motivates learners, and motivation is a well-established precursor to learning (Blumenfeld, 1992). Language, such as conversation, mediates the most effective social interaction (Eun, 2010; Tasker ; Mercer and Howe, 2012). For example, learning can arise in peer groups in a college biology classroom. These groups create time and space for students to talk about relevant concepts. The interpersonal dynamics motivate engagement with the material, and the conversations foster student development, as learners help each other explain and refine their ideas. Likewise, learning can arise in peer groups of college instructors. In this study, we investigated the impact of the social context of FLCs on college instructors’ learning.

Second, according to sociocultural theory, learning is a mediated action. Learners construct knowledge while interacting socially with others and artifacts in their environment (Vygotsky, 1978; Eun, 2010). The dynamic social interactions with these artifacts shape the learners’ minds, as language often is the mediator that facilitates learning (Wertsch, 1985). When knowledge and ways of thinking are external before internal, physical objects, whether cultural or otherwise, can mediate learning. Consequently, individual learning can be understood as a mediation process that is triggered by artifacts that are available in the learners’ social environment (Wertsch ). Artifacts can act as cultural amplifiers, passing on culture and ways of knowing, thinking, and acting through use. Vygotsky’s work addressed the notion that each higher psychological function, such as learning, is a social relationship with another person mediated by “elements,” such as signs, language, and artifacts (Roth, 2019). Because learning is culturally mediated, learners interact with one another, artifacts (such as learning materials), and their environment. These interactions leave behind traces, such as student work, that provide insight into student thinking (among other areas). For example, when an instructor and a group of students participate in purposeful dialogic discussions, such social interactions enable the teacher to change the instruction to match the developmental level of the students involved (Sherman ). To become effective, instructors must examine the cultural artifacts of student learning. Doing so enables instructors to see how students comprehend the teaching and learning environment as well as how the instructors can modify their practices. In this study, we examined the mediating role of a cultural artifact of student learning (i.e., AACR reports containing student writing) on the professional development of college instructors.

Third, sociocultural theory posits that learning and development in social settings lead to internalization. Vygotsky wrote, “Every higher mental function was external because it was social before it became an internal, strictly mental function; it was formerly a social relation” (Vygotsky, 1997, p. 105). All higher mental functions, such as attention and self-regulation, are internalized social relationships (Vygotsky, 1981; Kozulin ). Thinking begins on the social plane before it becomes individual knowledge (Samaras and Gismondi, 1998). Ideas exchanged through dialogue in social settings lead to reflection, which can subsequently spark mental activities, such as questioning and planning (Lantolf ). For example, imagine a student who interacts with an instructor and peers in a biology class focused on pathways for energy transformation. While studying, the learner continues to think about the pathways and related concepts discussed during class. The student’s reflection brings questions to mind (e.g., “What is the difference between the pathway we studied this week vs. last week?”). Questioning and further reflection lead the student to refine her/his ideas (e.g., “Oh, one pathway explains how the cell obtains energy in the form of ATP, and the other pathway utilizes this ATP for anabolism.”). Likewise, instructors can internalize their learning in the context of PD. During PD, instructors might talk about teaching without at first thinking deeply about their teaching, without internalization. However, with the right support, instructors can begin to reflect on ideas they encounter in PD and ways their pedagogy might change, and instructors can come to own for themselves the ideas and reflections shared in the social setting around cultural artifacts, that is, they can internalize the ideas and reflections. According to Vygotsky, once learners internalize their learning, they perform tasks consistently without assistance from others. In this study, we considered the internalization of knowledge by college instructors who participated in PD.

AACR FLCs were designed to facilitate internalization among participants. Specifically, we imagined that a social setting consisting of supportive peers (i.e., social interaction) combined with the review and discussion of AACR reports about students’ biological conceptions (i.e., mediation via student writing) would transform into new ways of thinking about teaching and learning. According to Vygotsky, the cognition within an FLC would serve as the basis for cognition within the individual (Wertsch, 1991). To this end, we hypothesized that the social interactions centered on students’ conceptions would serve as the basis for individual reflection, self-regulation, and planning.

The purpose of this paper was to examine individual interviews of AACR FLC participants for evidence of the socially mediated process of instructor learning. Specifically, we investigated two research questions: 1) What types of social interactions focused on artifacts of student learning did college instructors value in a PD focused on artifacts of student learning? 2) In what ways did social interactions focused on artifacts of student learning facilitate college instructors’ internalization of knowledge about teaching and learning?

METHODS

Study Context and Participants

Twenty-four instructors from six institutions with very high research activity participated in the AACR FLCs over 5 years from 2014 to 2019. Each local FLC consisted of college biology instructors who met three times per semester for 1 hour each meeting (McCourt ; Pelletreau ). The number of participants at each institution ranged from two to five members, and principal investigators from the AACR project facilitated the groups. The goal of the AACR FLCs was to provide data on student thinking from instructors’ own courses and opportunities for instructors to use this evidence to reflect on and plan for teaching. Students’ written responses to the AACR assessment questions were analyzed with a computer-based system that can predict experts’ scoring of students’ responses (Haudek ; Nehm ; Nehm and Haertig, 2012; Kaplan ; Prevost ). Following the analysis of their students’ written responses, instructors received AACR reports that included categorizations and various representations of their students’ ideas along with students’ actual written responses (Supplemental Figure S1). The FLCs supported biology instructors in discussing the AACR questions, interpreting reports about their students’ thinking, and sharing ideas to modify their teaching approaches to incorporate student thinking. All AACR FLCs received the support of local administrators, some of whom became participants themselves. All participants held full-time, tenure-track or teaching-intensive positions at their institutions. A summary of the instructors’ demographic information is included in Table 1. The female to male ratio of the 24 participants was 1:1. The courses the participants taught varied from introductory biology to upper-division courses, including genetics, evolution, and cell and molecular biology. This study was approved under exempt status by the University of Georgia IRB (protocol 00000257).

TABLE 1.

Participant demographic information organized by years of teaching experiencea

Position	N (female, male)	Average years of teaching experience
Associate professor	3, 6	17
Lecturer	21, 2	17
Professor with administrative roles	1, 1	18
Senior instructor	1	20
Professor	4, 2	26

Data on race and ethnicity were not collected. Participants ranged from 10 to 33 years of teaching experience and held titles from instructor to full professor with administrative roles such as assistant dean and department chair.

Data Collection

We employed a qualitative design using semistructured, in-depth interviews. Interviews provide in-depth information about participants’ experiences and perspectives (Turner, 2010). Throughout the longitudinal AACR FLCs, we conducted semistructured interviews with FLC participants four times, including three times during the project (years 1, 2, and 3) and once at the end of the project (year 5). In this study, we focused on participants’ year 3 interview data.

We organized the interview protocol (see Supplemental Figure S2) around a set of predetermined topics that built upon the prior work of the AACR project (McCourt ; Zagallo ). We asked follow-up questions during the interview to clarify participants’ responses and gain greater insights about their PD experiences (Whiting, 2008). We worded the questions in an open-ended form, such as “You mentioned that you learned about your students’ misconceptions in evolution. Could you elaborate more on that, please?” We scheduled all interviews in advance and conducted them via videoconference calls. Each interview session was audio-recorded and lasted between 1 and 1.5 hours.

Data Analysis

We conducted a qualitative thematic analysis of year 3 interviews (Braun and Clarke, 2006; Kuckartz, 2014). Three authors served as the primary (SJ) and secondary coders (JC, NB) in our analyses. The primary (SJ) and secondary coders (JC, NB) applied a priori codes from two prior studies of AACR FLC participants (McCourt ; Zagallo ). Using a priori codes allowed us to explore participants’ experiences as they continued to participate in these FLC meetings. When we encountered data that did not fit within the a priori codes but pertained to our research question, we added new codes or revised the definitions of existing codes. We based the code revisions on analytic memos that the primary (SJ) and secondary coders (JC, NB) independently maintained throughout the analysis (Saldaña, 2013). The coders (SJ, JC, NB) negotiated and discussed the definitions of the codes throughout this iterative process (Strauss and Corbin, 1998). We refined the codes throughout the process to describe the phenomena that emerged from the participants’ responses interview data. We revisited and clarified the definitions of the codes until coding was consistent among the researchers. We generated a final codebook and applied these codes to the entire data set. In a final step, we organized codes into larger subthemes and themes, which we present in the Results. The correspondence among codes, subthemes, and themes is diagrammed in Supplemental Figure S3. All coding was completed in MAXQDA18. We used pseudonyms for all participants and randomly assigned an alphabet designation (A–F) for institutions.

RESULTS

We designed AACR FLCs to include a social space for discussion of student thinking about biology as revealed in AACR reports. We hoped that this social activity would lead to internalization, that is, higher mental activity such as attention, questioning, and planning, among participants (Vygotsky, 1981; Kozulin ). We used interview analysis to determine the extent to which internalization occurred. In this section, we report the results of our analyses to address our specific research questions: 1) What types of social interactions did college instructors value in a PD focused on artifacts of student learning? 2) In what ways did social interactions focused on artifacts of student learning facilitate college instructors’ internalization of knowledge about teaching and learning? We present three themes from our analyses. Theme 1 substantiates that participants valued the social interactions for the camaraderie and diverse perspectives they facilitated. Themes 2 and 3 provide evidence of internalization, showing that social space and artifacts of student thinking led our instructors to new insights about student thinking (theme 2) as well as questions and plans for improving teaching (theme 3). Table 2 summarizes and illustrates these themes with quotes from the data. We describe each theme in the sections that follow. Quotes have been lightly edited for clarity and readability.

TABLE 2.

Summary of findings

Instructors’ perceptions	Evidence from quotes
Instructors valued social interactions that promoted camaraderie and diverse ideas.	“I guess there’s the departmental camaraderie that’s developed from the FLC meetings, and it becomes a lot easier to talk about problems that we’re having in classes because we can focus in on particular issues that are common to all of our courses. Regardless of the level or what not. It’s not just a session where we complain about students who don’t know anything or whatever. We can actually have a reasonable conversation about—It’s clear that none of our students at any of our classes get this one particular thing.”—Wallace, Institution D“Basically, having different perspectives on, for example, teaching evolution is helpful because we got a guy in the FLC that does evolution and teaches an upper-level evolution course. He’s like a different kind of biology guy than we are. That’s really, really helpful.”—Evelyn, Institution A
Instructors internalized new insights into student thinking.	“Students were still struggling with the concept of the stop codon not causing transcription to stop, but they got that it caused translation to stop. The other thing that the AACR questions revealed to me is that students didn’t know where to start transcription and start translation. Even though it said, ‘Here, it’s the plus one start site for transcription,’ some of them still started in the untranslated region. Another thing was that a lot of them forgot about the idea that translation starts somewhere else other than the beginning of the transcript. Those were the things that I saw in their responses, so then I would go back and talk to my students about them.”—Stephanie, Institution B
Instructors internalized questions and plans for improving teaching.	“[AACR questions and reports] give you some snapshot of student thinking before and after you did something in class. I can ask an AACR question before and after and use it as a particular tool to get an idea of whatever it’s measuring. Then, the report tells me, ‘Did we move the needle at all?’ I think it’s useful in that way so I can adjust my teaching. Because if it didn’t change at all, then you’ll have to ask yourself, ‘Is what I did in class really having an impact?’”—Kenneth, Institution B“I think it gives you a good idea of what the misconceptions are, what they don’t know and what you can focus on. Like when I teach the genetic code next year, … I’ll make sure to cover the misconceptions that came out from the AACR questions and the answers.”—Annalisa, Institution C

Theme 1. Instructors Valued Social Interactions That Promoted Camaraderie and Diverse Ideas

The AACR FLCs created a social setting for learning focused on artifacts of student thinking. The instructors’ perception of the importance and value of these PD components showed up in the interviews. We found that instructors experienced camaraderie through the diverse perspectives shared in their FLCs. As they reviewed reports on student writing with one another, which they found more motivating than working individually, they realized they were not alone in navigating the challenges of student thinking.

Instructors Valued Camaraderie in Their FLCs.

Instructors in the AACR FLC met with one another three times per semester over several years to review AACR questions and reports. Even though the conversations started with AACR, they typically moved into broader discussions of teaching and learning as well. These regular discussions fostered a sense of camaraderie and equity. Wallace and Jarvis from Institution D both explained this idea. Wallace, who was a new lecturer, described the camaraderie, equal treatment, and psychological benefits he found in his FLC:

Well, one it’s just camaraderie. As an early-career scientist and new lecturer…, it was good to be treated as an equal, and a psychological benefit as well, where everyone is treated equally. There is no feeling of inferiority or anything like that, and there’s although, we know, [our facilitator] is in charge, she doesn’t, in any way act like she’s the only one that has input. Everybody has equal input, and it’s been great getting to know other people and getting to know what they teach and what their expertise is in.—Wallace, Institution D

Jarvis described his FLC as a supportive space with affirmative feedback, a place where he could “sound ideas off of other people”:

[The FLC] was a place that one could sound ideas off of other people and were fairly supportive. They would probably just say, “Yes, go for it. Try it out,” and rather than them saying, “No, don’t do that.”—Jarvis, Institution D

Joyce from Institution B expressed a related sense that the FLC created a supportive, relational space for instructors who wanted to focus on teaching:

The FLC is nice because other people also think it’s important for us to get better at what we do, and therefore, just having that support group saying, “Yes, this is important. Yes, you should be spending your time there.” It’s very helpful.—Joyce, Institution B

Instructors’ Collective Review of AACR Reports Contributed to Their Sense Of Camaraderie.

Reports on student responses to AACR questions provided a computer-based analysis of student writing. The reports included categorizations of students’ ideas, maps showing the co-occurrence of ideas, and the actual constructed response of every student in the sample (see Supplemental Figure S1 for a sample report). AACR participants felt they could interpret the reports independently, but they preferred to do so with their FLCs. As Lourdes said, instructors were more motivated when they reviewed reports with their FLC colleagues, and the group prompted them to think more deeply about what they were seeing:

I think I could probably do it [review the AACR reports] on my own, but I like doing it with the group. Because it’s like when we go over the data, it makes us think a little bit more about it. For me, [reviewing the reports with my group] is motivating. Also, I just think … five heads are better than one.—Lourdes, Institution F

The collective review of AACR reports made our instructors realize they were not alone in navigating the challenges of student thinking. For example, Wallace from Institution D talked about the FLCs offering a place to discuss concerns that were common to all of their classes. He and his FLC members discovered that all of their students were struggling with a particular idea:

I guess there’s the departmental camaraderie that’s developed from the FLC meetings, and it becomes a lot easier to talk about problems that we’re having in classes because we can focus in on particular issues that are common to all of our courses. Regardless of the level or what not. It’s not just a session where we complain about students who don’t know anything or whatever. We can actually have a reasonable conversation about—It’s clear that none of our students at any of our classes get this one particular thing.—Wallace, Institution D

Joyce from Institution B shared a similar idea, explaining the encouragement she received from learning that other instructors were struggling to help their students with the same “misconception” she encountered:

As I said, I like the discussions we have during the FLC meetings. It’s multiple instructors and we’re talking about problems we encountered and things that we see… The FLC, it encourages you. You know you’re not the only one struggling with this and you’re not in a vacuum. And you can get the advice, and you can see that, oh yes, yes, yes. It’s not me that I’m doing something wrong. This is just a major misconception that students have across bunches of instructions.—Joyce, Institution B

Stephanie from Joyce’s FLC stated simply that her FLC gave her a sense that she was not the only one whose students were struggling with particular ideas:

I also like to hear about what others are doing, how they’re dealing with it. It also gives me a sense that I’m not the only one who’s teaching a class where students struggle with a particular idea.—Stephanie, Institution B

Instructors Appreciated the Diversity of Ideas in FLCs.

Instructors expressed how much they valued the camaraderie and support of their FLCs, and it seems this was enhanced by the diversity of people and perspectives in the FLC. At Institution C, Lucia and Ronald reported that having colleagues with various perspectives was a positive experience. Lucia focused on the value of hearing from a colleague who taught a different biology course and brought a broad view of biology topics to the group:

It’s great having [my colleagues’] input because the other two instructors are involved in teaching the same course that I do, but having an additional perspective from Sophie [who teaches a different course], too, is really good because she provides her perspective on more broad biology topics.—Lucia, Institution C

Ronald echoed Lucia’s response about the diversity of their group, and he noted in particular that members in their FLC had opposing views on the importance of content. He liked that this healthy disagreement could lead to balanced approaches to teaching:

We have a very diverse group with some very divergent ideas about what would be important in that we have at least a couple of people I can think of that are possibly polar opposites…. One thinks one thing, and another thinks the other thing…. This is where it’s going to be interesting because I think we have one person that at least I believe, who is very heavily in favor of content. We have another person who is the opposite…. I think it would be a really good discussion. I think there might be healthy disagreement, but together we could probably come up with a good balance.—Ronald, Institution C

At institution A, both Evelyn and Kimberly, who taught cell and molecular biology, discussed how helpful it was to their teaching to hear from a colleague in their FLC who focused on evolution:

Basically, having different perspectives on, for example, teaching evolution is helpful because we got a guy in the FLC that does evolution and teaches an upper-level evolution course. He’s like a different kind of biology guy than we are. That’s really, really helpful.—Evelyn, Institution A

Kimberly stated the same idea more succinctly:

It’s cool to have [another colleague] in the FLC, and it’s because—I think he’s in the evolution bio department, so he’s bringing in a different perspective.—Kimberly, Institution A

Our participants valued the camaraderie, equal treatment, and diverse ideas in their AACR FLCs. As they reviewed AACR reports together and exchanged stories about students struggling with concepts, their sense of camaraderie grew. This social setting led to internalization among participants. In the next two sections, we present two themes that describe the forms of internalization we discovered.

Theme 2. Instructors Internalized New Insights into Student Thinking

AACR reports included categorizations of students’ ideas and maps of the co-occurrence of ideas (Supplemental Figure S1), but instructors gravitated most strongly to the actual student responses contained in the reports. Reports showed every student response from the sample organized by category. FLCs often looked initially at the categories, and then quickly moved on to reading and discussing example student responses. It may seem surprising that FLC participants with so much teaching experience (Table 1) were so eager to read student writing, yet many of them had previously relied entirely on multiple-choice assessments (McCourt ). The AACR FLC marked the first time that some participants had reviewed and discussed their students’ writing, and even those who had reviewed students’ writing previously gained a deeper understanding of student thinking through the collective review of AACR reports in their FLCs. Here we document evidence that participants internalized the review and discussions conducted in their FLCs in the form of new insights into student thinking. They turned discussions with colleagues into greater personal awareness and attentiveness to students’ ideas.

For example, Stephanie and her FLC colleagues spent time reading their students’ written responses and realized that students were struggling not only with the concept of the stop codon but also with the idea of where transcription and translation start:

Students were still struggling with the concept of the stop codon not causing transcription to stop, but they got that it caused translation to stop. The other thing that the AACR questions revealed to me is that students didn’t know where to start transcription and start translation. Even though it said, “Here, it’s the plus one start site for transcription,” some of them still started in the untranslated region. Another thing was that a lot of them forgot about the idea that translation starts somewhere else other than the beginning of the transcript. Those were the things that I saw in their responses, so then I would go back and talk to my students about them.—Stephanie, Institution B

Stephanie emphasized in her interview that the “value in the [AACR] report” was that it highlighted what the students did or did not understand, as evidenced by their written responses.

“When that’s connected to the FLC,” she pointed out, “then it becomes an opportunity to figure out how to help students better achieve the learning outcomes for the course.” Clearly, the discussions of student writing stuck with Stephanie in the form of attention, awareness, and insight for teaching.

Afton also described the new insights she gained from AACR reports. She described how limited she was in the type of assessments she used in her large-enrollment courses and how AACR reports made her aware of “misconceptions” she had not previously encountered:

Yes. I think the greatest thing with AACR is that in the large-enrollment, lower-level courses, we’re always pretty limited with our types of assessments. The open-ended questions are really helpful. For one, getting students to formulate their thoughts without being prompted, which they don’t have to do often enough at that level. Then for me to realize some misconceptions that I might not understand are there, because I’m not thinking of that as I’m putting together my multiple choices. Seeing those problems in the way that the students understand can really help me address that better.—Afton, Institution E

Helen from Institution A echoed Afton’s response, explaining how AACR reports revealed the “common misconceptions that you see among students”:

I guess, another really interesting thing is AACR questions give you a good idea of the common misconceptions that you see among students. You can see that in a multiple-choice question if you have a really good question, but I think the AACR questions are a little better because you might see things that you didn’t think about.—Helen, Institution A

Other instructors conveyed genuine surprise about students’ ideas as revealed in AACR reports. Evelyn and her FLC colleagues could not believe it when they learned that students did not know about stop codons:

Students didn’t know what a stop codon was. We, as instructors, didn’t even know that this was a problem. To be honest, we just thought, “What? I’m sure they know what that is.” But no, they don’t. Those kinds of things were really helpful to find out.—Evelyn, Institution A

The AACR FLC discussions that centered around student writing stimulated learning and development among participants in the form of awareness and insight about student thinking about biology. Theme 3 documents how instructors internalized the interactions in their FLCs to an even greater degree in the form of questions and plans for improving teaching.

Theme 3: Instructors Internalized Questions and Plans for Improving Teaching

Instructors noted ways they could improve their teaching in response to what they learned from the student writing contained in AACR reports. Discussions of their observations of student thinking led to exchanging ideas for classroom activities, feedback on pedagogical approaches, and classroom research data. These discussions facilitated instructors’ reflections on their teaching and how it could be better. We provide evidence here of internalization in the form of participants questioning their own teaching effectiveness and planning to improve teaching.

Instructors Internalized Questions about Their Teaching.

Instructors gained new insights from their examination of student writing contained in AACR reports. We asked instructors to elaborate on the ways they used these insights in their teaching. We found that instructors used their internalized awareness of student thinking to question themselves. More specifically, questioning took the form of self-critique. Kenneth shared how AACR reports helped him determine whether he “moved the needle” on student learning:

[AACR questions and reports] give you some snapshot of student thinking before and after you did something in class. I can ask an AACR question before and after and use it as a particular tool to get an idea of whatever it’s measuring. Then, the report tells me, “Did we move the needle at all?” I think it’s useful in that way so I can adjust my teaching. Because if it didn’t change at all, then you’ll have to ask yourself, “Is what I did in class really having an impact?”—Kenneth, Institution B

Kimberly shared her reflections, which were triggered by the discussions in her AACR FLC. She explained that student responses in AACR reports sometimes led her to acknowledge that her teaching did not work:

It’s always good to get feedback and figure out what happened and where students miss stuff because you really think you’re doing a great job. Then you get their answers, and you think, “Oh God, that didn’t work.”—Kimberly, Institution A

Joyce described the process of learning how important terminology is in communicating with students. As she reflected on the student writing from AACR reports, it occurred to her that students were confused by the word “metabolize,” and she admitted that she should have “said it differently”:

A lot of the times AACR is actually quite informative. I think this is what we run into all the time: We don’t understand what students don’t know, right? What’s tripping them up is often something trivial, like on our last open-ended response question, it was quite clear that I had a group of students who didn’t understand what [it means] when I say “metabolize something.” They didn’t know what that meant. I told them, “Enzymes help metabolize lactose,” but in their minds that meant the enzymes were needed to make lactose, right? I didn’t realize Intro Biology students didn’t know what “to metabolize” means. So I should have said, “broke down, broken down, to break down.” They give you the enzymes and you need to “break down” lactose. I should have said it differently. Then that wouldn’t have tripped them up.—Joyce, Institution B

Instructors Internalized Plans for Improving Teaching.

Instructors not only internalized their new awareness of student thinking for self-critique. They also made plans to change their teaching, and some participants actually reported implementing their plans. Instructors like Annalisa and Jarvis spoke of changes they would make to their course content. Annalisa planned to address student misconceptions the next time she taught genetics:

I think it gives you a good idea of what the misconceptions are, what they don’t know and what you can focus on. Like when I teach the genetic code next year, … I’ll make sure to cover the misconceptions that came out from the AACR questions and the answers.—Annalisa, Institution C

Jarvis echoed the notion that AACR informed modifications to his energy unit:

In the past, we just talked about, “Oh, you’re going to eat a whole bunch of carbohydrates and then you extract the energy from it.” Then we talked about cellular respiration, and we never talked about what happened to the carbohydrate once you consumed it, before you broke it down. Now we spend a lot of time talking about putting sugar in water and then if you remove the water, what happens to the sugar?—Jarvis, Institution D

Other instructors, such as Byron and Tobias, used the knowledge gained from AACR reports to implement modifications to the assessments in their courses. Byron incorporated pre- and post-exam questions that aligned with an AACR question:

Well, for the topics that we asked those questions on…, it does validate my teaching on that. I had a set of slides for that topic that I was teaching before. I gave the AACR question and got the report [a] couple times, and during that process, I was able to hone the material a little bit and also incorporate pre- and post-exam questions to go along with the analyzed question.—Byron, Institution F

Tobias from Institution E shared an experience of evaluating his exam questions based on what he discovered about students’ understanding of organisms:

For example, when we assess using the AACR evolution questions, we have that result where students tend to answer differently when the organism is an animal or a plant, and whether it’s a gain or a loss of a trait, which wasn’t shocking when I heard it, but it wasn’t something that I would’ve ever come up with out of my own brain. Now we know that even though these questions look like equivalent questions to us the experts, they should not be treated as equivalent questions. We are trying to match our exam questions to some standard or learning objectives, but we realize that you can maybe change the animal, but you can’t change the animal to a plant for that question.”—Tobias, Institution E

Furthermore, participants reported that the space and activity created by AACR FLCs led them to consider the development of student thinking not only in their own courses but also across different courses. We observed this shift from individual to collaborative planning at multiple institutions. For example, at Institution A, two participants shared distinct plans about aligning evolution instruction across different courses. Lawrence planned to approach his FLC about how he could alter his course to prepare his students for their courses. He saw an opportunity to align learning goals across the department:

AACR FLC is providing me the opportunity to start thinking about how my course aligns with the other courses or how it’s taught by the people in the room. [Our FLC] is a really amazing group. Now, I’m in a position where we could start talking about, “Hey, now what are the things that you think are really important?” Or, “What can I do to start changing or altering what I do now to better prepare my students for the courses that they might be teaching, and vice versa.” I think we’re entering a phase where we’re really genuinely thinking about aligning how we teach relative to the important learning goals that we have established for our department.—Lawrence, Institution A

Evelyn shared even more specific plans from Institution A about teaching evolution in different courses. The FLC planned to collectively design activities for teaching evolution at the introductory and advanced levels:

We have definitely planned to figure out some kind of activities for teaching evolution. We decided what we needed to do to get that going. We’re going to first compare what sort of things we are doing to teach in the evolution part of our courses. And then everybody will be on the same page with that. Then we’ll decide how we can do some activities and sorts of things that would be useful for an intro bio-molecular class and the evolution class.—Evelyn, Institution A

Similarly, Emmanuel at Institution D revealed that the FLC decided to administer the same AACR questions in four courses that ranged from the introductory to graduate levels. Implementing AACR questions across four courses enabled the FLC to investigate student thinking across the entire departmental curriculum:

It was partly a group decision, because we also wanted to have a secondary goal of using AACR questions, which was to help assess how we’re teaching evolution as a unit and as a department. It was decided from a discussion with other faculty members and also those who were teaching courses that had some evolution. I think we ended up asking the [AACR] questions across four courses that ranged from intro to graduate level courses.—Emmanuel, Institution D

Related project plans were reported at Institution F. Both Heidi and Lourdes articulated their FLC’s intention to ask the same AACR questions across three courses in their biology program and to develop an activity addressing student thinking on these topics. Heidi described their FLC members’ intention to try the same AACR question in different courses:

For this coming Fall for the teachers of the three sections, we were talking about working together to all try the same question or a similar question just to see how the students do.—Heidi, Institution F

Lourdes elaborated more on the FLC’s intention to collaboratively develop activities to address student ideas:

The AACR group has people from all those three courses and that’s really useful. Because it brings us a little bit closer to actually compare and contrast what we teach and how much overlap there is and what we need to do to coordinate things better. That’s been really good, and I would like for it continue. I think that, last time, we talked about trying to identify, for each of our courses, what are some fundamental and really important misconceptions, and also trying to work a little bit on seeing whether we could develop those ideas into some activity, or perhaps a question that could be used.—Lourdes, Institution F

These findings indicate that AACR FLC participants not only began to question student learning in their own courses but also to consider the student learning progression from introductory to advanced courses. Internalizing the FLC discussions, participants planned to improve student learning by implementing new cross-course assessments and activities.

In summary, our results show that the opportunity to review and discuss student thinking led to internalization among participants. Instructors focused on noticing and interpreting student thinking and gained new awareness about students’ ideas. These new insights led them to question themselves and act on those questions with plans to improve teaching both individually and collaboratively. This shift toward collaborative inquiry occurred as instructors realized they were not alone in navigating challenges pertaining to student thinking. As a result, they discovered the potential for a concerted effort that could positively impact student learning across the curriculum. It is important to highlight that instructors’ internalization was facilitated by the positive social dynamics of AACR FLC meetings, where participants developed a sense of camaraderie. Camaraderie contributed to the exchange of ideas in the FLC. These experiences culminated in preparing participants for individual and collaborative plans to address student ideas in biology across programs and courses.

DISCUSSION

Informed by Vygotsky’s sociocultural theory of learning and development, this study addressed the research questions: 1) What types of social interactions did college instructors value in a PD focused on artifacts of student learning? 2) In what ways did social interactions focused on artifacts of student learning facilitate college instructors’ internalization of knowledge about teaching and learning? We found that the intentional design of PD to include social space and artifacts of student learning led to internalization in the form of new insights into student thinking and questions and plans for improving teaching. In the following paragraphs, we make three claims about the design of PD based on the current study and prior PD research. We also point out where further research is needed to strengthen the outcomes of PD.

First, PD for college science teaching should include social space to promote internalization. AACR FLCs offered a social space for colleagues to meet and discuss teaching. Sociocultural theory reveals that the reason social interaction is important is because it creates new knowledge. Ideas emerge through social exchanges that would never occur otherwise. As sociocultural theory posits, AACR FLCs launched learning for our participants. Participants conversed about student thinking, and this led to ideas for individual and collective work on teaching. We know from other studies and our previous work that creating a supportive environment motivates and accelerates faculty learning (Eun, 2010; Manduca, 2017). For instance, D’Avanzo (2013) called for the use of cooperative teams in PD experiences. Ebert-May emphasized mentorships and reflection between PD leaders and participants. Manduca highlighted peer instruction in which faculty members view their colleagues as trusted sources of information and feedback about their teaching. Further, Holton (2005) proposed a PD framework that emphasizes peer interactions with like-minded peers. Similarly, our previous work on the AACR project showed that a sense of camaraderie motivated instructors to continue to engage with the project (McCourt ). Thus, the preponderance of evidence on PD points to the importance of social interactions in providing a safe learning space for college science instructors. Instructors in our study were ready for these types of interactions and seemed predisposed to value such social spaces, which were grounded in mutual respect among participants (McCourt ; Zagallo ). This study’s data show that the social space in the AACR FLCs served as the seed for change in participants’ thinking and possibly in their classrooms. Social space is a necessity for college science PD.

Second, PD for college science teaching should include student-learning artifacts to promote internalization. These artifacts provide the link between student learning and instructor learning (Sykes, 1999; Simon and Campbell, 2012; Wallace and Loughran, 2012). Examples of artifacts of student learning include measures of student performance and systematic observations of student work that indicate how students are learning in a course. In our case, AACR reports, which contained student writing, served as cultural artifacts of student learning that mediated instructors’ learning. Participants in our study reported critical insights on student thinking gained by examining AACR reports. It is noteworthy that our participants gravitated more to actual student responses instead of the broad categorizations of students’ ideas. As they read student writing, they noticed patterns in their students’ thinking (i.e., “misconceptions”) and discussed these observations with their colleagues. As participants engaged with this artifact through discussion, their thinking moved to teaching changes to address student thinking. This social interaction among participants provided formative feedback and led to collaborative work, thereby creating positive learning environments. Moreover, the sharing of ideas surrounding the original artifact (i.e., AACR reports) became a new artifact, knowledge of how to enact instructors’ findings from the reports. Informed by these artifacts, participants potentially learned how to support and address student thinking through improved teaching practice. We did observe participants’ classes using the Classroom Observation Protocol for Undergraduate STEM (COPUS) protocol (Zagallo ), but COPUS alone is likely insufficient to see the types of changes our participants planned to make. Some PD programs have focused on learning from student artifacts, such as assessments (Ebert-May ; Derting ; Emery ). However, these programs do not explicitly explore the mediating role of the artifacts and their impact on instructor learning. We argue that our participants’ learning was inextricably linked to the learning of their students, whose thinking was captured in writing, and their colleagues, whose thinking was conveyed in the social setting. The significance of conceptualizing college instructor PD from Vygotsky’s learning theory resides not in foregrounding the instructor but in focusing on the students. The students are ultimately the center of faculty PD through the cultural artifacts of student learning, which impact instructor learning (Simon and Campbell, 2012). Thus, we favor an approach to college instructor PD that focuses on building knowledge about teaching by capitalizing on the connection between student thinking captured as an artifact (writing, videos, etc.) and instructor learning.

Third, PD for college science teaching that includes social space and artifacts of student learning will yield internalization. Internalization occurs when an idea or concept moves from outside the mind to a place inside it (Bruner, 1985; Wertsch and Stone, 1985). The dynamic social interactions between artifacts and a group of learners shape the learners’ minds. Our participants showed evidence of internalization through their statements about new insights on student thinking, questions about their teaching effectiveness, and plans for improving their teaching. The external event of reviewing, interpreting, and discussing student writing became internalized in the form of self-critiques, new lessons, and assessments that instructors planned to implement. Among our participants, collective thought and reflection, which are vital to college instructors, led to individual knowledge (Van Huizen ). In other words, discourse facilitated self-critique, questioning, and planning that prepared our participants for change in the future. For instance, our participants recounted during interviews the ways their teaching seemed to be falling short, and they noted plans for working solo or with their colleagues to improve their presentation of content and their assessments in future classes. Manduca (2017) also points to the learning that is possible through PD. If instructors enact knowledge from PD in their classrooms and then return to a PD setting to discuss it, they are likely to carry the learning process further. Thus, we see internalization as an expected outcome of a PD program that involves social interactions mediated by cultural artifacts of student learning.

Additional research is needed to expand our knowledge of the implications of sociocultural theory for college science PD. Prior research and the current study provide strong evidence that college science PD will lead to internalization if that PD includes social space and student learning artifacts. However, we do not know if PD of this type will lead to the ultimate indicator of internalization: consistent performance of tasks without assistance from others (Vygotsky, 1997). We know that the instructors in our study made plans to change their teaching, but we do not know the extent to which our participants’ internalization manifested in actual changes in teaching practice. It is possible that participants made subtle yet influential changes like the ones they discussed in interviews (e.g., changing verbiage used to describe a concept or revising assessments). Future research should examine the extent to which instructors undergoing PD enact fine-grained changes in teaching practice. Doing so will require moving beyond coarse-grained observation protocols to methods that examine the particular moves instructors make in response to student ideas.

Bruner (1985) argued that “new and enhanced forms of teaching” require the teachers’ commitment to self-development in their profession (p. 11). Bouwma-Gearhart (2012) also demonstrated that external motivators initially encourage faculty to participate in PD, but internal motivation sustains their participation and facilitates change in practice. Consequently, we hypothesize that PD programs like the ones described in this Discussion will lead to the changes we seek in college science instructors’ teaching. Undergraduate science instruction can become increasingly student centered, built on student knowledge, and can provide opportunities for knowledge development. Finally, this proposed approach to PD can promote self-reflective teaching that fosters continuous improvement.

Limitations

Two limitations of the current work could be strengthened with future research. First, we did not investigate the extent to which the AACR FLC facilitators served as more advanced peers to help learners learn and develop. Vygotsky’s work points to the importance of more knowledgeable peers or facilitators who helps the learner progress through their zone of proximal development (Vygotsky, 1997). We know that there were varying levels of teaching experience among our AACR FLC participants, even though on average the sample was highly experienced. We also know that the FLC facilitators possessed extensive knowledge of teaching and learning, most likely beyond that of the FLC participants. However, we never collected data about the extent to which the facilitator or more knowledgeable FLC members served as mediators for others’ learning. Second, we learned that there is a small yet growing body of research on assessment thinking that investigates the reasoning associated with instructors’ assessment of their science students’ written work (see Talanquer ). Our study did not consider the orientations of the instructors when they were discussing and interpreting their students’ pattern of ideas as revealed in their writing. Future research that incorporates these perspectives could expand our findings.

CONCLUSION

Our proposed approach to PD can be used in program design and research. PD designed according to the sociocultural perspective attends to college instructor learning, which should lead to improved practice. Future research should examine the teaching practice of instructors undergoing PD informed by sociocultural theory. In particular, to what extent do instructors enact small-grain changes to teaching practice. Moreover, future PD programming should include accountability and incentives for instructors to practice what they learn as they examine and discuss student work. To this end, future studies can respond to important questions, such as: To what extent will college instructors adopt new practices in theoretically grounded professional development?

Click here for additional data file.